Lime has it correct.

They easy way to remember it is
GREY PIECES BELONG TO THE OWNER OF THE TERRITORY THEY ARE IN WHEN THAT TERRITORY CHANGES HANDS.