Well if your going to think of it like that, then the moment you take your units off any territory then it would mean that you have NONE there at all, which if your Germans would be bad news because the Guerilla or Partisan forces would immediatelly retake it. So when a territory has no units on it, but is still claimed by a specific country i think of it as that country has garrison troops there but none of actual combat effectivness :)
Thats just how i rationalize it… (America actually did have troops stationed in Iceland during the war, it was there that a US destroyer engaged a German U-Boat, neither was sank and the report was made so close to Germany’s declaration of war that it is hardly made notice of)