From page 37 of the second edition manual:
“While not at war with Germany or Italy, the United States may end the movement of its sea units on the Europe map only in sea zones that are adjacent to U.S. territories, with one exception: U.S. warships (not transports) may also conduct long-range patrols into sea zone 102.”
Thus, yes you may drop off US units there and have US ships in sz 121 while not at war.
edit: also from page 15:
“A neutral power can’t move land or air units into or through neutral territories. It can’t move units into or through territories or onto ships belonging to another power”