Several people have made some solid points in this thread. I’ll give my own summary of why a Japanese invasion was not a real possibility. (While acknowledging that some of the below points have been made already.)
Problem 1: Japan had fewer people than the U.S., which puts a lower ceiling on its army size than on America’s army.
Problem 2: the Japanese Army was significantly less modern and less well-equipped than the American Army. During WWII, Japan produced less than 3,000 tanks, compared to 88,000 tanks for the United States. Compounding that problem was that Japan’s tanks were mostly obsolete light tanks which could not penetrate the armor of Shermans. The U.S. produced 257,000 artillery pieces during WWII, compared to just 13,000 for Japan. America’s artillery were better than Japan’s, especially after the U.S. started using proximity fused shells. The U.S. produced 325,000 military aircraft during WWII, compared to 76,000 for Japan. Later in the war, the U.S. began enjoying a qualitative advantage in that area as well.
Problem 3: Japan’s army used worse tactics than the American Army (as alluded to by Imperious Leader). U.S. infantry were about 80% as combat-effective as their German counterparts, making the U.S. Army one of the best in the world. (The British were 50% as effective as the Germans, the Soviets about 33% as effective, and the Italians were less effective than the Soviets.) In battles between Japanese and American soldiers, the U.S. generally achieved a 2:1 - 4:1 exchange ratio, with the ratio going in the U.S.'s favor. In the battle of Guadalcanal, Japan repeatedly threw waves of its infantry into the teeth of American machine gun defenses. Major Western nations had learned not to do this during WWI.
Problem 4: Japan’s army was tied down by a large number of other commitments. CWO Marc mentioned that a large portion of the Japanese Army was preoccupied by its bloody and unwinnable war in China. In addition, the Japanese had to maintain a large force in Manchuria to guard against a Soviet invasion. They also had to garrison the various islands and Southeast Asian land territory they had conquered, and protect that territory against British forces in India and Australia.
Problem 5: (which you alluded to). A successful invasion would have required near-complete Japanese naval supremacy in the Pacific. That supremacy would have been increasingly difficult to attain as the war went on, due to America’s faster rate of naval production than Japan.
Problem 6 (which Malachi Crunch pointed out). The D-Day invasion was very difficult for the Allies even though they only had to cross 30 miles of ocean. Japan would have had to cross a much larger distance. Its productive capacity was much less than that of the U.S., so building up the needed invasion fleet + transport capacity would have been difficult or impossible.
Problem 7 (pointed out by Imperious Leader). Once the invasion force had arrived, supplying it would have been very difficult or impossible; especially after it had begun making significant inland penetration. The supply line (which I assume would stretch all the way back to Japan) would be insanely long. Japan’s industrial capacity was wholly inadequate to provide the naval transports and military trucks required to make that supply line work.
It’s impossible for me to imagine the Japanese overcoming these problems and successfully invading the United States.