One way to consider the Caucasus as the German player is that by extending yourself to take it, you are outflanking yourself!!
I mean, in the classic Soviet defense, all forces are called toward Belorussia and adjoining support areas, to keep the Germans at least two territories away from Moscow, as long as possible. Now, the German answer to this must be to pressure Belorussia with maximum force as available.
If you as the German player take Ukraine – and retake it – well, you are spending well over 4 IPCs per turn to do so. Figure the Soviets will withdraw from Caucasus anyway by turn 4 at the latest, assuming you are doing all right in the center and north. So why spend infantry (and armor & air power) on assaulting this flank zone, when the same infantry can be used to greater effect in resisting the Soviet counterattacks? I say concentrate on holding the Ukraine with maximum forces against the Belorussian flank, rather than spending them on the admittedly juicy Caucausus region. You’ll soon get it for “free,” regardless!
The “other hand” in this case would, I suppose, be that by taking Caucasus with a small but credible force by turn 2 or 3, you may coax the Soviet player into sending some of his own dwindling forces there to retake. Even if he is successful, and eradicates some of your armor & infantry there, such a development may draw down the defensive strength of the Moscow-Belorussia salient. Good for Germany!
Forsooth, much of the decision-making may derive from the luck of the dicerolls in your invasion of Ukraine in the first place!!
What we know from history is… what? Hitler started Operation Barbarossa at least a month late, he ordered major thrusts toward the Caucasus in desperation, out of fear of running short of oil, and… the offensive in the center was blunted by good ol’ Zukhov & friends. Some historians figure that was it: the Nazis lost the war in 1941, followed by four more years of global carnage to certify the outcome.