The judgement applies to people who "willingly" worked in ghettos under German occupation and were paid wages, however meagre, as opposed to slave labourers, who are entitled to other compensation.
The Federal Social Court, which hears public insurance and entitlements claims, ruled that payments to plaintiffs would only be made retroactively to 2005 and not 1997, as demanded by the plaintiffs.
German pension authorities say the ruling will slash payouts by up to €500 million ($660 million).
The dispute resolved around Germany's so-called Ghetto Pensions Law of 2002 which applied to those who worked for pay in the ghettos.
About 70,000 Jews from around the world applied for payments under the law including around 30,000 from Israel and more than 10,000 from North America.
Monthly payments of between €100 and €300 were to have been paid retroactively going back to 1997 if applications were filed by mid-2003.
German authorities, however, rejected around 90 percent of the appeals on the grounds that they were invalid, leading to a comprehensive study of work conditions in the ghettos.
In 2009, the Federal Social Court eased qualification rules. However it said it had no authority to reverse the rejection of applications already filed.
Pension authorities argued that as the limit for retroactive payments is usually four years, those who re-applied could only claim payments back to 2005.
The judges on Tuesday upheld this view in a ruling based on an appeal by two Jewish women from Israel who had filed their applications by the original 2003 deadline but saw them rejected the first time.
"The court stresses that applications for retroactive payments for ghetto pensions fall under normal pension law," it said in a statement.
Germany says it has paid out the equivalent of more than €67 billion in reparations to victims of the Nazis including survivors of concentration camps, ghettos or forced work camps.