In the ruling, the court stated there was no need for video rental shops or call centres to be open on Sundays and it was not up to a state government to decide that.
The decision will have implications across the country, as other states have similar laws.
The case stems from a 2011 decision by the state of Hesse to allow broader exceptions to labour laws regulating who has to work on Sundays. The laws paved the way for call centres, libraries and ice cream makers, among others, to open on the legally mandated day of rest for Germans.
"This is a very positive success for us," said Bernhard Shiederig, the regional head of the labour union Verdi, which was one of the parties that brought the suit against Hesse.
Unions and churches are suing the state in the Leipzig court, saying the local government overstepped its bounds when forcing more people to work on Sundays.
"We need police, firefighters and hospitals, but we don't need to be able to oder things over the phone, or a library opened on Sunday or an open betting shop," said Schiederig when he launched the legal action.
Nurses, waiters, bus drivers and journalists are among those who are already exceptions to the rule. Destatis says that already more than one in four Germans work on Sundays.
The Call Center Association called the ruling a "slap in the face" - it's president, Manfred Stockmann, said that for many companies, being unreachable on the phone on Sundays is not an option.
The decision comes ahead of a particularly stingy year in terms of public holidays. Of the nine federal bank holidays, two of them will find themselves on the weekends in 2015. For those living in states that take bank holidays on church holidays, five holidays fall on weekends.
Even Christmas will be a stingy time for taking holidays as December 25 and December 26, both bank holidays here, are on a Friday and Saturday.
Unlike the UK or Canada, Germany doesn't offer a "make up" day when this happens, leaving German workers with fewer days off.