Preventive detention applies to criminals who have already served their sentences but are still considered dangerous. In March of this year, Germany was holding 475 prisoners, three of whom were women, in preventive detention.
In 1998, Germany extended from ten years to indefinite the period prisoners could be held in preventive detention. The change applied to crimes committed before that date too.
But in 2011, the European Court of Human Rights ruled that the move had been unconstitutional.
The four sex offenders had been sentenced in Baden-Württemberg between 1977 and 1985, when the ten-year rule for preventive detention still applied.
But the state court in Freiburg court ruled that their periods of preventive detention should be extended on the basis that they still posed a a threat to society.
The men took the case to the Higher Regional Court in Karlsruhe in 2010. It ruled in favour of their release and ordered the state of Baden-Württemberg to award between €49,000 and €73,000 in compensation.
The state had contended that the payments should come from federal funds.
Germany's Supreme Court on Thursday ruled that individual states must compensate prisoners who have been held for too long in preventive detention.