After decades of lobbying, victims and activists hailed a triumph in the struggle to clear the names of gay men who lived with a criminal record under Article 175 of the penal code.
An estimated 5,000 of those found guilty under the statute are still alive.
The measure overwhelmingly passed the Bundestag lower house of parliament, where Chancellor Angela Merkel's ruling right-left coalition enjoys a large majority.
It offers gay men convicted under the law a lump sum of €3,000 as well as an additional €1,500 for each year they spent in prison.
Germany's Article 175 outlawed "sexual acts contrary to nature... be it between people of the male gender or between people and animals".
Sex between women was not explicitly illegal.
Although it dated from 1871, it was rarely enforced until the Nazis came to power, and in 1935 they toughened the law to carry a sentence of 10 years of forced labour.
More than 42,000 men were convicted during the Third Reich and sent to prison or concentration camps.
In 2002, the government introduced a new law which overturned their convictions, but that move didn't include those prosecuted after World War II.
The article was finally dropped from the penal code in East Germany in 1968.
In West Germany, homosexual sex was decriminalized for men over 21 in 1969. But it was only fully repealed in 1994, and up until that year men were still being convicted under it.
'A stain removed'
"More than two decades after Article 175 was finally wiped from the books, this stain on democratic Germany's legal history has been removed," Sebastian Bickerich of the government's anti-discrimination office said in a statement.
Convicted under the law as a teenager in 1957, Fritz Schmehling, 74, told AFP: "Back then, you lived with one foot in prison."
Schmehling said he wished his long-time partner Bernd, who died in 2011,
had lived to see justice served.
"He told me, 'I don't think I'll ever see the day these convictions are lifted'. I think he would have been as happy as when the Berlin Wall fell."
Another beneficiary of the law, also 74, gave his name as "Heinz Schmitz" because of enduring shame about his conviction at the age of 19 for his family.
He said Article 175 robbed him of many of the best years of his life.
"I was as beautiful as a young god and men were always after me. But I was always afraid I would end up in prison," he said with a smile.
The vote marks a victory for the Social Democrats three months before a general election in which they plan to campaign on a more liberal stance on gay rights than Merkel's conservatives.
Germany legalised registered partnerships for same-sex couples in 2001 but has stopped short of granting full marriage rights - including adoption of children - common in many EU member states.