In a vote that caused lively and sometimes heated debate, 51.3 percent of those that voted were against allowing children in secondary schools the choice of taking religion instead of ethics, final results showed.
The opposing side scored 48.5 percent but even if it had inched ahead and won, turnout was too low for the referendum to have been valid with only 14.2 percent of Berliner's 2.4 million voters ticking the "yes" box.
Overall turnout was just 29.2 percent despite massive efforts and an expensive publicity campaign by religious groups across all faiths in the "yes" camp to get their supporters to get out and vote.
Berlin Mayor Klaus Wowereit said it was a “clear vote,” adding that the Berlin government was still willing to “do its part and provide a constructive cooperation and with the churches of the city,” news agency DDP reported.
Berlin has been described as the "capital of atheism" but 2005's "honour killing" shocked the left-wing city hall and made it decide that children needed to be taught more about the difference between right and wrong.
The resulting mandatory ethics classes are aimed at fostering common values and integration and giving children from different backgrounds a forum to discuss issues like sexuality, women's rights and abortion.
And in a city where more than 40 percent of children come from immigrant families, most of them Muslims, it was hoped that the lessons would nip any dangerous radicalism in the bud.
Germany, which opposed the 2003 US-led invasion of Iraq but has around 3,700 troops in Afghanistan under NATO command, has managed to escape terror attacks by Islamic radicals but authorities say there is still a serious threat.
But the ethics lessons and a resulting sharp drop in the number of children taking additional religion lessons caused considerable anger among Christians, Muslims and Jews alike.
A group called Pro Reli - "Reli" being what kids call religious classes - sprang up and managed to collect 265,000 signatures to force Sunday's referendum.
Its supporters wanted children to have the choice between ethics classes and religion lessons, as they do in most of the rest of Germany, with children of different faiths taught separately.
They say that the ethics classes are counter-productive when it comes to integration and that children must be steeped in teachings from their own faith before they can begin to understand the beliefs of others.
But the city's authorities stood firm.
"Particularly when it comes to fostering values, children should not be separated on religious grounds and classes split," said a leaflet from Berlin's ruling Social Democrats.
This "would be bad for integration in Berlin and bad for the education of our children," it said.
The campaign saw billboards plastered all over Berlin - many have been vandalised - and it has drawn in celebrities from the worlds of politics, sport and entertainment.
Chancellor Angela Merkel, the daughter of a pastor and leader of the Christian Democratic Union party, supported Pro Reli. On Saturday Pope Benedict XVI - who is German - underlined in Rome his support for religious education.
Twenty years after the fall of the Berlin Wall this November, final results also showed voters in the former communist East Berlin firmly in the "no" camp while those in the old West Berlin were very much on the other side.