Bearing posters with slogans like "Stop AfD", "My voice against incitement" or "My heart beats for diversity", the demonstrators rallied two days before AfD lawmakers will join other MPs at the first sitting of Germany's newly-elected parliament.
The anti-Islam and anti-migrant AfD garnered 12.6 percent of the vote in the watershed general election in September and became the country's third biggest party.
Its arrival in the Bundestag is a political earthquake for post-war Germany, as the AfD's top figures have repeatedly smashed taboos with their claims on German identity or by challenging Germany's culture of atonement over World War II.
But the party proved appealing to voters angry with Chancellor Angela Merkel's border policy, which allowed more than one million asylum seekers into the country since 2015.
Calling on people to join the protest on Sunday, the popular movement Campact urged Germans to "steal the show from the AfD".
"When the AfD sits in the Bundestag for the first time on October 24th, it needs to know that our parliament is not a stage for racism, discrimination and falsifying history!" said Campact.
Teacher Annette Saidler acknowledged at the protest that "it's now too late" to stop the AfD from entering parliament.
"We can't do anything other than demonstrate, to say that there are still many people who did not vote for the AfD."
Another protester, 25-year-old university student Bastian Schmidt said he was at the demonstration to "call on parliamentary parties to protest in parliament against the AfD".
"But above all, the people who are here, wherever they are in their daily lives -- in schools, universities or companies -- must fight against racism," said Schmidt, who turned up with a group of like-minded schoolmates.