Waving the Turkish flag and chanting "Turkey", the demonstrators turned the rally site next to the River Rhine into a sea of red as they began the demonstration by singing the Turkish and German national anthems. Some held banners saying "Erdogan, fighting for liberty".
"We are here because our compatriots in Germany are standing up for democracy and against the attempted military coup in Turkey," said Turkey's Sports Minister Akif Cagatay Kilic at the rally, Tagesspiegel daily reported.
"The message to be sent from the event is that in Turkey, all parties and NGOs want to stand together against the coup and to defend democracy," added the minister, who was born in Germany.
Police said the gathering, organised by groups including the pro-Erdogan Union of European-Turkish Democrats (UETD), broke up around 5 pm with the crowd estimated at 40,000.
Since the attempted July 15th power grab, Erdogan's government has launched a huge crackdown, detaining almost 19,000 people and sparking international concern.
Ratcheting up its clampdown on the military, Ankara on Sunday dismissed nearly 1,400 military personnel, including a top aide to Erdogan, and confirmed it would close military schools and academies.
Erdogan - who says a group within the military acted on the orders of US-based cleric Fethullah G[len - has also said he will bring the country's spy agency and military chief of staff directly under his control.
The drama has spilled over into Germany, home to three million ethnic Turks - the biggest Turkish diaspora in the world.
Hours before the Cologne rally Germany's constitutional court banned an application to show live speeches from Turkey by politicians including Erdogan, amid fears they could work the crowd up further.
The decision sparked anger in Turkey, with presidential spokesman Ibrahim Kalin calling the ban unacceptable and a "violation of the freedom of expression and the right to free assembly".
'Don't bring Turkey tensions here'
Meanwhile, skirmishes broke out at several smaller counter-demonstrations, with police moving in to separate around 80 right-wing nationalist Turks and 100 Kurds.
Some 250 far-right extremists, including many hooligans, had also come together before being dispersed by police. No injuries were reported.
The tension comes at a time when relations between Germany and Turkey are already strained over the German parliament's decision to brand as genocide the World War I-era Armenian massacre by Ottoman forces.
German politicians led by Chancellor Angela Merkel have also issued strongly-worded statements against Erdogan's crackdown following the failed putsch.
The hardline response "flouts the rule of law", Merkel's spokesman Steffen Seibert has said, also blasting "revolting scenes of caprice and revenge" in the wake of the failed coup.
At the same time, Ankara is demanding that Germany extradite suspects linked to Gulen. The 75-year-old cleric has strongly denied any involvement.
Erdogan enjoys large support among the diaspora in Germany, where around 1.5 million people with Turkish nationality can vote in Turkish elections.
His AKP party won 60 percent of votes cast in Germany in last November's election, a bigger share of the vote than in Turkey.
In the days following the botched coup, pro-Erdogan activists have stormed locations in Germany popular with Gulen's followers.
Critics of the Turkish president have also complained of abuse and threats against them on social media.
German Foreign Minister Frank-Walter Steinmeier on Saturday warned in an interview with Sueddeutsche Zeitung daily: "It is not right to bring Turkey's domestic political tensions here... and intimidate people who have other political convictions."
Meanwhile his Turkish counterpart has warnings for the European Union.
Mevlut Cavusoglu said in an interview with the Frankfurter Allgemeine Zeitung to appear Monday that Ankara could withdraw from the EU-Turkey accord on refugees if Europe fails to allow visa-free travel for Turks by October.