"The ballot box is your weapon," Erdogan told the crowd of over 14,000, who waved Turkish flags and shouted "We love you Erdogan, we are proud of you!" national news agency DPA reported.
Outside the congress centre near the western city of Karlsruhe, some of his supporters scuffled with critics, police said, while Kurdish and other opponents waved placards that read "Erdogan, you are anti-democratic".
Germany counts three million people of Turkish origin, the largest Turkish overseas expatriate community, making it one of the biggest constituencies after the Turkish cities of Istanbul, Ankara and Izmir.
They are mostly former "guest workers" who were invited decades ago to boost the labour force of post-war Germany, or their children and grandchildren.
Erdogan, 61, urged the Turkish expatriates to integrate in German society, but also not to forget their language and their religion, telling them: "the stronger our cohesion in the world, the stronger we all are".
The audience shouted the slogan "one nation, one flag, one fatherland, one state".
Officially, the event was not a campaign speech but a meeting with youths, said the Union of European Turkish Democrats which is close to Erdogan's ruling Justice and Development Party (AKP).
The AKP in June 7 parliamentary election hopes to gain a clear majority to be able to change the constitution and switch to a presidential system in which the head of state would enjoy full executive powers.
Erdogan - who ruled Turkey from 2003-2014 as premier, overseeing rapid economic growth - has already transformed the once largely ceremonial role since taking office in August.
Several other Turkish politicians have also visited the German cities of Berlin, Düsseldorf and Dortmund to rally the almost 1.4 million eligible Turkish voters living in Germany, who will be able to cast their ballots all this month in 13 consulates across the country.
Hamburg-based Turkey expert Yasar Aydin said canvassing in Germany is now done by all major Turkish political groupings and "is of existential importance for some of the parties".
Erdogan has been a frequent critic of Western powers and last month lashed out at the European Union after it and many member states recognised the 1915 massacres of Armenians by Ottoman forces as genocide.
Turkey has vehemently rejected the use of the term genocide, contending that hundreds of thousands of Muslims and Christians were killed on both sides in a wartime tragedy.
"No-one in the world can ignore you if you vote," Erdogan told the crowd. "That includes those who held a minute's silence in the EU for the Armenians, they can't ignore you either."