The anti-terror raids took place near the western city of Mainz.
The federal prosecutor's office said the men were "suspected of having taken part in Syria's civil war as members of a foreign terrorist group" but declined to name the jihadist group involved.
"There are no immediate indications that a concrete attack was planned," the prosecutor's office said, adding that it was not yet in a position to say whether police had arrested the men or if any items were seized from their homes.
German weekly Der Spiegel said on its website that one of the two was a 32-year-old "suspected former commander" of the Islamic State group who had fought in Syria and arrived in Germany as a refugee late last year.
Spiegel Online said it had spoken to the man before the raids, and that he had denied any link with IS.
In a similar police operation on Thursday, German officers arrested three Algerians suspected of having ties to the jihadist group.
Those arrests followed raids at several sites across Germany, including refugee shelters where some of the suspects lived.
Germany took in some 1.1 million asylum seekers last year, and fears are growing that jihadists have taken advantage of Europe's worst migrant crisis in decades to slip in undetected.
Security has already been tightened after a wave of sexual assaults in the western city of Cologne on New Year's Eve shocked the country. Many of the attacks were blamed on North African migrants.
SEE ALSO: None of 'Isis arrests' were for planning terror attack
Comments