The men, two aged 58 and one aged 33, stand under “strong suspicion of working as agents for an intelligence agency”, the Karlsruhe prosecutors said in a statement.
One of the 58-year-olds, Muhammed Taha G., is believed to have been the leader of the group.
Two of the men were arrested on arrival at Frankfurt airport by detectives from Rhineland-Palatinate, while the third was found at his home in North Rhine-Westphalia.
Prosecutors accuse them of sending information back to their superiors at home about Turks and Turkish organizations based in Germany.
The alleged spies were brought before an investigating magistrate, who ordered that they be held for questioning.
Rhineland-Palatinate detectives are leading the investigation.
SEE ALSO: Turkey demands 'stop spying on us'