Searches and seizures of materials belonging to the suspects took place on Tuesday at three apartments in Offenbach, according to the public prosecutor's office in Frankfurt am Main.
The men between the ages of 21 and 24 are alleged to have made preparations for an attack involving explosives or firearms.
According to the public prosecutor's office, a 24-year-old German of North Macedonian (formerly Macedonia) origin is the main suspect.
He had procured “basic components for the production of explosives” and searched the Internet for firearms, the authority announced.
Two men of Turkish citizenship aged 21 and 22 are also accused. All three men were to be brought before a judge on Tuesday, who would decide on detention awaiting a trial.
Terrorist attack planned in Rhine-Main region
According to the investigators, the men wanted to carry out an attack in the Rhine-Main area. It was alleged that they had already made themselves known to other people as Isis-followers.
They are accused of preparing for a serious crime that would endanger the state.
The raid involved some 170 officials, including special forces. In addition to data media and written documents, “explosives-relevant substances and equipment” were confiscated from the main suspect, according to the public prosecutor's office.
Germany has been on alert following several jihadist attacks in recent years claimed by terror group Isis.
The most deadly was committed in 2016 by a 23-year-old Tunisian who killed 12 people when he stole a truck and ploughed it through a Berlin Christmas market.