The searches targeted homes and religious schools linked to Salafist jihadist group Invitation to Paradise (EZP) in the northwestern cities of Braunschweig and Mönchengladbach, and the Islamic Cultural Center Bremen (IKZB) in northern German port.
"The EZP and the IKZB are accused of opposing the constitutional order with the aim of replacing it in Germany with an Islamic religious state," the ministry said in a statement.
The raids were part of a long-running investigation against the groups and had no link to warnings of potential impending terrorist attacks issued last month by Interior Minister Thomas de Maizière, it added.
The groups reject parliamentary democracy and believe that Islamic law should replace the constitution, the ministry said.
"In a well-fortified democracy it is advisable and necessary not to wait for jihad in the form of an armed struggle before taking action against anti-constitutional groups," the ministry said.
The raids will show whether or not government suspicions about the groups can be confirmed, it added.
A security official told news agency the Associated Press that officials had searched dozens of private homes, religious schools, a small publisher, and a store owned by EZP that sells traditional caftans and veils.
Meanwhile an EZP leader in Mönchengladbach condemned the raid, the news agency said.
"We're sad about this raid, we haven't done anything illegal," Sven Lau said.
But the leader of the anti-Salafist initiative “Citizens for Mönchengladbach” told AP his group welcomed the searches.
"We are happy that the Interior Ministry moved so fast against them and hope that it won't take much longer until the group will be banned altogether," Wilfried Schultz said.
Residents in Mönchengladbach have held frequent protests against attempts to build a religious school there by the Salafist group, whose strict Islamist ideology has been linked to several terrorist plots.
Their message has been particularly appealing to young Muslim immigrants and converts, the news agency reported.