The 31-year-old suspect, named as Akram A., was detained in the northeastern state of Mecklenburg-Western Pomerania, the federal prosecutor's office said in a statement.
Prosecutors allege that he manned a checkpoint in Syria that sought to prevent citizens from escaping Isis-controlled territory.
In early 2016, he stopped a woman who was trying to flee with her children, they said.
"Under the pretext of obtaining the necessary 'exit permit', the accused lured her to a house and raped her," prosecutors said in the statement.
The rape could amount to a war crime, they added.
The suspect will on Wednesday appear before a judge who will decide whether to keep him in custody.
German federal prosecutors have launched around a dozen investigations concerning alleged war crimes committed in Syria or Iraq, alongside dozens of cases of suspected membership of jihadist groups.
The investigations have gained momentum following the arrival of over a million asylum seekers since 2015, including hundreds of thousands of people from Syria and Iraq.
Last July, in the first such conviction in Germany, a German jihadist was sentenced to two years in prison on war crimes charges after posing for pictures in Syria with the severed and impaled heads of two government troops.