The German federal prosecutor's office said in a statement that the suspect identified only as Yusuf O. was arrested in Austria on May 31 on a German warrant.
Prosecutors say the 26-year-old travelled to the border region between Afghanistan and Pakistan from Germany in May 2009 and joined the "foreign terrorist organisation" German Taliban Mujahideen (DTM) in September of that year.
He allegedly trained in using explosives and firearms "and took part in jihad for the DTM," which plots attacks against government troops in the region as well as the NATO-led force in Afghanistan, the statement said.
"In addition he is accused of appearing in propaganda videos for the terrorist organisation," it said.
From early 2011, Yusuf O. is believed to have begun seeking new recruits for the DTM in Europe.
Austrian police arrested four terror suspects last week including a 25-year-old Austrian convert to Islam suspected of recruiting for terror training camps in Afghanistan and Pakistan and financing the DTM.
He is believed to have been in contact with Yusuf O., according to Austrian authorities.
The three other suspects, who were arrested at Vienna airport apparently on their way to a terrorist training camp in Pakistan, were released but the alleged Austrian recruiter remains in custody.
The daily Kronen Zeitung reported Saturday that the Austrian suspect, identified as Thomas al-J., was plotting to crash a plane into the Bundestag parliament building in Berlin and had been training on a flight simulator.
Neither Austrian nor German authorities have confirmed the report.