State prosecuter Thomas Steinkraus-Koch confirmed to news magazine Focus on Sunday that the 60-year-old from Kabul is being investigated for murder.
The Bavarian state police have been charged with the task of tracking him down, though Steinkraus-Koch said more police forces would have to get involved in the investigation.
According to Focus, the suspect was a department chief in the Afghan secret service before and after the Soviet Union's invasion of Afghanistan in December 1979.
He worked for the Soviet-backed regime of President Mohammad Najibullah and was responsible for investigations of political crimes and director of Afghanistan's prisons and interrogation and torture centres.
He is said to have ordered a mass-shooting of prisoners in the Pul-e-Charkhi prison in 1988, as a reprisal when inmates cheered the news that Soviet forces were withdrawing.
There are said to be more than 250 witness testimonies of the massacre, where inmates were shot in groups of 60 and tipped into mass graves. A communist-era mass grave was discovered at the prison in 2006, which is believed to contain 2,000 corpses.
The man has reportedly been living undercover in Germany for so long that he now has German nationality.
Two other German state prosecutors, in Hesse and North Rhine-Westphalia, are currently investigating five other ex-officers of the Afghan secret service thought to be living in Germany.