The order says that only portraits of "ambassadors appointed since the creation of the German Federal Republic's Foreign Ministry in 1951 can be displayed," a spokesman said late Tuesday.
Portraits of ambassadors named since 1951 but who had links to the 1933-1945 Nazi regime will be accompanied by an "explanatory text explicitly mentioning this" the spokesman added.
The instruction follows the publication of an official report in October which exposed active complicity by Nazi Germany's foreign ministry in the systematic slaughter of Jews in the Holocaust.
"Holocaust survivors welcome the removal of these portraits as a symbolic signal of modern Germany's moral and political maturity," the American Gathering of Holocaust Survivors and their Descendants said.
"This additional step demonstrates that present-day Germany is serious in its efforts to painfully and honestly confront its dark past.
"It is gratifying that Germany is in a sober and serious manner acknowledging the moral account it has with the Jewish people and humanity as a whole."