Ambassador Hüseyin Avni Karslioglu will fly back to Turkey on Thursday afternoon, the Turkish government has confirmed, according to N-tv.
After facing days of protests from Turkey, German MPs voted through a resolution to label the killing of Armenians by the Ottoman Empire a genocide.
Turkey slammed as "null and void" and a "historic mistake" the German parliament's resolution recognising the World War I killings of Armenians by Ottoman forces as genocide.
"The German parliament's recognition of 'distorted and groundless' allegations as 'genocide' is a historic mistake," Deputy Prime Minister and government spokesman Numan Kurtulmus said on his official Twitter account.
The Bundestag (German parliament) passed the resolution with just one vote against and one abstention.
It was a striking success for the joint motion from the governing Christian Democratic Union (CDU) and Social Democratic Party (SPD) coalition and the opposition Green Party.
But notable by their absence were Chancellor Angela Merkel (CDU) and Foreign Minister Frank-Walter Steinmeier (SPD) - the two leaders on the front lines of Germany's strained relationship with Turkey.
Merkel had earlier in the week said that she was in favour of the resolution but would be unable to attend and vote for it herself because of scheduling issues.
Western historical consensus is that the Ottoman government systematically massacred an estimated 1.5 million Armenians beginning in 1915.
Green party leader Cem Özdemir told MPs that Germany had a "historical obligation" to encourage Turks and Armenians to reconcile.
The connection to Germany was particularly pressing because of the German Empire's alliance with the Ottomans at the time of the genocide, he said.
CDU deputy leader Franz Josef Jung, meanwhile, said that while Turkey was an important international partner for Germany, the way had to be opened for dealing with the past.
SPD MP Rolf Mützenich said that the resolution was not a "bill of complaint" aimed at Turkey,adding that "the object of the debate is the genocide against the Armenians, not the judgement of [Turkish] President Erdogan".
Throughout the week leading up to Thursday's vote, President Recep Tayyip Erdogan and other officials in Ankara had sought to dissuade German lawmakers.
Erdogan himself called for Germans to show "common sense" when considering the resolution.
But while Turkey made clear that labeling the massacres in 1915 and 1916 a genocide would displease officials, politicians didn't go as far as threatening to overturn a hard-fought agreement they struck with Europe in March to limit refugee arrivals.
Meanwhile, the President of Armenia Serzh Sargsyan said on Wednesday that "It would not be fair to not call the genocide of the Armenians genocide, just because that makes the head of state of another country angry."