"To our Turkish partners, let us talk openly and critically, but let us remember the special meaning of our close relationship and let cool heads prevail," said Merkel's spokesman Steffen Seibert.
The spokesman sharply rejected Erdogan's claim.
"Nazi comparisons are always absurd and out of place, because they lead only to one thing - the trivialising of the crimes against humanity committed by the Nazis," said Seibert.
Merkel's chief of staff, Peter Altmaier, also said that the Nazi comparison was "absolutely unacceptable".
"The government will make this very clear" to Turkey, Altmaier told public broadcaster ARD.
"There is absolutely no reason to allow ourselves to be reproached over this," he added.
Ankara and Berlin are locked in a new dispute weighing on already frayed ties, this time over rallies in Germany by Turkish ministers to promote Erdogan's bid for greater powers.
The row erupted late last week after three German local authorities banned the ministers' scheduled rallies ahead of an April referendum over the plan to scrap the prime minister post in Turkey altogether.
The German authorities cited capacity problems in hosting the events, which they said would likely attract large crowds.
Merkel has said her government did not have a hand in the decisions, which fell under the jurisdiction of local governments.
But Ankara responded with fury to the cancellations, accusing Berlin of working against the referendum.
Lashing out against Berlin, Erdogan on Sunday told a rally in Istanbul: "Germany, you are not even close to democracy. Your practices are not different from the Nazi practices of the past."
He also warned Germany not to hinder him from making an appearance if he wished.
"If you don't let me in, or if you don't let me speak, I will make the whole world rise up," he said.
Big voter base
Germany is home to the biggest population of Turkish people abroad, with about three million people of Turkish origin living here, the legacy of a massive "guest worker" programme mainly in the 1960s and 70s.
With some 1.4 million Turks in Germany holding the right to vote in Turkey, it is not uncommon to see politicians travelling there from Ankara to seek to sway the significant voter base.
Undeterred by the numerous cancellations last week, Economy Nihat Zeybekci made a campaign appearance in Cologne on Sunday.
Turkey's Foreign Minister Mevlut Cavusoglu is also planning to hold a campaign rally on Tuesday in Hamburg.
Altmaier repeated Merkel's argument that campaign rallies by Turkish ministers, in general, should not be banned.
"But they must adhere to existing rules. They must be registered and subject to checks," he said.
Tensions spilling over
Ties between Ankara and Berlin have been strained over a series of disputes, particularly since the failed coup in July against Erdogan.
Berlin has emerged as a strident critic of Ankara's vast crackdown in the aftermath of the failed putsch, which has seen more than 100,000 people arrested, dismissed or sacked for alleged links to the plotters or to Kurdish militants.
With the significant Turkish and Kurdish populations in Germany, Berlin is anxious to prevent Turkish domestic disputes from spilling into the country.
Most recently, Ankara's arrest of Deniz Yucel, a Turkish-German dual national correspondent for Germany's Die Welt newspaper, has sparked an outcry in Berlin.
An Istanbul court has charged Yucel has been charged by an Istanbul court with spreading terrorist propaganda and inciting hatred.
Germany's Foreign Minister Sigmar Gabriel has said the case would make "everything harder" for Turkish-German relations.