More than 12 million voters are electing three new regional parliaments for the southwestern states of Baden-Wuerttemberg and Rhineland-Palatinate, as well as eastern Saxony-Anhalt in the so-called Super Sunday polls.
The elections are the biggest since a record influx of refugees to Germany, and are largely billed as a referendum on Merkel's decision to open the country's doors to people fleeing war.
"These elections are very important... as they will serve as a litmus test for the government's disputed policy" on refugees, Duesseldorf University political scientist Jens Walther told AFP ahead of the polls.
Surveys in the run-up to the vote show that support for the CDU and its junior coalition partner Social Democratic Party (SPD) dropping while the populist Alternative for Germany (AfD) was steadily gaining momentum and expected to record a surge in backing in all three states.
Merkel's Christian Democratic Union (CDU) was bracing for one of its poorest showings in years, particularly in its traditional stronghold of Baden-Wuerttemberg, with a poll published late Thursday by ZDF public television showing support plummeting by 10 percentage points to 29 percent -- putting it for the first time behind the Greens -- while the AfD snatched 11 percent.
Guido Wolf, the CDU's leading candidate in the southwest, has described Sunday's as the "most difficult election campaign" the party has had to run.
In Rhineland-Palatinate, where the fortunes of the CDU had been rising with the latest poll giving it 35 percent, the party is seen struggling to knock the Social Democratic Party, scoring 36 percent, off from the top of the list. The AfD meanwhile was hoping to crack the 10 percent mark.
In Saxony-Anhalt, where the CDU still commands a large lead in the poll with 32 percent, AfD has a stunning 18 percent, at the heels of the second-placed Left Party, on 21 percent.
Merkel has been under intense pressure to change course and shut Germany's doors after 1.1 million refugees -- many of them Syrians -- arrived in Europe's biggest economy last year alone.
But she has resolutely refused to impose a cap on arrivals, insisting instead on common European action that includes distributing refugees among the EU's 28 member states.
As dissent grew over her stance, AfD has capitalised on the darkening mood.
Founded in 2013 as an anti-euro party, AfD has since morphed into one that sparked a storm in January after suggesting police may have to shoot at migrants at the borders.
Although the upstart party has seats in five regional parliaments and is represented in the European Parliament, it has so far made its biggest gains in former communist eastern states that still lag western Germany in jobs and prosperity.
But its inroads into western states have sparked alarm in a Germany mindful of its Nazi past.
Story continues below…
With an eye on the upcoming polls, Vice-Chancellor Sigmar Gabriel warned voters that "we have a lot to lose if we deal carelessly with social stability and democracy".
Merkel herself described AfD as a "party that does not bring society together and offers no appropriate solutions to problems, but only stokes prejudices and divisions".
She has also shrugged them off as a temporary diversion.
"As we progress step by step on the question of refugees, our policies will show results. And I'm convinced that from there, the support that AfD is enjoying right now will drop off," she told Mitteldeutsche Zeitung in an interview.