The CDU's share of the vote fell by more than 10 percentage points in Thuringia and Saarland, which both currently have conservative state premiers. However, early results from public broadcaster ARD showed that the CDU would hold onto the eastern state of Saxony.
The two CDU premiers in Saarland and Thuringia, Peter Müller and Dieter Althaus, had both hoped to form coalitions with the pro-business Free Democrats (FDP). But now they face the possibility of being replaced by left-wing alliances between the centre-left Social Democrats (SPD), hard-line socialist party The Left, and the environmentalist Greens.
Because of Germany's complex coalition arithmetic, it could take several weeks until it is clear which alliances will rule Thuringia and Saarland.
So-called "red-red" coalitions between the SPD and The Left, a collection of former communists and disgruntled trade unionists, are common in the east of the reunited country. But they have been taboo in the west, where many see The Left as too radical and tainted by its historical links with the party that built the Berlin Wall. Saarland would be the first western state to see such a governing alliance.
Merkel, who grew up behind the Iron Curtain, warned in regional campaign appearances against the rise of The Left as Germany marks 20 years since the Wall's collapse.
But the Social Democrats, desperate for a boost ahead of the general election next month, also scored important victories in municipal elections in Germany's most populous state North Rhine-Westphalia. Exit polls showed SPD mayoral candidates winning in Cologne and Dortmund, even as the CDU remained the strongest party in municipal councils statewide.
Frank-Walter Steinmeier, the SPD's candidate struggling to oust Merkel from the Chancellery, immediate seized on the results as a chance to ignite his flagging campaign.
"I've heard and read a lot recently that the (general) election is already over," said Steinmeier at the SPD headquarters in Berlin. "This election night has shown that to be a huge mistake."
But the state results, largely expected, are unlikely to knock Merkel off course in her bid to win a second term as the CDU still enjoys around a 15-point lead ahead of the Social Democrats in national surveys.
CDU Secretary General Ronald Pofalla acknowledged disappointment at the elections' outcome but said the Social Democrats would gain no momentum from them for the national poll.
"The results from the state elections have both their bright and dark spots from the point of view of the CDU," Pofalla said. "We were the strongest party in all three states and left the SPD behind us in all three. We remain the only people's party in Germany."
Meanwhile, the neo-Nazi National Democratic Party, which stunned the country by scoring 9.2 percent in Saxony in 2004, appeared likely to be re-elected to a state parliament for the first time ever after just clearing the five-percent hurdle required for seats.