The result in Hesse, home to Germany's financial hub Frankfurt, gives the conservatives a boost in what is being billed as a super election year in Germany that Merkel hopes will culminate in her re-election this autumn.
According to public broadcaster ARD, the CDU captured around 37 percent, a slight increase from the last vote in 2008, ensuring Hesse's caretaker state premier Roland Koch will stay in office.
"This is a victory for Hesse," said Koch to cheering supporters in Wiesbaden. "There is once again a clear governing majority in Hesse."
But the SPD – Merkel's coalition partners at federal level – slumped some 13 points to less than 24 percent, marking a historic low in the state.
The pro-business Free Democrats (FDP), the CDU's preferred coalition partners both in Hesse and at national level, saw the biggest increase in their share of the vote, soaring some seven and points to around 16 percent.
The Greens also saw a strong increase, rising almost 7 percent to nearly 14 percent, while the far-left party The Left was largely unchanged just clearing the five percent hurdle needed to gain seats in Hesse's parliament.
Some 4.4 million people were called to the ballot boxes and polls had predicted a clear win for Merkel's party. Her conservatives and the SPD have been bound together at the national level in a fractious "grand coalition" since 2005 and are now together grappling with the country's worst post-war recession.
But with 16 elections planned this year on the regional, state, national and European level, the trick has been to press on with the business of running Europe's biggest economy at the same time they go head-to-head at the polls.
Despite criticism of Merkel's initially tentative reaction to the crippling slowdown, the Hesse outcome showed that the CDU appears not to be paying the price with voters, thanks largely to internal squabbling among the SPD.
CDU general secretary Ronald Pofalla said the conservatives were now well-placed to become the top party in the September 27 general election. "The result shows clearly that the CDU is the party of the political centre," Pofalla told ARD television. "With regard to this election year, we have every chance at the federal level to win more than 40 percent."
Although it was expected, political scientist Oskar Niedermayer of Berlin's Free University said the SPD's humiliating defeat could have a "psychological effect" and throw the party off its game in the upcoming races.
"The SPD runs the risk of piling up disastrous electoral results every two or three months, which could have an impact on the outcome of the general election," Niedermayer told AFP this week.
Merkel and her SPD challenger in September, Foreign Minister Frank-Walter Steinmeier, are being forced to walk a tightrope during this super election year. The two came together at a news conference and before parliament this week to defend a new €50-billion ($66-billion) economic stimulus package thrashed out by their two sides in a bid to pull the economy out of the mire.
But Hesse was already scorched earth for the SPD due to an inconclusive election one year ago.
The SPD's chief candidate Andrea Ypsilanti aimed to form a coalition with the Greens but also rely on the hard-line socialist party The Left for votes – breaking a campaign pledge not to work with the outfit. The bid failed when four rogue Social Democrats withdrew their support, saying their consciences would not allow them to vote for the new government.
The SPD's dealings with The Left, a loose-knit grouping of former East German communists and disaffected Social Democrats, have prompted an identity crisis for the party and Ypsilanti said she would step down as head of the party in Hesse.
"Comrades, this is a difficult day. I am not resigned, but I will assume responsibility for this defeat therefore I announce my resignation," she told a muted gathering of her supporters.