Merkel's conservatives (CDU/CSU) and the Free Democrats (FDP) only managed to muster enough support for Wulff in a third vote by the Federal Assembly, which is made up of MPs and representatives from Germany's 16 states.
"I will work hard to live up to the expectations and I look forward to the task ahead," said Wulff, after accepting the vote on the floor of the Bundestag following a nine-hour session. "Germany is a wonderful land. May God protect our country."
Wulff, 51, becomes the Germany's youngest-ever president after securing 625 votes. The candidate of the opposition Social Democrats (SPD) and the Greens, Joachim Gauck, the former East German dissident and pastor, won 494 votes.
Lukrezia Jochimsen from the socialist Left party withdrew her candidacy for the third vote, in which a candidate needed a simple plurality to become president.
A victory for Gauck, though always unlikely, could have dealt a potentially lethal blow to Merkel's government. But analysts said the failure to elect Wulff in the first two rounds of voting was tantamount to a public humiliation for the chancellor.
"The coalition has clearly failed to give a show of unity and the new start that is so badly needed to escape from the slump it has been in for weeks," Oskar Niedermayer from Berlin's Free University said.
Business daily Handelsblatt described the "debacle" as Merkel's "first vote of no confidence."
The first-round result indicated that several members of Merkel's coalition partners, the Free Democrats, opted against Wulff in the secret vote, since her government had - on paper - an absolute majority in the Federal Assembly.
“It was actually pretty clear that one or two were out to settle old scores,” said Dirk Niebel, FDP member and German development minister.
The big question was whether there were enough other dissenters in the closet to tip the vote in favour of Gauck in the third ballot.
SPD chairman Sigmar Gabriel said the repeated rejection of Wulff was proof Gauck was the better candidate, but the Social Democrats could not convince delegates from the far-left party The Left to back him in the third round of voting.
A decidedly relieved-looking chancellor on Wednesday evening was apparently hoping just to put the whole episode behind her.
"It was certainly a long process today. But I'm glad we will have a good president," Merkel said. "In my opinion Wulff is the right person at this time."
The presidential election was widely seen as a crucial test for Merkel, who is already weakened by her lukewarm performance and the constant bickering within her coalition since last September's election. Though the job as head of state is largely ceremonial, the controversies and fierce debate this time around have added a heavy political significance to the position.
The vote became necessary after the previous president, Merkel ally Horst Köhler, stepped down on May 31 after coming under fire for comments he made about the role of the German military abroad.