Merkel's Christian Democrats (CDU) and their Bavarian sister party the CSU - a team that last September celebrated a landslide win at the national level - between them secured 35.3 percent of votes cast.
The neo-Nazi National Democratic Party of Germany (NPD), won 300,000 votes, one percent of the total, and so wins its first seat in the 751-member European parliament.
The far-right party profited from a reform of Germany's electoral system, which removed a three percent threshold to gain seats.
And the removal of the three-percent hurdle also meant that the Animal Protection Party and a spoof party called Die Partei are set to win a seat each.
Die Partei’s leader Martin Sonneborn said: "I will spend the next four weeks in intensive preparation for my resignation."
The former editor of satirical magazine Titanic said his party follow a rotation principle. "We will try to resign once a month, so that we can smuggle 60 party members through the EU Parliament," he said. "So we'll be milking the EU like a small, southern European country."
The CDU-CSU result - though less triumphant than last year's 41.5 percent German election win, mainly because of CSU losses - was seen as another endorsement for Merkel, the only leader of a major EU member country to have weathered the fallout from the eurozone crisis.
Germany, the EU's most populous country, sends 96 members to the European legislature, which has demanded a bigger say in who takes over from outgoing European Commission president Jose Manuel Barroso.
Strong gain for centre-left SPD
Germany's SPD leader, Vice Chancellor Sigmar Gabriel, greeted the outcome at a joint appearance with Schulz, telling him: "We are super-proud that you are one of us."
Gabriel also praised the turnout of over 48 percent: "The people knew what this was about, and that's why they went out to vote: they wanted to decide for themselves who will be the next president of the European Commission."
The election also saw the debut of a new anti-euro party, the Alternative for Germany (AfD), which made its entry into the European Parliament with seven percent of the vote and seven seats.
"The AfD in this election blossomed into a new people's party in Germany, as a liberal party, as a social party and as a value-oriented party," said its leader Bernd Lucke, an economics professor.
The Green party took third place with 10.7 percent, followed by the far-left Die Linke with 7.4 percent. The pro-business Free Democrats, once Merkel's governing allies, remained in the doldrums at 3.4 percent.
See how the night unfolded in our live election blog.