Schulz won 387 of 670 votes, including those of the parliament's biggest group, the European People's Party (EPP), which had agreed to share at mid-term the five-year president's mandate with the Socialists and Democrats.
Also running for the top job were British MEPs Diana Wallis of the centrist Liberals and Democrats (ALDE), who won 141 votes, and Nirj Deva of the eurosceptic conservative ECR group, who won 142 votes.
In a statement after the vote, Schulz told members that "for the first time since it was founded, the failure of the European Union is a realistic possibility."
He said Europe needed a "shared understanding that the EU is not a zero-sum game, in which one person must lose so that another can win. The reverse is true: either we all lose – or we all win."
Schulz, 56, joined Germany's SPD aged 18, and was elected to the European Parliament in 1994. He took the helm of the Socialist group in 2004. Before entering politics full-time, he worked in the publishing industry.
Schulz made a name for himself in 2003 in a row at the assembly with Silvio Berlusconi.
The then Italian premier said Schulz, who had vexed Berlusconi in questions over his media empire, would be perfect in the film role of a "Kapo," a prisoner who worked for the Nazis in concentration camps.
The incident triggered a diplomatic spat between Italy and Germany.
Schulz has been known for his sharp tongue and dislike of extremist elements that have tried to play a role in parliamentary business – he once played a key role in blocking French right-wing extremist Jean-Marie Le Pen from presiding over a parliament session.
The European Parliament's president represents the parliament internationally and is in charge of keeping order during debates, although the position has been described by some as a vague one. Schulz will serve a two-and-half year term and can be re-elected once.