The choice could determine whether or not the centre-left SPD will stay in the coalition until 2021, when Merkel is planning to step down and retire from politics.
The vote was triggered by the departure of the Social Democrats' previous leader, Andrea Nahles, after the party's poor showing in European Parliament elections.
For the first time since the party was founded in 1890, a male-female duo will take over the party's leadership – following a model adopted by the Greens.
Another novelty is that the vote is no longer reserved to the 1,000 delegates attending the party conference but is open to all of the party's 26,630 members, who have until Friday to vote online or by post.
The result will be announced on Saturday ahead of the party conference on December 6-8th in Berlin.
Despite its importance, the election has failed to generate much excitement and the centre-left party is currently vying for third place in the polls with the far-right AfD after Merkel's centre-right CDU and the Greens.
In the first round of voting, which whittled down the pairs to two, only 53 percent of members took part.
The second round sees Olaf Scholz and Klara Geywitz, who got 22.7 percent in the first round, go up against Norbert Walter-Borjans and Saskia Esken (21 percent).
Olaf Scholz and Klara Geywitz stand next to Saskia Esken and Norbert Walter-Borjans after a live stream debate for the SPD leadership race. Photo: DPA
Victory for the first pair would be a relief for Merkel, who has been chancellor for 14 years.
Deputy Chancellor and Finance Minister Scholz and his running mate Geywitz want the coalition, which was put together with great difficulty in 2018, to stay.
Scholz lacks charisma and has been dubbed “Scholzomat” for his propensity to speak like an automaton but he has the full confidence of Merkel.
If the other duo wins, Merkel's fate is less clear.
Esken and Walter-Borjans have been highly critical of the coalition, although they have stopped short of calling for the SPD to pull out of it.
Conference delegates will in any case vote on whether to stay in the uneasy partnership after months of tensions.
The SPD scored a rare victory this month by getting the government to agree on the need to top up basic pensions.
But Esken and Walter-Borjans have called for an end to Scholz's “zero new debt” policies and the current government's timidity on battling climate change.
Esken and Walter-Borjans have the support of the youth wing of the party as well as the German branch of Fridays for Future, the environmental movement set up by Swedish teen Greta Thunberg which has a large following in Germany.
Young SPD-ers say Scholz, a former ally of Gerhard Schröder who later became mayor of Hamburg, hardly embodies the kind of renewal that the SPD needs.
But his rivals lack the kind of name recognition that could mean electoral success.
'Too rarely an alternative'
Whatever the result, the SPD's problems will not be resolved while the party is “in a bad state”, according to an editorial in the Spiegel weekly.
The SPD “offers no long-term political direction on the way in which it wants the society of tomorrow to function… on what the world of work will look like,” said Gero Neugebauer, a researcher at the Free University of Berlin.
Cut off from its “social roots”, the party seems “too rarely an alternative” to Merkel's CDU, he told AFP.
“The SPD cannot renew itself unless it is in opposition and has new people,” said fellow political expert Klaus Schröder.
After the next parliamentary elections scheduled for 2021, Schröder is predicting a coalition between the conservatives and the Greens.
By Mathieu Foulkes