Gabriel told Die Zeit newspaper that he would recommend that former European Parliament president Martin Schulz take his place as SPD party chairman and candidate for chancellor.
The 57-year-old Gabriel is currently vice chancellor and economy minister in Merkel's loveless "grand coalition" government, as well as chairman of the SPD, Germany's oldest party which is riven by disputes between its leftist and centrist wings.
Polls show Merkel's conservative Christian Democrats with a double-digit lead over the SPD, despite unease over her liberal asylum policy.
Asked by Die Zeit whether he was stepping aside because of a poll showing that the party rank-and-file overwhelmingly believed Schulz would have a better chance in the general election, Gabriel said: "Yes. It is my responsibility as (party) chairman."
In an interview with news weekly Stern ahead of a party meeting in Berlin later Tuesday, Gabriel was even more self-critical: "If I were to run, I would fail, and the SPD with me."
He added that the 61-year-old Schulz "would clearly have a better chance in the election" on September 24th.
Gabriel told Stern and Die Zeit that he now aimed to become foreign minister, replacing Frank-Walter Steinmeier who is expected to be elected German president next month.
Merkel, 62, announced in November that she would seek a fourth term at the helm of Europe's top economy.
Her popularity has endured despite misgivings about her refugee stance, as voters seek a safe pair of hands in the wake of Britain's vote to leave the European Union and the inauguration of Donald Trump as US president.
Gabriel had long hesitated to say whether he would carry the banner for the Social Democrats, who have seen their support plummet in Merkel's shadow.
The news nevertheless came as a shock to political observers in Berlin.
The top-selling Bild had reported earlier this month that Gabriel had made up his mind to run for the top job, on the advice of former chancellor Gerhard Schröder, 72.
Polls show that Merkel's Christian Democrats (CDU) as the clear frontrunners for the national election with around 37 percent support, followed at a distance by the SPD with about 20 percent.
An insurgent anti-immigration party, the AfD, is currently polling at around 15 percent.