In a short statement, the ECB said that the 63-year-old German had decided to resign "for personal reasons" before the end of his term of office in May 2014.
The announcement sent the markets into a tailspin, with the euro falling below $1.37 for the first time since February.
Stock markets across the single currency area nosedived, too, with the blue-chip DAX 30 index in Frankfurt shedding more than 4.4 percent, the CAC-40 index in Paris down more than 3.6 percent, the IBEX-35 in Madrid plummeting 4.4 percent and the FTSE Mib index in Milan down 4.9 percent.
Stark, who joined the bank's executive board in June 2006, "will stay on in his current position until a successor is appointed, which, according to the appointment procedure, will be by the end of this year," the ECB statement said.
Stark, one of the most hawkish ECB members who was even recently dubbed a "Rottweiler" by a Greek newspaper, has been critical of the bank's controversial programme of buying bonds of those countries - such as Greece - who find themselves unable to drum up financing by the markets.
The programme was put on hold until last month, when, according to a number of media reports, both Stark and Bundesbank president Jens Weidmann voted against its resumption.
On Thursday, ECB President Jean-Claude Trichet hit out at German criticism of the programme.
The bank only embarked on such a programme "because the governments in question had not behaved properly," the Frenchman said.
"I would very much like to hear congratulations for an institution which has delivered price stability in Germany over almost 13 years." But the ECB statement made no mention of such differences.
Instead, it said Trichet thanked Stark "wholeheartedly for his outstanding contribution to European unity over many years."
The ECB president expressed "particular gratitude for his exceptional and unwavering dedication as a member of the executive board and governing council for more than five years."
ECB watchers, however, suggested Stark's departure could point to deeper differences among EU and eurozone policy makers over the way out of the current debt crisis, especially as it was the second German resignation in Frankfurt in the past six months after Bundesbank President Axel Weber stepped down in February.
"It seems to confirm that there is a deep rift within the EU Council," said Holger Schmieding, chief economist at Berenberg Bank.
The resignation "will feed the unwarranted suspicion in global markets that Germany may eventually withdraw from monetary union. As such, it will probably exacerbate market tensions for a while," Schmieding said.
Carsten Brzeski, senior economist at ING Belgium, said that following the resignation of Weber at the Bundesbank earlier this year, "the second very outspoken critic of the ECB's bond purchasing programme is leaving the ship."
He believed the term "personal reasons" was a cover for "a deep-rooted disappointment at work," the analyst said. There is "clashing behind the scenes of the Eurotower."
Nevertheless, once the immediate shock had worn off, Stark's departure may not have a big impact on the ECB's monetary policy in the longer run, Brzeski said.
"The ECB as an institution has internalised the stability orientation of the Bundesbank and is more than its individual members, he argued.