Mustac, accompanied by German police, left the Croatian capital on a regular Lufthansa flight for Munich, the state-run HINA news agency reported.
Germany was seeking the 72-year-old Mustac under the European Arrest Warrant (EAW), which regulates extradition between European Union member states.
The issue of extraditing former spies sparked a row between the EU and Croatia right after the Balkan country joined the bloc in July.
Just days before becoming a member, Croatia changed its domestic law regarding European warrants, banning extradition of people wanted for crimes allegedly committed before August 2002, the date the EAW was introduced.
The move left Croatia sharply at odds with its 27 EU partners, notably Germany, which was seeking the extradition of Mustac and his deputy, Josip Perkovic.
In the wake of the dispute with Brussels, Zagreb amended the law in October.
Perkovic was eventually extradited to Germany in January for his alleged role in the 1983 murder in the southern German town of Wolfratshausen.
But the case of Mustac has been subject to a lengthy legal back-and-forth, until the Supreme Court, Croatia's top tribunal, ruled earlier this month that his extradition should go ahead.
Local media have speculated that Zagreb was reluctant to extradite the two former top spies as they may hold compromising information on influential people in Croatia.
But the government has denied this, saying its aim was to protect veterans of the 1990s independence war from being investigated by the EU.