The deal, which is expected to be signed in coming weeks, could snare nearly 1,000 tax cheats over two years. The banks will pay an upfront amount of two billion francs (€1.91 billion), some of which they will recoup from voluntary disclosures by German clients.
Swiss Finance Minister Eveline Widmer-Schlumpf said the deal "creates legal certainty and will strengthen the competitiveness and the reputation of Switzerland as a financial centre in the long run."
In Berlin, a German government source said the accord "should mark a real new start in the relationship with Switzerland."
Under the deal, German taxpayers would be given a one-off chance to make an anonymous lump sum tax payment, with the tax rate to vary between 19 and 34 percent of the assets, depending on how long it has been in the account and how much interest it has earned.
Any taxes collected from these voluntary disclosures would be offset against the two billion franc advance payment and refunded to the Swiss banks.
In future, all investment income and capital gains arising from assets held by German taxpayers would also be covered by a withholding tax of 26.375 percent.
To prevent new undeclared funds from entering Swiss banks, German authorities would be able to request information on suspected tax cheats.
The number of requests made is expected to reach between 750 to 999 for two years, said the Swiss ministry, which added that the accord should be in force by 2013.
Germany and Switzerland have locked horns over the small alpine state's banking secrecy rules, which Berlin says help to shield tax cheats.
Tensions between the two boiled over in July 2010, when German authorities raided branches of Credit Suisse in Germany in a tax evasion probe based on information they purchased for a reported €2.5 million.
Switzerland angrily condemned the move, saying that the data was stolen in violation of its banking secrecy law.
With Wednesday's deal, the two neighbours agreed to set aside their dispute over the stolen bank client data.
"Germany no longer sees any reason for the purchase of stolen bank client data. Switzerland undertakes to waive criminal prosecution of persons on account of involvement in illegally acquiring bank data," said the Swiss ministry.
In addition, German authorities would drop any plans to prosecute any bank employees who may have helped their clients to hide assets from the taxmen.
The latest dual taxation agreement counts among a series of bilateral deals the Swiss have been signing in order to meet tougher international standards after a post-crisis clampdown on offshore and banking havens.
While the German government has trumpeted the deal as a victory for tax justice, the opposition slammed it as insufficient.
Wolfgang Neskovic, justice spokesman for the socialist Left party, dismissed the arrangement as a “pay packet for German tax criminals.”
The centre-left Social Democrats deputy parliamentary leader, Joachim Poß, likewise dismissed such a lump-sum tax on invested money as “absurd.” The deal would reward the “unrepentant” perpetrator, he said. Allowing the tax collections to be ferried through the banks meant that the perpetrators would remain anonymous and the exact amounts they owed unclear.