The bank in question, Julius Bär, is said to have warned German clients who could be affected.
German state authorities have sparked controversy in the past for paying for stolen data on tax dodgers hiding money in Swiss banks.
Monday's Süddeutsche Zeitung reported that four public prosecutors in North Rhine-Westphalia (NRW) were planning a new wave of investigations into alleged tax evaders later this year.
These will be based on three CDs of stolen data and a comprehensive list of German customers who have invested their money in four financial institutions in Switzerland.
NRW State Premier Hannelore Kraft said earlier this month the state would continue to buy such data as and when possible, as she was convinced it was legal.
In October 2010, the NRW paid €1.4 million for a CD with data from Julius Bär on Germans who were allegedly hiding untaxed assets. The data sparked numerous investigations and voluntary disclosures.
Zurich banking insiders say Swiss banks have tightened their controls lately because of the repeated purchase of stolen data. The new measures are designed to expose data thieves and discourage further theft of customer information.
The bank did not comment on the nature and extent of customer data sold to German buyers. The thief was discovered "in the context of tighter controls and a comprehensive internal investigation," Julius Bär chief Boris Collardi told the Swiss Sonntags-Zeitung newspaper.
The suspect was arrested at his office in Zurich and has since been fired, bank spokesman Jan Vonder Mühll confirmed.
The row between Germany and Switzerland escalated earlier this month when German papers reported that tax authorities were investigating Julius Bär as well as the Swiss banking giant UBS, on suspicion of transferring German clients' assets to Asia.
The report prompted German Social Democrat Sigmar Gabriel to equate Swiss banks with organized crime syndicates for evading taxes
Switzerland's President Eveline Widmer-Schlumpf dismissed the charges, saying there was no evidence of banks engaging in the practice.