The court in Karlsruhe said in a statement the law was too broad and risked "grave dangers" to personal privacy.
"Data stockpiling effects almost every citizen with each use of the telecommunications system, meaning a great deal of sensitive information on practically every person is available for government access," the court said. If this is allowed, it could mean a "grave and irreversible invasion" of telecommunication privacy.
The ruling comes in response to a class-action suit filed by some 34,000 German citizens against the data-collection law, which has been in effect since January 1. It allows telephone and internet companies to save all information from their customers for six months.
This can still go on, the court said, but authorities can only access data in cases of criminal offence, which includes murder, theft, child pornography, money laundering, corruption, tax evasion and fraud. To use the data, authorities must have "specific evidence" to prove that their investigation would be hindered without the information.
Data can be saved, but only used in the most extreme cases, the court ruled. The injunction against the law is valid for the next six months pending further review.
Germany's government official for data protection Peter Schaar welcomed the high court's decision on Wednesday.
"For the moment, I'm happy," he said in Berlin, adding that the court must continue to protect civil rights. He said he was confident it would maintain its current opposition to weakening privacy protection in Germany. A court ruling in late February restricted secret online investigations by government authorities in all but the most extreme cases.