The legislation passed in 2008 and covering the activities of the federal police in surveillance of terror suspects must be reworked by June 2018, the Federal Constitutional Court in the southwestern city of Karlsruhe ruled.
The legislation gave investigators sweeping powers to use secret cameras and bugging devices in private homes and government-developed surveillance software on personal computers to prevent imminent attacks.
Controversially, it specifically allowed snooping in the bedroom and bathroom, and for people not targeted in an investigation to be observed if they are with a suspect.
The scarlet-robed judges said in their ruling in the high-profile case that while the thrust of the measures is covered by Germany's Basic Law, they found "several individual provisions disproportionate encroachment on privacy".
The court said that before the federal police may use data collected during surveillance operations, an "independent service" must review them to determine whether they contain "information of a highly private nature".
In addition, surveillance may only be carried out against a suspect outside his home if there is a "strong likelihood" he could commit a "terrorist act in the near future".
And the ruling bars data from being shared with other domestic intelligence services without a "specific threat of attack".
Germany has strict constitutional protections on state security powers to check the kind of egregious abuses committed by the Nazi and communist dictatorships.
The challenge was brought by former interior minister Gerhart Baum of the liberal Free Democratic Party and members of the opposition Greens, who argued that the level of intrusion into citizens' privacy was unconstitutional.
Interior Minister Thomas de Maiziere defended the law in testimony to the tribunal last July, saying the government had used the powers judiciously and claiming that they had helped prevent a dozen attacks on German soil.
Civil liberties groups have long complained about what they say is a creeping "surveillance state" in Germany.
Germany has thus far escaped a large-scale Islamic extremist attack but the secret services say the country is in the sights of jihadists.