The Federal Constitutional Court ruled that the database, which was set up in 2007 to help German security officials share information about suspected terrorists, did not break the rule about separating police and secret services laid down in Germany's constitution.
A retired judge had challenged the database because he believed it blurred the strict dividing line between law enforcement and intelligence established after World War II to stamp out the abuses of the Nazi period.
But judges in the southwestern city of Karlsruhe rejected the complaint, arguing that attacks by militants represented an assault on the "common good as a whole" and had to be "fought with the tools of the state under the rule of law."
However, the court said the database needed tighter security and that the data was currently available to too many authorities. Data stored on terrorism suspects includes names, dates of birth, addresses, bank details, religion as well as registered weapons and "skills relevant to terrorism."
The judges also called for Germany's independent data protection authorities to regularly check the database to make sure it kept to the country's strict data regulations.
The German police union (DPolG) said it was relieved at the ruling, which union head Rainer Wendt described in a statement as "a strong signal at the right time."
German Interior Minister Hans-Peter Friedrich also greeted the ruling and said the database was needed as an important instrument for the state in the fight against terror.
"I think we can be happy that the constitutionality of this law has been upheld," Friedrich told reporters, and pledged to implement the changes the court had ordered.
By extension, the ruling also confirms the legality of another database on far-right extremists - based on the design of the earlier anti-terrorist database - established last year in response to the discovery of the far-right NSU terrorist organisation.
Meanwhile, Justice Minister Sabine Leutheusser-Schnarrenberger also felt the ruling justified her criticism of the database and greeted the judges' demands for tighter security and limited data collection.
The court repeated critics' concerns that the current system makes it all too easy for innocent people with no links to extremism to end up on the database.
One in five of the over 17,000 people on the database are thought to be just an acquaintance, neighbour or relative of a suspect.
The interior ministry says 84 percent of those they have data on belong to radical Islamic groups abroad thought to have ties to Germany.