An Amnesty report cited three documented deaths and 12 cases of serious injury in recent years which it attributed to police action but said the actual figure was likely to be much higher.
"Law enforcement officials are not above the law -- they are subject to it. This means that the police must be accountable to the law, to the state and to the public," Europe and Central Asia Deputy Programme Director David Diaz-Jogeix said in a statement.
"Failure to live up to international standards ... is leading to a climate of impunity and a lack of accountability."
The report, "Unknown Assailant", recounts the use of what Amnesty calls excessive force during arrests, against protesters at demonstrations and during deportations.
It said a lack of information about how to lodge a criminal complaint over police abuse, difficulty in identifying officers involved and inadequate follow-up by the authorities were recurrent problems.
The report cited the infamous case of Oury Jalloh, an asylum-seeker from Sierra Leone who burned to death in 2005 after having been tied to a bed in a cell in the eastern city of Dessau.
"The accused police officers on duty when he burned to death remain in office but they are no longer working at the same police station," it said.
Amnesty called on German authorities to tackle the problem head-on with independent police complaint bodies, clear identification of police officers when they are on duty and regular training for police on the use of force.
"Officers responsible for criminal conduct must be brought to justice in full and fair proceedings," Diaz-Jogeix said.
The head of the German Police Union (GdP), Konrad Freiberg, rejected the group's call for special commissions to address brutality at the hands of the authorities, saying victims already had means of redress at their disposal.
He also welcomed AI's assurance that mistreatment of suspects was by far the exception in Germany, not the rule.
"Beginning with the selection of officers, to training and later the practice in the field, the police reflect the democratic understanding of the law set down in our constitution," he said.
But he dismissed the call to require all police officers to wear name tags, saying it would expose them to potential threats and harassment.
A separate police union, the DPolG, said that stress, exhaustion and fear on the job could push some officers over the edge. It said that the political class and justice authorities should do more to protect officers on the beat as a means of preventing brutality.