"This is a day we have long awaited," Berlin Mayor Klaus Wowereit said before an audience of around 600 guests at the city's Philharmonie concert hall near the Tiergarten park.
The hall is located near the now-demolished offices where more than 60 Nazi bureaucrats and doctors once worked in secret to implement the so-called T4 euthanasia programme.
Activists who had campaigned for a memorial since 2007 "had to fight not only against [people] forgetting but also against powerful opponents - science organisations that denied any participation in the 'euthanasia' murders and protected scientists who became criminals, Wowereit said.
The glass wall is the fourth and likely the last memorial to distinct groups of Nazi victims. It stands near a memorial to the six million Jews murdered in the Holocaust constructed in 2005, and two more to a million Roma and Sinti victims as well as homosexuals murdered by the Nazis.
"It was long overdue," added Lorena Endler, who works for Stolpersteine-Berlin, an organization that embeds small brass remembrance blocks into pavements outside houses where Holocaust victims lived before their deportation and murder.
First major extermination programme
The T4 programme began in 1939. Between January 1940 and August 1941 about 70,000 people died in what the Nazis deemed "mercy killings" of unfit members of society. Victims were mainly sent to gas chambers or killed by lethal injection in death camps in Germany and Poland.
The programme was ostensibly shut down in 1941, partly due to protests by the church, but continued in secret. Historians estimate that between 200,000 and 300,000 people were murdered.
Uwe Neumärker, director of the Holocaust memorial foundation in Berlin, said the slaughter of patients and residents of care homes marked "the first systematic mass crime of the National Socialist regime".
"It is considered a forerunner of the extermination of European Jews," he said before the inauguration.
Hartmut Traub fought back tears as he paid tribute to his uncle Benjamin, a schizophrenic who was murdered in a gas chamber in 1941 at the age of 27.