Bavaria State Premier Horst Seehofer promised that a memorial room would be set up in Munich, and told Israeli Deputy Prime Minister Silvan Shalom: “We cannot heal the wounds, but we can try to reduce the pain.”
Wreaths were laid at the Olympic village where the attack began, and then a service was held at the Fürstenfeldbruck Air Base, west of Munich, where the hostage drama ended in a bloodbath. Not only 11 Israeli athletes were killed, but also five Palestinian terrorists and one German police officer.
Flags flew at half-mast on southern Bavarian state public buildings as the ecumenical memorial service took place at the base, the site of the climax of the hostage-taking by members of a radical Palestinian group known as "Black September".
Six survivors and 11 relatives of the Israeli athletes and coaches taken hostage and subsequently killed were expected to attend the commemoration, to be addressed by, among others, Ankie Spitzer, widow of fencing coach Andre Spitzer.
The 40th anniversary has given rise to new research into the horrifying chain of events at the summer Munich Games, which were meant to showcase the new face of Germany nearly three decades after World War II.
On September 5, 1972, gunmen broke into the Israeli team's flat at the Olympic village, immediately killing two of the athletes and taking nine others hostage to demand the release of 232 Palestinian prisoners.
A bungled rescue operation resulted in all of the hostages being killed, along with a West German policeman and five of the eight hostage-takers.
The news sent shockwaves through Germany just 27 years after the Holocaust, and it caused a deep rift with Israel.
Israeli sprinter Esther Roth-Shachamorov relived the terror in an interview with AFP this week.
"I remember an exhausting and frightening day," she said.
"We saw the Germans conducting negotiations with the terrorists through the balcony. They were threatening every two hours that if 200 Palestinians were not released, they would throw an Israeli down on the street," she added.
Henry Hershkovitz, who was on the Olympic shooting team and has returned to the stadium, was quoted Wednesday by the Berliner Zeitung newpaper saying: "We were like a family and most of this family was killed."
Former fencer Yehuda Weinstain told the paper: "The Games admittedly went on but their spirit had been murdered."
Forty years later the events continue to provoke controversy.
Last week, Israel released official documents on the killings, including specially declassified material and an official account from the former Israeli intelligence head, lambasting the performance of the German security services.
The German police "didn't make even a minimal effort to save human lives," former Mossad head Zvi Zamir said at the time after returning from Munich.
He said elite German snipers had been equipped only with pistols, and that personnel carriers meant for the rescue operation had arrived late.
"They had no follow-up plan, nor any means of improvising an alternative," he said.
Meanwhile Der Spiegel in July accused the German government and Olympic organisers of covering up grave mistakes.
Months before the hostage-taking, the German interior ministry and the Bavarian state police warned federal authorities in vain of the possibility of "terrorist acts" at the Games, the magazine said.
The report recalled that the Olympic village was surrounded by a simple chain link fence without security reinforcements.
The head of the Munich police evidently feared that a robust security presence would revive ugly memories of the 1936 Games in Berlin, presided over by Adolf Hitler.