German public broadcaster ARD network reported Tuesday evening that a judge failed to produce a written document related to the three men's convictions within a prescribed time period meaning they had to be released according to Italian law.
The fourth man was sentenced to 13 years in prison for helping organise the 2006 murder of Maria Strangio, the wife of a mafia member. The man had been placed under house arrest due to poor health but escaped during a hospital stay, ARD said.
Strangio's murder is said to have provoked the 2007 killing of six mafia members outside a Duisburg pizzeria. The shooting shocked Germany and led to a cooperative police action with Italy to root out organized crime in the region.
Although it is not clear how the three released prisoners were involved in the killings, police said they were not the main organisers, who remain imprisoned.
Giovanni Strangio, a relative of Maria who is said to have been one of the primary masterminds of the massacre, was sentenced to life in prison this year.
The Federal Office of Criminal Investigation (BKA) said it could not rule out that the men could return to Germany, where they have lived before.
If that happens, Germany could once again be a centre of mafia clashes, ARD reported.