The court in the western city of Krefeld upheld the Chilean prison sentence of five years and one day for Hartmut Hopp, a German citizen in his 70s, for the abuses committed at the notorious Colonia Dignidad commune.
The ruling is not yet final as his lawyer has announced plans to appeal, the DPA news agency reported.
Hopp was the right-hand man of convicted paedophile Paul Schaefer, a former Wehrmacht soldier who in 1961 founded the commune where residents were indoctrinated and kept as virtual slaves over three decades.
Schaefer also collaborated with the dictatorship of Augusto Pinochet, whose secret police used the colony, about 350 kilometres south of the capital Santiago, as a place to torture and "disappear" his opponents.
The German court recalled that Hopp was convicted in Chile in 2011 of crimes including complicity in the sexual abuse of 16 minors, but fled to Germany before the final court ruling was imposed.
Last year prosecutors in Krefeld requested that Hopp serve out the prison term in his country of origin, in line with a Chilean request, and the court has now agreed.
The European Center for Constitutional and Human Rights greeted the ruling, calling it a long-overdue first step "in the resolution of the crimes of Colonia Dignidad, in particular rape and sexual abuse of minors".
It warned that Hopp had already fled Chilean justice once and said that authorities "must prevent a possible escape of Hartmut Hopp".
Germany last year said it was declassifying its files on the sect, and the foreign minister at the time, Frank-Walter Steinmeier, admitted that the diplomatic service had failed to stop the abuses.
The scale of the atrocities came to light only after the end of Pinochet's regime.
Schaefer in 1997 faced a series of lawsuits and fled Chile. He was arrested in Argentina in 2005 and subsequently convicted in Chile for sexual abuse of children, weapons possession and human rights violations.
He died in a Chilean jail in 2010 at the age of 88 while serving a 20-year sentence.
READ ALSO: Germany seeks extradition of Nazi sect paedophile doctor