The move was made quietly in Wunsiedel in northern Bavaria early on Wednesday morning, by choice of a church board that supervises the cemetery where Hess was buried, according to the Süddeutsche Zeitung.
His remains will be cremated and scattered at sea, the newspaper reported on Thursday.
The decision was made because the grave in the town of about 9,500 has become a neo-Nazi pilgrimage site over the last few years – much to the dismay of local politicians and many residents.
Despite a court order banning their assembly, skinheads have commemorated Heß – who died in a German prison in 1987 while serving a life sentence for crimes against humanity – by marching through Wunsiedel's streets every year and saluting Heß at his grave.
In an effort to stop this, the cemetery's owners terminated the lease on the grave, which was shared with some family members, and decreed that the body would only be allowed to remain in Wunsiedel until October.
Although relatives initially resisted – one family member even filed a lawsuit to stop the exhumation – intensive negotiations resulted in the agreement to remove Heß' remains, according to the Süddeutsche Zeitung.
Heß was originally allowed to be buried in Wunsiedel because he had asked for it in his will.
One of Hitler's closest aides, Heß was eventually made the Nazi dictator's deputy.
However, he was marginalized in the Nazi leadership over time, and he flew to Scotland in 1941 in a failed attempt to convince Great Britain to forge a peace agreement with Nazi Germany.
He was sentenced after the war during the Nuremberg Trials and imprisoned at Spandau Prison in Berlin.