The "Berghof", half way up a Bavarian mountain, was damaged by bombing at the end of World War II and US occupation forces blew up what was left.
Cyclists riding through the Bavarian forest take a track that runs straight through the site, where just a piece of wall remains.
No signpost shows the way there, but a notice board does tell visitors they have reached the right spot once they find this historical no-man's-land, neither quite on, nor off the map, a symbol of how Germany still finds it difficult to deal with its Nazi past.
In contrast, Hitler's "tea house", known as the "Eagle's Nest", a nearby mountain top lodge, is well on the beaten track.
Tens of thousands of visitors follow a dizzying road to the top for breath-taking views of peaks and valleys in both Germany and Austria.
The 1,800-metre-high (5,900 feet) Eagle's Nest, which now houses a restaurant, a cafe, and shops selling books with titles such as "Hitler's mountain", was a gift from the Nazi party to its leader on his 50th birthday.
It is said he was scared of heights and visited only infrequently.
Historians want both sites preserved for posterity, but others fear highlighting any site too closely associated with the Führer could encourage perverse pilgrimages or turn the area into a Nazi Disneyland.
"You must be very careful not to foster a fascination with Hitler," said Axel Drecoll, a 36-year-old historian in charge of the local documentation centre whose exhibits, visited by 160,000 people a year, depict Nazi crimes as well as Hitler's domestic life on the Obersalzberg mountain, overlooking Berchtesgaden.
"You've got to satisfy the curiosity of the tourists without pandering to sensationalism," he said.
Locals were appalled when a newspaper this year reported that a British company was offering a "Hitler holiday" to Berchtesgaden, along with other sites associated with the Führer.
"That really made waves," said local tourism director Michael Griesser who recalls how Bavarian authorities sought to stop the 'Face of Evil Tour' before realising they were dealing with ordinary tourists rather than neo-nazis.
Of course, there are still a few of those, who lay flowers and candles at the Berghof on Hitler's birthday, or wreaths on the anniversary of his death, but the documentation centre a short distance away quickly disposes of them, said Drecoll.
The real problem, he said, isn't right-wing radicals but the fact that, as people grow less afraid of the Nazi past, "the line between historical enquiry and commercial kitsch gets blurred."
"I don't want to see the Obersalzberg become a chamber of horrors" attraction, he said.
The US military kept a lid on the place for years by turning it into a recreational area for soldiers before handing the mountain back to the Bavarian government in 1995.
Until then all ruins, including some 12 kilometres of tunnels and bunkers, much closed to the public, had been subject to an official, but secret, preservation order.
But the order was then lifted "for political reasons", says Egon Johannes Greipl, head of Bavaria's Office for Ancient Monument Preservation. And authorities started carting off what remained of the rubble.
A few marble paving stones from Hitler's terrace did get recycled, by mistake, into a small outdoors chapel, newspapers said.
Greipl now wants the preservation order to be restored.
"After all nobody would think of demolishing the ruins of Olympia (Greece) and argue it would be better preserved in a documentation centre.
"It is original historical evidence you have on the Obersalzberg," he said.
Walter Schön, a regional official and the number two at the Bavarian justice ministry, opposes the idea of preserving the ruins. He said this, "would add no real, meaningful or worthwhile testimony to what took place" there, he said.
"Ascribing special cultural status to the Berghof" and surrounding Nazi homesteads "would only help create a sort of national-socialist hiking trail on the mountain", he adds.
Try as it may Berchtesgaden, like much of Germany, has trouble finding the proper balance when dealing with such history.
A five-star hotel recently build on the ruins of a house that belonged to Nazi leader Hermann Goering, near the Berghof, made sure all rooms were provided with a copy of a historical book on Hitler's crimes.
But such bed-time reading bothered some guests and the books were removed to the hotel library.