Frodeno, 27, passed Canada's 2000 Olympic champion Simon Whitfield with just metres left, finishing in 1 hour 48 minutes 53 seconds.
Athens silver-medallist Bevan Docherty of New Zealand took bronze while there was heartbreak for hot favourite Javier Gomez, triathlon's "Tiger Woods," who was relegated to fourth.
"I think I slept about two hours last night," Frodeno said. "I knew I had trained very well but these guys who were with me at the front were really the big guys."
The six-foot-three (1.94 metre) Frodeno was nobody's pre-race tip with a relatively modest record including 13 top-10 World Cup finishes and last year's German national title. But he put on a devastating burst of speed to run down Whitfield and leave Gomez and Docherty trailing.
"I just tried to execute my own race. As Simon went I knew it was going to be tough, I just had to bite and fight," Frodeno said. "This year I've lost all my races on sprints. It teaches you a lesson and I've learned at the right time I guess."
The 1.5 kilometre (0.93 mile) swim, 40 kilometre cycle and 10 kilometre run was held at the scenic Ming Tomb Reservoir near Beijing.
The four had been neck-and-neck entering the stadium but world champion Gomez faded at the final turn, dashing the Spaniard's hopes of a first Olympic medal.
"I gave it all I had and Jan just kept coming," said Whitfield. "What a spectacular performance by him."
Docherty described the race, held in steamy temperatures of 28 C (82 F), as "cat and mouse."
"I was a little bit nervous with just those four guys there but it depends how much you want it," he said. "These guys really wanted it so much more."
Russian Alexander Bryukhankov had led out the swim but it was New Zealand's Shane Reed who landed first and made it through the transition followed by Frederic Belaube of France with Gomez not far behind. Luxembourg's Dirk Bockel and Axel Zeebroek of Belgium broke away on the bike leg and established a lead of nearly a minute on the favourites going
into the run.
Their advantage was chopped to just 20 seconds by the end of lap one and it disappeared entirely in the next lap as Gomez and Spanish team-mate Ivan Rana hit the front. The smart money was on Gomez, 25, who has four World Cup victories this season after winning the series for the past two years running. He also won last year's World Cup race on this course.
But the expected surge never came as he was tracked all the way by Frodeno, Whitfield and Docherty, and ran out of steam at the last. "I didn't get it but sport is like that. It's not mathematics," Gomez said. "I gave it everything I had to give. I tried to charge in the third lap but it wasn't enough. I had nothing more to do."
Frodeno wins Germany's first triathlon gold after Stephan Vuckovic's silver behind Whitfield in 2000.
"In Sydney I just rolled along and I beat a German, and today he (Jan) got me back so there's a little bit of irony in that," Whitfield said.