Bobby Wood, who plays for Germany's third division side Erzgebirge Aue, scored the winner just before the final whistle to seal the Americans' deserved victory.
It capped a dream few days for the 22-year-old who also hit the winner when the USA fought back to stun the Netherlands 4-3 in Amsterdam last Friday.
Again the USA fought back from a goal down to cap a strong display as Germany suffered their third defeat since winning the World Cup last year.
“That was a fun little game with a good end for us,” beamed Klinsmann, who has coached the USA since 2011 and steered Germany to third at the 2006 World Cup.
“My players are very happy and the way things developed in the second half, the victory was well deserved.
“This success will give American football even more attention.”
The defeat was not what Germany needed before their Euro 2016 qualifier on Saturday when they face bottom side Gibraltar away in Faro, Portugal, as they look to claim top spot in their group.
But after their Dutch win, it was a confidence booster for the USA as they prepare to host the CONCACAF Gold Cup next month.
“Our energy levels dropped as the game went on and we couldn't respond so well,” rued Germany coach Joachim Löw.
“If we had taken our chances at the start of the game, the USA would certainly not have come back from two or three-nil down.”
Mario Götze's first-half goal for Germany in Cologne was cancelled out by Mikkel Diskerud's equaliser just before the half-time break before Wood struck the decisive blow late on.
“That was inconvenient, we must play better football on Saturday, then we won't get into trouble again,” said Götze.
It was Klinsmann's second win over his former side Germany, coached by his former assistant Löw, after America's friendly win in Washington in 2013.
But Löw's team won when in mattered in the group stages in Brazil last year.
After a bright start, especially from debutant Patrick Herrmann, Germany took the lead when Bayern Munich's Götze hit his third international goal since netting the winner in the World Cup final.
Herrmann finished a blistering run by cutting in from the right wing and rolled a pass across the box for the unmarked Götze to fire home on 13 minutes.
But the Stars and Stripes deservedly drew level on 41 minutes when midfielder Michael Bradley, a former Borussia Moenchengladbach player put in a superb diagonal pass.
It found Oslo-born Diskerud in space and the winger fired past Germany's second-choice goalkeeper Ron-Robert Zieler with Manuel Neuer rested.
Löw made changes soon after the break with Sami Khedira, whose transfer from Real Madrid to Juventus was announced on Tuesday, taking over from Germany captain Bastian Schweinsteiger.
The USA, who reached the last 16 of Brazil 2014, finished the stronger with only a reflex save from Zieler denying Bradley's thunderbolt strike from the edge of the area with ten minutes left.
Bradley started the move which led to Honolulu-born Wood's winner as he held off Shkodran Mustafi's challenge to beat Zieler from outside the area just before the whistle as the Hawaiian- native punched his shot home.
Khedira hit the crossbar in injury time as Germany chased in vain for an equaliser to spare their blushes.