Toni Kroos and Mario Gomez had put the World Cup holders 2-0 up at Berlin's Olympic Stadium on Saturday with an hour gone before the Three Lions roared back.
Harry Kane and Jamie Vardy netted for England before Eric Dier headed the winner a minute into stoppage time as Germany were punished for a sloppy defending and failing to create chances in attack.
"It's annoying as a coach when we lose after being 2-0 up, but you can't
say we didn't deserve it," admitted Löw, who steered Germany to win their fourth World Cup at Brazil 2014.
"Even when we were 2-0 up, we weren't in control.
"We had problems in our build-up and struggled up front."
Germany host Italy in Munich on Tuesday and Löw knows there is plenty of work to be done in his final international friendly before naming his squad for Euro 2016.
"We know the areas we have to work on, some of the combinations didn't work and left a lot to be desired," said Löw.
"In the second-half, we gave away a crazy amount of space.
"We didn't stand compact in defence, we lost balls on the counter and the English had more and more space for their quick attackers to exploit."
But his senior players were far from amused after surrendering their lead to vastly inexperienced England side.
Germany have failed to impress since winning the World Cup, losing Euro 2016 qualifiers to Poland and Ireland.
Despite finishing top of their qualifying group, Germany have much to do if they are to add the European crown to their world title in France in June.
With their captain Bastian Schweinsteiger facing a race to be fit for Euro 2016 after a knee injury, Germany's stand-in skipper in Berlin, Sami Khedira, was not amused.
"It was an unnecessary, dumb defeat," fumed the Juventus midfielder.
"Every single member of the team has to remember to keep working hard.
"We did a lot right up until we went 2-0 up, then a lot less after that. We played well for 60 minutes only. We have to play much better against Italy and at the European championships."
Toni Kroos had opened the scoring for Germany. Photo: Annegret Hilse/dpa
Goal-scorer Mario Gomez was also at a loss to explain the defeat.
"It's hard to explain how things worked out," said the striker.
"Up until we went 2-0 up, we played really well, although we didn't really use all our chances.
"After that, we left too much space and gave England the belief."
Real Madrid's Toni Kroos put the defeat down to poor defending throughout the German team as centre-back Jonathan Tah made his debut in the second-half in an inexperienced back four.
"When you play, you always want to win, so the result is important, even if it was only a friendly," said Germany's goal-scorer.
"We defended badly in the second half, even though we had a few new faces in defence, a performance like that still shouldn't have happened."