The German Football Federation (DFB) announced on Friday that Ballack would not play next month's Euro 2012 qualifiers against Belgium and Azerbaijan.
Ballack, who technically remains Germany's captain despite having missed the World Cup in South Africa, is still recovering from an ankle injury, though he has played for his new club Bayer Leverkusen.
The decision to leave Ballack out underscores the ongoing tension over who will hold the job of German captain – Ballack or defender Philipp Lahm.
Coach Löw said he told Ballack of his decision by phone.
“It was a good, open conversation,” Löw said. “Michael is on a good track three months after his serious injury and everyone is pleased with the recovery. It is now important that he finds his optimal form in the coming weeks at Leverkusen.”
Lahm replaced Ballack for the recent World Cup after Ballack suffered a serious ankle injury at the hands of German-Ghanain player Kevin-Prince Boateng during the English FA Cup final.
With Ballack absent, Lahm is likely to retain the captaincy in next month's qualifiers against Belgium and Azerbaijan.
Löw was in Dortmund last Sunday to see Ballack make his first appearance back in the Bundesliga after four years in the Premier League with Chelsea. His team Bayer Leverkusen beat Borussia Dortmund 2-0.
After his injury, Ballack only returned to training at the start of August and has played only a Bundesliga and Europa League match.
Looking ahead to the qualifiers, Löw said: "We want to win our first two games with the same football that we played at the World Cup."
The hot topic in the German media is the "C-frage" or who will captain Germany through their Euro 2012 campaign - Ballack or Lahm.
Ballack does not always cut a sympathetic figure: he is a dominant character and has clashed with both striker Lukas Podolski and team manager Oliver Bierhoff on the pitch during games for his country.
After former veteran midfielder Torsten Frings was dropped from the Germany squad in October 2008, Ballack accused Löw of not showing his senior players enough respect, but later apologised.
At that time, Ballack was in a powerful position as Germany's only true world-class player, while Löw was only two years into the job.
Two years on, Löw's position is far stronger, having guided a young Germany squad to third in South Africa, while Real Madrid's Sami Khedira is ten years younger than Ballack and is being groomed to replace him.
Lahm turns 27 in November and as well as being an outstanding defender, has a fresh-faced image and prefers to let his football speak for him.
Ballack is back in the Bundesliga after his four-year hiatus with Chelsea and is now under pressure to prove himself after failing to have his contract extended in London and losing his place in the Germany squad.
Germany has a Euro 2012 qualifier against Turkey at Berlin's Olympic Stadium on October 8 and then away to Kazakhstan on October 12. Should Ballack then be left out of the squad again, eyebrows will be raised.
Goalkeepers: Manuel Neuer (Schalke 04), René Adler (Bayer Leverkusen), Tim Wiese (Werder Bremen)
Defenders: Holger Badstuber (Bayern Munich), Marcell Jansen (Hamburg), Philipp Lahm (Bayer Munich), Per Mertesacker (Werder Bremen), Sascha Riether (Wolfsburg), Heiko Westermann (Hamburg)
Midfielders: Sami Khedira (Real Madrid/ESP), Toni Kroos (Bayern Munich), Marko Marin (Werder Bremen), Mesut Özil (Real Madrid/ESP), Bastian Schweinsteiger (Bayern Munich), Christian Trasch (Stuttgart)
Forwards: Cacau (Stuttgart), Mario Gomez (Bayern Munich), Stefan Kiessling (Bayer Leverkusen), Miroslav Klose (Bayern Munich), Thomas Müller (Bayern Munich), Lukas Podolski (Cologne)