Joachim Löw's side remain unbeaten and are favourites to claim the group four's single guaranteed place for the finals in South Africa, but the matter is likely to be decided when they face main rivals Russia in Moscow on October 10.
"We lost our way a bit in the second half," said Löw. "We would have controlled the game a bit better, but we made too many errors in our structured play. We still have some room for improvement.
It was Löw 's 40th game in charge and his 28th win with six draws and six defeats, but there are bigger tests ahead against Finland and Russia in the next few months.
Löw got a boost before kick off when captain and Chelsea star Michael Ballack was declared fit to play despite damaging a thigh muscle in training on Tuesday.
And there was a slight surprise in Löw's selection with Stuttgart's Serdar Tasci chosen ahead of Berlin captain Arne Friedrich.
Bayern Munich midfielder Schweinsteiger opened the scoring after just 12 minutes when he received the ball just outside the penalty area and rifled his shot home from 30 metres for his 19th goal on his 68th appearance.
Germany were rarely tested and they doubled their lead just after the break when Bayern Munich striker Mario Gomez's shot at Azerbaijan goalkeeper Farhad Veliyev was headed into the empty net on the rebound by Klose.
It was his 89th goal on his 45th appearance and made up for his first-half miss when the Bayern Munich stalwart drove his header over the bar with an empty net at his mercy.
Azerbaijan, coached by ex-Germany boss Berti Vogts, had precious few chances to score in the second-half as the German defence, led by Bayern's Philipp Lahm, closed down their opponents' attack.
The sides will clash again in Hannover on September 9 in the return qualifier.
Russia have the chance to close the gap back to a single point before then when they face minnows Liechtenstein in St Petersburg on September 5.