Australian officials who made the announcement said they had opted against picking a "big name" to help rebuild the team after a disappointing World Cup.
Osieck, 61, who was Franz Beckenbauer's assistant when Germany won the 1990 World Cup and coached Canada to the 2000 CONCACAF title, beat a host of better known candidates to head Asia's top-ranked team.
"It was obvious to me that it was a great challenge. I went for the challenge," Osieck told journalists by videolink from Slovenia, where the Socceroos are playing a friendly.
"I hope that together we will get where we want to be."
The appointment met with some bemusement from Australian media, which had linked former Brazil coach Carlos Alberto Parreira, Dutch great Leo Beenhakker and even ex-Aston Villa boss Martin O'Neill to the role.
"Holger who?" read the headline on www.adelaidenow.com.au's story, while one journalist quizzed the new coach on the pronunciation of his name.
Osieck has also led Turkish giants Fenerbahce during a 33-year coaching career spanning 11 jobs, and won the 2007 AFC Champions League title with Japan's Urawa Red Diamonds.
However, he was fired after a poor start to the 2008 J-League season and has since been working with FIFA's technical department.
He replaces Dutchman Pim Verbeek, whose contract expired after the World Cup in South Africa, where Australia failed to reach the knock-out stages.
The Socceroos have also been coached by top-rated Dutch coach Guus Hiddink, who took them to the 2006 World Cup knock-out stages, and former England boss Terry Venables.
Football Federation Australia chairman Frank Lowy said he wanted a coach who could manage the Socceroos' transition from the current crop of ageing stars, including Harry Kewell and Lucas Neill, to a new generation.
"It wasn't how famous he was, it was what history he had, what kind of person he is, does he get on with the team, what kind of contribution will he make to the game," Lowy said. "This was the criteria, rather than a big name."
Osieck's first major assignment is January's Asian Cup in Qatar, where Australia will be among the favourites despite only reaching the quarter-finals at the last edition in 2007.
"We're not going to Qatar just to have an enjoyable time - I'm the kind of person who wants to win," he said.
The German, who unlike Verbeek has pledged to live in Australia, will watch Wednesday's Slovenia game as a spectator and is expected to take charge for the first time for the September 3 away friendly against Switzerland.
He is contracted to work until the end of the 2014 World Cup with Australia, which is in the running to host the tournament's 2022 edition.