"In terms of the return from Germany, half our personnel should be back by 2015 and the remainder by 2020," he told parliament as he announced a widespread shake-up of the military and significant cuts to its budget.
Britain and the United States stationed troops in Germany in the aftermath of World War II as a visible show of support for their NATO ally, and Britain currently has about 20,000 military personnel based there.
"The presence of the British military has played an important role in demonstrating Alliance solidarity, and has also been a symbol of steadfast UK-German friendship," the strategic defence and security review said.
"But there is no longer any operational requirement for UK forces to be based there, and the current arrangements impose financial costs on the UK, disruption on personnel and their families and opportunity costs in terms of wider army coherence."
Some of the returning troops could be housed in the three Royal Air Force bases to be closed under the defence review, which is being implemented along with cuts to the defence ministry budget of eight percent. British forces in Germany are predominately stationed in the northwest of the country in Lower Saxony and North Rhine-Westphalia.
But the review said Britain would retain its permanent joint operating bases in Gibraltar, Cyprus and the Falklands Islands among other locations. British troops will also continue to train in Canada, Kenya and Brunei.