President Nicolas Sarkozy and German Chancellor Angela Merkel agreed on the deal during a meeting in Paris earlier this week, government spokesman Thomas Steg told a press conference on Wednesday in Berlin.
The two countries share a joint army brigade of some 5,000 soldiers - 2,800 of which are German. Until now, they have been stationed only in south west Germany.
"Germany has agreed in principle to transfer members of the Franco-German Brigade to France, that includes German troops," Steg said, calling the move "highly symbolic and historically significant".
A handful of German officers are already based in Strasbourg, east France, directly engaged with the NATO mission Eurocorps.
However, no German military unit has been stationed in the country since the end of hostilities in World War II.
The Franco-German Brigade was set up in 1989 by the then French President Francois Mitterand and former German Chancellor Helmut Kohl to increase military cooperation between the two countries.
Steg added that both Merkel and Sarkozy believe the joint initiative should be kept going. The country's defence ministers are set to meet next year to thrash out a clear plan for the brigade's future.