Chancellor Olaf Scholz said at the signing ceremony in the city of Görlitz that the plant's conversion "ensures greater security for Germany".
"Instead of train carriages, parts for the defence industry will be manufactured here from next year," Scholz said.
Russia's full-scale invasion of Ukraine in 2022 showed the need to build up European defence capabilities, the chancellor said.
The factory, located close to Germany's border with Poland, was one example of how "production is being ramped up", Scholz said.
Under the agreement, KNDS will complete the takeover of the factory from its current owner, the French rail manufacturer Alstom, in 2027.
The defence group will retain around half of the plant's 700 employees and offer places at different production sites to a handful of others.
The factory has manufactured rolling stock for over 175 years, dating back to the time of the German Kaisers, and currently produces double-decker train carriages.
With KNDS in control, it will turn out parts for the Leopard 2 main battle tank, the Puma infantry fighting vehicle and the armoured Boxer unit.
As the defence industry has boomed, Germany has seen a growing trend of converting production from civilian to military purposes.
German defence group Rheinmetall signed a deal last year to hire 100 employees from a plant set to be closed by auto supplier Continental.
German media have also reported that defence groups had eyed up some Volkswagen factories, as the crisis-hit carmaker looks to scale back production.
Comments