Excavation work at the new Pempelforter Strasse underground station was stopped immediately while the 1.20-metre-long tusk was gently removed and taken away for scientific study.
A search was conducted to see if the rest of the skeleton was somewhere nearby, but only the tusk could be found.
The theory is that the tusk was probably carried to its final resting spot by melt waters from a glacier thousands of years ago when the Rhineland area was still tundra.
DNA analysis should help determine how old the mammoth it came from was. The Ice Age creatures died out about 10,000 years ago.
Rolf Lommerzheim from Düsseldorf authorities said that workers had initially thought the tusk was an old piece of pipe. “But it quickly became clear that it was something special,” he told Die Welt newspaper.
The tusk will eventually be put on display in an underground station, when everything is finished, the paper said.