The large bomb, believed to be dropped by the US in World War II, was discovered on Monday under the Mülheim bridge in the North of the city by the RheinEnergie construction company, city authorities in Cologne reported.
The city has organized a large-scale evacuation plan in advance of the bomb's controlled detonation on Tuesday.
Residents of the Mülheim, Niehl and Riehl neighbourhoods were instructed to leave their homes by 9am on Tuesday. They have been told to head to shelters arranged by the city where drinks, snacks and information about the bomb disposal will be available.
Approximately 1,300 people live in city social housing in the affected area, many of whom require nursing care.
Beginning at 6am, those in need of assistance were transported to a nearby hospital in fire trucks and ambulances, newspaper Köllner Stadt Anzeiger reported.
City officials hope to have the area secured in the afternoon, after which the bomb can then be detonated.
The city plans to keep the heavily-trafficked Mülheim bridge open as long as possible in advance of the detonation.