Police spokesman Michael Maass said the bomb was found during building work on a bridge over the Spree River.
It was discovered in the river, next to the Oberbaum Bridge, and would be dragged onto dry ground to be defused, he added.
Underground and commuter train services have been disrupted in the area. Delays and cancellations were expected to last at least until the evening.
World War II-era bombs are found frequently in Germany.
In April 2009, Chancellor Angela Merkel had to leave her apartment in the city centre due to the discovery of an unexploded Soviet bomb.
Authorities believe there are still some 3,000 bombs buried beneath the capital alone.