The police could not say whether underground works in the area were connected to the collapse, but witnesses said they saw the archive and two surrounding buildings slowly slide into the tunnel after an earthquake-like tremor, news agency DDP reported.
"The buildings collapsed one after another along a 50-metre stretch," Günter Heimann, a motorist who witnessed the accident, told the Kölner Stadt-Anzeiger newspaper.
He said several people were pulled into the depths below: "I felt tremors. Then the navigation computer fell from my windshield. I saw how the first house fell down in my rear-view mirror. Then there was just a dust cloud. I pulled over and ran back to help."
Retiree Peter Steinkamp told DDP he had been walking along the street on the Waidmarkt in the Südstadt district when there was an "enormous bang" that sent clouds of dust several stories in the air.
Police in the North Rhine-Westphalia city evacuated neighbouring buildings and closed the street for search and rescue operations. The Cologne fire department sent all available units to the scene.
At least three people were still missing on Tuesday evening after the structures collapsed. Rescue workers had originally been looking for nine people with search dogs, but officials later reduced the number thought to be buried under the rubble. The authorities said the chances of anyone surviving overnight are considered slim.
Fire department director Stephan Neuhoff told DDP that sounds within the archive alarmed those inside and they were able to get out before the building collapsed.
But Eberhard Illner, a long-serving archive official, made serious accusations that there had been plenty of warnings of an impending collapse. He said he reported finding cracks in the cellar or the archive last summer and there was evidence the building was sinking as recent as last week.
"There will have to be an investigation into who's responsible for this," Illner told DDP.
Meanwhile authorities attempted to secure a historic Catholic church tower near the collapse that was also in danger of crumbling, news agency DPA reported.
The historical archive was among the largest in the country holding 65,000 priceless documents, thousands of maps and half a million photos about Cologne's history. The oldest document dates back to 922. The structure was built in 1971.