The man, identified only as Khalil G., lived in an apartment next to the archive, which is thought to have collapsed due to underground work on a nearby metro tunnel.
“The evidence points to it being the second missing person,” said a police spokeswoman late on Thursday, adding that confirmation would come after an autopsy.
Police and rescue workers had struggled to get to the 24-year-old design student amid the archive's debris. His body was found some 9.5 metres below street level by a search dog.
The other victim of the accident, a 17-year-old baker's apprentice, was pulled from the rubble on Sunday.
“The rescue teams have done everything humanly possible to find the other person who was missing as quickly as they can,” said Cologne Mayor Fritz Schramma. “The entire city now mourns the second victim that this horrible misfortune has taken.”