The building was considered too dangerous for fire fighters to enter, so a digger was used to tear off parts of the roof and pull down walls to create access, Der Spiegel magazine reported on Thursday.
More than 220 fire trucks were used to blast 11,000 litres of water per minute into the building in the Niehl area of Cologne, while fire fighters used machinery to haul out the burning rubbish so it could be hosed down outside the warehouse.
Around a thousand tonnes of waste had been stored in the facility.
While the blaze was brought under control by about 8 am, authorities have voiced concerned about the cloud of smoke drifting from the south east of Cologne towards the centre, over built-up residential areas.
So far, no toxic substances have been detected in the air but eight vehicles kitted out with measuring devices have been dispatched around the city to monitor for toxins.
People in the Nippes, Mülheim and Kalk areas in particular were advised to keep their doors and windows shut and to block off any ventilation shafts until told otherwise.
Police believe that the fire started inside a shredding device, Der Spiegel reported. One theory is that there was a flammable liquid in a delivery of rubbish that, when put through the machine, caught on fire.
No-one was injured but a police spokesman said that the blaze had caused around millions of euros of damage.
A private company is working to contain and dispose of of the tens of thousands of litres of contaminated water now draining from the building.