Just 66,000 new flats and houses were built in large cities with more than 100,000 inhabitants in 2014 out of a total of 245,000 new builds, the economists found.
But that was 50 percent short of the actual number that were needed, which the IW said was around 102,000.
Berlin alone will need 20,000 new homes every year from now until 2020 to keep up with demand, although just 8,744 were completed in 2014.
Meanwhile, in less-populated areas like the Eifel mountains, the Black Forest and large parts of former East Germany, too many homes were built.
That's a symptom of local governments in rural areas trying to stimulate business and attract more people by selling off cut-price land for construction, the economists said.
More choose to live in cities
Populations in Germany's big cities are swelling, with ever-larger numbers of students, immigrants and "young pensioners" choosing urban life over peace and quiet in the countryside or smaller towns.
"Favourite cities have to relax the regulations affecting how tall buildings can be," IW housing expert Michael Voigtländer suggested in a statement on Wednesday.
He added that unused plots should be built up to make the most of the available space.
Another policy option was improving the attractiveness of areas with an over-supply of homes that are close enough for commuters to reach crowded big cities.
Voigtländer pointed to the example of the neighbouring North Rhine-Westphalia towns of Düsseldorf, where there aren't enough homes, and Wuppertal, where flats and houses are standing empty.
"Politicians have to invest more in infrastructure in towns so as to attract commuters," he said.
'Wrong types of homes being built'
Berlin Tenants' Association (Berliner Mieterverein) manager Reiner Wild argues that the biggest housing problem - at least in the capital - is that the rent on many new-build apartments is too high.
"What's really lacking are affordable rental homes," Wild told The Local.
While there were very few homes being built, compared with the roughly 100,000 people who had moved to the capital in the past three years, prices were the decisive factor, he believes.
"The homes being built right now are unaffordable for most people already in Berlin and for the people who are moving here," Wild said.
"It's a mistake in the supply."
In some districts of the capital, there were also three or four times as many homes being offered for sale as for rent, Wild said - massively limiting prospective tenants' options.
He added that Voigtländer's suggestion of removing limits on high buildings would be wrong for Berlin, too.
"We think that steps like removing planning restrictions don't work. Property developers always try and get the rules changed at times like this, but we don't think it's the right answer.
"We have to think of the future of the city as well."