A survey by public broadcaster ZDF found that 60 percent of respondents believe Germany cannot cope with the large numbers of new arrivals, which reached 1.1 million in 2015.
The Cologne attacks clearly had an impact, the broadcaster said, as only 46 percent of people surveyed in December felt that way.
The poll of 1,203 people over January 12 to 14 also found that a majority (56 percent) are now dissatisfied with German Chancellor Angela Merkel's refugee policies, up from 49 percent in December.
Seven people in 10 fear the influx will lead to more crime, compared to 62 percent in October.
And a growing minority, now at 42 percent from 33 percent in October, say they fear their cultural values are under threat.
A separate survey by the group DeutschlandTrend for the state TV ARD found that 51 percent of German adults said they do not believe Merkel's repeated claim - "we will manage" - that Germany can absorb the inflow.
In October, 48 percent of respondents said they had this view.
The poll also found that 48 percent of those surveyed said they were afraid of refugees, while 50 percent said they did not have this view. That question was not asked in October.
The survey covered 1,000 people who were interviewed by phone on January 12 and 13.
Hundreds of women were groped and robbed in a throng of mostly Arab and North African men outside the main railway station of Cologne.
The tally of criminal complaints reached 652 by Thursday, including 331 sex-related crimes, Cologne prosecutors said.
The case has inflamed tensions in Germany, which took in nearly 1.1 million asylum seekers in 2015, mainly from Syria, Afghanistan and Iraq, and put pressure on Merkel for her welcoming stance toward refugees fleeing war.