The monthly Ifo business climate index plunged to 108.7 points from 112.9 points in July - the measure's lowest level since June 2010 - underscoring a sharp slowdown seen also in eurozone consumer confidence.
German business expectations for the coming six months fell as well, a development noted with particular concern by many analysts.
The latest Ifo poll was well below expectations of a decline to 111 points and was the biggest drop since the index lost 4.9 points in November 2008.
"Recent financial market turmoil and the never-ending eurozone sovereign debt crisis have befogged the outlook for the German economy," ING senior economist Carsten Brzeski commented.
A sub-index that measures companies' expectations for the next six months dropped to 100.1 points from 105. A figure below 100 indicates a downturn.
It was the expectations' lowest level since October 2009 and was singled out for concern because it "has the best track record of anticipating" growth, Markit chief economist Chris Williamson said.
UniCredit counterpart Andreas Rees noted that "it was the sixth consecutive decline in expectations, i.e. there can absolutely be no denying that this is a downward trend."
An Ifo statement quoted president Hans-Werner Sinn as saying: "The current business situation, however, continues to be assessed overall as good, although the situation appraisals in recent months were significantly more favourable."
The sub-index of companies' assessment of their current situation fell to 118.1 points from 121.4 points in July.
The survey of around 7,000 firms showed falls in all categories - manufacturing, retailing and construction.
In retailing, "the optimism of the distributing firms regarding the six-month business outlook has largely dissipated," Ifo said, backing up an EU Commission survey of consumer confidence that was released on Tuesday.
That poll's fall of 5.4 points to minus 16.6 points was a sharper drop than one seen in October 2008, after US investment bank Lehman Brothers collapsed, setting off the global financial crisis and recession.
"The Ifo survey data, therefore, add to a growing picture of rapidly deteriorating business and household confidence in Germany," Williamson said.
"Worries have intensified in recent weeks as economic data have shown growth faltering, financial markets have slumped and policymakers appear unable to resolve the region's financial crisis," he added.
Meanwhile, eurozone industrial orders also surprised economists on the downside, showing a decline of 0.7 percent in June instead of an anticipated gain of 0.5 percent, EU data showed.
On Tuesday, Markit's purchasing managers index for eurozone private sector activity showed it stagnating in August while the ZEW index of German investor confidence slumped for the sixth month running to a level last seen in December 2008.
Commerzbank chief economist Joerg Kraemer said the debt crisis would decide if Germany faced only a soft patch or a more serious downturn.
"If it escalates and Italy and Spain are cut off from the capital markets and default, this would likely trigger a global uncertainty shock similar to after the Lehman bankruptcy. The economy would then enter a serious recession," he warned.
Berenberg Bank senior economist Christian Schulz said that "further hits to confidence cannot be excluded and the risk of a recession remains high."
Economists agreed that the European Central Bank would refrain from raising its main interest rate again this year, having hiked it twice by 0.25 percentage points to 1.50 percent.