The monthly business climate index calculated by Munich-based economic research institute Ifo fell to 97.5 points from 101.3 points in June. The last time it was below that level was in September 2005.
Analysts polled by Dow Jones Newswires had forecast a more modest drop in July to 100.2 points.
The index had already fallen sharply in June, and Ifo president Hans-Werner Sinn said the firms questioned "are much more dissatisfied with their current business situation and they are clearly more reserved regarding the six-month outlook.
"These results suggest that the economic upswing is coming to an end," he said.
Ifo's current situation sub-index dropped to 105.7 points from 108.3 in June, while the expectations indicator fell to 90.0 points from 94.6 points. Broken down further, the manufacturing business climate index "has fallen noticeably," the Ifo statement said.
"Export business will no longer expand so rapidly, in the estimation of the manufacturers" and they were les likely to take on additional workers, Ifo found.
Germany is the world's leading exporter and the latest findings are bad news because a hoped for pick-up in domestic consumption has failed to materialize owing to high inflation brought on by surging energy and food prices.
In other key sectors covered by the Ifo index - construction, retailing and wholesaling - "the business climate index is clearly trending downwards," the statement added. "The trading firms have assessed both their current business situation as well as the six-month outlook clearly more pessimistically than in June."
Ifo surveys roughly 7,000 German companies each month, and its index is considered the most reliable indicator of business sentiment. Although the index remained above its long-term average, "the magnitude of the recent falls is clearly a worry," said Capital Economics economist Ben May.
Taken together with a recent fall in the euro zone purchasing manager's index (PMI), he said available data "suggest that there is a chance that GDP (gross domestic product) contracted in the second quarter and ... there might even be another fall in the third quarter."
That would signal a recession in the 15-nation euro-zone economy.