"My visit has convinced me that much remains to be done," Githu Muigai, UN special rapporteur for racism, said after a 10-day fact-finding trip across the country.
People with a migrant background "are under-represented in important institutions, including the political system, the police and the courts," Muigai said.
Last November, the Greens became the first major party to elect a Turkish-German as leader, Cem Ozdemir, but ethnic minorities are generally a rarity in German politics, making up fewer than 15 national MPs.
Muigai, who was paying a routine visit to Germany, also said that due to the historical experience of Germany with Nazism, there is a "tendency to equate racism with extremist politics."
"I think that this great country needs to turn its attention to the problems of daily racism and discrimination, that is the racism and discrimination to be found in daily life in schools, public places, in housing, in employment etcetera," he said.
He added however that he was "encouraged by the open recognition by authorities that Germany is nowadays a country of migrants and that migrants make a positive contribution to society."
Kenyan-born Muigai, special rapporteur since August, also said that far-right groups in Germany "demand constant vigilance" and should be banned "within the constitutional legal mechanism provided by German law."
There are regular calls for a ban on Germany's biggest far-right party, the NPD, which has no seats in the Bundestag national parliament but has representatives in two of Germany's powerful regional assemblies.
The previous centre-left government had sought to ban it, but the case was thrown out in 2003 by the Constitutional Court.
Germany also needs to do more to speed up the processing of asylum-seekers, Muigai said.
Muigai said that his full report, due to be presented at the Human Rights Council in 2010, will also focus on the plight of Muslims in Germany following the increase in security measures aimed at tackling Islamic extremism.
"The Muslim community does have very special concerns, particularly in the aftermath of 9/11 and in the manner in which the security apparatus have responded to ... the challenge of the fight against terrorism," he said.
A government study published in June found that Germany had between 3.8 and 4.3 million Muslims - between 4.6 and 5.2 percent of the population – much higher than the previous estimates of 3.1-3.4 million.