"I think we have to work on that," Zuckerberg told Merkel during their meeting on Saturday on the fringe of the United Nations Sustainable Development Summit in New York.
Facebook has been much criticized in Germany for not doing more to shut down hate speech on its network during the refugee crisis, with Justice Minister Heiko Maas a leading voice calling for better moderation on the site.
At a meeting with Maas earlier in September, when the minister demanded faster procedures to get hate comments removed, Facebook agreed to set up a working group to look at the problem.
Beyond the government, other leading voices have been calling not just for social networks to improve their moderation, but for ordinary people to fight back against hate speech.
In August, TV news anchor Anja Reschke made headlines with a direct call to viewers to argue against those spreading racial hatred against refugees online.
Earlier this month broadcaster SWR announced that it had reported one Facebook user to police after he left threatening comments on stories about refugees.