Speaking to the Rheinische Post, the conservative Bavarian politician said that it is in Germany's interest to reach an agreement with Turkey, but that "we must not concede too much to them."
Herrmann noted that when Germany did away with visa requirements for Balkan states in 2009, the number of asylum applications from these countries "exploded".
Turkey is pushing for a deal with the EU which would impose tighter controls along its sea border to prevent refugees entering Greece, in exchange for a relaxation of visa restrictions for Turkish citizens seeking to enter the 28-member union.
"We shouldn't bring a new refugee wave into the country by agreeing to abolish visa restrictions for all Turkish citizens including Kurds," Herrmann said.
Turkey is currently engaged in a violent conflict with the separatist Kurdish Workers Party (PKK) in the southeast of the country.
The violence is the worst since 1990, with Turkish air force helicopters striking PKK positions within its own borders as well as northern Iraq, Reuters reports.
The mooted deal with Turkey has also been criticized from the left of German politics.
Die Linke leader Sahra Wagenknecht said on Wednesday that "human trafficking should not provide an entry card for a dictator into the EU or visa exemption."
Merkel-Erdogan-Deal stoppen: Menschenhandel darf keine Eintrittskarte für eine Diktatur in EU oder Visafreiheit sein https://t.co/0g2y1zIkcc- Sahra Wagenknecht (@SWagenknecht) March 9, 2016
Turkish president Recep Tayyip Erdogan has been severely criticized by rights groups for his autocratic style of governance.
Earlier in March, Human Rights Watch criticized Erdogan for a sustained campaign to "silence critical media” after a court appointed government-controlled trustees to run the mass-circulation and critical Zaman newspaper.
Merkel 'underestimated' refugee crisis: Denmark
Danish foreign minister Kristian Jensen complained on Wednesday that German Chancellor Angela Merkel had underestimated how many refugees would flood into her country under a former open door policy.
Denmark and Sweden have temporarily reintroduced border checks in an attempt to control the migrant flow, but Jensen said much of the problem stemmed from poor planning in their larger southern neighbor.
"I think that Merkel has been surprised by the number of refugees coming to Germany. She thought this was a crisis that would fade out," Jensen told guests at a US think tank during a visit to Washington.
"She has been accused actually of being the one sending out invitations to refugees around the world, 'Please come to Germany, you are welcome here, we have a house and a job and it's a great place to be'," he said.
Merkel is under intense domestic political pressure to put a cap on refugee arrivals after more than a million - many fleeing war in Syria and Afghanistan - arrived in Germany last year alone.
Instead, she has taken the lead in negotiating a deal with Turkey to send back migrants arriving on the shores on Greece in exchange for EU help in dealing with the 2.7 million refugees on Turkish soil.
"So after being too late too little she is now taking on responsibility, and I think she is doing a tremendously good job," Jensen said, complaining that other EU members were being kept in the dark.
"Sometimes she perhaps should remember to inform and listen because even given how strong she can be and how large and strong Germany is, they can't decide for the rest of the 27."
Jensen also met with his US counterpart Secretary of State John Kerry, who assured him that Washington does not regard the refugee crisis as Europe's problem alone.