The vote during the CDU party conference signals the party's intention to reverse a compromise that the CDU agreed to with its coalition government partner the Social Democrats (SPD) in 2014.
The CDU and SPD agreed to change the law as of December 2014 to allow immigrant children to keep both German citizenship and that of their parents.
The reform entails certain requirements for those who want to maintain their dual citizenship, such as showing proof that they had lived in Germany for eight years, or attended a German school for at least six years.
Previously children born to foreign parents from certain countries were allowed to have dual citizenship up until the age of 23, but then had to choose which nationality they would keep.
But Merkel herself criticized her party's vote to undo the reform.
“We should not go back on this law,” Merkel told broadcaster N-tv.
The law particularly affects the large Turkish minority in Germany, many of whom came over through a guest worker programme started in the 1960s. But it also applies to children of parents from many non-EU countries.
Before the reform was passed, exceptions were made for children of parents from other EU states, Switzerland and countries that do not allow people to renounce their citizenship, like Iran. But still an estimated 40,000 young people were having to choose between nationalities each year.
The CDU vote was close: 319 for and 300 against.
Interior Minister Thomas de Maizière said during the debate that he did not believe it was good to go back on an agreed compromise, adding that any party that the CDU will try and build a coalition with after the 2017 election will not accept a reversal of the legislation.
Fellow CDU member Jens Spahn applauded the vote and said that while the party must make compromises within a coalition, “we are here at a party conference”. He added that it is not unreasonable to ask young people to make a clear choice.