Russia's annexation of Crimea in March "called the whole of the European peaceful order into question, and it has continued by Russia exporting its influence to destabilize eastern Ukraine," Merkel said after attending the G20 summit in Brisbane.
Western leaders, including British Prime Minister David Cameron and US President Barack Obama, hammered home their determination to rein in Moscow at the talks, where Russian President Vladimir Putin faced a hail of criticism.
Seven months of fighting in eastern Ukraine has claimed the lives of more than 4,100 people, according to UN figures. The West accuses Russia of stoking the crisis by supporting separatist rebels, an accusation Moscow denies.
"Who would've thought that 25 years after the fall of the Wall, after the end of the Cold War, after the end of the division of Europe and the end of the world being divided in two, something like that can happen right at heart of Europe?" Merkel said in a speech in Sydney.
"Old thinking, thinking in terms of theories of influence, where international law is violated, this must not be allowed to prevail. I'm convinced it will not prevail."
Merkel - who held lengthy talks with Putin on the G20 sidelines - said the West was working hard towards a diplomatic solution to the conflict, but was also committed to economic sanctions against Moscow.
The European Union, United States and Australia have slapped the toughest sanctions on Russia since the collapse of the Soviet Union over what they say is its meddling in Ukraine.
Merkel, who grew up in communist East Germany, said the West could draw lessons from history in how to approach discussions with Russia.
"We need to have the necessary patience for an uphill battle," she said. "My personal experience from the history of the German Democratic Republic is that one should not lose hope too quickly.
"The biggest danger is that we allow ourselves to be separate, to be divided, that a wedge will be driven between us."
Merkel was set to lay a wreath later Monday at the Anzac Memorial in Sydney, which commemorates Australians who served and died in conflict zones.