According to German media, the stolen coin is the "Big Maple Leaf", a commemorative piece issued by the Royal Canadian Mint in 2007.
The coin, 53 cm across and three cm thick, features the portrait of Queen Elizabeth II.
Bode Museum gave the face value of the coin at €920,000, though the market price of 100 kg of gold is around €3.7 million.
German police said on Twitter that the robbers likely used a ladder found at a nearby rail track to break into the museum at around 3:30 am.
Suburban rail traffic was interrupted as investigators combed the area for clues.
The Bode Museum, located on the German capital's UNESCO-listed Museum Island, houses one of the world's biggest coin collections.
The holding includes 102,000 coins from ancient Greece and about 50,000 Roman coins.