Gauck, who is the first head of state from the former communist east, delivered the impassioned appeal for more European "solidarity" to members of both houses of parliament, in the sunlight-filled main chamber of the Reichstag building.
"We must keep saying yes to Europe," he said. "Particularly in times of crisis, we must dare to have more Europe."
Languishing under the Nazi and then the communist dictatorships had fuelled his dream a united Europe living in peace and prosperity, the 72-year-old said.
"Europe was the promised land for my generation."
"For my grandchildren, Europe has long been part of their day-to-day reality, with freedom across borders and the opportunities and concerns of an open society. This reality is an achievement, not only for my grandchildren."
Germany, as the rescue-fund paymaster for Europe's stricken member states, has come under pressure throughout the sovereign debt crisis to boost its contributions to the bailouts in the face of Chancellor Angela Merkel's resistance.
The turmoil has also exposed old rifts, with protesters on the streets of Athens branding Merkel a "Nazi" for her hard-line stance.
Gauck, a Lutheran pastor and former East German democracy activist, said he would make the integration of Germany's large immigrant population and the struggle against the far right priorities of his presidency.
In a rousing passage of the 23-minute inaugural speech, Gauck said Germany would stand united in the fight against extremists.
"Your hatred will only spur us on," he thundered in reference to a group of neo-Nazi militants thought to have killed 10 people – mostly shopkeepers of Turkish origin - over a seven-year period.
The murder spree has sparked a new drive to ban the anti-immigrant National Democratic Party of Germany, which had ties to the suspected murderers.
Story continues below…
Gauck will make his first trip abroad as president to Poland next week.