The visit by Merkel, who grew up in the communist state, was part of this year's 20th anniversary commemorations marking the unexpected opening of the Berlin Wall on November 9, 1989 in a peaceful revolution.
Merkel said the Hohenschönhausen jail, named for the northeastern district where it is located, was a symbol of a "sad chapter in the history" of the Stalinist state.
"This drives home the brutality with which people's dignity was abused," she said, after meeting with former prisoners of the facility. "It is crucial that this chapter of the history of the East German dictatorship is not covered up or forgotten."
The Hohenschönhausen prison was located in a military zone of East Berlin to which civilians had no access.
Wiped from all city maps, the jail held thousands of political prisoners, as well as ordinary citizens who fell foul of the regime, in degrading conditions from 1951 until national unification in 1990.
The site, reopened as a memorial in 1994, draws a quarter of a million visitors each year, most guided through the facility by former prisoners.
Its director, Hubertus Knabe, thanked Merkel for her visit, calling it "an important gesture" as she was the first chancellor to make the trip.
Merkel toured the grim cells and a windowless torture cellar known to detainees as "the submarine" and laid a wreath of flowers in front of a memorial for "victims of the communist tyranny."
She also met with a group of pupils and underlined the importance of instruction in schools about East German history as today's graduates would have no personal memory of the country's four-decade-long division.