“We look forward to rescheduling this important set of meetings,” State Department spokeswoman Morgan Ortagus said.
“The Secretary looks forward to being in Berlin soon.”
She gave no further details or explanation for the sudden change of plans.
Pompeo had been due to meet German Chancellor Angela Merkel and Foreign Minister Heiko Maas later Tuesday.
It was not immediately clear whether he would still travel to Britain on Wednesday, when he was to meet with Prime Minister Theresa May, or on Thursday to Greenland, where he was to complete his European tour.
Pompeo started the trip attending a meeting of ministers of the Arctic Council in the northern Finnish city of Rovaniemi.
Merkel's office referred all queries about the reasons for the cancellation to the US embassy in Berlin, which could not immediately be reached for comment.
The embassy had earlier tweeted the Ortagus remarks.
Merkel has not hidden her differences with US President Donald Trump, who has repeatedly taken the unusual step of criticizing the close ally, especially over what he has repeatedly called German's insufficient NATO
Berlin and Washington are also at odds over a range of trade issues and the Nord Stream 2 project to build a new gas pipeline between Russia and Germany under the Baltic Sea.