Accompanied by Germany's Foreign Minister Heiko Maas, he left flowers outside the kebab shop, adding to a sea of bouquets and candles there, and exchanged a few words with the owner.
Deploring the scourge of anti-Semitism, Pompeo also visited the synagogue where he met with members of the Jewish community.
The suspect in the attack, 27-year-old German Stephan Balliet, had sought to storm the synagogue in the eastern German city of Halle on Yom Kippur, one of the holiest days on the Jewish calendar.
When he failed, he shot dead a female passerby outside the synagogue, and later gunned down a man in the kebab shop a short distance away.
Police subsequently captured the Halle suspect after a gun battle that left him wounded.
America's top diplomat is on a two-day tour of Germany ahead of the 30th anniversary of the fall of the Berlin Wall.
On Thursday, he was in Leipzig meeting with demonstrators from the 1989 peaceful revolution that brought the communist regime down, and will on Friday give a speech in the German capital.