The man with the knife, a 53-year-old Austrian, was “yelling and causing damage” before being shot in the leg shortly after 4;00 pm, police said in a statement.
When the incident began, about 100 people were evacuated from the landmark Berlin Cathedral in the heart of the German capital.
When officers arrived they urged the man to put down the knife and tried to neutralise him with irritant gas, to no success, police said.
Finally, they faced the assailant near the altar of the cathedral, and “the man was wounded in the leg by a shot from a policeman,” a spokesman said.
The man was hospitalised and underwent surgery, the spokesman added.
A policeman was also slightly injured by a ricocheting bullet from another officer, but was able to leave the hospital in the evening.
Witnesses told local media that the man appeared “very confused”.
“Based on what we know so far, we have no information that the suspect in any way had a terrorist or Islamist motive,” a police spokesman said.
The iron-domed Protestant cathedral, one of the city's top tourist attractions, is on Museum Island off east Berlin's main Unter den Linden boulevard and close to the Alexanderplatz shopping district.
An AFP reporter said the entrances to the church were blocked off with red-and-white police tape and several officers with automatic weapons were patrolling the scene.
DPA news agency said some witnesses were taken away to receive psychological counselling.
The shooting comes with authorities in a state of high alert for jihadis attacks after several assaults claimed by the Islamic State group in the country.