The 41-year-old man had been convicted in 2008 of planning an attack in Berlin against former Iraqi prime minister Iyad Allawi, a prosecution spokesman told AFP.
Thursday's incident began when four police cars were called to the western Berlin district of Spandau because the man was reported acting aggressively and threatening passers-by, police said.
When a policewoman approached him, he stabbed her with a knife with a nine-centimetre (3.5 inch) blade in the neck area, before one of her colleagues opened fire, killing the Iraqi man and "suspected Islamist", prosecutors said.
The 44-year-old woman was also hit accidentally by one of the bullets fired by her police colleague, said the prosecution service.
The Iraqi man had in the morning removed an electronic ankle monitor he had been ordered to wear after being released from prison.
National news agency DPA quoted chief prosecutor Dirk Feuerberg as saying it was too early to speculate about a "terrorist motive", and that the man's apartment was being searched.
Berlin police said on Twitter about the policewoman, who had been taken to hospital by helicopter, that "the condition of our colleague is stable, she remains in intensive care".
The attacker died in an ambulance shortly after being shot, despite attempts to revive him.
Prosecution service spokesman Martin Steltner identified the Iraqi man as "Rafik Y.", saying he was sentenced in 2008 to an eight-year prison term for his role in a plot against Allawi.
In the court case in the southwestern city of Stuttgart, Rafik Mohamad Yousef was one of three Iraqi men sentenced to jail terms, including time already spent behind bars during their trials.
News site Spiegel Online reported he had returned to Berlin in 2013, and was kept under surveillance.
Die Welt daily wrote that the convict could not be deported to Iraq under German law because he would face the death penalty there.
The three men had been convicted of belonging to a foreign terrorist organisation - Iraqi militant group Ansar al-Islam - and attempted conspiracy to commit murder.
Ansar al-Islam, a predominantly Kurdish group, was believed to have links to Al-Qaeda in Iraq.
Yousef was believed to have volunteered to carry out the attack on Allawi, the court heard.
Presiding judge Christine Rebsam-Bender described Yousef as "hot-tempered and aggressive" and cited his frequent outbursts, including an attack on a prison guard that broke the officer's rib.
"Because they are Nazis!" Yousef shouted at the judge.
Intelligence services at the time estimated the group had about 100 members in Germany connected to a wider network of supporters across western Europe.
The court found that the plot to assassinate Allawi had been hatched only days before the premier's brief trip to Berlin in December 2004.