The 50-year-old officer was shot on a rural country road in the Groß-Gerau region, south-west of Frankfurt am Main, on Wednesday evening.
He died of his injuries on Ulmenstraße in Bischofsheim, police in Darmstadt said in a statement released in the early hours of Thursday.
A large scale manhunt is now underway for the killer, who was able to flee the crime scene.
The suspect was last seen heading towards the town centre after the shooting at around 7.30pm.
Police said on Thursday morning they had no further leads as to the fugitive's whereabouts, but were searching for a man aged between 50 and 60.
An e-fit of the man has been released based on eyewitness accounts. He was wearing a grey baseball hat and a light coloured top.
His hair was described as "greying" and he had a "three-day-old" beard.
The dead policeman, who had worked the southern Hesse region, had lived not far from the scene of the crime and had been off duty on Wednesday, a Darmstadt public prosecutor spokeswoman told the Frankfurter Rundschau.
There was no indication yet of a clear motive for the crime and investigators were looking into the victim's private and professional background for possible leads, she said.
Since releasing the e-fit, the police had received several tip-offs from the public, but had not yet got onto the man's trail. Police said he could still be armed.
"We're appalled by the murder of our colleague in southern Hesse and are stunned by this cruel, cowardly and senseless act," said Heini Schmidt, head of the regional chapter of the DPolG police union.
"We hope that the brutal murderer will be caught very soon and given his just punishment, even though this can be of no comfort and will not bring our colleague back."