The perpetrator, a tax advisor in his mid-50s who was already in dispute with the tax authorities, entered the office at around 10am and asked to see the manager.
In mid-conversation with the worker the man took out his gun and shot his victim, who later died in hospital due to his injury. How many shots were fired is still not known.
The tax office was locked and forensic scientists and investigators have begun working in the building.
The shooter is still in detention but was examined in hospital for a light injury.
"I am shocked that violence has burst into the life of a peaceful person," First Minister of Schleswig-Holstein Torsten Albig said.
"My thoughts are with his family."
Schleswig-Holstein Finance Minister Monika Heinold visited the scene in Rendsburg to find out what happened.
"We are all very affected in the state government," a visibly upset Heinold told journalists.
There have been several cases of violence against government workers in Germany in recent years, although only occasionally have people been killed.
In April 2013 a pensioner shot the district administrator of Hameln-Pyrmont in Lower Saxony before shooting himself after years of judicial disputes.
Stabbings occurred in job centres in Neuss in the Lower Rhine and Frankfurt in 2011 and 2012.