The 28-year-old perpetrator inflicted multiple stab wounds on the Jobcenter worker, who died on the scene despite first responders' efforts to resuscitate him.
Investigators have not yet determined the motive for the crime.
The 28-year-old had an appointment at the Jobcenter with the victim at noon on Wednesday. He drew the knife suddenly and attacked before other workers in the office were able to drag him off and restrain him.
But his victim was already lying on the ground and died shortly after police arrived on the scene.
The victim was not an employee of the Jobcenter, but an expert who was sometimes called to assist in special cases in the district.
"The Jobcenter office of Rothenburg remains closed until further notice," a note on the door read on Thursday.
"Whether there was an established relationship between the perpetrator and the victim, as well as when the offender made the decision to act - before or after or during the appointment - has yet to be determined by investigators," said Hermann Lennert, detective chief in nearby Ansbach.
The body has been transferred to a forensics lab in Nuremburg to undergo autopsy on Thursday.
Rothenburg is a city in shock after the crime.
"It's incredibly sad that something like this can happen in a small city," said a 17-year-old who works in Rothenburg.
"People can't even feel safe here," said an elderly resident.
This is the third murder in a German Jobcenter in recent years.
Federal Labour Minister Andrea Nahles said that everything was being done to protect state employees.
"They should not be the target of aggressive attacks," she said.
SEE ALSO: Man jailed for job centre hammer attack