The 36-year-old nurse from Bochum brought the case against her employer, a hospital sponsored by the Evangelical Church.
She requested to wear the headscarf, which she had not done previously, on her return after a long period of maternity leave and illness.
But the hospital refused to allow her to return to work wearing the headscarf and did not pay her any compensation.
According to her lawyer, the hospital discriminated against her on religious grounds, as it did not have the authority to prevent her from wearing the religious symbol at work.
Lawyers for the hospital argued that it should be allowed to remain true to its religious principles and employees should respect that.
The court decided that workers in Church-funded institutions should remain at least religiously neutral in their behaviour, a court spokeswoman told reporters.
Judges found that the headscarf, as a "symbol of belonging to the Islamic faith" was not compatible with working in a Church institution, she said.
However, exceptions might be made in certain cases, for example in jobs where employees would have little contact with the public.
The nurse had previously been awarded compensation in her first case before a local labour court, but the decision was overturned at appeal by the state court.
In comments before the hearing, the nurse's lawyer said that she would consider appealing to the Constitutional Court if the case went against her.
This is the first time the federal court in Erfurt had heard a case involving a headscarf wearer in dispute with a religious employer.
A 2009 study by the Federal Migration Office (BAMF) found that around 25 percent of Muslim women over 16 wear the veil regularly, while 70 percent never wear it at all.