The high administrative court in Münster ruled against the state law, which declared that women with “essentially the same skills, qualifications and professional accomplishments” should be favoured for promotion in public jobs.
The law, in effect since last July for civil servants in the western state, had been defended by the state government before several other lower courts.
According to the Rheinische Post, the law had in practice been interpreted so that, in some cases, women with lesser qualifications than their male co-workers were promoted.
The Münster court ruled that this law violated the German constitutional concept of choosing the best candidate. This principle means that the person selected for a position should simply be the person who is best qualified for the job.
Several men had filed urgent motions to courts around the state to stop promotions because they felt they had a disadvantage.
The latest court ruling pointed out that equal treatment for women can be promoted while also observing the best selection principle.
It noted that certain gender disadvantages, such as women more often feeling torn between family and work life, can be balanced out if employers put more emphasis on other qualities for promotion - such as life experience, character, aptitude, and general knowledge.
"Promoting women only on sex lacks a legal basis, while neglecting these aspects," the court stated.
But the state of North Rhine-Westphalia plans to appeal the decision.
“It is important for all female and male public employees that the law's constitutionality - of which we are convinced - be quickly clarified,” said state interior minister Ralf Jäger in a statement.
The state's equality minister, Barbara Steffens, also defended the law.
“Each step towards more equality has thus far had to be fought for rigorously,” Steffens said.
“As long as women are still underrepresented in higher positions, we need an effective quota system in the public sector.”
The German federal government has passed laws in recent years to help shrink both the gender achievement gap and the pay gap for women. Chancellor Angela Merkel's cabinet approved a wage transparency law proposal earlier this year that would force companies to disclose salaries paid to male and female employees.
Germany also implemented a so-called "women's quota" last year for larger businesses to help get more women into higher positions. But a recent report found that the country's boardrooms are still largely male-dominated, even with the quota system calling for supervisory boards to have 30 percent women.