The pair, who were working with Nigeria's National Commission for Museum and Monuments, were seized at about 8:55 am as they worked at an excavation site in southern Kaduna on Wednesday.
Federal police chief Ibrahim K. Idris said the men had been assigned security personnel for their own protection but that they did not go with them to the dig site into ancient Nok culture.
Surveillance helicopters and anti-kidnapping and counter-terrorism officers have been deployed in the hunt for the perpetrators, he added.
The foreign ministry in Berlin made no comment on the kidnapping but residents said two women, also thought to be German, were with the men when they were attacked.
"The abductors came wielding guns and machetes and asked the two Germans to follow them into the bush," said a resident who asked not to be identified because of security fears.
"Surprisingly, they did not kidnap the two female German colleagues of the men. They fired shots into the air to scare off the villagers before abducting the Germans," he added.
Police said two locals were killed.
The kidnapping took place off the main road linking Kaduna airport, to the north of the state capital, Kaduna city, with the national capital, Abuja, 220 kilometres away.
Safety on the road has come under intense scrutiny in recent weeks since the federal government announced the closure of Abuja's only airport for essential runway repairs.
All domestic flights will be diverted to Kaduna from March 8th for six weeks but international carriers have refused to re-route their services, reportedly because of fears about capacity.
In July last year, Sierra Leone's defence attache to Nigeria was kidnapped by armed men with AK-47 assault rifles in military fatigues at a fake check-point on the Abuja-Kaduna road.
Nigeria's government has promised to increase security on the route but many foreign missions and companies have advised staff to limit their travel during the closure period.
Extra checkpoints could add several hours onto journeys following a renewal of long-standing violence between herdsmen and farmers in southern Kaduna since December last year.
Some estimates say the clashes between the Muslim, largely Hausa-speaking Fulani cattle drivers and mainly Christian farmers have left as many as 800 dead. But the official toll is about 200.
Kidnapping for ransom was previously almost exclusively seen in the oil-rich south, where criminal gangs singled out the wealthy and expatriate workers in targeted or opportunistic attacks.
But it has gradually spread northwards as Nigeria's oil-dependent economy imploded, to the extent that the Control Risks consultancy said it was "entrenched" countrywide.
In the first half of 2016, Nigeria was the fourth riskiest place for kidnap in the world, the risk management firm said in September last year.
In the north, a government crack-down on cattle rustling has been blamed for a rise in abductions.
Nnamdi Obasi, senior Nigeria researcher at the International Crisis Group, said poverty had made "anybody, just anybody" a target and had led to a rise in "spontaneous" kidnappings.
But so-called "express kidnappings", while harder to stop, tended to be less violent and most victims were released without harm, he said.
Abductions were hurting foreign investment but also Nigerians, said Obasi.
"They don't want to go to some places... they think twice before travelling by road and sometimes don't have the money to travel by plane. It's very harmful for the local economy."
Security consultant Don Okerere said kidnapping was under-reported.
"For every 10 reported incidents of kidnapping, there are probably six or more that aren't reported," he said. "A lot of victims will rather 'play ball' than involve the police."