Rasmussen told a press conference in Copenhagen that his move was "a reaction to a decision made in Sweden", which on Monday brought in similar controls on all forms of transport arriving from Denmark.
The new rules took effect at midday and will initially last for 10 days. After that, the controls can be extended by 20 days at a time depending on the situation, Rasmussen said.
"I regret that Denmark has decided to introduce temporary border controls," Torsten Albig, minister-president of Schleswig-Holstein, Germany's northernmost state, said in a statement.
"This could negatively affect the good communal life in the German-Danish border region and especially be a burden on commuters."
But Albig saw a "ray of hope" in the fact that the controls were only random checks, not comprehensive border controls, and that the Danish government was not insisting that transport personnel carry them out.
"The state government [of Schleswig-Holstein] has called on the Danish government to only implement this measure for as short a time as possible, to minimize the damage for social and economic life in the border area," Albig concluded.
Meanwhile, the federal government warned that the passport-free Schengen zone was "in danger" after the Nordic countries announced their new controls.
"Freedom of movement is an important principle -- one of the biggest achievements (in the European Union) in recent years," foreign ministry spokesman Martin Schaefer told reporters.
"Schengen is very important but it is in danger,"
Chancellor Angela Merkel's spokesman, Steffen Seibert, said that the Schengen accord, which provides for borderless travel across most of the 28-nation EU, was dependent on better protection of the EU's frontiers.
"We need a common strategy," he said.
Seibert said that Danish Prime Minister Lars Lokke Rasmussen had informed Merkel of his decision before making the announcement.
Story continues below…
Denmark's move could see an increased number of refugees backed up in cities in northern Germany, where many have passed through in recent months on their way to northerly neighbour countries.
Schleswig-Holstein has taken in around 50,000 of the more than one million refugees who arrived in 2015.
In November, Sweden asked those on their way to the Nordic nation to stay in Germany, saying that it was running out of beds to accommodate new arrivals.