A 24-year-old German of Lebanese origin and a 28-year-old Palestinian were detained and an Islamic cultural club in the northern part of the city had been searched, the authorities said.
A police spokesman confirmed a Berliner Morgenpost report that the two men were stockpiling chemicals that could be used to make an explosive device.
Apart from the club in the northern district Wedding, the men's flats in the districts of Neukölln and Kreuzberg are also being searched.
According to the newspaper, the two men had collected large quantities of coolant and acid used in agriculture. Police said the suspects intended to mix the substances to make a bomb.
The men are said to have regularly spent time in the club, and sometimes stayed there over night. Police have apparently been keeping them under surveillance for two months. There are as yet no indications they had ties to al-Qaida or other terrorist groups.
The investigation began when chemical companies in Berlin and the state of Baden-Württemberg alerted police to the suspicious orders, the newspaper said.
"They could have created a highly explosive device with it," a police spokesman said.
Agricultural chemicals were also used by right-wing extremist Anders Behring Breivik for his 500-kg bomb attack on the Norwegian government in Oslo in July, which killed eight people.
It is not clear what the men were planning to do with the bomb. The spokesman said there was no immediate connection with the tenth anniversary of the September 11 terrorist attacks in the United States this Sunday or the Pope's visit to Berlin later in the month.
"We have no indication that a bomb was meant to explode here on September 11," the spokesman said.
Der Tagesspiegel newspaper said on its website that authorities had had the
men under round-the-clock surveillance.
"We barely had mobile operations commando forces available for other duties because they were all committed to the terror cell," an unnamed investigator told the paper.
The GdP police union said the arrests underlined the risk factor posed by Islamists in Germany.
"Those who think that the situation has returned to normal 10 years after the terrible attacks in the United States and the death of Osama bin Laden once stand corrected," GdP chairman Bernhard Witthaut said.
Germany has around 5,000 troops in Afghanistan under NATO command. Authorities say the Islamist scene is large and dangerous, with about 1,000 members across the country.
A 21-year-old Kosovar went on trial last week for killing two US airmen who were heading to Afghanistan from the western city of Frankfurt in March, in what has been called the first jihadist attack on German soil.
In 2007 police thwarted a major plot to attack US soldiers and civilians in Germany and Islamic militants in 2006 placed suitcases with homemade bombs on two regional trains at Cologne's main station. They failed to detonate, averting an almost certain bloodbath.
And three of the September 11, 2001 suicide hijackers including their ringleader Mohammed Atta, who flew the first plane into the World Trade Center, lived in the northern port city of Hamburg before moving to the United States.