Organisers said there were up to 30,000 people, but police estimated the crowd to be around 17,000.
Gathering at the north wing of the central station, they marched through town to the Schloßplatz for a rally, where they listened to speakers and then held a loud minute of protest.
Head of the Green party Renate Künast was at the rally, saying she was surprised by the size of the crowd and telling them she was sure the project would fail.
Her party has opposed the project to the fury of conservatives who insist it will go ahead.
The massive €4.5-billion Stuttgart 21 project aims to transform the southwestern city and the surrounding region into an important European railway crossroads.
The plan is to blast 16 tunnels and cuttings into the many surrounding hills, build 18 new bridges, lay 60 kilometres (38 miles) of new train track and create three new stations.
Stuttgart's main station will be utterly transformed from a terminus into an underground through-station on the major European connections between Rome and Berlin, and Paris-Vienna-Budapest. There will also be a rail link to Stuttgart airport and a new line between Wendlingen and Ulm.
"Around 2.5 million people live in the Stuttgart region and they are stuck in traffic morning, noon and night," Stuttgart 21's spokesman Wolfgang Drexler told AFP.
"There have been no major construction projects here for 20 years."
But many in Stuttgart say the project is far too disruptive and expensive and that the rail network could be improved in other, cheaper ways. They also fear it will go over budget.
In particular they object to the side wings of the historic train station building, an interwar modernist classic designed by Paul Bonatz, being demolished.
The conservative newspaper Frankfurter Allgemeine Zeitung, likened it to destroying St. Peters in the Vatican City but leaving the dome intact.
Opponents also say that doing so much underground construction is potentially dangerous, and that the current station need only be modernised. Others say that many of the new lines will be unsuitable for cargo trains.
"It's been neglected a bit, but it's still the second most punctual terminus station in Germany," says Axel Wieland from the BUND green pressure group, one of the many organisations against the project.
But supporters say that when the decision to go ahead was taken in 1995, a majority supported it. And a whole new district will be created where the rail lines now stand, freeing up land for parks and valuable real estate.
The traditionally conservative state of Baden-Württemberg, where Stuttgart is the capital, will hold elections in March, and some observers believe the unpopular rail project could affect support for Chancellor Angela Merkel's Christian Democrats.