“Bolts had been loosened along a stretch around 80 metres (yards) long,” prompting the immediate closure of the line, police in Wiesbaden said in a statement.
“Investigations are proceeding in all directions, and we cannot rule out that this was a possible attempted attack,” they added.
State-owned rail operator Deutsche Bahn described the incident as “sabotage”.
A high-speed Intercity Express (ICE) train driver had noticed something unusual in the early hours while crossing a bridge near Niedernhausen, just outside Frankfurt.
Several trains had already passed over the damaged section of track before the tampering was uncovered, police said.
“Luckily no harm came to any trains or passengers,” they added.
News website Der Spiegel reported that the rails were standing five centimetres (two inches) further apart than usual along the section where the bolts had been loosened.
This could have derailed further trains passing along the line, it added, sending them plunging off the bridge.
Investigators led by Frankfurt prosecutors searched a wide area around the damaged track, and used a police helicopter to take aerial photos.
“A focus of our measures is on making sure there is no further danger,” police said.