The men were arrested after an Indian fisherman died in what his colleagues claimed was a collision with the freighter being captained by the Germans, Der Spiegel reported on Wednesday.
The incident took place 11 days ago off the east Indian coast near Chennai. One fisherman died while two more jumped off their fishing boat into the water.
The police have accused the captain and first officer of the Grietje freighter of negligent manslaughter, saying they simply carried on after the accident.
But the Germans say that there was no collision and that underwater examination of their vessel backed them up, something which Der Spiegel said was confirmed by the Indian police.
But the two Germans were detained in India, had their passports confiscated, and have been told not to leave the port of Chennai. The alleged incident occurred within the 12-mile zone which means it was within Indian jurisdiction.
The concern is that the Germans could become caught up in a lengthy procedure as a result of an ongoing diplomatic spat between India and Italy which on Tuesday led to the resignation of Italian Foreign Minister Guilio Terzi.
Der Spiegel said the Italians were protecting a merchant ship last year when they shot dead two Indian fishermen, fearing they were pirates preparing to attack. Although this incident happened in international waters, Indian police arrested the soldiers and preparations were made to put them on trial.
Hefty diplomatic wrangling enabled them to return to Italy for Christmas, and then again in February so they could vote in the general election. The last time the marines were home, the Italian government refused to return them to India, provoking furious claims that the government was aiding and abetting their flight from justice.
The row even affected Italy's ambassador to India Daniele Mancini, who was told he could not leave the country - that his diplomatic immunity was no longer valid as he had made promises on behalf of the marines.
Story continues below…
Last week the Italian government reversed its course and sent the soldiers back to India - prompting Terzi to resign on Tuesday.