The remains have been uncovered in a mass grave in Snagovo, near the town of Zvornik, where forensics were searching for corpses of Muslims killed in the 1992-1995 inter-ethnic war.
Initially the authorities said they thought the grave contained Muslim victims from the latest conflict, but then changed their opinion.
"The human remains of some 20 persons exhumed at the locality of Snagovo most likely date back to World War II," the prosecutor's office said in a statement, adding that it would inform the embassies of Germany and Italy in Sarajevo about the discovery.
Close to the remains, the forensic experts found Italian and German currency from the time as well as personal items, namely watches, that led them to "assume with a huge probability that these were German and Italian citizens," the statement said.
The remains have been transported to a laboratory for samples to be taken so attempts can be made to identify the dead through DNA tests.
During World War II, Bosnia was part of the pro-Nazi "Independent State of Croatia" and divided into two zones of interest, Italian in the southwest and German in the northeast.
The Bosnian war in the 1990s claimed about 100,000 lives, and about 7,800 people are still unaccounted for.