The couple's other five-year-old son reportedly told police on Tuesday that he had been taken to the cave in the municipality of Adeje in the southwest of the island with his mother and brother but had managed to escape.
A couple found the boy on a mountain and then alerted the authorities who then arrested his father on Tuesday night, a Guardia Civil police force spokesman told AFP.
Officers arrested the father, 43, who is also German, in an apartment in downtown Adeje.
The spokesman did not give details as to what the boy said he had seen. The cause of death of the boy's mother and older brother had yet to be established, he added.
“The bodies were found in a mountainous area, inside a cave, between two ravines,” the spokesman said.
Local officials say everything is pointing to a case of domestic violence.
Socialist Prime Minister Pedro Sanchez, who faces a general election on Sunday, tweeted that “a mother and her son have been murdered. Gender-based violence has struck again… we must end this.”
Spanish daily newspaper El Pais, citing unnamed sources close to the investigation, said the woman and her two children were visiting the boys' father in Tenerife.
Tenerife, the biggest of Spain's Canary Islands located off the coast of Morocco, is especially popular with British and German tourists and expats.