Neighbours called the emergency services when the saw the 38-year-old and the boy lying in the courtyard of their building on Schönhauser Allee in Prenzlauer Berg.
Both father and son were seriously injured and had to be taken to hospital, although their fall was broken by a canopy below the window.
Police found the man's girlfriend, who is not the mother of the boy, in a state of shock when they entered his apartment.
Tagesspiegel reported that the family had been under supervision by the youth welfare service (Jugendamt) after the man climbed onto the roof of the nearby Colosseum cinema with his son in February.
That time police called to the scene were able to persuade him to come down and nobody was hurt.
The man was later charged with crimes under parenting laws and the youth welfare office became involved, taking the child into foster care.
The man dropped out of sight of the authorities after returning to his homeland, Israel, and did not inform either the police or the youth welfare office of his return to Berlin.
Christine Keil, the local councillor responsible for youth issues, said the incident was the first the authorities knew of his return to Berlin.
State murder police have taken over the investigation.