The larger than life plexiglas sculptures, on display in Rathenauplatz in the centre of Germany's finance hub, were of Lilli Henoch and Walther Bensemann.
According to the Jewish Museum in Frankfurt they were ripped from their pedestals on two successive nights over the weekend.
Deputy head of the museum, Fritz Backhaus, said the criminals had used extreme violence to destroy the sculptures.
The exhibition, which has been on display in several German cities, portrays 17 successful Jewish athletes who were persecuted by the Nazis from 1933 onwards.
Lilli Henoch was a track and field athlete who was German national champion on several occasions. Walther Bensemann was one of the founders of the German Football Federation (DFB) as well as the football club Eintracht Frankfurt. He fled to Switzerland when the Nazis came to power and died there in 1934.
Police say they have no indication at this stage as to who the perpetrators were.
The exhibition will continue as planned until July 7th.