Police said there was no evidence the perpetrators had any neo-Nazi ties but were rather common thieves with previous convictions for armed robbery and assault.
“These are simple thieves,” said Krakow police chief Andrzej Rokita.
The men were arrested in the Polish cities of Gdynia and Wloclawek on Sunday night and were questioned on Monday over the theft which took place Friday morning.
The men arrested were aged between 20 and 39.
Krakow police spokeswoman Katarzyna Padlo said the sign had been cut into three pieces. The thieves had carried it 400 metres and taken it through a hole they had cut in the fence before loading it onto a vehicle. But it remains unclear how they got it out without being seen.
The sign is about 5 metres long and made of heavy cast iron.
The cynical statement, which means ''Work shall set you free,'' has come to symbolise the tragic fate of the 1.1 million Jews murdered at Auschwitz during the Second World War. It was crafted by Polish prisoners at the camp in 1940 under order of their German captors. The phrase was also used by the Nazis at other concentration camps.
The theft sparked outrage around the world. Avner Schalev, president of the Israel Holocaust memorial Jad Vaschem, branded it an attack on the memory of the Holocaust.
Auschwitz committee president Noach Flug described the theft as a very disturbing sign.
Israel Prime Minister Benjamin Netanyahu had publicly called on Poland to catch the perpetrators, while the Polish Prime Minister Donald Tusk had promised to make the case an “absolute priority.” Several dozen police officers were assigned to the case.