The man, 37, was allegedly an acquaintance of the victim's mother and had been a guest in the family's home in 2010. Prosecutors said at a press conference late on Saturday that he lived just a few metres from the girl's residence in Zella-Mehlis, a town near Suhl.
Identified as Tino L., the man was also on probation for narcotics and road traffic violations, according to prosecutors.
Zella-Mehlis Mayor Kai-Uwe Panse said he hoped for a speedy trial and a severe sentence for the alleged killer.
"The man has disgraced our town," he added. "Hopefully he will be put away for a long time and will never come here again." Panse said the man's arrest should bring a bit of peace to Zella-Mehlis.
Mary Jane disappeared on June 24 on her way home from school. The following day, hikers found her fully-clothed body in a forest not far from the girl's home.
The head of the special commission investigating the case, Andreas Beez, said the girl was sexually abused in the alleged killer's apartment but died in the woods nearby.
Beez said Mary Jane sustained injuries to the neck and died after drowning in a stream.
Authorities were tipped off by local residents, who told police that they had often seen Tino L. watching children from his apartment balcony.
During their investigation of the murder, police interviewed Tino L., who voluntarily submitted a DNA sample. "Through intensive investigations since the middle of this week, we were able to determine that the alibi this person gave apparently was not true," Beetz said.
Police said Mary Jane's red school bag was found in the man's apartment building.