The Royal Blues of Gelsenkirchen will discover their group stage opponents in Thursday's Champions League draw, but threw away a 2-1 lead at Hannover, who have not lost at home for 18 months.
The hosts took the lead via a 43rd-minute header from Brazilian defender Felipe, who was left unmarked in the area.
But Dutchman Klaas-Jan Huntelaar, the league's top scorer last season, levelled for Schalke on 52 minutes with a long-range drive before midfielder Lewis Holtby put the guests ahead 12 minutes later with a superb header.
Hanover's Swiss midfielder Adrian Nikci, who had only been on the pitch for three minutes, became the hosts' hero after converting a crisp pass from ex-Germany striker Jan Schlaudraff 10 minutes from time to claim a point.
"I am very disappointed with the 2-2 draw. We were clearly the better team, had the better chances and we controlled the game," said Holtby.
Holtby's team-mate, ex-Germany goalkeeper Timo Hildebrand, was more direct.
"We led 2-1 and we should never, ever have allowed the game to escape our grasp," he said.
Defending champions Borussia Dortmund kicked off the new campaign with a 2-1 home win over Werder Bremen on Friday night thanks to goals from Germany stars Marco Reus and Mario Götze.
Bavarian giants Bayern Munich also got their campaign off to a flying start with a 3-0 win at newly promoted Greuther Fürth on Saturday, as goals from Thomas Müller, Mario Mandzukic and Arjen Robben secured the win.
Bayern are still smarting after Dortmund won last season's Bundesliga title, while Munich lost both the German Cup and Champions League finals.
There was drama on Saturday night as Wolfburg claimed a late 1-0 victory over Stuttgart.
After Wolves' goalkeeper Diego Benaglio saved a penalty from Stuttgart striker Vedad Ibisevic in the 88th-minute, Wolfsburg's 23-year-old Dutch forward Bas Dost headed in a 90th-minute winner on his Bundesliga debut.
Promoted Eintracht Frankfurt came from behind for a 2-1 win at home to Bayer Leverkusen thanks to an equaliser from Stefan Aigner and a late header from Martin Lanig.
Borussia Mönchengladbach responded after their home defeat by Dynamo Kiev in Tuesday's Champions League play-off first leg with a 2-1 victory at home to Hoffenheim.
Striker Mike Hanke headed home on 33 minutes, only for Hoffenheim's Roberto Firmino to equalise with a header on 66 minutes.
Gladbach's Venezuela midfielder Juan Arango then unleashed a spectacular free-kick on 79 minutes to claim the points.
Hamburg's season got off to a bad start with a 1-0 defeat at home to Nuremberg, whose new signing Hanno Balitsch, freshly arrived from Leverkusen, grabbed the winner 22 minutes from time.
Striker Dani Schahin had an afternoon to remember with both second-half goals for promoted Fortuna Duesseldorf in their 2-0 win at Augburg, while Freiburg were held to a 1-1 draw at home to Mainz.