Kroos was named man of the match and goalkeeper Neuer made several key saves as Ukraine threatened to expose a makeshift German defence weakened by injuries to Mats Hummels and Antonio Ruediger before half-time.
However, Shkodran Mustafi's early header and a first international goal in five years by Bastian Schweinsteiger, deep into stoppage time, gave Joachim Löw's men a perfect start in their bid to match their success in Brazil two years ago.
"We played well and kept up our good standards in defence," said Bayern Munich's Neuer. "We haven't shown yet everything that we have to show, we will keep improving."
Kroos's brilliant cross set up the opener, but he admitted Ukraine were unlucky not to have at least levelled before the break as Neuer's heroics and a spectacular goalline clearance from Jerome Boateng kept them at bay.
"It is always good for the confidence to win 2-0 and keep a clean sheet," said Kroos, who shone fresh from winning the Champions League with Real Madrid.
"The last 15 minutes before half-time we didn't concentrate well, so we couldn't complain if we had conceded.
"In the second half we were much stronger, we dominated the game and didn't concede any chances.
"We've learned a lot, we're not at the level we want to achieve during this tournament yet, but it is a good start."
It was a special night for Schweinsteiger, who had only played 23 minutes since tearing a knee ligament in March, but battled back to fitness just in time to make the squad.
And he made a huge impact in just three minutes on the field as he sprinted forward to slam home Mesut Ozil's low cross in the 93rd minute.
"I ran so hard in the celebrations that I'm a bit out of breath still," said the Manchester United midfielder.
"It was amazing for that to happen, it's the kind of thing you can only wish for, but the most important thing is the team wins."
Neuer was captain for the night in Schweinsteiger's absence, but he is hoping his former Bayern team-mate will be fit to play a bigger role in the rest of the tournament.
"Basti was class when he came on and made the goal," said Neuer. "I hope he can play from the start soon."
Löw will be charged with the decision as to whether Schweinsteiger can oust Sami Khedira for a place alongside Kroos in midfield.
And the German coach insisted the goal was just reward for Schweinsteiger's sacrifice in battling back to fitness.
"I am very happy for Bastian that after all this work he has done throughout the last weeks and months he could score the second decisive goal," added Löw.
Next up for Germany is a clash with age-old rivals Poland.
Löw expects the meeting in Paris to decide who wins Group C after the Poles saw off Northern Ireland 1-0 earlier on Sunday, and is especially wary of Bayern's prolific striker Robert Lewandowski.
"Poland are a very strong team. We played twice against them in qualification, we lost away, but won at home.
"They have quite strong strikers with Lewandowski, who we know well and who scores a lot of goals."