A strike after just ten minutes from Germany forward Miroslav Klose - his 39th goal in 75 internationals - and a first-half own-goal gave the home side a commanding 2-0 lead at the break before Bulyga's double.
Having out-foxed the home defence to beat Germany goalkeeper Jens Lehmann on 61 minutes, Bulyga struck again to leave the Arsenal man and his defenders red-faced.
"It is disappointing that we allowed the lead to slip from our hands, you could clearly see the hard work from the training camp was missing," said Germany coach Joachim Löw whose squad spent last week training in the Spanish island of Majorca. "In twelve days, all the players will be in much better form, I can promise you that."
Several players now face a nervous wait to see if they will make the final Euro 2008 squad. It was their last chance to impress Loew before Wednesday's announcement when the three players who will not be in the 23-man squad travelling to Austria and Switzerland are revealed.
"Of course, that result gives us food for thought," added Löw. "The decision of who to leave out will be difficult, we will work hard again in Majorca. We have 26 good players in the squad and those who don't make it are not wanting for talent."
Having played in last Wednesday's Champions League final, Germany captain Michael Ballack looked comfortable in the first half before making way for Jermaine Jones.
And teenager Marko Marin - who has yet to play a Bundesliga match having been an integral part of Borussia Moenchengladbach's return to the German top table this season - impressed on his full debut, coming on for Bastian Schweinsteiger at the start of the second-half.
With strikers Mario Gomez and Kevin Kuranyi amongst the six players nursing injuries, Bayern Munich forward Lukas Podolski was handed the chance to impress alongside club-partner Klose in attack.
The pair combined when Podolski cut in from the right to set up Klose who opened the scoring with a goal after ten minutes. The home side went 2-0 up when midfielder David Odonkor whipped in a cross which ricocheted off the boot of Belarus midfielder Vladimir Korytko and into the back of his own net on 20 minutes.
Both Podolski and Klose made way after 54 minutes as Borussia Dortmund's Oliver Neuville and Patrick Helmes, who will play for Bayer Leverkusen next season, were given their chance to impress, but failed to find the net.
Belarus pulled a goal back when Bulyga smashed his shot at Lehmann on the hour and the ball looped into the visitor's net after hitting the goalkeeper's leg. And with just two minutes left, the striker beat Lehmann again with a curling shot at the near post to claim the draw.
In their last warm-up game, Germany will meet Serbia in Gelsenkirchen on Saturday and open their Euro 2008 campaign against Poland in Klagenfurt on June 8.