Ambassador Ali Masednah al-Khotany takes over from Jamal al-Barag who has been given until September 15 to leave the country, the German foreign ministry said in a statement.
Earlier this month the foreign ministry declared another senior Libyan official, identified as Hisham Al Sharif, 'persona non grata' shortly after he was sent by the Qaddafi regime to act as charge d'affaires.
Germany recognised the rebel National Transitional Council as the legitimate representative of the Libyan people in June during a visit to Benghazi by Foreign Minister Guido Westerwelle.
A German foreign ministry spokesman said Berlin had already earmarked €100 million euros' worth of aid for Libya, of which two-thirds had been spent.
Germany is currently in the process of revising its policy towards Libya after Chancellor Angela Merkel's government decided to abstain from a United Nations vote on a NATO mission in the country.
Merkel is expected to attend a conference on Libya's future in Paris on Thursday.
The foreign ministry also suggested the international community would move to speed up the release of Libyan assets frozen abroad, adding that €1 billion of these were held in Germany alone.