The proposed ban would apply to around 100,000 flats in 20 so-called "social conservation areas" in the central areas.
Berlin's ruling Social Democrats (SPD) agreed to push through the ban at a closed meeting on Sunday evening, Welt newspaper reported. A law will be prepared in the coming weeks.
The idea is to slow down rapid developments in Berlin's property market which has led to thousands of tenants being pushed out of the capital's central areas, said SPD regional chairman Jan Stöß.
"The Berlin property market is changing so quickly that we have to fully exploit the powers of the regional government," Stöß told the meeting.
Berlin's Senate recently raised the rental notice threshold to a maximum of ten years for long-term tenants, but this is not enough to help residents affected by the changes, he said.
"The transformation [of rented property] into private flats has proved itself to be the decisive factor in driving out of long-term tenants," said Stöß.
The ban will only apply in certain in-demand areas particularly threatened by rapidly rising rents such as parts of Prenzlauer Berg, Friedrichshain-Kreuzberg, Mitte and Tempelhof-Schöneberg.
Berlin's tenants association has warmly greeted the proposal whereas the German Real Estate Association (IVD) has rejected the intervention in the property market.
"It's very concerning if the SPD wants to tell people what to do with their property," head of the regional chapter Dirk Wohltorf told the Welt. The ban would effectively punish those who had bought property as an investment for their old age, he added.
SEE ALSO: Rental cap for busy areas draws closer