Gurlitt, 80, told Der Spiegel news weekly in an interview that his father, a powerful Nazi-era art dealer, had acquired the priceless works legally and that he as his heir sees himself as their rightful owner.
"I will not give anything back voluntarily," he told a reporter who said she spent 72 hours with the eccentric loner last week.
"I hope this gets resolved soon and I finally get my pictures back."
Gurlitt, who suffers from a heart condition, said he had given state prosecutors investigating him on charges of tax evasion and misappropriation of assets "enough" documents to prove his innocence.
He said he was shocked by all the unwanted attention, including photographers besieging him outside his home and while grocery shopping.
"I am not Boris Becker, what do these people want from me?" he said, referring to the German former tennis great.
"I just wanted to live with my paintings."
Gurlitt is the son of Hildebrand Gurlitt, one of a handful of dealers tasked by the Nazis with selling confiscated, looted and extorted works in exchange for hard currency.
While he sold many of the works, he kept a vast trove for himself. Most of the collection was believed lost or destroyed but surfaced during a routine customs investigation at Gurlitt's flat in February 2012.
They are currently in storage at a secret location.
But German authorities kept the case under wraps, arguing that they did not want to set off a deluge of fraudulent ownership claims for hoard, which includes works by Picasso, Matisse, Chagall, Renoir and Delacroix.
Jewish families and museums which have come forward to say that paintings in the collection were taken from them more than 60 years ago have criticised the fact that it took a German magazine, Focus weekly, to bring the spectacular find to light this month.