The Werkstatt Deutschland organization cited "massive criticism in the media and the political world" over its plans to give Putin the Quadriga Award that recognizes "role models for enlightenment, dedication and the public good."
The group said it "profoundly regretted the news that (former Czech president) Vaclav Havel wanted to return his prize received in 2009" in protest at Putin's nomination.
Putin was to receive the award in part because of progress he'd made in improving relations between Russia and Germany, the board had said earlier, adding that this "is one of the great achievements of Vladimir Putin."
But many German politicians have charged that his human rights record makes him an unacceptable candidate.
Cem Özdemir, the head of Germany's Green party, announced he was stepping down from the board of the non-profit Quadriga in protest at the decision.
"In my view, the Quadriga prize should go to people who have done a particular service to democracy and its promotion," he said in a statement. "I do not see Vladimir Putin among those ranks."
"During his tenure as head of state and prime minister, Putin has dismantled democracy, restricted freedom, undermined the rule of law and given Russia corruption," Human Rights Commissioner Markus Löning told Der Spiegel magazine.
Erika Steinbach, a Bundestag member with the centre-right Christian Democratic Union added: "During his presidency, human rights were and are being systematically violated."