The demonstration in Germany coincides with the 15th anniversary of the European Central Bank (ECB) and will be followed later Saturday by similar rallies in both recession-hit Spain and Portugal.
"We would clearly like to say that the policies of the European Central Bank and the troika ... are not the solution," said Roland Suss, a spokesman for the Blockupy movement which organised the demonstration in Germany.
The troika of international lenders -- the International Monetary Fund, the European Union and the European Central Bank -- have imposed strict conditions on countries such as Greece and Portugal in exchange for bailout funds.
Protesters say the belt-tightening measures applied by many governments to try to meet these conditions unfairly penalise the poor as some governments have been forced to slash jobs and raise taxes.
Sixty-two-year-old Marica Frangakis from Greece said: "It's good to see that people (in Germany) also feel concerned about the crisis."
In Greece, she said, people "are desperate after a five-year crisis".
According to police, around 5,000 people took part in the Frankfurt rally which will end in front of the ECB building.
In Lisbon, demonstrators are symbolically set to march to the International Monetary Fund's (IMF) representational office.
"This time, there won't only be Portugal pounding the pavement, but all the people in Europe will demonstrate against the austerity recipe," Portugal's "Que se lixe la troika" (Screw the Troika) movement said in a statement. Rallies are also expected in 17 other Portuguese cities.
In Madrid, protestors plan to march past the stock market, central bank and the Madrid delegation of the European Commission, carrying slogans saying "We don't owe, we won't pay" and "Troika Go Home".