The out-of-work figure fell to 5.8 percent this month, the Federal Labour Agency said, beating analysts' forecasts.
A 0.1-percentage point fall from the level it had held since January placed it at a new all-time low since German reunification in 1990.
"The number of unemployed sank significantly as spring begins to breathe life into the economy," said the agency's chief Detlef Scheel in a statement.
"Growth in employment is holding steady and demand from companies for new employees remains high."
In seasonally-adjusted figures, the number of people registered as out of work fell by 30,000 in March.
Jobs, wages and working conditions are set to be a key battleground in German elections slated for September, with left-wing politicians pushing for higher wages and more unemployment protection.
Right-wingers contend that tough employment law reforms in the mid-2000s built the foundation for Germany's present economic success and should not be rolled back.