Ignace Murwanashyaka, head of the Democratic Forces for the Liberation of Rwanda (FDLR), received 13 years in prison, while his deputy Straton Musoni was given eight years, judge Jürgen Hettich told the court after a trial that lasted more than four years.
Both men were convicted of holding leadership roles in a foreign terrorist organization, while Murwanashyaka was also sentenced for a charge of aiding and abetting war crimes.
Prosecutors said during their closing argument last week that the two men were guilty of "the full range of atrocities that one can imagine in a civil war" and demanded life imprisonment for Murwanashyaka.
They had initially been accused of 26 counts of crimes against humanity and 39 counts of war crimes committed by militias under their command between January 2008 and November 2009, although those charges were gradually whittled down to avoid forcing traumatized witnesses to testify.
While Murwanashyaka will be jailed immediately, Musoni will serve no further time in jail, as he has served his time in custody during the investigation and trial.
The trial was hailed by the United Nations and human rights groups as a breakthrough in pursuing war criminals when it opened four years ago, as it is more difficult to prove crimes against humanity than terrorism charges.