"We mourn the death of our company founder, Karl Albrecht," Aldi said in a statement, adding that he died on July 16th.
Albrecht was born on February 20th, 1920, in the northwestern city of Essen, where his parents owned a small corner shop.
After learning his trade in the family business, he and Theo, who died in 2010, took over the store in 1946 and by 1948 had quickly expanded it to a network of more than 300 shops in the Germany's most populous state of North Rhine-Westphalia.
They then transformed the network into the Albrecht Diskont or Aldi chain in the 1960s, which grew rapidly.
They divided the stores between Aldi Nord in the north of the country and Aldi Süd in the south, with Theo responsible for Aldi Nord and Karl in charge of Aldi Süd.
Aldi does not publish annual accounts, but it currently has around 10,000 stores in 17 countries, including Germany, Austria, the United States, Australia and Britain.
Annual sales amount to an estimated €40 billion, with Aldi seizing market share off established retailers in many countries.
As many as 75 percent of German households are estimated to shop regularly at Aldi.
Both brothers were highly private people, never appearing in public or giving media interviews or releasing any photos.
According to Forbes magazine Karl Albrecht was Germany's richest man, and 24th richest in the world, with an estimated net worth of €19 billion.
He continued to head Aldi Sued until he was 75, seeking to ensure the company's continuing success and independence by transferring the assets into two foundations.
"Karl Albrecht wanted no public attention and always turned down any honours, leading a secluded life," the company statement said.
"He was fair and dependable entrepreneur and a man who lived by his Christian values," the statement said.
Given the family's habitual secrecy, as few details have emerged about his death as about his life.
The daily Frankfurter Allgemeine Zeitung reported that Karl Albrecht died in his villa in Essen surrounded by his family.
He was then buried on Monday alongside his late brother.
The newspaper described him as "the last post-war patriarchs, the most singular, but also the greatest of Germany's company founders."
Karl Albrecht shied away from the usual trappings of wealth and owned no private jet, yacht or holiday villas.
He held just a couple of hundred thousand euros on his private bank account right up until his death, the newspaper said.
In 1971, his brother Theo was kidnapped and held hostage for a total of 17 days and was only freed on payment of a ransom of seven million deutschmarks, an incident which traumatized the family and made them more secretive than ever.
Karl Albrecht was married for 67 years until the death of his wife last year.
He leaves two children, a son and a daughter.