"It won't work out without targeted immigration," Weise told Die Welt newspaper, adding that some industries and regions were already suffering from a shortage of workers.
Weise said that current prognoses suggested that by 2025, Germany will be lacking between six and seven million skilled workers. "We might be able to fill half of this gap by mobilizing the domestic workforce," he said.
He also warned, "The worst case scenario is that companies start re-locating to where the workforce is, or restructuring so that they don't need so many skilled workers."
In March, Chancellor Angela Merkel's government approved legislation to recognize foreign qualifications, making it easier for non-EU people to find work in Germany.
Unemployment is currently dipping to an historical low, and is expected to go below three million people this month.
But the ranks of Germany's employed include a rising number carrying out so-called "mini-jobs", whose salary must be less than €400 per month.
There were nearly five million people in this position in April, a rise of 0.6 percent on the year, according to the BA. A further 2.4 million were doing a "mini-job" on top of another job.