Germany welcomed news of the releases while a relative of the family said they presumed the brother is dead. There was no word on the fate of the parents or of a Briton abducted along with them.
Saudi interior ministry spokesman General Mansour al-Turki told AFP that the two rescued girls are in Saudi Arabia.
"Their condition is okay. But they are in the hospital to make sure they get any medical care they might need," he said.
A ministry statement released on the SPA news agency said the girls were freed on Monday.
"Security forces in communication with Yemen counterparts were able to recover two German girls in the border region between the two countries on Monday," the statement said. "They were part of a group kidnapped by criminal elements last year," it added.
According to Turki, the girls were part of a group of seven Germans, a Briton and a South Korean seized in Yemen's northern mountains in June last year.
They were captured in Saada province, the scene of a longstanding conflict between government forces and Zaidi Shiite rebels.
The bodies of two German women, both of them Bible students, and the South Korean were found soon after the abduction, raising fears for the safety of the remaining hostages - the Briton and the German family.
German Foreign Minister Guido Westerwelle said the girls would return to Germany on Wednesday.
"We are relieved that the Saudi security forces have succeeded in freeing two of our five compatriots kidnapped in Yemen," he said in a statement.
"The two little girls are currently under the protection of the Saudi authorities. They are doing well given the difficult circumstances. They should be returning to Germany tomorrow," he added.
German officials in January gave the ages of the three children as one, three and five.
Turki said he had no information on the four remaining hostages, but indicated that the operation by Saudi special forces and intelligence is ongoing.
"We don't have any idea about the others," he said. "I believe the operation still has some objectives to achieve."
However, a family spokesman in Germany said that the brother was probably dead while there was no word on the fate of the parents.
"There is one missing ... We have to assume that the boy is no longer alive," said Reinhard Poetschke.
The rescue announcement came after unconfirmed reports from Yemen on Tuesday that Saudi forces had crossed the border into a Yemeni village and that Saudi military helicopters were flying over the area.
Turki said he could not provide any details of the operation.
"This was a special security forces and intelligence operation," he said. "It was a humanitarian effort and a job that we could take on."
In January, Yemeni Foreign Minister Abu Bakr al-Kurbi said that the government had determined that the five Germans and the Briton were being held in Saada, the northern city at the centre of fighting between government troops and Zaidi rebels.
He said at the time that the government was conducting negotiations for their release.
Then in early April, after the rebels and the government had struck a ceasefire deal on February 12, Yemen's deputy prime minister for defence Rashad al-Alimi said the government was still pressing efforts to secure their release.
"Efforts are continuing, in coordination with the German and British g overnments, to liberate the hostages and end their ordeal," Alimi said, according to the defence ministry's official website.
He also reassured the hostages' families about the health of their loved ones, the website added without elaborating.