According to reports by broadcaster NDR, the officer under investigation had also allegedly threatened a colleague with his gun and asked him to perform sexual acts.
NDR reported on Monday that in August of 2013, the officer took out his gun, went into the police station common room and held the gun to the temple of a fellow policeman. The officer then ordered the other man to perform sexual acts.
Inside sources told NDR that five other officers witnessed the incident.
"Something like this is criminal and shows contempt for other people," said Dietmar Schilff, Lower Saxony state police union chairman, to NDR.
Another source told NDR that this was not an isolated incident and that the policeman handled his weapon "loosely".
The officer had on other occasions reportedly pointed his gun at other officers, held the gun to someone's head and would also repeatedly unholster and holster the weapon.
NDR had previously reported that the policeman is under investigation for alleged abuse of refugees, including choking an Afghan refugee and dragging him through the cells with his feet in chains.
In another incident, the officer also reportedly humiliated a Moroccan refugee by force-feeding him rotten meat.
He also allegedly bragged about his abuses to a friend through instant messaging app WhatsApp, saying "I stuck my finger in his nose" and "I choked him, that was funny". A picture he reportedly sent shows a man bound in an unnatural position with a pained expression on his face.
Other sources told NDR that within the police department, there is a very low tendency to report incidents of misconduct to higher officials.
Hamburg Professor Rafael Behr, who studies policing, said that even if the gun incidents were intended to be a joke, they still were dangerous.
"It is not simply fun, there is also aggressiveness behind it. And that's what makes it so threatening," Behr told NDR.
According to Bild, the officer and others connected to the refugee incidents have been suspended and their weapons taken away.
Investigators also searched the man's house and the police station on Friday.
"Regardless of current accusations, such behavior has no place within the police force," Federal Police President Dieter Romann told Bild.
"More than 40,000 officers and employees work hard every day and every night to fulfill their difficult duties for the public welfare in the best way possible, despite adverse conditions, and have achieved for themselves and the police a high reputation.
"We will not let this be ruined by a few people."