Streets around the Roßmarkt were closed off as the building was evacuated. But some people were told it was safer to stay where they were and lock themselves in their offices.
The incident started at lunchtime when two businessmen took tens of thousands of euros out of a bank and got into a car. The armed man got into the back seat and threatened them with a pistol.
One of the men got out of the car and ran, followed by the would-be robber, and then the other businessman. A woman then saw the armed man run into the office building.
The structure has shops on the ground floor including a bag store, as well as doctors offices, two flats and the offices of an airline.
Central Roßmarkt was closed off to keep the hundreds of passersby well away from the building, while helicopters could also be seen above the city.
A police commando unit entered the six-storey building at around 3 pm and started to search for the man.