The police priority is to end the Sydney cafe siege peacefully, with the safety of the hostages "paramount", New South Wales Police deputy commissioner Catherine Burn has said.

She confirmed that officers had made contact with the hostage-taker, who seized control of the Lindt Chocolat Cafe at the height of the rush-hour this morning with dozens of people inside.

What appeared to be a black Islamic State flag was put in the window of the cafe soon afterwards, fuelling speculation that the attack was the work of Islamic extremists.

Holding a press conference near the scene, Ms Burn told reporters: "We are in contact ... the aim is a peaceful resolution."

"The good news is that we have five people out of the building," she said, but declined to comment on how many were still inside.

"We still have people being held, we still have a person that we are dealing with."

The freed hostages were seen fleeing from the building towards heavily armed police who had flooded the area.

Officers refused to confirm if they had been released or had escaped themselves.

Local media reported that one of the hostages was treated at St Vincent's Hospital in nearby Darlinghurst but that he had not suffered any physical injuries.

"We have had five people emerge from the location," Ms Burn said.

"Those people are now being assessed to ensure that their health is OK, and then police will speak with them."

She added: "Our approach is to resolve this peacefully. It might take a bit of time, but that is our approach.

"Please be assured that all police resources are in place. We are dealing with this. We have the best police negotiations in the world."

Ms Burn would not not confirm the number of hostages being held: "It's not something that I can confirm at the moment and it's not particularly useful."

Although an exclusion zone remains in place at the heart of the city's legal, financial and political district, Ms Burn urged Sydneysiders: "Continue your business as usual. If you had plans to come into the city then you should go about your plans as usual."

She described negotiations as "sensitive" and said police were prepared to continue into the night to resolve the crisis peacefully.

The scenes as the five hostages fled were captured on TV.

Footage showed two men running towards a group of armed police huddled at what looked like a fire escape door. A separate shot showed another man - wearing a brown apron - running out through that door.

A short while later, footage of what looked like two women running from the scene was shown. Both were wearing brown aprons and one of them appeared to be crying.

Ms Burn said nobody has been harmed at the moment, adding: "We have been working through our negotiations to try to make sure that the people inside are cared for, that they have what they need, so that they don't become ill or injured, so we will continue to do that."

She said negotiators have had contact and they "continue to have contact".

New South Wales Police Commissioner Andrew Scipione had earlier told reporters that an "armed offender" was "holding an undisclosed number of hostages".

Asked if the incident is terrorism-related, he said: "It's certainly something we couldn't rule out."

He said they are "not tagging it at this stage", but added: "Certainly we've moved to a footing that would be consistent with a terrorism event."

Mr Scipione said there is "at least one" offender.

He said people were working on what the black flag with Arabic writing - seen in a photo from the scene on Twitter - "stands for".

Australian Prime Minister Tony Abbott said he could not think of a more "distressing or terrifying" event to be caught up in than the unfolding incident.

An exact motive has yet to be established, but Mr Abbott said there were "some indications" that it could be politically motivated, while police said they could not yet rule out terrorism.

Martin Place is a plaza in the heart of the city's financial and shopping district which is packed with holiday shoppers this time of year.

A photo of hostages with their hands up was posted on Twitter by Australia's 7 News, as well as a photo of a black flag with Arabic writing being hung up at the window of the cafe.

Sydney Opera House is also believed to have been evacuated.

Mr Abbott said: "Our thoughts and prayers must above all go out to the individuals who are caught up in this.

"I can think of almost nothing more distressing or terrifying than to be caught up in such a situation, and our hearts go out to those people."

He said there are "some indications" that the incident could be politically motivated, but said the exact motive is yet to be established.

Mr Abbott said: "We don't yet know the motivation of the perpetrator. We don't know whether this is politically motivated, although obviously there are some indications that it could be.

"We have to appreciate that, even in a society such as ours, there are people who would wish to do us harm.

"That's why we have police and security organisations of the utmost professionalism that are ready and able to respond to a whole range of situations and contingencies including the situation that we are now seeing in Sydney.

"The whole point of politically motivated violence is to scare people out of being themselves. Australia is a peaceful, open and generous society. Nothing should ever change that and that is why I would urge all Australians today to go about their business as usual."

Mr Abbott described it is an "unfolding situation", adding that the ordinary business of government will continue today.

"Finally, this is a very disturbing incident. I can understand the concerns and anxieties of the Australian people at a time like this. But our thoughts and prayers must above all go out to the individuals who are caught up in this," the premier said.

In a post on Twitter later, he said: "I'm proud of the way police and security bodies have responded to the disturbing events today.

"It's been a difficult day, which has tested us, but, like Australians in all sorts of situations, we have risen to the challenge."

A man who works at the cafe, and arrived for work just after the siege started, said he saw one of his colleagues pressed up against the window.

Nathan Grivas said the building was already locked when he arrived.

''I didn't get a good look at them - as soon as I saw the gun, all I wanted to do was get out,'' he said.

Journalists from broadcaster Channel Seven, which has its Sydney newsroom in Martin Place, a few metres from the Lindt Cafe, were evacuated as the crisis unfolded but have now been allowed to return to the building.

Senior journalist Chris Reason said he and his colleagues could see across to the cafe and had counted 15 hostages inside - not the 50 that had been speculated.

"From inside the Martin Place newsroom, we can see food is being delivered to the hostages. Staff bringing it out from kitchen at back," he tweeted.

Martin Place is home to the state premier's office, the Reserve Bank of Australia, and the headquarters of two of the nation's largest banks. The state parliament house is a few streets away.

On the Lindt Chocolate Cafe Australia's Facebook page, a post appeared around five hours into the ongoing incident which said: "We would like to thank everyone for their thoughts and kind support over the current situation at the Lindt Chocolate Cafe at Martin Place.

"We are deeply concerned over this serious incident and our thoughts and prayers are with the staff and customers involved and all their friends and families.

"The matter is being dealt with by the authorities and we are waiting for any updates from them."

Prime Minister David Cameron said he had been briefed overnight on the siege, adding: "It's deeply concerning and my thoughts are with all those caught up in it."

The NSW MP for the inner city seat of Sydney, Alex Greenwich, moved to reassure locals.

"I know many CBD residents are scared and anxious this evening. We have world-class police working hard for our safety, as they always do," he said.