Prime Minister David Cameron says he is “completely fed up” about the release on bail of Abu Qatada after the Muslim cleric won his deportation appeal.
He said ministers had “moved heaven and earth” to try to send Abu Qatada to Jordan where he is accused of bomb plots.
Abu Qatada’s lawyers told a UK court he would not get a fair trial in Jordan.
The court approved the appeal after deciding witness evidence obtained by torture might be used at trial.
“I am completely fed up with the fact this man is still at large in our country, he has no right to be there, we believe he’s a threat to our county,” Mr Cameron told BBC News.
“We have moved heaven and earth to try and comply with every single dot and comma of every single convention to get him out of this country.
“It’s extremely frustrating and I share the British people’s frustration at the situation we find ourselves in,” Mr Cameron said.
He said the government would lodge an appeal and added: “We will do everything we can to make sure that we do have the power to expel and deport people from our country who have no right to be there and mean us harm.”
‘Politically unthinkable’
BBC legal correspondent Clive Coleman said Home Office lawyers had 21 days to find a point of law to take to the Court of Appeal.
If the appeal was knocked out, Abu Qatada’s stringent bail conditions would “simply fall away”, he said, leaving the home secretary with a stark choice.
Theresa May could do the “politically unthinkable” and release Abu Qatada or she could impose a new Terrorism Prevention and Investigation Measures (TPim) control order which itself could be subject to legal challenge.
Faced with this difficult problem, there would also be furious diplomatic efforts to beef up assurances from the Jordanian government, our correspondent added.
Abu Qatada – whose real name is Omar Othman – arrived back at his home in London earlier after he was released from Long Lartin prison, in Worcestershire.
He was greeted outside his home by a small group of protesters – holding a “get rid of Abu Qatada placard” – who chanted, “Out, out, out.”
Earlier this year, judges at the European Court in Strasbourg ruled the cleric would not face ill-treatment if returned to Jordan, citing assurances outlined in a UK-Jordan agreement.
But the judge did not believe he would get a fair trial because a Jordanian court could use evidence against Abu Qatada that had been obtained from the torture of others.
On Monday, despite the UK obtaining additional assurances from Jordan, the Special Immigration Appeals Commission chairman Mr Justice Mitting ruled he was not satisfied Abu Qatada would be tried fairly.