It was removed by crowbar by the leader of a nearby youth club within hours of being found.
Dennis Stinchcombe said he hoped to raise £100,000 for the struggling Broad Plain Boys' Club by auctioning it.
But Bristol mayor George Ferguson asked for the work to be put back on the city council-owned wall.
It is now expected to go on display in a museum in the city while the ownership is debated.
The work, showing a couple embracing while checking their mobile phones, was found on Tuesday - but removed soon after.
A smaller copy of the original has been pinned up in its place.
The youth club leader (pictured right) temporarily held the board in his clubMr Stinchcombe, 58, said: "Now we've ended up with a Banksy on our doorstep. It is a dream come true."
"If we hadn't taken it, someone would have ripped it from the wall or vandalised it."
But Mr Ferguson said it was "against the spirit of Banksy and street art" to remove it.
"He [Mr Stinchcombe] has got a job to do supporting his boys' club, but I don't think it is in the right spirit to remove it, and I would ask that he puts it back.
"I think it was an unwise thing to do.
"On the face of it that would be theft, so I think he should take a lot of care.
Police viewed the removed artwork at the boys' club on Wednesday"He should be careful and make sure he's on the right side of the law. I'm pretty sure that it belongs to us."
The BBC has asked a spokesman for Banksy for a comment but has not yet received a reply.
Mr Stinchcombe said the Banksy would be displayed at the Bristol Museum and Art Gallery until the different opinions over its legal ownership could be settled. He previously said he would make contact with Bonhams auction house about selling it.
A spokesman for Bonhams said so far it had not been approached by anyone wanting to sell the artwork.
He said if the auction house was approached its experts would first validate the item by checking with a company that authenticates Banksy's work.
"After this we would make a number of checks as to whether it was on any stolen artworks database and whether the seller is entitled to sell it, as part of our due diligence process.
The youth club is struggling financially and believes the Banksy work could raise funds for its survival"Until then we could not put any value on it," he added.
Mr Stinchcombe said he had received death threats after removing the work.
"I had to make a decision as to whether to allow it to stay there and come in today to find it gone or damaged, or take it yesterday and think what can we do with this to make it a bit of a reality and be good for the community," he said.
Police officers who visited the boys' club on Wednesday afternoon to view the artwork advised club officials to seek legal advice because there could be issues surrounding the sale of the artwork if they did not own it.
The youth club is struggling financially and believes the Banksy work could raise funds for its survivalAlison Bevan, of the Royal West of England Academy in Bristol, described the piece as "brilliant".
"It is great to have Banksy back here," she said.
"I think that the fact it was screwed on to a wall makes it extremely portable and would have increased the likelihood of someone taking it.
"From my point of view, the fact it has come to an organisation like this, that is supporting the local community, is better than it going straight to some wealthy collector."
Another visitor to the site, Paul Messenger, from Bristol, said: "I don't mind at all [that it was removed], I think it was a sensible thing to do.
"It can make money for a good cause. I would suggest it was the right thing to do."
Youth worker Jordan Powell, from Warmley, Bristol, stayed in the club overnight to help protect the original.
"Everyone can still see it here and it is safe," he said.
"Everyone has their own opinions, some are probably jealous, but it is going to a great cause. It is going to help the youngsters in Banksy's home town."