Fire broke out near the cafe in Scotland where JK Rowling wrote the first Harry Potter novels.
One person was injured in the fire, which started at Edinburgh's George IV Bridge.
A cleaner at the Elephant House cafe, where Rowling wrote sections of her books, raised the alarm at 6am.
The fire did not spread to Elephant House but the cafe suffered water and smoke damage.
“I feel absolutely gutted about this,” he said.
“At best, we can hope for opening in a few weeks time, but if there is structural damage too then it isn't bearable to think about.”
The cafe is a draw for tourists and Harry Potter fans, and trading had only returned to normal days ago after Scotland lifted Covid-19 restrictions.
“It's really quite disturbing because we had just got back to some semblance of normality after two years, although we were only running at about 50 per cent of our festival trade,” Mr Taylor added.
On Tuesday, commuter and tourist buses were diverted as fire crews blocked the road for hours.
Scottish Fire and Rescue Service said nine fire engines were sent to tackle the blaze.
“Operations control mobilised nine appliances and two height vehicles. Crews are still in attendance,” a spokesman said.
A Scottish Ambulance Service spokesman said: “We received a call at 6.39am on Tuesday to attend an incident at George IV Bridge in Edinburgh with our first resource arriving on scene at 6.46am.
“We dispatched two ambulances, one paramedic response unit and our special operations team to the scene.
“We transported one patient to Edinburgh Royal Infirmary.”