Azure outage: Microsoft says customer configuration change triggered cloud glitch

As user frustration builds, experts say it's time to revisit 'legacy internet components'

For the second time in less than a month, a cloud computing glitch affected users around the world. Photo: Corbis

Microsoft has pointed to one of its Azure customers as having triggered a widespread outage affecting users around the world.

The company based in Redmond, Washington, sought to explain Wednesday's issues with a preliminary post-incident review, the technology world's equivalent to a postmortem.

According to Microsoft's Azure status page, "an inadvertent tenant configuration change" led to the widespread outage.

Microsoft said that a "tenant configuration" caused an Azure cloud outage felt by hundreds of thousands of users around the world on Wednesday.

That is a polite way of saying one of Microsoft's cloud customers inadvertently caused the massive disruption.

Microsoft also accepted responsibility. "Our protection mechanisms, to validate and block any erroneous deployments, failed due to a software defect," the Azure status page explained. Microsoft has since corrected the problem and hopefully prevented similar issues from reoccurring, it added.

The outage, which affected hundreds of thousands of people trying to access apps, games, web portals and other services dependent on Microsoft's Azure service, is the second major cloud outage in less than a month.

Frustration toward cloud service providers has ratcheted up in recent weeks with outages proving costly.

Last week, a large Amazon Web Services (AWS) outage rendered many online tools useless.

In terms of market share, Microsoft and Amazon are among the top providers of data centres and cloud services, causing some politicians to call for more diversity and market competition.

It also has some firms looking at the possibility of changing the current cloud computing infrastructure to prevent similar incidents from happening.

A day before the Azure outage, Nvidia chief executive Jensen Huang touched on the need for the current cloud-computing framework to evolve. He praised cloud computer offerings from companies such as AWS but said Nvidia's new partnership with telecoms firm Nokia would offer improvements, options and eventually 6G technology.

"We’re going to do the same thing on top of the wireless telecommunications network," Mr Huang said, outlining the company's plan to push the envelope with 6G technology and new wireless communication consoles.

"Cloud computing will be able to go right to the edge where data centres are not because we have base stations all over the world," he said, providing a tantalising glimpse of a world where bricks-and-mortar data centres might be less significant.

Yet even the most robust visions for more reliable cloud computing options are not perfect. When widespread internet outages often occur, wireless systems are frequently overwhelmed because of the sheer number of users suddenly rushing to use the services amid disruptions.

There is no quick fix on the horizon, with so many customers relying on so few companies with the ability to build cloud-computing facilities at the scale required.

"The recent outages at Microsoft’s Azure platforms and Amazon Web Services make clear that digital resilience is no longer optional and plans for cloud-based disaster recovery and high availability should be a part of every organisation’s business continuity plan," said Morey Haber, chief security adviser at BeyondTrust.

Mr Haber said the burden from the potential fallout looks set to fall on businesses, governments and other organisations relying on cloud providers.

Morey Haber, chief security officer with BeyondTrust, said only a change to legacy internet systems would help to bolster the stability of cloud services.

"The best recommendation is to take proactive steps for business continuity including offline backups, alternative providers, multi-region deployments and testing continuity plans on a regular basis," he said.

The prevailing wisdom that more cloud competition and choices would help blunt the outages is not a silver bullet solution, he added.

"The issue is not just market share but rather systemic concentration and shared dependencies that make up the entire internet like domain name systems, network time protocols and certificates," he said. Only a long-term overhaul of "legacy internet components" would help to improve the stability of cloud services, he added.

The ongoing worries about cloud glitches have also created a gap for newer companies to fill. "Acting as the 'Switzerland of data', we enable organisations to synchronise and replicate their data across multiple clouds," said Levent Ergin, chief AI strategist for Informatica, a data platform company.

"These outages have exposed a deeper systemic issue, digital dependence without digital redundancy", and companies like Informatica will prove to be instrumental amid so many fears about outages, he added.