Facebook on Monday began restoring access to its platform as well as to Instagram, WhatsApp and Messenger after a global outage lasting about six hours shut out many of its 2.7 billion users, left some of the company’s employees idle and prompted a public apology.
"Facebook services coming back online now - may take some time to get to 100 per cent," its chief technology officer Mike Schroepfer said in a tweet. "To every small and large business, family, and individual who depends on us, I'm sorry."
The error-reporting website Downdetector showed the services first stopped working around 11:45am Eastern Time (7:40pm in the UAE). Across the globe, users of Facebook and its sister sites were unable to load content or were greeted with error messages.
Facebook and its affiliated apps began to return online for some users about 5:45pm ET, around six hours after the incident began.
"We've been working hard to restore access to our apps and services and are happy to report they are coming back online now," the company's spokesman, Andy Stone, said in a tweet.
It was one of the longest failures in recent memory. Downdetector, which monitors internet problems, said the Facebook outage is the largest it has seen, with more than 14 million reports worldwide.
The disruption came a day after a whistleblower accused Facebook of repeatedly prioritising profit over clamping down on hate speech and misinformation.
Several users logging in to third-party apps such as Pokemon Go and Match Mastersusing using their Facebook credentials to were also facing issues.
Facebook staff, who usually communicate using software developed in-house, were reportedly forced to use Zoom and Discord as they scrambled to identify the cause of the problem.
The outage also left staff unable to access buildings and conference rooms at the company's offices after their electronic entry keys stopped working, the New York Times reported, citing an internal memo.
The memo also revealed a team of employees was dispatched to a California data centre to attempt to manually restart the social media company's servers.
"Given the level of back-ups, regional co-location servers, the Facebook FNA node network, their data centre fabric, their neural-network fabric interconnecting data centre and machine learnings applications – this kind of global outage should, in 2021, be inconceivable — or at least lasting minutes," Neil Campling, co-head of Mirabaud Securities' Global Thematic Group, told The National.
Why did Facebook go down?
Facebook blamed a "faulty configuration change" for the issues in a post on Tuesday morning from its vice president of infrastructure.
Posting on Facebook, Santosh Janardhan said sorry for the outage and blamed “configuration changes on the backbone routers that co-ordinate network traffic between our data centres”.
He said: “To all the people and businesses around the world who depend on us, we are sorry for the inconvenience caused by today’s outage across our platforms.
“We’ve been working as hard as we can to restore access and our systems are now back up and running. The underlying cause of this outage also impacted many of the internal tools and systems we use in our day-to-day operations, complicating our attempts to quickly diagnose and resolve the problem.
"Our engineering teams have learned that configuration changes on the backbone routers that co-ordinate network traffic between our data centres caused issues that interrupted this communication. This disruption to network traffic had a cascading effect on the way our data centres communicate, bringing our services to a halt.”
During the outage, security experts had identified a configuration change as a more likely cause than an outside hack. A massive denial-of-service attack that could overwhelm one of the world's most popular sites would require either co-ordination among powerful criminal groups or a very innovative technique.
Facebook has previously been coy about the causes of its service interruptions, which affect three of the world’s most-used apps.
In 2019, after most of its services were inaccessible for almost 24 hours in the worst disruption it has faced, the company blamed a “server configuration change” for a “cascading series of issues”.
Facebook also experienced widespread outages with its apps in March and July this year.
On Sunday, a whistleblower revealed her identity and accused the company of putting profit over the safety of its users.
Facebook shares suffered their worst day in nearly a year, closing at $326.31 on Monday, down 4.87 per cent for the day amid a broad sell-off of tech stocks.
The revelations have ignited a firestorm for Facebook in Washington as politicians accuse the company of covering up internal research about its negative effects.
Katie Paul, director of the Tech Transparency Project, said the huge impact of Monday's outage was the "perfect illustration" of the problem with giving one company monopolistic control over much of the world's digital communication.
"Especially in developing nations, where Facebook is the gateway to the internet, and where WhatsApp serves as the primary communication method for hundreds of millions, Facebook's overwhelming power was on full display through its absence," Ms Paul told The National.
The users of dozens of smartphone models will be unable to use WhatsApp from November 1, when the Facebook-owned platform will stop working on phones that run on systems older than Android OS 4.1, Apple’s iOS 10 and KaiOS 2.5.1, according to its FAQ section.