Human error was the cause of last night’s major outage at Facebook, WhatsApp and Instagram. The company has admitted this in a detailed statement about the incident.
The outage started with a configuration error. A system administrator accidentally gave a command that shut down Facebook’s global backbone network. This is a network that connects Facebook servers worldwide. This led to a domino effect, as a result of which those servers were no longer connected to the internet.
Due to the malfunction, Facebook’s servers were therefore inaccessible, even for the employees of Facebook itself. As a result, outage crews at Facebook data centers had to physically access the servers to investigate and fix the flaw. Due to the strict security of the servers, this was a time-consuming task.
Carefully reachable again
Once the bug was fixed, Facebook gradually made its apps available again. This was to prevent the systems from going upside down again because billions of phones worldwide tried to reach Facebook, WhatsApp or Instagram at the same time.
“Things like this can always happen. People make mistakes,” says tech editor Joost Schellevis: “Although it seems that Facebook was relying a lot on the functioning of its own infrastructure. So if things go wrong there, things go wrong right away. “
The outage started yesterday around 5.30 pm Dutch time and lasted until just after midnight. Here it led to complaining users, but not to major problems. The timing was quite painful for Facebook: today a whistleblower testified in the American congress about abuses at the company.