WEBSITES DOWNED BY ONE USER’S CHANGE
Bug crashes Fastly’s network of proxy servers
ON JUNE 8, a little before 9.47 a.m., one of Fastly’s customers decided to change their settings. The change was valid, but it triggered a bug in the Fastly software, and things started to go wrong rapidly. About 85 percent of Fastly’s CDN (content delivery network, a geographically distributed set of proxy servers ) crashed. At 9.58 a.m., a status report from the company said it was “currently investigating potential impact to performance with our CDN services.” By 10.44 a.m., it reported that “the issue has been identified and a fix is being implemented.” The fix was in place at 11.36 a.m., and most services restored. By 12.41 p.m., all services had been recovered, although low cache hit rates remained a problem. The major outage lasted about an hour. This isn’t a long time for one person to be without the Internet, but it is an age for many of the world’s biggest online businesses to be offline. Victims included Reddit, Pinterest, Amazon, Etsy, Spotify, Twitch, PayPal, Quora, HBO Max, Stripe, and many major international news outlets.