Google Cloud has bounced back from a major outage Sunday that impacted its own services, including YouTube and Gmail, as well as those of some of its clients.
The network outage primarily occurred on the East Coast of the U.S., but services that radiated out from there were also affected. After about four hours of problems, Google said the issues were resolved on Sunday.
"The network congestion issue in eastern USA, affecting Google Cloud, G Suite, and YouTube has been resolved for all affected users as of 4:00 p.m. Pacific," according to a Google Cloud status dashboard. "We will conduct an internal investigation of this issue and make appropriate improvements to our systems to help prevent or minimize future recurrence. We will provide a detailed report of this incident once we have completed our internal investigation. This detailed report will contain information regarding SLA credits."
Other than "network congestion," Google Cloud hasn't provided additional details on what caused the widespread outage across its own sites, including YouTube, Nest and Gmail, and social media sites such as Snapchat, Vimeo and Discord.
Whether the outage was due to a network congestion issue or a large distributed denial-of-service attack along the Eastern seaboard, it's still a black eye for Google, and a lesson to some of its customers that putting all of your eggs in one cloud provider's basket isn't a good practice.
The outage also comes on the heels of last week's news that the U.S. Department of Justice is reportedly preparing an antitrust probe against Alphabet subsidiary Google, according to The Wall Street Journal.
Aside of social media apps being down, the outage meant that some of Google's Nest customers couldn't use their Nest thermostats, Nest smart locks, and Nest cameras, according to Twitter posts on Google Nest.
Monitoring company ThousandEyes confirmed Monday morning that the outage was likely caused by network congestion, and that the issues rippled across the globe.
"We started seeing elevated packet loss in Google’s network as early as 12 p.m. PT between sites on the eastern U.S., including Ashburn, Atlanta and Chicago, and various Google-hosted services," a company spokesman said in an email to FierceTelecom. "These issues started to impact users globally approximately 20 minutes prior to their public announcement of the issue, showing an early indication of what was to come. For the majority of the duration of the 4-plus hour outage, ThousandEyes detected 100% packet loss for certain Google services from 249 of our global vantage points in 170 cities around the world. Starting at around 3:30 p.m. PT, we started to see services slowly become reachable again, and the issue appeared to fully resolve by 4:45 p.m. PT.”
Google first announced on Sunday that it was experiencing multi-region issues with Google Compute Engine. At 9:25 a.m. PT, Google reported "high levels of network congestion in the eastern USA, affecting multiple services in Google Cloud, G Suite and YouTube. Users may see slow performance or intermittent errors," the company said in a statement around 3 p.m. PT. "We believe we have identified the root cause of the congestion and expect to return to normal service shortly."
Google announcement that the issues had been resolved appeared around 5 p.m. PT on Sunday.
Google has been down this road before. In November it lost control of several million of its IP addresses for more than hour, which impacted its search engine and other outside services, according to Ars Technica.