Google Cloud outage traced to engineer accidentally disconnecting all fiber links
A routine maintenance mistake by a Google engineer caused a full fiber disconnection in the us-central1-b region, leading to a temporary loss of connectivity for a portion of the G-Cloud.
During a scheduled hardware maintenance operation on September 1, a Google Cloud engineer mistakenly pulled out all fiber-optic connections in the us-central1-b region, causing a complete network outage for that zone. The fault resulted in 100% traffic drop rates for affected resources, rendering virtual machines inaccessible to external users while they could still communicate internally. Google’s design, which normally isolates failures across multiple routing devices, was overwhelmed by the simultaneous disconnection of every fiber path.
After detecting the issue, the company automatically shifted traffic to healthy capacity elsewhere in the region and dispatched technicians to physically reconnect the cables. By 11:52 PT, the optical links were restored, traffic normalized, and services returned to normal operation. The event was recorded in Google’s service health report, highlighting the importance of strict procedural adherence during maintenance.
Why it matters
The outage shows how a single human error can disrupt cloud services despite built-in redundancy, affecting many customers.
In this story
