A mystery bug at Google’s end caused a mass outage on Wifi and OnHub routers connected to networks that were operating normally.
Google has apologized for annoying Google Wifi and OnHub customers after accidentally triggering a mass factory reset on the routers.
A bunch of Wifi and OnHub owners panicked yesterday after discovering the devices were returned to a factory state, leaving owners offline despite the absence of any disruption on carrier networks.
Frustratingly for customers, the unrequested factory reset required users to reconfigure device names and network settings, which Google admits isn’t that simple to do.
The outage highlighted a problem with OnHub and Wifi’s reliance on Google cloud services for network connectivity.
“Not a good sign…so these access points depend on something in the cloud to work? Or even worse someone/something at Google can just decide to shut down my network? I reinstalled my old dumb AP, my internet connection is just fine,” one user commented on Google’s Wi-Fi forum.
Indeed, as Google eventually explained, it wasn’t a buggy update that caused the mass crash but “an issue with the Google Accounts engine”, which signed users out of their network device and in turn reverted them to a factory state.
“Unfortunately, when a device factory-resets, all network settings and data is erased. This means that you will need to set back up your network as you did before, including device names, DNS settings,” Google’s Wifi support team said in a support page.
“We know that this is not necessarily an easy or quick process for many of you with complicated network setups, but unfortunately this is the only way to setup your network again.”
The incident tested Google’s capacity to support customers. Google started looking into the issue at 2:38pm PST on Thursday but customers waited until 7:24pm for instructions on how to get back online again with their own Google credentials. However, Google did provide steps earlier to get online using credentials on the bottom of the device.
Still, many customers were stuck on hold at Google’s support line for nearly an hour only to be told by Google’s Wifi care team not to wait on hold, but rather keep hitting refresh on its support page.
One user wasn’t happy with that suggestion: “So you bricked our OnHub routers with a software update and you will reach out with an explanation when you have one?”