Some of you might have tried unsuccessfully to read us on Tuesday.
Between the hours of 4:00 pm and 8:00 pm (US, Eastern Standard Time) our blog host, TypePad and many other services such as Red Envelope and CraigsList got shut down by a Power Outage in San Francisco.
I thought it would be useful to share this piece of information with readers outside the US who could have wondered about the reasons for this ‘blackout’.
Usually things work smoothly until a glitch happens.
Nothing is perfect!
Should the soundtrack for this include Gary Numan ‘Are Friends Electric‘, Orchestral Maneuvres in The Dark ‘Electricity’, Jimi Hendrix ‘Electric Ladyland‘ and Kraftwerk ‘Electric Cafe‘
Related stories: Off the Electric Grid again yesterday
There are no surprises here. San Francisco’s Mission St. Substation feeds half a dozen significant datacenters (365main, Level3, Coloserve, 400 Mission, and 650 Townend) and has suffered 3 serious outages in the past 7 years. California itself had 2 straight summers of rolling blackouts, which only subsided thanks to the dot-com crash. California is running out of duct-tape.
365main, usually runs a good operation, and is one of the best datacenters in California.. However, it’s also the most expensive datacenter in California, and should have a better track record than it’s lower-cost competitors like 200 Paul and Coloserv.
In May, 2007 we moved our infrastructure out of 365, off of California’s cancerous power grid, and into a more reliable, greener, and cheaper grid.. Yeah, we moved to Seattle. This was the best decision we ever made.
Most of our experience with 365 was extremely positive, however pricing, and power density problems forced us to move. I can’t list all of the good things 365main did, but here’s a list of 365’s power problems as we experienced them:
In April, 2005 365main had an outage that affected all customers for 50 minutes due to a failed EPO valve. 365 handled that outage spectacularly, claling all of their customers within 15 minutes of the outage.
In February, 2006 365main experienced a partial outage for 3 seconds that only affected some customers, but caused problems in their Telco spine, affecting connectivity.
In October, 2006 365main had a backup generator fail, but supposedly no customers were directly affected, but customers were not allowed to enter the building between 3:29 PM and 4:40 PM.