IE 11 is not supported. For an optimal experience visit our site on another browser.

News sites knocked out as NYC data center floods

Gizmodo redirect
Tech blog Gizmodo redirected readers to anGizmodo

At 4:21 p.m. ET Monday, the tech blog Gizmodo tweeted that its downed site would "be back soon! There was a data center battery failure after the power went down in Lower Manhattan. Generators powering up." But almost a day later, the site — and its sister sites in the Gawker network — were still limping.

Gawker was not alone. The Huffington Post and BuzzFeed experienced outages and trouble publishing. CNET reported that Livestream, host of the live "Sandycam" video feed, had a "major outage." And MarketWatch, a property of the Wall Street Journal, was " severely disrupted by Hurricane Sandy," according to a post on its Tumblr site.

When you see a website go down, the blame usually falls on hackers or, more often, software bugs. We don't often think of the Web as having a physical existence, but sure enough, Sandy has shown that nature can impact websites with greater force than any team of malicious code wizards can summon up.

The problem was explained by BuzzFeed — not in a post on its own site, but on a BuzzFeed channel on the Tumblr blogging network:

Datagram, the ISP whose Manhattan servers host BuzzFeed, Huffington Post, Gawker, and other sites, has lost power, an official there told us via text this evening."Basement flooded, fuel pump off line — we got people working on it now. 5 feet of water now," the official wrote.

Even Tuesday, Datagram was still struggling to recover. At 11:20 a.m. ET, the company's website reported: "We are still flooded, as soon as the basement is cleared we will be allowed to operate the generator and restore power." Then at 11:53 a.m. came the update: "The building has slowly begun to pump water out of the basement. They are unable to provide an ETA however."

Nationally, the picture was not so grim. Most of the country's top websites "appear to have weathered the storm remarkably well," wrote Aaron Rudger, senior marketing manager of Web performance for Keynote, a mobile and website monitoring and testing service, in an email to NBC News. "Overall performance slowed by about 7 percent, with average availability of 98 percent."

Rudger said that only one of 40 biggest websites tracked by Keynote had an outage in the timeframe of Sandy's impact, job site Monster.com. "That site experienced major failures at around 8 p.m. Pacific Time last night, persisting throughout the evening and into the morning. The site appears to have recovered now."

Gawker founder and publisher Nick Denton "was among Internet honchos who engages a data center in downtown Manhattan, which, unshockingly to anyone who's read a weather report in the last five days, didn't make it," wrote The Awl's Choire Sicha, who added, "The rest of us made data center preparations; even then, some of us went down after midnight."

But given the scope of the outage, it's not clear what quick preparations could have been made to avoid the problems, especially for sites that are hosted in a single physical location. 

"We had plans for a second data center in 2013 which we will now be moving up," Scott Kidder, Gawker's executive director of operations, told NBC News in an email. "We — as other publishers — had counted on Datagram's ability to withstand anticipated natural disasters, which seems to have been misplaced."

Like other sites facing data failures in the wake of Hurricane Sandy, Gawker turned to alternative publishing. (You can now find Gawker sites at updates.gawker.comupdates.gizmodo.com, updates.kotaku.com, updates.jalopnik.com, and so on). Denton urged his staff to turn to Facebook and Twitter as well.

"While we're obviously disappointed with Datagram," said Kidder, "our priority has been getting back online for our readers with an alternate publishing platform, which we've now done with all sites thanks to Tumblr."

"Elements of BuzzFeed’s site and many story pages are back online, thanks to a Content Delivery Network, Akamai, which hosts the content at servers distributed around the world," said BuzzFeed's Tumblr page. "We aren’t able to update the site right now, however," so the site provided its Tumblr, Twitter and Facebook info for readers.

During its outage, The Huffington Post was able to redirect traffic to an alternate server at status.huffingtonpost.com, but appeared to have recovered sooner than the others. 

Re-directing readers to social channels and new Web addresses is a messy proposition, but in today's twitterfied news world, it's not as crazy as it sounds. At least not for BuzzFeed, according the site's tech editor, Matt Buchanan.

"I think BuzzFeed is in a unique position because it's a social publisher at its core — so it's totally natural for us to move to these other media," Buchanan told NBC News. "I think a lot of our readers are totally comfortable following us as natives of the social Web. Also, it helps that we had strong presences on Twitter and Tumblr to begin with."

Despite intermittent Tumblr outages, Buchanan said traffic was "picking up" on BuzzFeed's alternatives, particularly buzzfeedpolitics.tumblr.com.

But getting BuzzFeed's main site back on track has taken more Herculean efforts: It is rebuilding the whole operation on a cloud-based server run by Amazon. "The team have been working non-stop since we went down yesterday," said BuzzFeed's Ben Smith. One of BuzzFeed's New York-based developers who coded through the night for the emergency site transplant did so with a tree smashed through the roof of his home.

Wilson Rothman is the Technology & Science editor at NBC News Digital. Catch up with him on Twitter at @wjrothman, and join our conversation on Facebook.