The Cost of Forward Thinking

September 16, 2009

This blog post is *ancient*, and preserved only for historical record.

In the last two weeks, I’ve seen at least two websites fall off the internet because of a distinct lack of forward planning

Firstly, there was Derren Brown’s blog

After Derren did his “The Events” trick with the lotto balls and dark magic, the number of fans hitting his page daily looking for clues, news, and gossip, caused the server to fall over. It even caused some of the channel 4 servers some traffic troubles (and they’ve got a lot of nodes!)

Derren’s blog was down for at least 2 days, as far as i could see. If his producers/agents/IT manager had said “hey, this stunt might turn out to be popular, let’s move onto a cloud infrastructure, with a CDN cache, we might have to invest a bit of money now, but we’ll have better uptime than if we’re just serving from a single 1U Dedi in a rack” then the site may have remained up and serving for far longer, to endure the wave of traffic generated by the publicity on tv.

The second one of these, was caused tonight by Dragons’ Den Online, a cut-down version of the popular Dragons’ Den format.

The final segment was dedicated to a web startup, introducing Yet Another Social Network for families.. Something about sharing photos, videos, calendars and wishlists Personally, I do all this with Flickr, Google Apps, and Amazon Wishlist.

It was remarked to me at least once, that this could be breaking down the nature of the family unit, because everyone spends their time in front of the computer instead of actually interacting with each other

But I Digress

About 20 minutes ago, I was looking at their site, Family Fridge, and noticed that it winked out of existence as soon as the web address was mentioned.

Yes, they got Slashdotted by the BBC

I’ve seen many a site get taken down by getting a FryTweet, that’s a pretty effective way to kill a webserver, when 50,000+ followers all open the site at once, it’s not good for any website

I suppose there’s that old adage about “no such thing as bad publicity”… I can’t help but apply the same scenario as before “If we spend a little money now, get a cloud computing services infrastructure, then we can use the Dragons’ Den as advertising and get a whole stack of new members in one night” Sure, upgrading the platform isn’t free, but the potential in increased revenue from such a “publicity stunt” is significant, and should be enough to offset the cost of the new infrastructure.

Moreover, I think it proves to some extent that the investment might not be quite so sound

Scalability is something of a buzzword of the times we live and work in, but it’s also very important, the moment you launch a product on twitter or facebook, you’ve instantly got a far wider audience than perhaps you initially anticipated

In my opinion, it looks kinda bad on the developers of this site, that either they never anticipated that this would happen, or they don’t care.

On a technical note, they probably wouldn’t need to go as far as a cloud-computing infrastructure, or even a CDN. Simple page optimisations and front-end caching can make a world of difference to generating a new dynamic page for every single visitor.

Knowing my luck, someone on Twitter or Facebook will pick this up as “Interesting” and i’ll get a hundred requests a second, and my poor overworked hosting account at Streamline will get overwhelmed.

Profile picture

Written by Tom O'Connor, an AWS Community Builder and Senior Security Engineer, with background in DevOps and scalability. You should follow them on Twitter  or Mastodon