Strategy

Strategy: Three Techniques to Survive Traffic Surges by Quickly Scaling Your Site

General Chicken

— 2 min read

Matthew Might, as a first responder to a surprise traffic surge on his inexpensive linode hosted blog, took emergency steps that you might find useful in a similar situation:

  1. Find the bottleneck. Reloading the page in firebug showed the first page took 24 seconds to load and after that everything else loaded quickly. In retrospect this burst meant the site was thread limited as the CPU was idle.
  2. Cut image sizes in half with a shell script using ImageMagick's convert. Load time is now 12 seconds.
  3. Turn dynamic content into static content using a static index.html file  copied using the browser's "view source" feature. Load time is now 6 seconds.
  4. Added threads to the Apache configuration file. Load time is now 2 seconds. Crises averted.

Because of this quick thinking and quick action the patient survived to serve pages another day.

And in fine post-mortem tradition some of the future changes are:

  1. Run a cron job to trigger an earlier alert. Email when requests exceed a per second threshold as calculated from a httpd log file. Google analytics did not alert on the problem because the pages didn't get to fully load and thus didn't generate statistics. A linode iPhone app also enables checking server load and bandwidth utilization in real time.
  2. Use AWS as a temporary mirror in times of stress. Given there's no charge until needed it's free insurance. No details on how this is done.
  3. Use lighttpd, a high-performance event-based httpd server, for static content. Serving files is a high-IO, low-CPU activity, so event-based httpd servers aren't crippled artificially by having too few threads.

It's a well written article and worth a read.

Some other ideas from comments made on the post:

  • jasonkester: Never ever scale your blog. It sounds like the author was making the same mistake that pretty much everybody makes: Treating your blog as though it were dynamic content. But it's not. It's static HTML, and you should never have to make any modifications to anything to make it scale.
    • Have your blog export all entries to plain HTML.
    • Move your imagery out to S3/Cloudfront.
  • Outsource comments to Disqus (or others).
  • Use a blog service that may or may not scale.
  • Ditch Apache on a VPS, switch to Nginx (or others).
  • Use mod_rewrite to redirect people to a static version of a page. 

Read more