Election night is a big traffic boost for news and social sites. Yahoo expects up to 400 million page views on Election Day. Data Center Knowledge has an excellent article how various sites are preparing to handle spikes in election night traffic. Some interesting bits:
Prepare ahead. Don't wait to handle spikes, plan and prepare before the blessed event.
Use a CDN. Daily Kos puts images on a CDN, but the dynamic nature of their site means the can't use CDN for their other content.
Scale up. Daily Kos "to handle the traffic better, we moved to a cluster of six quad core Xeons with 8GB RAM for webheads that all boot off a central NFS (Network File System) root, with the capability of adding more webheads as needed,” . They also "added two 16GB eight-core Xeons and a 6×73GB RAID-10 array for database files running a MySQL master/slave setup."
Add Cache. Daily Kos added 1GB instances memcached to each webhead.
Change Caching Strategy. Daily Kos puts fully rendered pages into memcached.
Change Serving Strategy. Daily Kos directly serves cached pages from memcached directly to anonymous users from lighttpd running as the front end proxy. The moves a lot of work off the backend and distributes work on the new hefty webheads. Site performance has improved greatly.
Add Capacity. Limelight expanded its network capacity to over 2 Terabytes per second.
Tonight is a big night for a lot of sites. It's interesting to see how some are responding to the challenge. A lot of what they are doing will work for you too.
Comments
Re: How Sites are Scaling Up for the Election Night Crush
Is there a good percentage increase figure to help users determine how much their system will need to scale for "the big day"?
Re: How Sites are Scaling Up for the Election Night Crush
Trackback
Re: How Sites are Scaling Up for the Election Night Crush
Re: Daily Kos directly serves cached pages from memcached directly to anonymous users from lighttpd running as the front end proxy.
How are they actually doing this? I would assume a script accesses memcached and pulls the content, but the way that is written it makes it sound like something more magical is happening. Please explain!
How much? Lighttpd
We are int he news bizz and saw 30% increased... millage will vary. Always expect 50%
Now the lighttpd to memcache thing is cool - facebook is doing the same thing. Everyone says its custom code in lighttpd to do this - I would like to see a module or native support.. we may consider doing something like this as well...
If anyone has a working example they would like to share.. let me know.
Re: How Sites are Scaling Up for the Election Night Crush
Looks like the guys over at "Daily Kos" posted their solution, mod_mcpage.
http://www.dailykos.com/story/2008/9/29/13317/3981/898/614381
It is a patch to lighttpd, that allows serving of pre-rendered pages from memcached.
Post new comment