Example
Update 2: Stack Overflow Architecture Update - Now At 95 Million Page Views A Month Update: Startup – ASP.NET MVC, Cloud Scale & Deployment shows an interesting alternative approach for a Windows stack using ServerPath/GoGrid for a dedicated database machine, elastic VMs for the front end, and a free
Database
Update 8: Introducing MongoDB by Eliot Horowit. Update 7: The Future of Scalable Databases by Robin Mathew. Update 6: NoSQL : If Only it Was that Easy. BJ Clark lays down the law on which databases are scalable: Tokyo - NO, Redis - NO, Voldemort - YES, MongoDB - Not Yet,
EC2
This tutorial will show you how to use Amazon EC2 and Cloudera's Distribution for Hadoop to run batch jobs for a data intensive web application. During the tutorial, we will perform the following data processing steps.... read more on Cloudera website
Strategy
These are from Laura Thomson of OmniTi: 1. Profile early, profile often. Pick a profiling tool and learn it in and out. 2. Dev-ops cooperation is essential. The most critical difference in organizations that handles crises well. 3. Test on production data. Code behavior (especially performance) is often data driven.
humor
Another fun one... Should I be Worried About Scaling? What happens when the comfortable SQL world of yesterday changes out from under you? You feel like this (it uses some language picante so be advised): Here's a link to the source: http://browsertoolkit.com/fault-tolerance.png A great
Strategy
Aaron Kimball of Cloudera gives a wonderful 23 minute presentation titled Cloudera Hadoop Training: Thinking at Scale Cloudera which talks about "common challenges and general best practices for scaling with your data." As a company Cloudera offers "enterprise-level support to users of Apache Hadoop." Part of
Strategy
Update: Offloading ALL JS Files To Google. Now you can let Google serve all your javascript files. This article tells you how to do it (upload to Google Code Project) and why it's a big win (cheap, fast, caching, parallel downloads, save bandwidth). Don't have a
Strategy
Virtualization offers a lot of benefits, but it also comes with a cost (memory, CPU, network, IO, licensing). If you are in or running a cloud then some form of virtualization may not even be an option. But if you are running your own string of servers you can choose
Example
Update 7: Basecamp, now with more vroom. Basecamp application servers running Ruby code were upgraded and virtualization was removed. The result: A 66 % reduction in the response time while handling multiples of the traffic is beyond what I expected. They still use virtualization (Linux KVM), just less of it now.
Strategy
This post on www.ilovebonnie.net documents some impressive system performance improvements by the addition of Squid Cache (a caching proxy) and APC Cache (opcode cache for PHP). * Apache is able to deliver roughly 700% more requests per second with Squid when serving 1KB and 100KB images. * Server load is
Strategy
Update 8: The Cost of Latency by James Hamilton. James summarizing some latency info from Steve Souder, Greg Linden, and Marissa Mayer. Speed [is] an undervalued and under-discussed asset on the web. Update 7: How do you know when you need more memcache servers?. Dathan Pattishall talks about using memcache
Paper
There have been reports that software engineering is dead. Maybe, like the future, software engineering is simply not evenly distributed? When you read this paper I think you'll agree there is some real engineering going on, it's just that most of the things we need to