« Stuff The Internet Says On Scalability For July 8, 2011 | Main | 11 Common Web Use Cases Solved in Redis »

Myth: Google Uses Server Farms So You Should Too - Resurrection of the Big-Ass Machines

For a long epoch there was a strategy of scaling up by making ever bigger super computers. I had the pleasure of programming on a few large massively multi-processor machines from SGI and DEC. Beautiful, highly specialized machines that were very expensive. These met the double-tap extinction event of Moore's law and a Google inspired era of commodity machine based clusters and extreme software parallelism. Has the tide turned? Does it now make more sense to use big machines instead of clusters?

In Big-Ass Servers™ and the myths of clusters in bioinformatics, Jerm makes the case that for bionformatics, it's more cost effective to buy a Big-Ass Server instead of using a cluster of machines and a lot of specialized parallel programming techniques. It's a classic scale-up argument that has been made more attractive by the recent development of relatively inexpensive large machines. SeaMicro has developed a 512 core machine. Dell has a new 96 core server. Supermicro has 48 core machines. These are new options in the scale-up game that have not been available before and could influence your architecture choice.

Jerm's reasoning for preferring big-ass servers is:

  • The development of multicore/multiprocessor machines and memory capacity has outpaced the speed of networks. NGS analyses tends to be more memory-bound and IO-bound rather than CPU-bound, so relying on a cluster of smaller machines can quickly overwhelm a network.
  • NGS has forced the number of high-performance applications from BLAST and protein structure prediction to doing dozens of different little analyses, with tools that change on a monthly basis, or are homegrown to deal with special circumstances. There isn't time or ability to write each of these for parallel architectures.

Jerm then goes on to tackle several myths about why a cluster is not necessarily the best solution, well worth reading. The key seems to be the type of problem you are solving. For Google solving search, investing in a large permanent cluster to do one thing very well makes a lot of sense. For someone trying to solve a particular problem and then move on to the next problem, investing a lot of time in writing a parallelized solution doesn't make as much sense. It makes more sense to find a big-ass machine, program simply, and let it rip. 

If agility and flexibility are key then clusters may not be right tool for your big data or big computation problem. It's interesting how this right tool for the right job idea keeps popping up and how technological innovation brings different options in and out of fashion.  Now might be an inflection point in tool choice with the rise of the new big machines.  

Reader Comments (5)

The SeaMicro box is not a single (cache-coherent) system: it's a bunch of separate systems linked by SeaMicro's own interconnect. This means that you _cannot_ write simple code expecting it to run on 512 CPUs managed by a single Linux image; you still need to write code distributed over 256 different machines in SeaMicro's case, and worry about network I/O, fault-tolerance, etc..

July 7, 2011 | Unregistered Commenterx

Just be damn sure you aren't EVER going to need to scale into a parallel networked architecture, or your site will die before it can be rewritten. Seems to me planning for that eventuality saves a lot of bacon.

July 7, 2011 | Unregistered CommenterDan Shick

@dan not every scaling problem is to do with scaling a website.

July 8, 2011 | Unregistered CommenterAndy

"The key seems to be the type of problem you are solving."

Absolutely and also how you expect your problem to evolve over time. In this particular case, there's a built-in assumption that all the data required can be crammed into one big memory on one box. Although, as another reader points out, at least in the case of Sea Micro this isn't relevant as it isn't either a high-core count NUMA or SMP box:

"The SeaMicro SM10000-64 is the First Server Purpose Built for Scale Out Workloads
Designed to replace 40 1 RU Dual Socket Quad Core servers, the SM10000-64 integrates 512 Intel Atom low power cores (256 Dual Core Intel x86-64 processors), top of rack Ethernet switching, server management, and application
load balancing in a single 10 RU “plug and play” standards-based server. The SM10000-64 uses 1/4 the power and takes 1/4 the space of the best in class volume server without requiring any modifications to existing software."

Note also the Atom processors - perhaps not the CPU horsepower one might have been wishing for.

This is also interesting:

"Myth: Our development setup should mimic our production setup

An experimental computing structure with a BAS™ allows for researchers to freely explore big data without having to think about how to divide it efficiently. If an experiment is successful and there is the need to scale-up to a clinical or industrial platform, that can happen later."

Two points:

(1) At least for web operations, if you have to do work in a development environment, having something that is a scale unit of production is immensely useful. It makes predicting performance/capacity requirements easier and allows for authentic failure testing and the like.

(2) I think the vast majority of evidence out there suggests that one must spend a large amount of time tuning/optimising code to scale on SMP/NUMA boxes if the workload is anything other than embarrassingly parallel (i.e. no co-ordination and thus locking or similar is required). Should the workloads that Jerm encounters not be embarrassingly parallel they will hit scale out problems requiring substantial revision of code. Something that one would always prefer to account for early than late for anything but the smallest codebase.

July 8, 2011 | Unregistered CommenterDan Creswell

Hey. I don't know anything about computers or anything, but I wanted you to know that the phrase "double-tap extinction event" is awesome. Well played.

July 8, 2011 | Unregistered CommenterBodine Wilson

PostPost a New Comment

Enter your information below to add a new comment.
Author Email (optional):
Author URL (optional):
Some HTML allowed: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <code> <em> <i> <strike> <strong>