Stuff The Internet Says On Scalability For April 20, 2012

It's HighScalability Time:

• 100 Billion PVs/Week: Plenty of Fish; 695k TPS: Node.js & VoltDB; 131ms: Response from EC2 EU
• Quotable quotes:
  • Markus Frind:  Scaling is not very fun these days.
  • Mike Krieger: Scaling is like replacing all components on a car while driving it at 100 mph.
  • @jaykreps: We built servers that could handle 100k connections by building servers that could handle 10k connections and waiting a decade.
  • @RichExperiences: Twitter alone generates more than 7 terabytes of data every day, Facebook 10 TB...
  • @HectorESoto: Scalability is about building wider roads, not about building faster cars.” – Steve Swartz
  • @jasongorman: Put our logic in the client and our data in the "cloud"? What does that remind me of? Micrsoft Visual Basic.
  • ‏ @jasobrown: #nasa's use of AWS/cloud is quite similar to netflix. long term storage = s3, spin up pre-make (baked) amis, using VPC for extended firewall
• Netflix never used its $1 million algorithm due to engineering costs. Ease of implementation often trumps absolute measures of performance. That's keeping it simple.
• MySQL in 2012: Report from Percona Live. MySQL is alive and kicking. Nice overview of the conference and description of how Craigslist and Pinterest use MySQL in very different ways. Some of the talk videos are now on-line: Keynote: The MySQL Evolution, Making LAMP a Cloud, The New MySQL Cloud Ecosystem,  What Comes Next, and The Road Ahead for MySQL. 
• If the key to scaling is specialization, then the advantage of Software Defined Networking is it will allow you to build to your exact requirements, shifting spending from CapEx to OpEx, it won't mean cheaper networking. That's what Greg Ferro thinks. Also, Going With The Flow: Google’s Secret Switch To The Next Wave Of Networking.
• Martin Thompson with great advice on the process to adopt when starting on an existing project:  create appropriate load tests and get the profilers running - lock-contention came out as the biggest culprit limiting both latency and throughput; work on the most limiting factor  because when it is removed the list below it can change radically as new pressures are applied - make new lock-free Executor to replace the standard Java implementation - 16X more throughput and the latency histogram has become much more compressed;  Azul Zing had the most best latency profile with virtually no long tail; develop new IPC transport based on shared memory via memory-mapped files in Java; optimise the call stack for message handling to remove any shared dependencies; Do not use locks in the main transaction flow; Never have more threads that need to run than you have cores available; Set affinity of threads to cores; and much more. Also, Beginner's Guide to Hardcore Concurrency. 
• Zookeeper is probably the coolest software you've never used because it's hard to know how to use it correctly. There's a Netflix for that: Introducing Exhibitor - A Supervisor System for Apache ZooKeeper. Exhibitor provides a number of features that make managing a ZooKeeper ensemble much easier: Instance Monitoring, Log Cleanup, Backup/Restore, Cluster-wide Configuration and much more.
• Concurrency is not Parallelism (it's better) - A talk by Rob Pike. That these ideas are so hard to keep separate in a mind might indicate how useful a distinction is being made. Good discussion on G+.
• Living with SQL and NoSQL at Craigslist. Jeremy Zawodny talking about Craigslist uses a variety of data storage systems in its backend systems: in-memory, SQL, and NoSQL. They make use of the filesystem, MongoDB, Memachedb, MySQL, Redis, and Sphinx. Choose the right tool based on what you need to do. In practice they just don't care.
• Distributed Podcast. Excellent podcast with lots of good topics. In particular, an interview with Mike Barker who helped architect the LMAX Disruptor project, a high-performance, ultra-low latency structure for producer/consumer operations.  
• Assessing and improving an approach to delay-tolerant networking. The Bundle Protocol specifies a new way of transmitting data in a complex protocol format that is assembled from different blocks for different purposes. Blocks and header information can be inserted, removed and modified by intermediate nodes.  
• Good comments on Ask For Forgiveness Programming - Or How We'll Program 1000 Cores
  • Shaper_pmp: One can argue, however, that no data structure is ever general purpose, because they all come with inherent assumptions, optimisations or limitations (hash tables' lack of key-ordering, trees' inherent hierarchical structure, etc), big-O characteristics for inserting/retrieving/deleting values, etc. It's possible to view "correctness" not as a moral absolute which must always be striven for, but merely another axis along which data structures may be optimised or sorted...
  • sketerpot: Game code divides, roughly, into three parts. There's graphics code that does the same operations to a huge amount of data in parallel. There's physics and AI code which is not so simply parallel, but heavily numerical. And there's game state code, usually involving a bunch of objects with mutable state interacting with one another. The graphics code has a fundamentally easy-to-exploit form of parallelism. The others are trickier.
• Rackspace keeping it real by running their own cloud on OpenStack. It's still in beta, but there is the promise of a battle tested platform in the future, and that's good for everyone. Though they'll have to move faster to keep up with Amazon.
• Improving Per-Node Efficiency in the Datacenter with New OS Abstractions: Traditional operating system abstractions are ill-suited for high performance and parallel applications, especially on large-scale
  SMP and many-core architectures. We propose four key ideas that help to overcome these limitations. These ideas are built on a philosophy of exposing as much information to applications as possible and giving them the tools necessary to take advantage of that information to run more efficiently.
• I Ain't Afraid of No Downtime: Scaling Continuous Deployment. In the land of mobile does downtime exist? Cody Powell makes the case for no:  in mobile, you get in the habit of trying and retrying everything related to the network, because the coverage can be so spotty.
• The myth of five nines – Why high availability is overrated. Sean Hull making argument that service interruption could be phyric victory for most applications, costing more than than the return. Add testing and monitoring before complicating an architecture beyond managability. Also, Autoscaling MySQL on Amazon EC2 with lots of good advice on your scaling options on EC2. Did you know by using RDS that you do not have access to the slow query log?
• What does node.js stack look like? Here's what Beyond Fog uses: Web Framework: Express; Templating: Jade; Auth: Everyauth; Database:  MongoDB; Session Store: connect-mongo; Email: Sendgrid; Control-Flow: Step; and many more. 
• Extracting Performance Scalabilty Metrics from TCP. Baron Schwartz looks at how to capture TCP information, create metrics like index of dispersion; how to use the information to find performance problems, like unexpected locks; findig what causes non-linear scalability; and how to understand the Universal Scaling Law.
• Great interview with Jonathan Ellis on the deep integration of Solr with Cassandra, it looks like a really strong search solution: you insert your data once, and access it via Cassandra, Solr, or Hadoop. DSE offers elastic workload partitioning: your analytics jobs run against their own copies of the data, kept in sync by Cassandra replication, so they don’t interfere with your real time queries.
• Dumb interview questions that could get you hired at Google.