Stuff The Internet Says On Scalability For December 12th, 2014

Hey, it's HighScalability time:

We've had a wee bit of a storm in the bay area.

• 33 million: things in the American Museum of Natural History; 40 million: Target credit cards compromised; 300 Million: Instagram monthly users; 1 billion: Telegram daily messages
• Quotable Quotes:
  • Judge Richard Posner - 7th Circuit Court: I’m shocked at the thought that a
    company would be permitted to manufacture an electronic product that the government
    would not be able to search. 
  • @arclight: Why did it take 2 weeks to write 200 lines of code? Because I had to write 1000 lines of code to learn which 800 to throw away.
  • @aspinall: Beyond the 99th percentile latency can (and often does) increase 1000x @giltene #yow14
  • wmf: Maybe the true genius of AWS is the pricing model that makes expensive look cheap.
  • jacques_chester: queues are not queues. You need to compare delivered apples with lossy oranges.
  • @elastisys: Predictive #autoscaling saves 70% of #cloud costs - @adrianco at @UCC2014_London
  • @recoil: "Synchronous communication is the crystal meth of programming" #yow14 keynote with Thompson & Montgomery
  • kwitcherbichen: Matching the requirements to the system and setting expectations, especially expectations around how and when it goes wrong, is so much more important the benchmarks.
  • @stilkov: This is one of those @unclebobmartin post’s I disagree with almost completely: http://blog.cleancoder.com/uncle-bob/2014/10/01/CleanMicroserviceArchitecture.html …. The deployment model is NOT a detail.
  • @paul_okeeffe: My thought from Amanda Laucher’s #yow14 talk - types are maths, tests are science. They go well together.
  • @peakscale: Yeah, try doing millions of messages per second with unix pipes, I will wait 


• A new category is in play and the race to be the Linux of the Datacenter OS is on! Mesosphere Announces First Data Center OS and $36M in Funding. Then there's Rocket vs. Docker and the Myth of the “Simple, Lightweight Enterprise Platform” and Docker extensions: the new Docker plugins model. It may seem like we know the winners, but not so fast says Brett Slatkin, nobody has won yet: Docker vs. Rocket vs. Vagrant — Still anyone's game. And when money is involved we see the inevitable breaks ups: Why Docker and CoreOS’ split was predictable. 


• I'm asked a lot to recommend a good introduction to building scalable systems. Here you go: Distributed systems for fun and profit. Mikito Takada makes a difficult topic accessible, with a very straightforward style. The chapters: Distributed systems at a high level; Up and down the level of abstraction; Time and order; Replication; Replication: weak consistency model protocols; Further reading and appendix. 


• Remember, in the Game of AI you are the jackpot. Google’s Intelligence Designer. 


• Artisan achieves nearly 5x better performance using Redis Pipelining and Scripting: Redis scripting allows you to build custom commands which can be invoked like any other Redis command. Combinedwith pipelining, this becomes an incredibly powerful capability.


• We're One Step Closer To Superfast Computers That Run On Light Instead Of Electricity: According to researchers at Stanford, roughly 80% of the power used by a computer is lost thanks to the use of these copper wires.


• What is the flip side of technical debt? Working, revenue generating code. The failed economics of our software commons, and what you can about it right now.


• The Economics of Scaling Cassandra: fast for both read and write (~200k ops/s, single digit latency); avoid virtualization; no more than 2 Docker containers per physical machine; more nodes means the less efficient you are performance/price; the bigger the nodes the less efficient you are on performance/price. 


• I'm not sure what you do for fun and enjoyment, but Edward Capriolo builds a Column Family store. And during the process he is explaining what's going on. Looks interesting.


• Varnish goes over the design for their new Massive Storage Engine...uses ‘Least Frequently Used/Least Recently Used Hybrid’ cache eviction algorithm providing smarter selection criteria in the cache...Objects are placed in tiers–the “bottom” tier holds candidates in queue for purging when space is needed...available storage space [split] into segments. When a thread needs to allocate storage it is granted an exclusive lock on the segments...eliminates fragmentation by selectively taking out objects that are placed adjacent to each other.


• Copenhagen first to be enervated with sensors: The system, still in its early stages, has put Copenhagen on the leading edge of a global race to use public outdoor lighting as the backbone of a vast sensory network capable of coordinating a raft of functions and services: whether easing traffic congestion, better predicting where to salt before a snowstorm or, to the alarm of privacy advocates, picking up on suspicious behavior on a busy street corner.


• Have a lot of analytics to store? There's a new database for that InfluxDB: An open-source, distributed, time series database with no external dependencies.


• Just because a site is static doesn't mean it's fast. Speeding Up Octopress by 15x. Load times can still be slow, there can be too much unused javascript, there can be too many unused features, fancy fonts can be used for not particular reason, css may not be minified. 


• Very well said. Adam Wiggins explains My Heroku values: Make it real; Ship it; Do it with style; Intuition-driven balances data-driven; Divide and conqueor; Timing matters; Throw things away; Machete design; Small sharp tools; Put it in the cloud; Results, not politics; Decision-making via ownership, not consensus or authority; Do-ocracy / intrapreneurship; Everything is an experiment; Own up to failure; Gradual rollouts; Design everything; Do less; Question everything; Interfaces matter; Names matter; Maniacal focus on simplicity; CLI 4 Life; Ignore the competition; Write well; Strong opinions, weakly held; Candor; Programming literacy for all; 


• Eric Hammond warns Before You Buy Amazon EC2 (New) Reserved Instances: You should forget what you knew about Reserved Instances and read all the fine print before making any further Reserved Instance purchases. One of the big differences between the old and the new is that you are always committing to spend the entire 1-3 years of cost even if you are not running a matching instance during part of that time. 


• The only thing worse than too few alerts is too many alerts. Which is why StackExchange has built and open sourced Bosun, their monitoring & alerting system. OpenTSDB is used for the time series database. An expression language supports sophisticated alerting conditions. A templating language creates more uniform alerts. Alerts can be tested against history to test their noisiness. Why another such system? Kyle Brandt:  I honestly struggle to boil it down since there are lot of things. But the main result is “Far less alert noise”


• Great discussion of SurgeMQ: MQTT Message Queue @ 750,000 MPS on Hacker News. Where MQTT "is a Client Server publish/subscribe messaging transport protocol. It is light weight, open, simple, and designed so as to be easy to implement." In particular jacques_chester shares some battle scars about how different features impact performance and not everything is just about messages per second.


• If you like exercises in analytic thinking the you might like The Opportunity Cost of War: After adding up the cost of each year of war over time totals to $437.62 trillion in 1990 U.S. dollars. This figure only considers the opportunity cost using the method above; however, it is still astronomical. To put this into perspective, the total cost of the Apollo program was approximately $65.5 billion in 1990 U.S. dollars.


• Good explanation behind the process of Building a Scalable Event Service with Cassandra.


• Probably good general advice. Impact of Shared Storage on Cassandra: Simply put, shared storage cannot keep up with the amount of disk io placed onto a system from Cassandra.  It’s not recommended.  Don’t use shared storage with Cassandra and be happier for it.


• StorageMojo on The 30% solution: The upshot is that most storage capacity will become a 30% gross margin business in the next 10 years. There will always be specialized applications that require high-margin gear, but that won’t be the bulk of the market.


• pg_shard: a sharding extension for PostgreSQL. It shards and replicates your PostgreSQL tables for horizontal scale and high availability. The extension also seamlessly distributes your SQL statements, without requiring any changes to your application.


• Sorenso Puts Amazon SQS to the test. SQS is simple, reliable, cost effective, no setup is required, messages are replicated. SQS has a problem with handling more than 2.3K message insert/sec; not a replacement for Kafka or Amazon Kinesis; it has limitation in message size and retention; it's not a streaming service.


• Looking into Heterogeneity: When Simple is Faster: Cores based on a complex architecture are generally much faster than those based on a simple architecture. On the other hand, cores are more energy-efficient when based on simple architecture. Since the recent mobile environments require both the high performance and low power consumption, HMP (heterogeneous multi-processing) architecture with both the complex cores and the simple cores in a chip has been introduced. In the HMP architecture, complex cores are used to enhance the performance, and the simple cores are used to save energy. However, in contrary to the conventional belief, complex cores do not enhance the performance for all applications, neither do the simple cores save energy for all applications. Consequently, the running application should be carefully considered to fully benefit from the HMP architecture.