hot links

Stuff The Internet Says On Scalability For May 2nd, 2014

High Scalability

High Scalability

— 7 min read

Hey, it's HighScalability time:


Google's new POWER8 server motherboard

  • 1 trillion: number of scents your nose can smell; millions of square feet: sprawling new server farms
  • Quotable Quotes:
    • Gideon Lewis-Kraus: As the engineer and writer Alex Payne put it, these startups represent “the field offices of a large distributed workforce assembled by venture capitalists and their associate institutions,” doing low-overhead, low-risk R&D for five corporate giants. In such a system, the real disillusionment isn’t the discovery that you’re unlikely to become a billionaire; it’s the realization that your feeling of autonomy is a fantasy, and that the vast majority of you have been set up to fail by design.
    • @aphyr: I was already sold on immutability, pure functions, combinators, etc. What forced me out of Haskell was the impenetrable, haphazard docs.
    • @monicalent: Going to be migrating away from Rackspace to save some cash. Recommendations? Both Digital Ocean and Linode are half the price for 1GB RAM.
    • Linus Torvalds: So while I'd love for 'make' to be super-efficient, at the same time I'd much rather optimize the kernel to do what make needs really well, and have CPU's that don't take too long either.
    • Joe Landman: when the networking revolution comes, the cheap switches will be the first ones against the wall
    • @etherealmind: Buying public cloud can say I can't afford a house so I'll buy a tent. Because that works just as well, right ?
    • Neil DeGrasse Tyson: The act of doing it perfectly is the measure of it going unnoticed.[....] when it's done perfectly it goes unnoticed or, at best, it's just taken for granted.
  • How we scaled Freshdesk (Part I) – Before Sharding. Requests per week boomed from 2 million to 65 million. They scaled vertically for as long as they could to handle the increased load. Increasing RAM, CPU and I/O. First using read slaves for their heavily read weighted traffic, then assigning queries to particular slaves. Writes still needed to scale, so they turned on MySQL partitioning. Then caching of objects and html partials. They also used different storage engines for different functions. RedShift for analytics and data mining. Redis for state and as a job queue. But in the end they had to embrace the shard.

  • This took some guts. Fouresquare uses data driven decision making to decide to deportalze their app: We looked at the session analysis and saw that only 1 in 20 sessions had both social and discovery. Why not actually just split those apart, because 19 out of 20 times, tapping on one icon or the other, you have satisfied your need completely.

  • The blockchain story is bullshit: Looking at the blockchain from a realist’s standpoint, it is not obvious that there is a need for a worse-performing database, that an unregulated oligarchy has disproportionate power over, that isn’t improved with administrator arbitration. It looks like a technology looking for a problem to solve, rather than a technology created to solve a problem.

  • How Pinterest handles 20 billion events/day with Apache Kafka and Secor. Pinterest is open sourcing Secor, a zero data loss log persistence service whose initial use case was to save logs produced by our monetization pipeline. Secor persists Kafka logs to long-term storage such as Amazon S3. It’s not affected by S3’s weak eventual consistency model, incurs no data loss, scales horizontally, and optionally partitions data based on date.

  • Peter Lawrey on Writing and testing high frequency trading engines in java. Looks at low latency trading, thread affinity, lock free code, ultra low garbage and low latency persistence. Good stuff.

  • Large scale testing is always fun. It often can take more effort than the original product itself. Here's how you go about Testing Cassandra 1000 Nodes at a Time. And it's not just the product that gets tested, but all the services inbetween. Good on them for doing it.

  • A lot of people including me were wrong wrong wrong. But who even knew about app-installs when Facebook IPO'ed? Thoughts on Facebook Q1 2014 earnings:  like Google Facebook is now a company that derives over 90% of its revenue from ads, so that’s where we’ll focus. The most striking thing about Facebook’s past year or so is the amazingly rapid shift of growth from desktop to mobile advertising.

  • This is amazing...Video: Formula 1 Pit Stops 1950 & Today… a Huge Difference. It's astonishing the improvements that can be made to processes once people start thinking of them as a things-in-themselves.

  • The new hotness won't always treat you nice. Why VividCortex Uses MySQL: MySQL is Mature and Proven; MySQL Has a Flourishing Ecosystem; MySQL is Excellent, Good, or Usable For Many Things; MySQL is High-Performance and Scalable; MySQL Is Reliable.

  • Preach it brother. Jepsen: Testing the Partition Tolerance of PostgreSQL, Redis, MongoDB and Riak: First, clients are an important part of the distributed system, not objective observers. Consider extending consistency algorithms through the boundaries of your systems: hand TCP clients ETags or vector clocks, or extend CRDTs to the browser.

  • CoreOS is a minimal Linux-based operating system aimed at large-scale server deployments. Can it work for you? Marcel de Graaf is a mad scientist Experimenting with CoreOS, confd, etcd, fleet, and CloudFormation try to find out: I’m extremely impressed by CoreOS and the tooling ecosystem around it. I wouldn’t be surprised if this is going to be the next big step in setting up highly available and scalable web applications.

  • So you add slaves to your database hoping to fix problems and you get another problem in return, replication lag. How to identify and cure MySQL replication slave lag is a very helpful article on the subject. Just don't tell Daenerys Targaryen.

  • Good design advice when designing your own process infrastructure. Differences between nanomsg and ZeroMQ: POSIX Compliance, implemented in C, Pluggable Transports and Protocols, better threading, organization around state machines, IOCP Support, Level-triggered Polling, and much more. Though a question still burns. Is nano really smaller than zero?

  • Russian doll virtualization is becoming a thing. Multi-node OpenStack RDO IceHouse on AWS/EC2 and Google. Using double nested virtualization, with 4 levels involved: Amazon runs Xen (bare metal OS), We run our hypervisor on top of Amazon (first level guest), Customer runs OpenStack compute node (second level guest, first level of nested), Instance runs on Openstack (third level guest, second level nesting).

  • Now available videos from Craft 2014 - Software Craftsmanship Matters - in beautiful Budapest. To see all the videos make sure to click on the room links near the top.

  • CloudFactory talks about Engineering breakthroughs continue to help us scale: We decided to decouple this task dispatcher from our core platform and use Redis (RAM database) as our backend technology. Decoupling this way helped us to make it more cohesive and focus on its sole job i.e. task dispatching. This also made it very light-weight and blazingly fast.

  • Slides are available fron NoSQL Matters @ Cologne, Germany.

  • Slides for GopherCon 2014 are now available.

  • When ideology trumps pragmatic design: I am not claiming Apple is not long for this world. They make great kit. But at the same time, like all other companies, sometimes they mess up. And do so in profound ways. Ways that completely open the door to competition, and effectively exclude them from playing there. I hope it was unintentional. They need to rethink the no-PCIe card design. Cool doesn’t matter for squat if you can’t do your work on it.

  • Nice detailed article on how to calculate running totals in SQL. So who needs NoSQL?

  • Creating a static blog is like making your own light saber. Blogging With Jekyll and Linode Part 1: Jekyll: This is the first post in a multi-part series on creating a new blog from scratch in Jekyll and hosting it yourself with Linode and nginx.

  • StorageMojo: The biggest surprise of the Hacker News comments was how reductionist most views of the issue were. Cheap storage? Powered down disks. But power isn’t the major driver of cost at Internet scale. 

  • Yammer manages to move to weekly release schedules for mobile apps: Projects at Yammer follow a rule of thumb: 2 to 10 people, 2 to 10 weeks. We like to run a lot of concurrent smaller projects, not a few big monoliths. Today, 80% of our iOS users are running the latest version of app 2 days after it is released. As a team we agreed that it’s no longer ok for a project to knowingly temporarily regress the master branch. And we’re steadily increasing our test coverage.

  • Scalability in a Reactive World: Two things really matter: Maximising locality reference making sure data stays local to its processing context is extremely important. Contention is the second scalability killer e.g. overuse of synchronous locks. To truly scale we also need a way of adding resources to our system, scale out is about managing elasticity, adding nodes or resources. 

  • 6 Distributed File Systems For Big Data: HDFS, Quantcast File System, Ceph, Lustre, GlusterFS, PVFS.

  • The Scalable Commutativity Rule: Designing Scalable Software for Multicore Processors: Can scalability opportunities be identified even before any implementation exists, simply by considering interface specifications? To answer these questions this paper introduces the following rule:Whenever interface operations com-mute, they can be implemented in a way that scales.

  • Lightweight Contention Management forEcient Compare-and-Swap Operations: Our performance evaluation establishes that lightweight contention management support can greatly improve performance under medium and high contention levels while typically incurring only small over- head when contention is low.

  • Pinocchio: Nearly Practical Verifiable Computation: To this end, we introduce Pinocchio, a built system for efficiently verifying general computations while relying only on cryptographic assumptions. With Pinocchio, the client creates a public evaluation key to describe her computation; this setup is proportional to evaluating the computation once. The worker then evaluates the computation on a particular input and uses the evaluation key to produce a proof of correctness. 

Read more