Strategy
From a talk by @aysylu22 at QCon London on modern computer science applied to distributed systems in practice. "CS research is timeless." Lessons learned are always pertinent. @aysylu22 #qconlondonMarch 8, 2016 There has been a renaissance in the appreciation of computer science papers as a relevant source of
Paper
Making a system work in one datacenter is hard. Now imagine you move to two datacenters. Now imagine you need to support multiple geographically distributed datacenters. That’s the journey described in another excellent and thought provoking paper from Google: High-Availability at Massive Scale: Building Google’s Data Infrastructure for
Paper
Underutilization and segregation are the classic strategies for ensuring resources are available when work absolutely must get done. Keep a database on its own server so when the load spikes another VM or high priority thread can't interfere with RAM, power, disk, or CPU access. And when you
Paper
It's amazing what you can accomplish these days on a single machine using SSDs and smart design. In the paper FlashGraph: Processing Billion-Node Graphs on an Array of Commodity SSDs they: demonstrate that FlashGraph is able to process graphs with billions of vertices and hundreds of billions of
Paper
"From the dawn of civilization until 2003, humankind generated five exabytes (1 exabytes = 1 billion gigabytes) of data. Now we produce five exabytes every two days and the pace is accelerating." -- Eric Schmidt, Executive Chairman, Google, August 4, 2010. Where are we going to store the deluge
Paper
Joe Beda (@jbeda): Borg paper is finally out. Lots of reasoning for why we made various decisions in #kubernetes. Very exciting. The hints and allusions are over. We now have everything about Google's long rumored Borg project in one iconic Google style paper: Large-scale cluster management at Google
Paper
How will your OLTP database perform if it had to scale up to 1024 cores? Not very well according to this fascinating paper: Staring into the Abyss: An Evaluation of Concurrency Control with One Thousand Cores, where a few intrepid chaos monkeys report the results of their fiendish experiment. The
Paper
With Reactive Systems becoming the new old hotness, it will help to have a thorough grounding in the Actor Model. Here's a good start. Carl Hewitt in Actor Model of Computation: Scalable Robust Information Systems gives a very thorough and relatively concise explanation of the Actor model. Here&
Database
What data structure does your database use? It's not something the typical database user spends much time pondering. Since data structure is destiny, what data structure your database uses is a key point to consider in your selection process. We know CouchDB uses a modified B+ tree. We&
Paper
Do you really need to roll your own? ZooKeeper: Wait-free coordination for Internet-scale systems: The ZooKeeper interface enables a high-performance service implementation. In addition to the wait-free property, ZooKeeper provides a per client guarantee of FIFO execution of requests and linearizability for all requests that change the ZooKeeper state. These
Paper
So how do you knit multiple datacenters and many thousands of phones and other clients into a single cooperating system? Usually you don't. It's too hard. We see nascent attempts in services like Firebase and Parse. SwiftCloud, as described in SwiftCloud: Fault-Tolerant Geo-Replication Integrated all the
Paper
Famous computer scientist John Backus, he's the B in BNF(Backus-Naur form) and the creator of Fortran, gave a Turing Award Lecture titled Can programming be liberated from the von Neumann style?: a functional style and its algebra of programs, that has layed out a division in programming