Stuff The Internet Says On Scalability For August 17th, 2018

Hey, it's HighScalability time:


The amazing Zoomable Universe from 10^27 meters—about 93 billion light-years—down to the subatomic realm, at 10^-35 meters.


Do you like this sort of Stuff? Please lend me your support on Patreon. It would mean a great deal to me. And if you know anyone looking for a simple book that uses lots of pictures and lots of examples to explain the cloud, then please recommend my new book: Explain the Cloud Like I'm 10. They'll love you even more.


  • 2.24x10^32T: joules needed by the Death Star to obliterate Alderaan, which would liquify everyone in the Death Star; 13 of 25: highest paying jobs are in tech; 70,000+: paid Slack workspaces; 13: hours ave american sits; $13.5 million: lost in ATM malware hack; $1.5 billion: cryptocurrency gambling ring busted in China; $8.5B: Auto, IoT, Security startups; 10x: infosec M&A; 1,000: horsepower needed to fly a jet suit; 30% Google's energy savings from AI control of datacenters;

  • Quotable Quotes:
    • The Jury Is In: From the security point of view, the monolithic OS design is flawed and a root cause of the majority of compromises. It is time for the world to move to an OS structure appropriate for 21st century security requirements.
    • @coryodaniel: Rewrote an #AWS APIGateway & #lambda service that was costing us about $16000 / month in #elixir. Its running in 3 nodes that cost us about $150 / month. 12 million requests / hour with sub-second latency, ~300GB of throughput / day. #myelixirstatus !#Serverless...No it’s not Serverless anymore it’s running in a few containers on a kubernetes cluster
    • @cablelounger: OH: To use AWS offerings, you really need in-house dev-ops expertise vs GCP, they make dev ops transparent to you     I've a lot of experience with AWS and mostly agree with the first point. I haven't really used GCP in earnest. I'd love to hear experiences from people who have?
    • @allspaw: engineer: “Unless you’re familiar with Lamport, Brewer, Fox, Armstrong, Stonebraker, Parker, Shapiro...(and others) you don’t know distributed systems.” also engineer: “I read ‘Thinking Fast and Slow’ therefore I know cognitive psychology and decision-making theory.”
    • alankay1: To summarize here, I said I love "Rocky's Boots", and I love the basic idea of "Robot Odyssey", but for end-users, using simple logic gates to program multiple robots in a cooperative strategy game blows up too much complexity for very little utility. A much better way to do this would be to make a "next Logo" that would allow game players to make the AI brains needed by the robots. So what I actually said, is that doing it the way you are doing it will wind up with a game that is nxot successful or very playable. Just why they misunderstood what I said is a bit of a mystery, because I spelled out what could be really good for the game (and way ahead of what other games were doing). And of course it would work on an Apple II and other 8 bit micros (Logo ran nicely on them, etc.)
    • Michael Malone: Nolan was the first guy to look at Moore’s law and say to himself: You know what? When logic and memory chips get to be under ten bucks I can take these big games and shove them into a pinball machine.
    • @hichaelmart: To be honest, I think the main lesson from this is that API Gateway is expensive – 100% agree. We have a GAE app doing a very similar thing, billions of impressions/mth – and *much* cheaper than if it were on API Gateway.
    • Keep on reading for many more quotes hot off the internet. You'll be a better person.
Don't miss all that the Internet has to say on Scalability, click below and become eventually consistent with all scalability knowledge (which means this post has many more items to read so please keep on reading)...

Click to read more ...


What do you believe now that you didn't five years ago?



Decentralized systems will continue to lose to centralized systems until there's a driver requiring decentralization to deliver a clearly superior consumer experience. Unfortunately, that may not happen for quite some time.

I say unfortunately because ten years ago, even five years ago, I still believed decentralization would win. Why? For all the idealistic technical reasons I laid out long ago in Building Super Scalable Systems: Blade Runner Meets Autonomic Computing In The Ambient Cloud.

While the internet and the web are inherently decentralized, mainstream applications built on top do not have to be. Typically, applications today—Facebook, Salesforce, Google, Spotify, etc.—are all centralized.

That wasn't always the case. In the early days of the internet the internet was protocol driven, decentralized, and often distributed—FTP (1971), Telnet (<1973), FINGER (1971/1977),  TCP/IP (1974), UUCP (late 1970s) NNTP (1986), DNS (1983), SMTP (1982), IRC(1988), HTTP(1990), Tor (mid-1990s), Napster(1999), and XMPP(1999).

We do have new decentalized services: Bitcoin(2009), Minecraft(2009), Ethereum(2104), IPFS(2015), Mastadon(2016), and PeerTube(2018). We're still waiting on Pied Piper to deliver the decentralized internet

On an evolutionary timeline decentralized systems are neanderthals; centralized systems are the humans. Neanderthals came first. Humans may have interbred with neanderthals, humans may have even killed off the neanderthals, but there's no doubt humans outlasted the neanderthals.

The reason why decentralization came first is clear from a picture of the very first ARPA (Advanced Research Projects Agency) network, which later evolved into the internet we know and sometimes love today:

Click to read more ...


Stuff The Internet Says On Scalability For August 10th, 2018

Hey, it's HighScalability time (out Thur-Fri, so we're going early):


London Maker Faire 1851—The Great Exhibition—100,000 objects, displayed along more than 10 miles, by over 15,000 contributors.


Do you like this sort of Stuff? Please lend me your support on Patreon. It would mean a great deal to me. And if you know anyone looking for a simple book that uses lots of pictures and lots of examples to explain the cloud, then please recommend my new book: Explain the Cloud Like I'm 10. They'll love you even more.


  • 90%: accuracy predicting gender from retinal image; $1 billion: Ebay sales per quarter from AI; $78 billion: global AI software market by 2025; $75m: penalty for botched SAP upgrade; 35 million: m^3 of mud dredged out of the Dutch waterways; 138 terabytes: memory per square inch; 500 million: Uber metrics per second; 22x: new faster JSON Sparser; 

  • Quotable Quotes:
    • @IanColdwater: The JIRA tickets will continue until morale improves
    • @david_perell: Three crazy stats from @mikedariano’s newsletter. 1. People watch more Minecraft hours than the NBA, NHL, NFL, and MLB combined.  2. Only 26 countries have more people than PewDiePie has subscribers.  3. Only 20% of YouTube’s traffic is from the United States. 
    • Charlie Demerjian: Why does SemiAccurate say that Intel knows? We have seen their internal documents that show exactly how frightened the company is. The documents go into specifics we don’t feel are appropriate to discuss publicly but there is one thing we can say, Intel knows their position. One of the documents says in no uncertain terms that the company understands they will not be competitive in the server market until AFTER Sapphire Rapids, the 2022 server part. AMD has a clear run in Intel’s core market for at least 4 years.
    • @ScottMcGready: Can we just take a moment to remember that one company I worked for backed up their stuff on tapes religiously- all tapes sent to a warehouse nightly. Years later someone tested a tape... turns out nothing had been written... ever. We had a (paid) warehouse full of empty tapes
    • Uber: Since 2016, Uber has added several new lines of business to its platform, including Uber Eats, Uber Freight, and Jump Bikes. Now, we complete over 15 million trips a day, with over 75 million monthly active riders. In the last eight years, the company has grown from a small startup to 18,000 employees across the globe.
    • @mims: Here is a super important thing that we don't talk about enough: Almost all of the increase in income inequality from 1978 to the present can be accounted for by the difference in wages between top performing firms and everyone else. And now we have some idea what's driving unequal growth in productivity of top-performing firms -- it's how they build and use *their own, proprietary software and other IT/technology
    • @taotetek: Distributed systems tip: Write your system without any queues first. You might find you don't need queues. If you end up needing queues, the retry and reliability code you wrote in order to function without queues will still make your system more reliable.
    • @theburningmonk: I think the visual flow is sometimes under-appreciated - our app support team can easily look at it and figure out what went wrong without knowing ins & outs of implementation details. I can also show the diagram to a product person and he/she would get it as well
    • John Mark: It’s time to understand something about open source software development: it is not going to save us. Using or developing more open source software is not going to improve anyone’s lives. Developing open source software is not a public good. It’s not going to result in a fairer or more equitable society. In fact, as currently structured, open source development is part of the problem. 
    • There are a few more quotes. Don't miss them.
Don't miss all that the Internet has to say on Scalability, click below and become eventually consistent with all scalability knowledge (which means this post has many more items to read so please keep on reading)...

Click to read more ...


Case Study: Pokémon GO on Google Cloud Load Balancing


There are a lot of cool nuggets in Google's New Book: The Site Reliability Workbook. If you haven't put it on your reading list, here's a tantalizing excerpt from CHAPTER 11 Managing Load by Cooper Bethea, Gráinne Sheerin, Jennifer Mace, and Ruth King with Gary Luo and Gary O’Connor.


Niantic launched Pokémon GO in the summer of 2016. It was the first new Pokémon game in years, the first official Pokémon smartphone game, and Niantic’s first project in concert with a major entertainment company. The game was a runaway hit and more popular than anyone expected—that summer you’d regularly see players gathering to duel around landmarks that were Pokémon Gyms in the virtual world.

Pokémon GO’s success greatly exceeded the expectations of the Niantic engineering team. Prior to launch, they load-tested their software stack to process up to 5x their most optimistic traffic estimates. The actual launch requests per second (RPS) rate was nearly 50x that estimate—enough to present a scaling challenge for nearly any software stack. To further complicate the matter, the world of Pokémon GO is highly interactive and globally shared among its users. All players in a given area see the same view of the game world and interact with each other inside that world. This requires that the game produce and distribute near-real-time updates to a state shared by all participants.

Scaling the game to 50x more users required a truly impressive effort from the Niantic engineering team. In addition, many engineers across Google provided their assis‐ tance in scaling the service for a successful launch. Within two days of migrating to GCLB, the Pokemon GO app became the single largest GCLB service, easily on par with the other top 10 GCLB services.

As shown in Figure 11-5, when it launched, Pokémon GO used Google’s regional Network Load Balancer (NLB) to load-balance ingress traffic across a Kubernetes cluster. Each cluster contained pods of Nginx instances, which served as Layer 7 reverse proxies that terminated SSL, buffered HTTP requests, and performed routing and load balancing across pods of application server backends.

Figure 11-5. Pokémon GO (pre-GCLB)

Click to read more ...


Sponsored Post: NationBuilder, Twitch, InMemory.Net, Triplebyte, Etleap, Scalyr, MemSQL

Who's Hiring? 

  • NationBuilder — if you’re a systems engineer, SRE or DevOps focused developer and have been looking for a place where you can help other people while still working in tech? We can give that opportunity. Please apply here

  • Twitch's commerce team in San Francisco is looking to hire senior developers to keep up with rapidly increasing demand for our Subscriptions and Payment platform. Engineers will be tasked with building new products and features to solve business and ecommerce challenges as we're dealing with engaging problems at a massive scale and will create solutions that impact millions of people around the world. Apply here

  • Triplebyte lets exceptional software engineers skip screening steps at hundreds of top tech companies like Apple, Dropbox, Mixpanel, and Instacart. Make your job search O(1), not O(n). Apply here.

  • Need excellent people? Advertise your job here! 

Fun and Informative Events

  • Advertise your event here!

Cool Products and Services

  • InMemory.Net provides a Dot Net native in memory database for analysing large amounts of data. It runs natively on .Net, and provides a native .Net, COM & ODBC apis for integration. It also has an easy to use language for importing data, and supports standard SQL for querying data. http://InMemory.Net
  • For heads of IT/Engineering responsible for building an analytics infrastructure, Etleap is an ETL solution for creating perfect data pipelines from day one. Unlike older enterprise solutions, Etleap doesn’t require extensive engineering work to set up, maintain, and scale. It automates most ETL setup and maintenance work, and simplifies the rest into 10-minute tasks that analysts can own. Read stories from customers like Okta and PagerDuty, or try Etleap yourself.

  • Scalyr is a lightning-fast log management and operational data platform.  It's a tool (actually, multiple tools) that your entire team will love.  Get visibility into your production issues without juggling multiple tabs and different services -- all of your logs, server metrics and alerts are in your browser and at your fingertips. .  Loved and used by teams at Codecademy, ReturnPath, Grab, and InsideSales. Learn more today or see why Scalyr is a great alternative to Splunk.

  • MemSQL envisions a world of adaptable databases and flexible data workloads - your data anywhere in real time. Today, global enterprises use MemSQL as a real-time data warehouse to cost-effectively ingest data and produce industry-leading time to insight. MemSQL works in any cloud, on-premises, or as a managed service. Start a free 30 day trial here:

  • Advertise your product or service here!

If you are interested in a sponsored post for an event, job, or product, please contact us for more information.

Make Your Job Search O(1) — not O(n)

Triplebyte is unique because they're a team of engineers running their own centralized technical assessment. Companies like Apple, Dropbox, Mixpanel, and Instacart now let Triplebyte-recommended engineers skip their own screening steps.

We found that High Scalability readers are about 80% more likely to be in the top bracket of engineering skill.

Take Triplebyte's multiple-choice quiz (system design and coding questions) to see if they can help you scale your career faster.

The Solution to Your Operational Diagnostics Woes

Scalyr gives you instant visibility of your production systems, helping you turn chaotic logs and system metrics into actionable data at interactive speeds. Don't be limited by the slow and narrow capabilities of traditional log monitoring tools. View and analyze all your logs and system metrics from multiple sources in one place. Get enterprise-grade functionality with sane pricing and insane performance. Learn more today

If you are interested in a sponsored post for an event, job, or product, please contact us for more information.


Stuff The Internet Says On Scalability For August 3rd, 2018

Hey, it's HighScalability time:


Everything starts with Doug Engelbart — Jane Metcalfe.
It was the very first time (1968) the world had ever seen a mouse, seen outline processing, seen hypertext, seen mixed text and graphics, seen real-time video conferencing. — Doug Engelbart (Valley of Genius).
ARPA funded the demo at a cost of $1 million. Most importantly? It was the first use of a todo list as an example. A tradition unlike any other.


Do you like this sort of Stuff? Please lend me your support on Patreon. It would mean a great deal to me. And if you know anyone looking for a simple book that uses lots of pictures and lots of examples to explain the cloud, then please recommend my new book: Explain the Cloud Like I'm 10. They'll love you even more.


  • $1 trillion: Apple; 45: mean founder age for fastest growing new ventures; 1 trillion: files created by Trinity in 2 minutes;  $3.93 billion: Baidu's AI driven quarterly revenues; 13.7: microsecond error in GPS timestamps for nearly 14 hours; 240 million: Apple CarPlay cars by 2023; 99%: confidence setting to use when face matching politicians to criminals; 31%: Apple services revenue increase to ~$10B; 4 billion: containers Google launches each week; 0,1,2: likely per person mutations on a coding gene; 1,000x: solid-state memory density; $10 billion: Pentagon cloud contract up for bid; 661Tbps: through a single optical fiber; $49bn: last 6 months of M&A; $1.1 trillion: projected 2040 space economy revenue; 

  • Quotable Quote:
    • @BrianRoemmele: This shift away from affiliate compensation for IOS app sales will have a rather large impact on niche apps. Many are high ticket and require rather involved promotion and support by third parties that were partly compensated by the affiliate compensation.
    • Paco Nathan: Frankly, I’d feel a lot more comfortable sending my kids off to school in a self-driving bus if the machine learning models hadn’t been trained solely by Google’s proprietary data. Instead, let’s get every possible edge case understood by mingling Google’s training data with that from the other manufacturers.
    • David Rosenthal: The margins on AWS, averaging 24.75% over the last twelve quarters, are what enables Amazon to run the US retail business averaging under 3% margin and the international business averaging -3.7% margin over the same period.
    • @asymco: Apple Q4 Sales Guidance $60B-$62B vs $52.58B year ago, Gross Margin 38-38.5% vs. 37.9% year ago.
    • Valley of Genius: The best way to think about Silicon Valley is as one large company, and what we think of as companies are actually just divisions. Sometimes divisions get shut down, but everyone who is capable gets put elsewhere in the company: Maybe at a new start-up, maybe at an existing division that’s successful like Google, but everyone always just circulates. So you don’t worry so much about failure. No one takes it personally, you just move on to something else. So that’s the best way to think about the Valley. It’s really engineered to absorb failure really naturally, make sure everyone is taken care of, and go on to something productive next. And there’s no stigma around it.
    • David Gerard: Companies are shocked to realise that blockchain — an expensive and useless idea that has soaked up millions of dollars for zero return — may not be a good technology. “Many companies will halt their blockchain tests this year. The pullback could hurt IBM and Microsoft, analyst says … The expectation was we’d quickly find use cases.”
    • Tullis: The U. S. Department of Homeland Security has designated 16 sectors of infrastructure as 'critical', and 14 of them depend on GPS.
    • Valley of Genius: The people who really create things that change this industry are both the thinker and the doer in one person.
    • Joanna Hoffman [General Magic]: If you have a bunch of self-motivated and smart people and you put them together they’ll produce something incredible. But you can’t minimize the importance of management. It’s a dirty word. It’s prosaic. It’s not vision. It’s not dream. It’s not technological excellence. But unfortunately, it makes all the difference.
    • @timoreilly: OH: "We can't entirely eliminate our technical debt. My goal is to refinance it at a lower interest rate."
    • Alok Pathak: While both (Multi-AZ and Read replica) maintain a copy of database but they are different in nature. Use Multi-AZ deployments for High Availability and Read Replica for read scalability. You can further set up a cross-region read replica for disaster recovery.
    • Stacey Higginbotham: a startup in France called GreenWaves Technologies has built a dedicated chip for the Internet of Things. The company chose the RISC-V architecture because it wanted to avoid raising the crazy amounts of money typically needed for chip startups. GreenWaves CEO Loic Lietar said the company has raised €3.1 million (US $3.6 million) and has already managed to produce a sample of its silicon.  
    • ...
Don't miss all that the Internet has to say on Scalability, click below and become eventually consistent with all scalability knowledge (which means this post has many more items to read so please keep on reading)...

Click to read more ...


Stuff The Internet Says On Scalability For July 27th, 2018

Hey, it's HighScalability time:


Startup opportunity? Space Garbage Collection service. 18,000+ known Near-Earth Objects. (NASA)


Do you like this sort of Stuff? Please lend me your support on Patreon. It would mean a great deal to me. And if you know anyone looking for a simple book that uses lots of pictures and lots of examples to explain the cloud, then please recommend my new book: Explain the Cloud Like I'm 10. They'll love you even more.


  • 143 billion: daily words Google Translated; 73%: less face-to-face interaction in open offices; 10 billion: Uber trips; 131M: data breach by Exactis; $123 billion: Facebook value loss is 4 Twitters and 7 snapchats; $9.1B: spent on digital gaming across all platforms; 20-km: width of lake on mars; 1 billion: Google Drive users; $32.7 billion: Alphabet revenues; $110bn: Microsoft total revenue; $1.9 million: buy the Brady Bunch house; 5.623 trillion: Amazon Sable requests handle on prime day; 91%: Facebook's advertising revenue on mobile; $18 billion: deep learning market by 2024; 

  • Quotable Quotes:
    • Ryan Cash: The App Store has changed the world so drastically it’s hard to even imagine sometimes. In some ways, the world feels kind of the same as it did 10 years ago. But only for a second. In almost every single way, with almost every single thing we do, the iPhone and the App Store have changed how we live as humans. It’s changed how we communicate, how we share, and how we express ourselves artistically. It’s changed how we travel, do business, and how we eat. It’s made us healthier, wiser, and a little goofier. It truly kickstarted a massive global revolution that, while even 10 years in, feels like just the beginning.
    • yaypie~ I've often wondered what the computing world would look like today if Apple had bought Be. Somewhere out there is a parallel universe where BeOS, rather than OpenStep, became the basis for Apple's new OS. Would it have been able to compete with Windows? Without macOS's BSD underpinnings, would it have been as popular with developers as Mac OS X was? I wonder. 
    • Peter: DARPA foresee a third one in which context-based programs are able to explain and justify their own reasoning.
    • @chrismunns: "hey Ops, we're launching next week, can we make sure we can handle 1000 #serverless function invocations?" Ops: "Sorry, 3-5 month lead time on DC hardware and our switches are near capacity" - coming soon to an on-prem "serverless" project near you.
    • Aaron Frank: In the world of real estate, as Brad Inman puts it, “the company has gone viral.” Incredibly, this growth is largely the result of eXp Realty’s use of an online virtual world similar to Second Life. That means every employee, contractor, and the thousands of agents who work at the company show up to work—team meetings, training seminars, onboarding sessions—all inside a virtual reality campus.
    • Lance Gutteridge: The reason why almost no one encrypts their databases is one of the dirty secrets of IT.
    • @AndreaPessino: It's finally happening - after >30 years of pro use, 20 of which quite reluctantly, I am officially DONE with C/C++. Only maintenance from now on, everything new will be in @rustlang. THANKS Rust team for refining modern concepts into such a practical, elegant system. I love it.
    • Peter J. Denning: These analyses show that the conditions exist at all three levels [chip, system, and adopting community] of the computing ecosystem to sustain exponential growth. They support the optimism of many engineers that many additional years of exponential growth are likely. Moore's Law was sustained for five decades. Exponential growth is likely to be sustained for many more.
    • Quotes are back baby. Last week was slow. Not this week. Click through to read more.
Don't miss all that the Internet has to say on Scalability, click below and become eventually consistent with all scalability knowledge (which means this post has many more items to read so please keep on reading)...

Click to read more ...


Google's New Book: The Site Reliability Workbook


Google has released a new book: The Site Reliability Workbook — Practical Ways to Implement SRE.

It's the second book in their SRE series. How is it different than the previous Site Reliability Engineering book?

David Rensin, a SRE at Google, says:

It's a whole new book.  It's designed to sit next to the original on the bookshelf and for folks to bounce between them -- moving between principle and practice.

And from the preface:

The purpose of this second SRE book is (a) to add more implementation detail to the principles outlined in the first volume, and (b) to dispel the idea that SRE is implementable only at “Google scale” or in “Google culture.”

The Site Reliability Workbook weighs in at a hefty 508 pages and roughly follows the structure of the first book. It's organized into three different parts: Foundations, Practices, and Processes. There are three appendices: Example SLO Document, Example Error Budget Policy, and Results of Postmortem Analysis.

The table of content is quite detailed, but here are the chapter titles:

  1. How SRE Relates to DevOps.  
  2. Implementing SLOs.
  3. SLO Engineering Case Studies.
  4. Monitoring.
  5. Alerting on SLOs.
  6. Eliminating Toil.
  7. Simplicity.
  8. On-Call.
  9. Incident Response.
  10. Postmortem Culture: Learning from Failure.
  11. Managing Load.
  12. Introducing Non-Abstract Large System Design.
  13. Data Processing Pipelines.
  14. Configuration Design and Best Practices.
  15. Configuration Specifics.
  16. Canarying Releases.
  17. Identifying and Recovering from Overload.
  18. SRE Engagement Model.
  19. SRE: Reaching Beyond Your Walls.
  20. SRE Team Lifecycles.
  21. Organizational Change Management in SRE.

What makes this book a tour de force are all the examples and case studies. You aren't just stuck with high level principles, you're given worked examples that make the principles concrete. That's hard to do and takes a lot of work.

In Chapter 2—Implementing SLOs—there's a detailed example involving the architecture for a mobile phone game. First, you must learn how to think "about how users interact with the system, and what sort of SLIs (Service Level Indicators) would measure the various aspects of a user’s experience." You're then taken through a number of SLIs and how to implement and measure them. Given the SLIs you learn how to calculate SLOs (Service Level Objectives). And once you have the SLO you're shown how to derive the error budget. That's not the end. You have to document the SLO and error budget policy. Then you need reports and dashboards that provide in-time snapshots of the SLO compliance of your services. Is that the end? No. You must continuously improve your SLO targets and learn how to make decisions using that information. And that's not the end either, but for the rest you'll need to read the book.

In Chapter 3—SLO Engineering Case Studies—Evernote and The Home Depot tell the story of their journey into SRE.

In Chapter 4—Monitoring—there are examples of moving information from logs to metrics, improving both logs and metrics, and keeping logs as the data source.

In Chapter 6—Eliminating Toil—there are detailed case studies on Reducing Toil in the Datacenter with Automation and Decommission Filer-Backed Home Directories.

And so it goes through nearly every chapter.

As you can see it's a very detailed and thorough book. The preface modestly contends it's a necessarily limited book, but I'd hate to see how many pages would be in the unlimited version.

Like the first book, the writing is clear, purposeful, and well organized. For a company well known for its influential publications, this is another winner.

Best of all? It's free until August 23rd!


Stuff The Internet Says On Scalability For July 20th, 2018

Hey, it's HighScalability time:



World History Timeline from 3000BC to 2000AD. Yet we still program with text—in files (Schofield and Sims)



Do you like this sort of Stuff? Please lend me your support on Patreon. It would mean a great deal to me. And if you know anyone looking for a simple book that uses lots of pictures and lots of examples to explain the cloud, then please recommend my new book: Explain the Cloud Like I'm 10. They'll love you even more.


  • $150 billion: Bezos Prime; 49%: Amazon's share of US e-commerce; 1,000 terabytes: image size to represent one cubic millimeter of brain tissue;  7x: 4 year reduction in cost of computing power; 25x: faster code using SIMD; 4TB: RAM in GCE “ultramem” instance type; 4 months: half-life of an ICO; 80%: cost savings moving from AWS to DO; 130,000: square feet in biggest vertical farm; 14x: price increase for Google Maps;  

  • Quotable Quotes:
    • Chappell: In [CPU] architectures, we believe that aggressive specialization is a part of the answer to what happens next. That’s mapping applications to the specific architectural choices. And you already see that in machine learning, where there’s a really hot field in terms of deep neural nets and other implementations. The third wing of the architecture piece is the “domain specific system-on-chip.”
    • Troy Hunt: I love the way these two services work in unison: Azure Functions to ensure you can scale immediately without being bound by logical infrastructure, deployment slots that make it easy to test and rollover new releases with zero downtime, then of course Cloudflare Workers to give you heaps of control over traffic flow for testing and rollover purposes and all protected using their rate limit service. 
    • Cliff Click: The JVM is very good at eliminating the cost of code abstraction, but not the cost of data abstraction. That means multiple data indirections mean multiple cache misses. They are very expensive. This is where your performance goes. 
    • jaybo_nomad: The Allen Institute for Brain Science is in the process of imaging 1 cubic mm of mouse visual cortex using TEM at a resolution of 4nm per pixel. The goal is to complete this in about 4 months running in parallel on 5 scopes. 
    • @lowrykoz: Stolen from a co-worker "Every company has a test environment. Some are lucky enough to also have a production environment."  Thanks Charlie for my laugh today!
    • DonHopkins: NeWS differs from the current technology stack in that it was all coherently designed at once by James Gosling and David Rosenthal, by taking several steps back and thinking deeply about all the different problems it was trying to solve together. So it's focused and expressed in one single language, instead of the incoherent fragmented Tower of Babel of many other ad-hoc languages that we're stuck with today.
    • tef: You can use a message broker to glue systems together, but never use one to cut systems apart.
    • @CompSciFact: 'Fancy algorithms are slow when n is small, and n is usually small.' -- Rob Pike
    • @Grady_Booch: I interviewed John Backus shortly before his death. He told me his work in functional programming languages failed, and would likely always fail, because it was easy to do hard things but incredibly difficult to do simple things.
    • Mark LaPedus: MRAM, a next-generation memory type, is being touted as a replacement for embedded flash and cache applications. MRAM works in consumer applications, but it’s still unclear if it will ever meet the temperature requirements for automotive. Some say MRAM will never work in automotive.
    • crabbone: This is the prism through which Java programmers view the world. They never question this belief. It also works well to justify an acquisition of more servers to investors. Investing tons of efforts into IT, building complicated deployment and clustering software etc. This is what happens in the company I work for, and couple of mid-size to big companies I worked for before. The truth about it is that Java only gets you a good bang for your buck just a wee bit before it hits OOM. It doesn't scale down to the point that is ridiculous and painful. A typical example of modern "microservices-inspired" Java application would function along these lines:
    • Netflix: We observed during experimentation that RAM random read latencies were rarely higher than 1 microsecond whereas typical SSD random read speeds are between 100–500 microseconds. For EVCache our typical SLA (Service Level Agreement) is around 1 millisecond with a default timeout of 20 milliseconds while serving around 100K RPS. During our testing using the storage optimized EC2 instances (I3.2xlarge) we noticed that we were able to perform over 200K IOPS of 1K byte items thus meeting our throughput goals with latency rarely exceeding 1 millisecond. This meant that by using SSD (NVMe) we were able to meet our SLA and throughput requirements at a significantly lower cost.
    • paulddraper: The biggest lesson HN teaches for designing large scale systems is "use a large scale system someone else has already designed".
    • There are a few more quotes. Not so many this week.
Don't miss all that the Internet has to say on Scalability, click below and become eventually consistent with all scalability knowledge (which means this post has many more items to read so please keep on reading)...

Click to read more ...


Stuff The Internet Says On Scalability For July 13th, 2018

Hey, it's HighScalability time:


Steve Blank tells the Secret History of Silicon Valley. What a long, strange trip it is.


Do you like this sort of Stuff? Please lend me your support on Patreon. It would mean a great deal to me. And if you know anyone looking for a simple book that uses lots of pictures and lots of examples to explain the cloud, then please recommend my new book: Explain the Cloud Like I'm 10. They'll love you even more.


  • $27 billion: CapEx invested by leading cloud vendors in first quarter of 2018; $40 billion: App store revenue in 10 years; $57.5 billion: venture investment first half of 2018; 1 billion: Utah voting system per day hack attempts; 67%: did not deploy a serverless app last year; $1.8 billion: made by Pokeman GO; $13 billion: Netflix's new content budget; 

  • Quotable Quotes:
    • @davidbrunelle: The best developers and engineering leaders I've personally worked with do *not* have a notable presence on GitHub or public bodies of speaking or writing work. I worry that a lot of folks confuse celebrity and visibility with talent and ability.
    • Bernard Golden: The tech industry has never seen this level of investment [in datacenters]. The investment we’re seeing in cloud capacity really has no precedent, save perhaps Henry Ford’s manic factory building for his Model T, the US government’s armaments efforts in WWII, and Foxconn’s manufacturing support for smartphones. As Ford’s efforts presaged the boom growth of the industrial economy, so too do (cloud) investments augur the explosion of the digital economy.
    • @kellabyte: The issue with microservices is it’s taught people to stop thinking about cohesiveness.  Cohesiveness is really important. When you fight against it you experience major pain and Service Autonomy works much better as a cohesive unit not just making “micro” things everywhere.
    • Leslie Lamport: Today, programming is generally equated with coding. It's hard to convince students who want to write code that they should learn to think mathematically, above the code level, about what they’re doing. Perhaps the following observation will give them pause. It's quite likely that during their lifetime, machine learning will completely change the nature of programming. The programming languages they are now using will seem as quaint as Cobol, and the coding skills they are learning will be of little use. But mathematics will remain the queen of science, and the ability to think mathematically will always be useful.
    • Nolan Bushnell, founder of Atari: What everybody wanted was a party and some beer and some pizza, and they ended up going home with each other.
    • @Grady_Booch: Ada Lovelace devised the first program. Grace Hopper wrote the first complier. Margaret Hamilton started the field of software engineering. Women have always been and will always be essential to the advance of computing.
    • Hossein Fateh: The largest deal of 2001 was 3.5 megawatts. That same company leased 35 megawatts from us in 2016. The decimal place moved by a column. The next decimal place will move in 2022. Deals will be 350 megawatts.
    • Matt Alderman: Based on our estimates, AWS Fargate deployments should save you 5 percent to 10 percent in your compute bill as compared to highly optimized AWS ECS or EKS deployments.
    • Memory Guy: Conventional wisdom holds that SSDs will someday displace all HDDs, but in reality SSDs are proving to be more of a challenge to the DRAM market than to the HDD market...So, if you have a fixed budget, SSDs can help you get the most out of your system and are a better alternative than additional DRAM.
    • @JoeEmison: Given this context, it is insane to say that something like AWS AppSync is bad lock-in. If I want AppSync functionality, I have two choices: use AppSync, or write AppSync myself. Given those, how is using AppSync until it makes sense for me to write my own a bad move?
    • More, more, more, more, more...
Don't miss all that the Internet has to say on Scalability, click below and become eventually consistent with all scalability knowledge (which means this post has many more items to read so please keep on reading)...

Click to read more ...