advertise
Wednesday
Aug222018

What do you believe now that you didn't five years ago? Centralized wins. Decentralized loses.

 

 

Decentralized systems will continue to lose to centralized systems until there's a driver requiring decentralization to deliver a clearly superior consumer experience. Unfortunately, that may not happen for quite some time.

I say unfortunately because ten years ago, even five years ago, I still believed decentralization would win. Why? For all the idealistic technical reasons I laid out long ago in Building Super Scalable Systems: Blade Runner Meets Autonomic Computing In The Ambient Cloud.

While the internet and the web are inherently decentralized, mainstream applications built on top do not have to be. Typically, applications today—Facebook, Salesforce, Google, Spotify, etc.—are all centralized.

That wasn't always the case. In the early days of the internet the internet was protocol driven, decentralized, and often distributed—FTP (1971), Telnet (<1973), FINGER (1971/1977),  TCP/IP (1974), UUCP (late 1970s) NNTP (1986), DNS (1983), SMTP (1982), IRC(1988), HTTP(1990), Tor (mid-1990s), Napster(1999), and XMPP(1999).

We do have new decentalized services: Bitcoin(2009), Minecraft(2009), Ethereum(2104), IPFS(2015), Mastadon(2016), and PeerTube(2018). We're still waiting on Pied Piper to deliver the decentralized internet

On an evolutionary timeline decentralized systems are neanderthals; centralized systems are the humans. Neanderthals came first. Humans may have interbred with neanderthals, humans may have even killed off the neanderthals, but there's no doubt humans outlasted the neanderthals.

The reason why decentralization came first is clear from a picture of the very first ARPA (Advanced Research Projects Agency) network, which later evolved into the internet we know and sometimes love today:

Click to read more ...

Tuesday
Aug212018

Sponsored Post: NationBuilder, Twitch, InMemory.Net, Triplebyte, Etleap, Stream, Scalyr, MemSQL

Who's Hiring? 

  • NationBuilder — if you’re a systems engineer, SRE or DevOps focused developer and have been looking for a place where you can help other people while still working in tech? We can give that opportunity. Please apply here

  • Twitch's commerce team in San Francisco is looking to hire senior developers to keep up with rapidly increasing demand for our Subscriptions and Payment platform. Engineers will be tasked with building new products and features to solve business and ecommerce challenges as we're dealing with engaging problems at a massive scale and will create solutions that impact millions of people around the world. Apply here

  • Triplebyte lets exceptional software engineers skip screening steps at hundreds of top tech companies like Apple, Dropbox, Mixpanel, and Instacart. Make your job search O(1), not O(n). Apply here.

  • Need excellent people? Advertise your job here! 

Fun and Informative Events

  • Advertise your event here!

Cool Products and Services


  • InMemory.Net provides a Dot Net native in memory database for analysing large amounts of data. It runs natively on .Net, and provides a native .Net, COM & ODBC apis for integration. It also has an easy to use language for importing data, and supports standard SQL for querying data. http://InMemory.Net
  • Build, scale and personalize your news feeds and activity streams with getstream.io. Try the API now in this 5 minute interactive tutorialStream is free up to 3 million feed updates so it's easy to get started. Client libraries are available for Node, Ruby, Python, PHP, Go, Java and .NET. Stream is currently also hiring Devops and Python/Go developers in Amsterdam. More than 400 companies rely on Stream for their production feed infrastructure, this includes apps with 30 million users. With your help we'd like to ad a few zeros to that number. Check out the job opening on AngelList.
  • Scalyr is a lightning-fast log management and operational data platform.  It's a tool (actually, multiple tools) that your entire team will love.  Get visibility into your production issues without juggling multiple tabs and different services -- all of your logs, server metrics and alerts are in your browser and at your fingertips. .  Loved and used by teams at Codecademy, ReturnPath, Grab, and InsideSales. Learn more today or see why Scalyr is a great alternative to Splunk.

  • MemSQL envisions a world of adaptable databases and flexible data workloads - your data anywhere in real time. Today, global enterprises use MemSQL as a real-time data warehouse to cost-effectively ingest data and produce industry-leading time to insight. MemSQL works in any cloud, on-premises, or as a managed service. Start a free 30 day trial here: memsql.com/download/.

  • Advertise your product or service here!

If you are interested in a sponsored post for an event, job, or product, please contact us for more information.


Make Your Job Search O(1) — not O(n)

Triplebyte is unique because they're a team of engineers running their own centralized technical assessment. Companies like Apple, Dropbox, Mixpanel, and Instacart now let Triplebyte-recommended engineers skip their own screening steps.

We found that High Scalability readers are about 80% more likely to be in the top bracket of engineering skill.

Take Triplebyte's multiple-choice quiz (system design and coding questions) to see if they can help you scale your career faster.


The Solution to Your Operational Diagnostics Woes

Scalyr gives you instant visibility of your production systems, helping you turn chaotic logs and system metrics into actionable data at interactive speeds. Don't be limited by the slow and narrow capabilities of traditional log monitoring tools. View and analyze all your logs and system metrics from multiple sources in one place. Get enterprise-grade functionality with sane pricing and insane performance. Learn more today


If you are interested in a sponsored post for an event, job, or product, please contact us for more information.

Friday
Aug172018

Stuff The Internet Says On Scalability For August 17th, 2018

Hey, it's HighScalability time:

 

The amazing Zoomable Universe from 10^27 meters—about 93 billion light-years—down to the subatomic realm, at 10^-35 meters.

 

Do you like this sort of Stuff? Please lend me your support on Patreon. It would mean a great deal to me. And if you know anyone looking for a simple book that uses lots of pictures and lots of examples to explain the cloud, then please recommend my new book: Explain the Cloud Like I'm 10. They'll love you even more.

 

  • 2.24x10^32T: joules needed by the Death Star to obliterate Alderaan, which would liquify everyone in the Death Star; 13 of 25: highest paying jobs are in tech; 70,000+: paid Slack workspaces; 13: hours ave american sits; $13.5 million: lost in ATM malware hack; $1.5 billion: cryptocurrency gambling ring busted in China; $8.5B: Auto, IoT, Security startups; 10x: infosec M&A; 1,000: horsepower needed to fly a jet suit; 30% Google's energy savings from AI control of datacenters;

  • Quotable Quotes:
    • The Jury Is In: From the security point of view, the monolithic OS design is flawed and a root cause of the majority of compromises. It is time for the world to move to an OS structure appropriate for 21st century security requirements.
    • @coryodaniel: Rewrote an #AWS APIGateway & #lambda service that was costing us about $16000 / month in #elixir. Its running in 3 nodes that cost us about $150 / month. 12 million requests / hour with sub-second latency, ~300GB of throughput / day. #myelixirstatus !#Serverless...No it’s not Serverless anymore it’s running in a few containers on a kubernetes cluster
    • @cablelounger: OH: To use AWS offerings, you really need in-house dev-ops expertise vs GCP, they make dev ops transparent to you     I've a lot of experience with AWS and mostly agree with the first point. I haven't really used GCP in earnest. I'd love to hear experiences from people who have?
    • @allspaw: engineer: “Unless you’re familiar with Lamport, Brewer, Fox, Armstrong, Stonebraker, Parker, Shapiro...(and others) you don’t know distributed systems.” also engineer: “I read ‘Thinking Fast and Slow’ therefore I know cognitive psychology and decision-making theory.”
    • alankay1: To summarize here, I said I love "Rocky's Boots", and I love the basic idea of "Robot Odyssey", but for end-users, using simple logic gates to program multiple robots in a cooperative strategy game blows up too much complexity for very little utility. A much better way to do this would be to make a "next Logo" that would allow game players to make the AI brains needed by the robots. So what I actually said, is that doing it the way you are doing it will wind up with a game that is nxot successful or very playable. Just why they misunderstood what I said is a bit of a mystery, because I spelled out what could be really good for the game (and way ahead of what other games were doing). And of course it would work on an Apple II and other 8 bit micros (Logo ran nicely on them, etc.)
    • Michael Malone: Nolan was the first guy to look at Moore’s law and say to himself: You know what? When logic and memory chips get to be under ten bucks I can take these big games and shove them into a pinball machine.
    • @hichaelmart: To be honest, I think the main lesson from this is that API Gateway is expensive – 100% agree. We have a GAE app doing a very similar thing, billions of impressions/mth – and *much* cheaper than if it were on API Gateway.
    • Keep on reading for many more quotes hot off the internet. You'll be a better person.
Don't miss all that the Internet has to say on Scalability, click below and become eventually consistent with all scalability knowledge (which means this post has many more items to read so please keep on reading)...

Click to read more ...

Monday
Aug132018

What do you believe now that you didn't five years ago?

 

 

Decentralized systems will continue to lose to centralized systems until there's a driver requiring decentralization to deliver a clearly superior consumer experience. Unfortunately, that may not happen for quite some time.

I say unfortunately because ten years ago, even five years ago, I still believed decentralization would win. Why? For all the idealistic technical reasons I laid out long ago in Building Super Scalable Systems: Blade Runner Meets Autonomic Computing In The Ambient Cloud.

While the internet and the web are inherently decentralized, mainstream applications built on top do not have to be. Typically, applications today—Facebook, Salesforce, Google, Spotify, etc.—are all centralized.

That wasn't always the case. In the early days of the internet the internet was protocol driven, decentralized, and often distributed—FTP (1971), Telnet (<1973), FINGER (1971/1977),  TCP/IP (1974), UUCP (late 1970s) NNTP (1986), DNS (1983), SMTP (1982), IRC(1988), HTTP(1990), Tor (mid-1990s), Napster(1999), and XMPP(1999).

We do have new decentalized services: Bitcoin(2009), Minecraft(2009), Ethereum(2104), IPFS(2015), Mastadon(2016), and PeerTube(2018). We're still waiting on Pied Piper to deliver the decentralized internet

On an evolutionary timeline decentralized systems are neanderthals; centralized systems are the humans. Neanderthals came first. Humans may have interbred with neanderthals, humans may have even killed off the neanderthals, but there's no doubt humans outlasted the neanderthals.

The reason why decentralization came first is clear from a picture of the very first ARPA (Advanced Research Projects Agency) network, which later evolved into the internet we know and sometimes love today:

Click to read more ...

Thursday
Aug092018

Stuff The Internet Says On Scalability For August 10th, 2018

Hey, it's HighScalability time (out Thur-Fri, so we're going early):

 

London Maker Faire 1851—The Great Exhibition—100,000 objects, displayed along more than 10 miles, by over 15,000 contributors.

 

Do you like this sort of Stuff? Please lend me your support on Patreon. It would mean a great deal to me. And if you know anyone looking for a simple book that uses lots of pictures and lots of examples to explain the cloud, then please recommend my new book: Explain the Cloud Like I'm 10. They'll love you even more.

 

  • 90%: accuracy predicting gender from retinal image; $1 billion: Ebay sales per quarter from AI; $78 billion: global AI software market by 2025; $75m: penalty for botched SAP upgrade; 35 million: m^3 of mud dredged out of the Dutch waterways; 138 terabytes: memory per square inch; 500 million: Uber metrics per second; 22x: new faster JSON Sparser; 

  • Quotable Quotes:
    • @IanColdwater: The JIRA tickets will continue until morale improves
    • @david_perell: Three crazy stats from @mikedariano’s newsletter. 1. People watch more Minecraft hours than the NBA, NHL, NFL, and MLB combined.  2. Only 26 countries have more people than PewDiePie has subscribers.  3. Only 20% of YouTube’s traffic is from the United States. 
    • Charlie Demerjian: Why does SemiAccurate say that Intel knows? We have seen their internal documents that show exactly how frightened the company is. The documents go into specifics we don’t feel are appropriate to discuss publicly but there is one thing we can say, Intel knows their position. One of the documents says in no uncertain terms that the company understands they will not be competitive in the server market until AFTER Sapphire Rapids, the 2022 server part. AMD has a clear run in Intel’s core market for at least 4 years.
    • @ScottMcGready: Can we just take a moment to remember that one company I worked for backed up their stuff on tapes religiously- all tapes sent to a warehouse nightly. Years later someone tested a tape... turns out nothing had been written... ever. We had a (paid) warehouse full of empty tapes
    • Uber: Since 2016, Uber has added several new lines of business to its platform, including Uber Eats, Uber Freight, and Jump Bikes. Now, we complete over 15 million trips a day, with over 75 million monthly active riders. In the last eight years, the company has grown from a small startup to 18,000 employees across the globe.
    • @mims: Here is a super important thing that we don't talk about enough: Almost all of the increase in income inequality from 1978 to the present can be accounted for by the difference in wages between top performing firms and everyone else. And now we have some idea what's driving unequal growth in productivity of top-performing firms -- it's how they build and use *their own, proprietary software and other IT/technology
    • @taotetek: Distributed systems tip: Write your system without any queues first. You might find you don't need queues. If you end up needing queues, the retry and reliability code you wrote in order to function without queues will still make your system more reliable.
    • @theburningmonk: I think the visual flow is sometimes under-appreciated - our app support team can easily look at it and figure out what went wrong without knowing ins & outs of implementation details. I can also show the diagram to a product person and he/she would get it as well
    • John Mark: It’s time to understand something about open source software development: it is not going to save us. Using or developing more open source software is not going to improve anyone’s lives. Developing open source software is not a public good. It’s not going to result in a fairer or more equitable society. In fact, as currently structured, open source development is part of the problem. 
    • There are a few more quotes. Don't miss them.
Don't miss all that the Internet has to say on Scalability, click below and become eventually consistent with all scalability knowledge (which means this post has many more items to read so please keep on reading)...

Click to read more ...

Wednesday
Aug082018

Case Study: Pokémon GO on Google Cloud Load Balancing

 

There are a lot of cool nuggets in Google's New Book: The Site Reliability Workbook. If you haven't put it on your reading list, here's a tantalizing excerpt from CHAPTER 11 Managing Load by Cooper Bethea, Gráinne Sheerin, Jennifer Mace, and Ruth King with Gary Luo and Gary O’Connor.

 

Niantic launched Pokémon GO in the summer of 2016. It was the first new Pokémon game in years, the first official Pokémon smartphone game, and Niantic’s first project in concert with a major entertainment company. The game was a runaway hit and more popular than anyone expected—that summer you’d regularly see players gathering to duel around landmarks that were Pokémon Gyms in the virtual world.

Pokémon GO’s success greatly exceeded the expectations of the Niantic engineering team. Prior to launch, they load-tested their software stack to process up to 5x their most optimistic traffic estimates. The actual launch requests per second (RPS) rate was nearly 50x that estimate—enough to present a scaling challenge for nearly any software stack. To further complicate the matter, the world of Pokémon GO is highly interactive and globally shared among its users. All players in a given area see the same view of the game world and interact with each other inside that world. This requires that the game produce and distribute near-real-time updates to a state shared by all participants.

Scaling the game to 50x more users required a truly impressive effort from the Niantic engineering team. In addition, many engineers across Google provided their assis‐ tance in scaling the service for a successful launch. Within two days of migrating to GCLB, the Pokemon GO app became the single largest GCLB service, easily on par with the other top 10 GCLB services.

As shown in Figure 11-5, when it launched, Pokémon GO used Google’s regional Network Load Balancer (NLB) to load-balance ingress traffic across a Kubernetes cluster. Each cluster contained pods of Nginx instances, which served as Layer 7 reverse proxies that terminated SSL, buffered HTTP requests, and performed routing and load balancing across pods of application server backends.


Figure 11-5. Pokémon GO (pre-GCLB)

Click to read more ...

Tuesday
Aug072018

Sponsored Post: NationBuilder, Twitch, InMemory.Net, Triplebyte, Etleap, Scalyr, MemSQL

Who's Hiring? 

  • NationBuilder — if you’re a systems engineer, SRE or DevOps focused developer and have been looking for a place where you can help other people while still working in tech? We can give that opportunity. Please apply here

  • Twitch's commerce team in San Francisco is looking to hire senior developers to keep up with rapidly increasing demand for our Subscriptions and Payment platform. Engineers will be tasked with building new products and features to solve business and ecommerce challenges as we're dealing with engaging problems at a massive scale and will create solutions that impact millions of people around the world. Apply here

  • Triplebyte lets exceptional software engineers skip screening steps at hundreds of top tech companies like Apple, Dropbox, Mixpanel, and Instacart. Make your job search O(1), not O(n). Apply here.

  • Need excellent people? Advertise your job here! 

Fun and Informative Events

  • Advertise your event here!

Cool Products and Services


  • InMemory.Net provides a Dot Net native in memory database for analysing large amounts of data. It runs natively on .Net, and provides a native .Net, COM & ODBC apis for integration. It also has an easy to use language for importing data, and supports standard SQL for querying data. http://InMemory.Net
  • For heads of IT/Engineering responsible for building an analytics infrastructure, Etleap is an ETL solution for creating perfect data pipelines from day one. Unlike older enterprise solutions, Etleap doesn’t require extensive engineering work to set up, maintain, and scale. It automates most ETL setup and maintenance work, and simplifies the rest into 10-minute tasks that analysts can own. Read stories from customers like Okta and PagerDuty, or try Etleap yourself.

  • Scalyr is a lightning-fast log management and operational data platform.  It's a tool (actually, multiple tools) that your entire team will love.  Get visibility into your production issues without juggling multiple tabs and different services -- all of your logs, server metrics and alerts are in your browser and at your fingertips. .  Loved and used by teams at Codecademy, ReturnPath, Grab, and InsideSales. Learn more today or see why Scalyr is a great alternative to Splunk.

  • MemSQL envisions a world of adaptable databases and flexible data workloads - your data anywhere in real time. Today, global enterprises use MemSQL as a real-time data warehouse to cost-effectively ingest data and produce industry-leading time to insight. MemSQL works in any cloud, on-premises, or as a managed service. Start a free 30 day trial here: memsql.com/download/.

  • Advertise your product or service here!

If you are interested in a sponsored post for an event, job, or product, please contact us for more information.


Make Your Job Search O(1) — not O(n)

Triplebyte is unique because they're a team of engineers running their own centralized technical assessment. Companies like Apple, Dropbox, Mixpanel, and Instacart now let Triplebyte-recommended engineers skip their own screening steps.

We found that High Scalability readers are about 80% more likely to be in the top bracket of engineering skill.

Take Triplebyte's multiple-choice quiz (system design and coding questions) to see if they can help you scale your career faster.


The Solution to Your Operational Diagnostics Woes

Scalyr gives you instant visibility of your production systems, helping you turn chaotic logs and system metrics into actionable data at interactive speeds. Don't be limited by the slow and narrow capabilities of traditional log monitoring tools. View and analyze all your logs and system metrics from multiple sources in one place. Get enterprise-grade functionality with sane pricing and insane performance. Learn more today


If you are interested in a sponsored post for an event, job, or product, please contact us for more information.

Friday
Aug032018

Stuff The Internet Says On Scalability For August 3rd, 2018

Hey, it's HighScalability time:

 

Everything starts with Doug Engelbart — Jane Metcalfe.
It was the very first time (1968) the world had ever seen a mouse, seen outline processing, seen hypertext, seen mixed text and graphics, seen real-time video conferencing. — Doug Engelbart (Valley of Genius).
ARPA funded the demo at a cost of $1 million. Most importantly? It was the first use of a todo list as an example. A tradition unlike any other.

 

Do you like this sort of Stuff? Please lend me your support on Patreon. It would mean a great deal to me. And if you know anyone looking for a simple book that uses lots of pictures and lots of examples to explain the cloud, then please recommend my new book: Explain the Cloud Like I'm 10. They'll love you even more.

 

  • $1 trillion: Apple; 45: mean founder age for fastest growing new ventures; 1 trillion: files created by Trinity in 2 minutes;  $3.93 billion: Baidu's AI driven quarterly revenues; 13.7: microsecond error in GPS timestamps for nearly 14 hours; 240 million: Apple CarPlay cars by 2023; 99%: confidence setting to use when face matching politicians to criminals; 31%: Apple services revenue increase to ~$10B; 4 billion: containers Google launches each week; 0,1,2: likely per person mutations on a coding gene; 1,000x: solid-state memory density; $10 billion: Pentagon cloud contract up for bid; 661Tbps: through a single optical fiber; $49bn: last 6 months of M&A; $1.1 trillion: projected 2040 space economy revenue; 

  • Quotable Quote:
    • @BrianRoemmele: This shift away from affiliate compensation for IOS app sales will have a rather large impact on niche apps. Many are high ticket and require rather involved promotion and support by third parties that were partly compensated by the affiliate compensation.
    • Paco Nathan: Frankly, I’d feel a lot more comfortable sending my kids off to school in a self-driving bus if the machine learning models hadn’t been trained solely by Google’s proprietary data. Instead, let’s get every possible edge case understood by mingling Google’s training data with that from the other manufacturers.
    • David Rosenthal: The margins on AWS, averaging 24.75% over the last twelve quarters, are what enables Amazon to run the US retail business averaging under 3% margin and the international business averaging -3.7% margin over the same period.
    • @asymco: Apple Q4 Sales Guidance $60B-$62B vs $52.58B year ago, Gross Margin 38-38.5% vs. 37.9% year ago.
    • Valley of Genius: The best way to think about Silicon Valley is as one large company, and what we think of as companies are actually just divisions. Sometimes divisions get shut down, but everyone who is capable gets put elsewhere in the company: Maybe at a new start-up, maybe at an existing division that’s successful like Google, but everyone always just circulates. So you don’t worry so much about failure. No one takes it personally, you just move on to something else. So that’s the best way to think about the Valley. It’s really engineered to absorb failure really naturally, make sure everyone is taken care of, and go on to something productive next. And there’s no stigma around it.
    • David Gerard: Companies are shocked to realise that blockchain — an expensive and useless idea that has soaked up millions of dollars for zero return — may not be a good technology. “Many companies will halt their blockchain tests this year. The pullback could hurt IBM and Microsoft, analyst says … The expectation was we’d quickly find use cases.”
    • Tullis: The U. S. Department of Homeland Security has designated 16 sectors of infrastructure as 'critical', and 14 of them depend on GPS.
    • Valley of Genius: The people who really create things that change this industry are both the thinker and the doer in one person.
    • Joanna Hoffman [General Magic]: If you have a bunch of self-motivated and smart people and you put them together they’ll produce something incredible. But you can’t minimize the importance of management. It’s a dirty word. It’s prosaic. It’s not vision. It’s not dream. It’s not technological excellence. But unfortunately, it makes all the difference.
    • @timoreilly: OH: "We can't entirely eliminate our technical debt. My goal is to refinance it at a lower interest rate."
    • Alok Pathak: While both (Multi-AZ and Read replica) maintain a copy of database but they are different in nature. Use Multi-AZ deployments for High Availability and Read Replica for read scalability. You can further set up a cross-region read replica for disaster recovery.
    • Stacey Higginbotham: a startup in France called GreenWaves Technologies has built a dedicated chip for the Internet of Things. The company chose the RISC-V architecture because it wanted to avoid raising the crazy amounts of money typically needed for chip startups. GreenWaves CEO Loic Lietar said the company has raised €3.1 million (US $3.6 million) and has already managed to produce a sample of its silicon.  
    • ...
Don't miss all that the Internet has to say on Scalability, click below and become eventually consistent with all scalability knowledge (which means this post has many more items to read so please keep on reading)...

Click to read more ...

Thursday
Jul262018

Stuff The Internet Says On Scalability For July 27th, 2018

Hey, it's HighScalability time:

 

Startup opportunity? Space Garbage Collection service. 18,000+ known Near-Earth Objects. (NASA)

 

Do you like this sort of Stuff? Please lend me your support on Patreon. It would mean a great deal to me. And if you know anyone looking for a simple book that uses lots of pictures and lots of examples to explain the cloud, then please recommend my new book: Explain the Cloud Like I'm 10. They'll love you even more.

 

  • 143 billion: daily words Google Translated; 73%: less face-to-face interaction in open offices; 10 billion: Uber trips; 131M: data breach by Exactis; $123 billion: Facebook value loss is 4 Twitters and 7 snapchats; $9.1B: spent on digital gaming across all platforms; 20-km: width of lake on mars; 1 billion: Google Drive users; $32.7 billion: Alphabet revenues; $110bn: Microsoft total revenue; $1.9 million: buy the Brady Bunch house; 5.623 trillion: Amazon Sable requests handle on prime day; 91%: Facebook's advertising revenue on mobile; $18 billion: deep learning market by 2024; 

  • Quotable Quotes:
    • Ryan Cash: The App Store has changed the world so drastically it’s hard to even imagine sometimes. In some ways, the world feels kind of the same as it did 10 years ago. But only for a second. In almost every single way, with almost every single thing we do, the iPhone and the App Store have changed how we live as humans. It’s changed how we communicate, how we share, and how we express ourselves artistically. It’s changed how we travel, do business, and how we eat. It’s made us healthier, wiser, and a little goofier. It truly kickstarted a massive global revolution that, while even 10 years in, feels like just the beginning.
    • yaypie~ I've often wondered what the computing world would look like today if Apple had bought Be. Somewhere out there is a parallel universe where BeOS, rather than OpenStep, became the basis for Apple's new OS. Would it have been able to compete with Windows? Without macOS's BSD underpinnings, would it have been as popular with developers as Mac OS X was? I wonder. 
    • Peter: DARPA foresee a third one in which context-based programs are able to explain and justify their own reasoning.
    • @chrismunns: "hey Ops, we're launching next week, can we make sure we can handle 1000 #serverless function invocations?" Ops: "Sorry, 3-5 month lead time on DC hardware and our switches are near capacity" - coming soon to an on-prem "serverless" project near you.
    • Aaron Frank: In the world of real estate, as Brad Inman puts it, “the company has gone viral.” Incredibly, this growth is largely the result of eXp Realty’s use of an online virtual world similar to Second Life. That means every employee, contractor, and the thousands of agents who work at the company show up to work—team meetings, training seminars, onboarding sessions—all inside a virtual reality campus.
    • Lance Gutteridge: The reason why almost no one encrypts their databases is one of the dirty secrets of IT.
    • @AndreaPessino: It's finally happening - after >30 years of pro use, 20 of which quite reluctantly, I am officially DONE with C/C++. Only maintenance from now on, everything new will be in @rustlang. THANKS Rust team for refining modern concepts into such a practical, elegant system. I love it.
    • Peter J. Denning: These analyses show that the conditions exist at all three levels [chip, system, and adopting community] of the computing ecosystem to sustain exponential growth. They support the optimism of many engineers that many additional years of exponential growth are likely. Moore's Law was sustained for five decades. Exponential growth is likely to be sustained for many more.
    • Quotes are back baby. Last week was slow. Not this week. Click through to read more.
Don't miss all that the Internet has to say on Scalability, click below and become eventually consistent with all scalability knowledge (which means this post has many more items to read so please keep on reading)...

Click to read more ...

Wednesday
Jul252018

Google's New Book: The Site Reliability Workbook

 

Google has released a new book: The Site Reliability Workbook — Practical Ways to Implement SRE.

It's the second book in their SRE series. How is it different than the previous Site Reliability Engineering book?

David Rensin, a SRE at Google, says:

It's a whole new book.  It's designed to sit next to the original on the bookshelf and for folks to bounce between them -- moving between principle and practice.

And from the preface:

The purpose of this second SRE book is (a) to add more implementation detail to the principles outlined in the first volume, and (b) to dispel the idea that SRE is implementable only at “Google scale” or in “Google culture.”

The Site Reliability Workbook weighs in at a hefty 508 pages and roughly follows the structure of the first book. It's organized into three different parts: Foundations, Practices, and Processes. There are three appendices: Example SLO Document, Example Error Budget Policy, and Results of Postmortem Analysis.

The table of content is quite detailed, but here are the chapter titles:

  1. How SRE Relates to DevOps.  
  2. Implementing SLOs.
  3. SLO Engineering Case Studies.
  4. Monitoring.
  5. Alerting on SLOs.
  6. Eliminating Toil.
  7. Simplicity.
  8. On-Call.
  9. Incident Response.
  10. Postmortem Culture: Learning from Failure.
  11. Managing Load.
  12. Introducing Non-Abstract Large System Design.
  13. Data Processing Pipelines.
  14. Configuration Design and Best Practices.
  15. Configuration Specifics.
  16. Canarying Releases.
  17. Identifying and Recovering from Overload.
  18. SRE Engagement Model.
  19. SRE: Reaching Beyond Your Walls.
  20. SRE Team Lifecycles.
  21. Organizational Change Management in SRE.

What makes this book a tour de force are all the examples and case studies. You aren't just stuck with high level principles, you're given worked examples that make the principles concrete. That's hard to do and takes a lot of work.

In Chapter 2—Implementing SLOs—there's a detailed example involving the architecture for a mobile phone game. First, you must learn how to think "about how users interact with the system, and what sort of SLIs (Service Level Indicators) would measure the various aspects of a user’s experience." You're then taken through a number of SLIs and how to implement and measure them. Given the SLIs you learn how to calculate SLOs (Service Level Objectives). And once you have the SLO you're shown how to derive the error budget. That's not the end. You have to document the SLO and error budget policy. Then you need reports and dashboards that provide in-time snapshots of the SLO compliance of your services. Is that the end? No. You must continuously improve your SLO targets and learn how to make decisions using that information. And that's not the end either, but for the rest you'll need to read the book.

In Chapter 3—SLO Engineering Case Studies—Evernote and The Home Depot tell the story of their journey into SRE.

In Chapter 4—Monitoring—there are examples of moving information from logs to metrics, improving both logs and metrics, and keeping logs as the data source.

In Chapter 6—Eliminating Toil—there are detailed case studies on Reducing Toil in the Datacenter with Automation and Decommission Filer-Backed Home Directories.

And so it goes through nearly every chapter.

As you can see it's a very detailed and thorough book. The preface modestly contends it's a necessarily limited book, but I'd hate to see how many pages would be in the unlimited version.

Like the first book, the writing is clear, purposeful, and well organized. For a company well known for its influential publications, this is another winner.

Best of all? It's free until August 23rd!

Page 1 ... 2 3 4 5 6 ... 227 Next 10 Entries »