advertise
Monday
Jan182016

Use Google For Throughput, Amazon And Azure For Low Latency

Which cloud should you use? It may depend on what you need to do with it. What Zach Bjornson needs to do is process large amounts scientific data as fast as possible, which means reading data into memory as fast as possible. So, he made benchmark using Google's new multi-cloud PerfKitBenchmarker, to figure out which cloud was best for the job.

The results are in a very detailed article: AWS S3 vs Google Cloud vs Azure: Cloud Storage Performance. Feel free to datamine the results for more insights, but overall his conclusions are:

Click to read more ...

Friday
Jan152016

Stuff The Internet Says On Scalability For January 15th, 2016

Hey, it's HighScalability time:


Space walk from 2001: A Space Odyssey? Nope. A base jump from the CN Tower in Toronto.

 

If you like this Stuff then please consider supporting me on Patreon.
  • 13.5TB: open data from Yahoo for machine learning; 1+ exabytes: data stored in the cloud; 13: reasons autonomous cars should have steering wheels; 3,000: kilowatt-hours of energy generated by the solar bike path; 10TB: helium-filled hard disk; $224 Billion: 2016 gadget spending in US; 85: free ebooks; 17%: Azure price drop on some VMs; 20.5: tons of explosives detonated on Mythbusters; 20 Billion: Apple’s App Store Sales; 70%: Global Internet traffic goes through Northern Virginia; 12: photos showing the beauty of symmetry; 

  • Quotable Quote:
    • @WhatTheFFacts: Scaling Earth's 'life' to 46 years, the industrial revolution began 1 minute ago -- In that time we've destroyed half the world's forests.
    • David Brin: The apotheosis of Darth Vader was truly disgusting. Saving one demigod—a good demigod, his son—wiped away all his guilt from slaughtering billions of normal people.
    • Brian Brazil: In today’s world, having a 1:1 coupling between machines and services is becoming less common. We no longer have the webserver machine, we have one machine which hosts one part of the webserver service. 
    • @iamxavier: "Snapchat is said to have 7 billion mobile video views vs Facebook’s 8 bil.The kicker: Fb has 15x Snapchat’s users."
    • Charlie Stross: Do you want to know the real reason George R. R. Martin's next book is late? it's because keeping track of that much complexity and so many characters and situations is hard work, and he's not getting any younger. 
    • @raju: Unicorn-Size Losses: @Uber lost $671.4 million in 2014 & $987.2 million in the first half of 2015
    • @ValaAfshar: 3.8 trillion photos were taken in all of human history until mid-2011. 1 trillion photos were taken in 2015 alone
    • @ascendantlogic: 2010: Rewrite all the ruby apps with javascript 2012: Rewrite all the javascript apps with Go 2014: Rewrite all the Go apps with Rust
    • @kylebrussell: “Virtual reality was tried in the 90s!” Yeah, with screens that had 7.9% of the Oculus Rift CV1 resolution
    • @kevinmarks: #socosy2016 @BobMankoff: people don't like novelty - they like a little novelty in a cocoon of familiarity, that they could have thought of
    • @toddhoffious: The problem nature has solved is efficient variable length headers. Silicon doesn't like them for networks, or messaging protocols. DNA FTW.
    • @jaykreps: I'm loving the price war between cloud providers, cheap compute enables pretty much everything else in technology. 
    • The Confidence Game: Transition is the confidence game’s great ally, because transition breeds uncertainty. There’s nothing a con artist likes better than exploiting the sense of unease we feel when it appears that the world as we know it is about to change.
    • @somic: will 2016 be the year of customer-defined allocation strategies for aws spot fleet? (for example, through a call to aws lambda)
    • beachstartup: i run an infrastructure startup. the rule of thumb is once you hit $20-99k/month, you can cut your AWS bill in half somewhere else. sites in this phase generally only use about 20% of the features of aws.
    • @fart: the most important part of DevOps to me is “kissing the data elf”
    • @destroytoday: In comparison, @ProductHunt drove 1/4 the traffic of Hacker News, but brought in 700+ new users compared to only 20 from HN.
    • @aphyr~ Man, if people knew even a *tenth* of the f*cked up shit tech company execs have tried to pull... Folks are *awfully* polite on twitter.
    • @eric_analytics: It took Uber five years to get to a billion rides, and its Chinese rival just did it in one
    • lowpro: Being a 19 year old college student with many friends in high school, I can say snapchat is the most popular social network, followed by Instagram then Twitter, and lastly Facebook. If something is happening, people will snap and tweet about it, Instagram and Facebook are reserved for bigger events that are worth mentioning, snapchat and Twitter are for more day to day activities and therefore get used much more often.
    • Thaddeus Metz: The good, the true, and the beautiful give meaning to life when we transcend our animal nature by using our rational nature to realize states of affairs that would be appreciated from a universal perspective.
    • Reed Hastings: We realized we learned best by getting in the market and then learning, even if we’re less than perfect. Brazil is the best example. We started [there] four years ago. At first it was very slow growth, but because we were in the market talking to our members who had issues with the service, we could get those things fixed, and we learned faster.

  • Why has Bitcoin failed? From Mike Hearn: it has failed because the community has failed. What was meant to be a new, decentralised form of money that lacked “systemically important institutions” and “too big to fail” has become something even worse: a system completely controlled by just a handful of people. Worse still, the network is on the brink of technical collapse. The mechanisms that should have prevented this outcome have broken down, and as a result there’s no longer much reason to think Bitcoin can actually be better than the existing financial system.

  • Lessons learned on the path to production. From Docker CEO: 1) IaaS is too low; 2) PaaS is too high: Devs do not adopt locked down platforms; 3) End to end matters: Devs care about deployment, ops cares about app lifecycle and origin; 4) Build management, orchestration, & more in a way that enables portability; 5) Build for resilience, not zero defects; 6) If you do 5 right, agility + control

  • Is this the Tesla of database systems? No Compromises: Distributed Transactions with Consistency, Availability, and Performance: FaRMville transactions are processed by FaRM – the Fast Remote Memory system that we first looked at last year. A 90 machine FaRM cluster achieved 4.5 million TPC-C ‘new order’ transactions per second with a 99th percentile latency of 1.9ms. If you’re prepared to run at ‘only’ 4M tps, you can cut that latency in half. Oh, and it can recover from failure in about 60ms. 

  • Uber tells the story behind the design and implementation of their scalable datastore using MySQL. Uber took that path of many others in writing an entire layer on top of MySQL to create the database that best fits their use case. Uber wanted: to be able to linearly add capacity by adding more servers; write availability; a way of notifying downstream dependencies; secondary indexes; operation trust in the system, as it contains mission-critical trip data. They looked at Cassandra, Riak, and MongoDB, etc. Features alone did not decide their choice. What did?: "the decision ultimately came down to operational trust in the system we’d use."  If you are Uber this is a good reason that may not seem as important to those without accountability. Uber's design is inspired by Friendfeed, and the focus on the operational side inspired by Pinterest.

Don't miss all that the Internet has to say on Scalability, click below and become eventually consistent with all scalability knowledge (which means this post has many more items to read so please keep on reading)...

Click to read more ...

Wednesday
Jan132016

Live Video Streaming At Facebook Scale

With 1.49 billion monthly active users, operating at Facebook scale is far from trivial. Facebook's new live video streaming services present a fascinating use case for designing streaming service in global distribution and massive scale.

Click to read more ...

Monday
Jan112016

A Beginner's Guide to Scaling to 11 Million+ Users on Amazon's AWS

How do you scale a system from one user to more than 11 million users? Joel Williams, Amazon Web Services Solutions Architect, gives an excellent talk on just that subject: AWS re:Invent 2015 Scaling Up to Your First 10 Million Users.

If you are an advanced AWS user this talk is not for you, but it’s a great way to get started if you are new to AWS, new to the cloud, or if you haven’t kept up with with constant stream of new features Amazon keeps pumping out.

As you might expect since this is a talk by Amazon that Amazon services are always front and center as the solution to any problem. Their platform play is impressive and instructive. It's obvious by how the pieces all fit together Amazon has done a great job of mapping out what users need and then making sure they have a product in that space. 

Some of the interesting takeaways:

  • Start with SQL and only move to NoSQL when necessary.
  • A consistent theme is take components and separate them out. This allows those components to scale and fail independently. It applies to breaking up tiers and creating microservices.
  • Only invest in tasks that differentiate you as a business, don't reinvent the wheel.
  • Scalability and redundancy are not two separate concepts, you can often do both at the same time.
  • There's no mention of costs. That would be a good addition to the talk as that is one of the major criticisms of AWS solutions.

The Basics

Click to read more ...

Sunday
Jan102016

Uptime Funk - Best Sysadmin Parody Video Ever!

This is so good! Perfect for your Monday morning jam.

 

Uptime Funk is a music video (parody of Uptown Funk) from SUSECon 2015 in Amsterdam.
My favorite: 
I'm all green (hot patch)
Called a Penguin and Chameleon
I'm all green (hot patch)
Call Torvalds and Kroah-Hartman
It’s too hot (hot patch)
Yo, say my name you know who I am
It’s too hot (hot patch)
I ain't no simple code monkey
Nuthin's down
Friday
Jan082016

Stuff The Internet Says On Scalability For January 8th, 2016

Hey, it's HighScalability time:


Finally, a clear diagram of Amazon's industry impact. (MARK A. GARLICK)

 

If you like this Stuff then please consider supporting me on Patreon.
  • 150: # of globular clusters in the Milky Way; 800 million: Facebook Messenger users; 180,000: high-res images of the past; 1 exaflops: 1 million trillion floating-point operations per second; 10%: of Google's traffic is now IPv6; 100 milliseconds: time it takes to remember; 35: percent of all US Internet traffic used by Netflix; 125 million: hours of content delivered each day by Netflix's CDN;

  • Quotable Quotes:
    • Erik DeBenedictis: We could build an exascale computer today, but we might need a nuclear reactor to power it
    • wstrange: What I really wish the cloud providers would do is reduce network egress costs. They seem insanely expensive when compared to dedicated servers.
    • rachellaw: What's fascinating is the bot-bandwagon is mirroring the early app market.
      With apps, you downloaded things to do things. With bots, you integrate them into things, so they'll do it for you. 
    • erichocean: The situation we're in today with RAM is pretty much the identical situation with the disks of yore.
    • @bernardgolden: @Netflix will spend 2X what HBO does on programming in 2016? That's an amazing stat. 
    • @saschasegan: Huawei's new LTE modem has 18 LTE bands. Qualcomm's dominance of LTE is really ending this year.
    • Unruly Places: The rise of placelessness, on top of the sense that the whole planet is now minutely known and surveilled, has given this dissatisfaction a radical edge, creating an appetite to find places that are off the map and that are somehow secret, or at least have the power to surprise us.
    • @mjpt777: Queues are everywhere. Recognise them, make them first class, model and monitor them for telemetry.
    • Guido de Croon:  the robot exploits the impending instability of its control system to perceive distances. This could be used to determine when to switch off its propellers during landing, for instance.
    • @gaberivera: In the future, all major policy questions will be settled by Twitter debates between venture capitalists
    • Craig McLuckie: It’s not obvious until you start to actually try to run massive numbers of services that you experience an incredible productivity that containers bring
    • Brian Kirsch: One of the biggest things when you look at the benefits of container-based virtualization is its ability to squeeze more and more things onto a single piece of hardware for cost savings. While that is good for budgets, it is excessively horrible when things go bad.
    • @RichardWarburto: It still surprises me that configuration is most popular user of strong consistency models atm. Is config more important than data
    • @jamesurquhart: Five years ago I predicted CFO would stop complaining about up front cost, and start asking to reduce monthly bill. Seeing that happen now.
    • @martinkl: Communities in a nutshell… • Databases research: “In fsync we trust” • Distributed systems research: “In majority vote we trust”
    • @BoingBoing: Tax havens hold $7.6 trillion; 8% of world's total wealth
    • @DrQz: Amazon's actual profits are still tiny, relying heavily on its AWS cloud business.
    • hadagribble: we need to view fast storage as something other than disk behind a block interface and slow memory, especially with all the different flavours of fast persistent storage that seem to be on the horizon. For the one's that attach to the memory bus, the PMFS-style [1] approach of treating them like a file-system for discoverability and then mmaping to allow them to be accessed as memory is pretty attractive.

  • EC2 with a 5% price reduction on certain things in certain places. Not exactly the race to the bottom one would hope for in a commodity market, which means the cloud is not a commodity. Happy New Year – EC2 Price Reduction (C4, M4, and R3 Instances).

  • Since the locus of the Internet is centering on a command line interface in the form of messaging, chatbot integrations may be giving APIs a second life, assuming they are let inside the walled garden. The next big thing in computing is called 'ChatOps,' and it's already happening inside Slack. The advantage chatops has over the old Web + API mashup dream is that messaging platforms come built-in with a business model/app store, large amd growing user base, and network effects. Facebook’s Secret Chat SDK Lets Developers Build Messenger Bots. Slack apps. WeChat API. Telegram API. Alexa API. Google's Voice Actions. How about Siri or iMessage? Nope. njovin likes it: I've worked with the new Chat SDK and our customers' use cases aren't geared toward forcing (or even encouraging) users into using Facebook Messenger. Most of them are just trying to meet demand from their customers. In our particular case, we have customers with a lot of international travelers who have access to data while abroad but not necessarily SMS. IMO it's a lot better than having a dedicated app you have to download to interact with a specific brand.

  • The world watched a lot of porn this year. If you like analytics you'll love Pornhub’s 2015 Year in Review: In 2015 alone, we streamed 75GB of data a second; bandwidth used is 1,892 petabytes; 4,392,486,580 hours of video were watched; 21.2 billion visits.

  • A very interesting way to frame the issue. On the dangers of a blockchain monoculture: The Bitcoin blockchain: the world’s worst database. Would you use a database with these features? Uses approximately the same amount of electricity as could power an average American household for a day per transaction. Supports 3 transactions / second across a global network with millions of CPUs/purpose-built ASICs. Takes over 10 minutes to “commit” a transaction. Doesn’t acknowledge accepted writes: requires you read your writes, but at any given time you may be on a blockchain fork, meaning your write might not actually make it into the “winning” fork of the blockchain (and no, just making it into the mempool doesn’t count). In other words: “blockchain technology” cannot by definition tell you if a given write is ever accepted/committed except by reading it out of the blockchain itself (and even then). Can only be used as a transaction ledger denominated in a single currency, or to store/timestamp a maximum of 80 bytes per transaction. But it’s decentralized!

Don't miss all that the Internet has to say on Scalability, click below and become eventually consistent with all scalability knowledge (which means this post has many more items to read so please keep on reading)...

Click to read more ...

Wednesday
Jan062016

Let's Donate Our Organs and Unused Cloud Cycles to Science

There’s a long history of donating spare compute cycles for worthy causes. Most of those efforts were started in the Desktop Age. Now, in the Cloud Age, how can we donate spare compute capacity? How about through a private spot market?

There are cycles to spare. Public Cloud Usage trends:

  • Instances are underutilized with average utilization rates between 8-9%

  • 24% of instance reservations are unused

Maybe all that CapEx sunk into Reserved Instances can be put to some use? Maybe over provisioned instances could be added to the resource pool as well? That’s a lot of power Captain. How could it be put to good use?

There is a need to crunch data. For science. Here’s a great example as described in This is how you count all the trees on Earth. The idea is simple: from satellite pictures count the number of trees. It’s an embarrassingly parallel problem, perfect for the cloud. NASA had a problem. Their cloud is embarrassingly tiny. 400 hypervisors shared amongst many projects. Analysing all the data would would take 10 months. An unthinkable amount of time in this Real-time Age. So they used the spot market on AWS.

The upshot? The test run cost a measly $80, which means that NASA can process data collected for an entire UTM zone for just $250. The cost for all 11 UTM zones in sub-Sarahan Africa and the use of all four satellites comes in at just $11,000.

“We have turned what was a $200,000 job into a $10,000 job and we went from 100 days to 10 days [to complete],” said Hoot. “That is something scientists can build easily into their budget proposals.”

That last quote, That is something scientists can build easily into their budget proposals, stuck in my craw.

Imagine how much science could get done if you didn’t have the budget proposal process slowing down the future? Especially when we know there are so many free cycles available that are already attached to well supported data processing pipelines. How could those cycles be freed up to serve a higher purpose?

Netflix shows the way with their internal spot market. Netflix has so many cloud resources at their disposal, a pool of 12,000 unused reserved instances at peak times, that they created their own internal spot market to drive better utilization. The whole beautiful setup is described Creating Your Own EC2 Spot Market, Creating Your Own EC2 Spot Market -- Part 2, and in High Quality Video Encoding at Scale.

The win: By leveraging the internal spot market Netflix measured the equivalent of a 210% increase in encoding capacity.

Netflix has a long and glorious history of sharing and open sourcing their tools. It seems likely when they perfect their spot market infrastructure it could be made generally available.

Perhaps the Netflix spot market could be extended so unused resources across the Clouds could advertise themselves for automatic integration into a spot market usable by scientists to crunch data and solve important world problems.

Perhaps donated cycles could even be charitable contributions that could help offset the cost of the resource? My wife is a tax accountant and she says this is actually true, under the right circumstances.

This kind of idea has a long history with me. When AWS first started, I like a lot of people wondered, how can I make money off this gold rush? That’s before we knew Amazon was going to make most of the tools to sell to the miners themselves. The idea of exploiting underutilized resources fascinated me for some reason. That is, after all, what VMs do for physical hardware, exploit the underutilized resources of powerful machines. And it is in some ways the idea behind our modern economy. Yet even today software architectures aren’t such that we reach anything close to full utilization of our hardware resources. What I wanted to do was create a memcached system that allowed developers to sell their unused memory capacity (and later CPU, network, storage) to other developers as cheap dynamic pools of memcached storage. Get your cache dirt cheap and developers could make some money back on underused resources. A very similar idea to the spot market notion. But without homomorphic encryption the security issues were daunting, even assuming Amazon would allow it. With the advent of the Container Age sharing a VM is now way more secure and Amazon shouldn’t have a problem with the idea if it’s for science. I hope.

Tuesday
Jan052016

Sponsored Post: Netflix, StatusPage.io, Redis Labs, Jut.io, SignalFx, InMemory.Net, VividCortex, MemSQL, Scalyr, AiScaler, AppDynamics, ManageEngine, Site24x7

Who's Hiring?

  • Manager - Site Reliability Engineering: Lead and grow the the front door SRE team in charge of keeping Netflix up and running. You are an expert of operational best practices and can work with stakeholders to positively move the needle on availability. Find details on the position here: https://jobs.netflix.com/jobs/398

  • Senior Service Reliability Engineer (SRE): Drive improvements to help reduce both time-to-detect and time-to-resolve while concurrently improving availability through service team engagement.  Ability to analyze and triage production issues on a web-scale system a plus. Find details on the position here: https://jobs.netflix.com/jobs/434

  • Manager - Performance Engineering: Lead the world-class performance team in charge of both optimizing the Netflix cloud stack and developing the performance observability capabilities which 3rd party vendors fail to provide.  Expert on both systems and web-scale application stack performance optimization. Find details on the position here https://jobs.netflix.com/jobs/860482

  • Senior Devops Engineer - StatusPage.io is looking for a senior devops engineer to help us in making the internet more transparent around downtime. Your mission: help us create a fast, scalable infrastructure that can be deployed to quickly and reliably.

  • UI EngineerAppDynamics, founded in 2008 and lead by proven innovators, is looking for a passionate UI Engineer to design, architect, and develop our their user interface using the latest web and mobile technologies. Make the impossible possible and the hard easy. Apply here.

  • Software Engineer - Infrastructure & Big DataAppDynamics, leader in next generation solutions for managing modern, distributed, and extremely complex applications residing in both the cloud and the data center, is looking for a Software Engineers (All-Levels) to design and develop scalable software written in Java and MySQL for backend component of software that manages application architectures. Apply here.

Fun and Informative Events

  • Your event could be here. How cool is that?

Cool Products and Services

  • Turn chaotic logs and metrics into actionable data. Scalyr is a tool your entire team will love. Get visibility into your production issues without juggling multiple tools and tabs. Loved and used by teams at Codecademy, ReturnPath, and InsideSales. Learn more today or see why Scalyr is a great alternative to Splunk.

  • Real-time correlation across your logs, metrics and events.  Jut.io just released its operations data hub into beta and we are already streaming in billions of log, metric and event data points each day. Using our streaming analytics platform, you can get real-time monitoring of your application performance, deep troubleshooting, and even product analytics. We allow you to easily aggregate logs and metrics by micro-service, calculate percentiles and moving window averages, forecast anomalies, and create interactive views for your whole organization. Try it for free, at any scale.

  • Turn chaotic logs and metrics into actionable data. Scalyr replaces all your tools for monitoring and analyzing logs and system metrics. Imagine being able to pinpoint and resolve operations issues without juggling multiple tools and tabs. Get visibility into your production systems: log aggregation, server metrics, monitoring, intelligent alerting, dashboards, and more. Trusted by companies like Codecademy and InsideSales. Learn more and get started with an easy 2-minute setup. Or see how Scalyr is different if you're looking for a Splunk alternative or Sumo Logic alternative.

  • SignalFx: just launched an advanced monitoring platform for modern applications that's already processing 10s of billions of data points per day. SignalFx lets you create custom analytics pipelines on metrics data collected from thousands or more sources to create meaningful aggregations--such as percentiles, moving averages and growth rates--within seconds of receiving data. Start a free 30-day trial!

  • InMemory.Net provides a Dot Net native in memory database for analysing large amounts of data. It runs natively on .Net, and provides a native .Net, COM & ODBC apis for integration. It also has an easy to use language for importing data, and supports standard SQL for querying data. http://InMemory.Net

  • VividCortex goes beyond monitoring and measures the system's work on your servers, providing unparalleled insight and query-level analysis. This unique approach ultimately enables your team to work more effectively, ship more often, and delight more customers.

  • MemSQL provides a distributed in-memory database for high value data. It's designed to handle extreme data ingest and store the data for real-time, streaming and historical analysis using SQL. MemSQL also cost effectively supports both application and ad-hoc queries concurrently across all data. Start a free 30 day trial here: http://www.memsql.com/

  • aiScaler, aiProtect, aiMobile Application Delivery Controller with integrated Dynamic Site Acceleration, Denial of Service Protection and Mobile Content Management. Also available on Amazon Web Services. Free instant trial, 2 hours of FREE deployment support, no sign-up required. http://aiscaler.com

  • ManageEngine Applications Manager : Monitor physical, virtual and Cloud Applications.

  • www.site24x7.com : Monitor End User Experience from a global monitoring network.

If any of these items interest you there's a full description of each sponsor below. Please click to read more...

Click to read more ...

Monday
Jan042016

Server-Side Architecture. Front-End Servers and Client-Side Random Load Balancing

Chapter by chapter Sergey Ignatchenko is putting together a wonderful book on the Development and Deployment of Massively Multiplayer Games, though it has much broader applicability than games. Here's a recent chapter from his book.

Enter Front-End Servers

[Enter Juliet]
Hamlet:
Thou art as sweet as the sum of the sum of Romeo and his horse and his black cat! Speak thy mind!
[Exit Juliet]

— a sample program in Shakespeare Programming Language

 

 

Front-End Servers as an Offensive Line

 

Our Classical Deployment Architecture (especially if you do use FSMs) is not bad, and it will work, but there is still quite a bit of room for improvement for most of the games out there. More specifically, we can add another row of servers in front of the Game Servers, as shown on Fig VI.8:

Click to read more ...

Friday
Jan012016

Stuff The Internet Says On Scalability For January 1st, 2016

Hey, Happy New Year, it's HighScalability time:


River system? Vascular system? Nope. It's a map showing how all roads really lead to Rome.

 

If you like Stuff The Internet Says On Scalability then please consider supporting me on Patreon.
  • 71: mentions of innovation by the Chinese Communist Party; 60.5%: of all burglaries involve forcible entry; 280,000-squarefoot: Amazon's fulfillment center in India capable of shipping 2 million items; 11 billion: habitable earth like planets in the goldilocks zone in just our galaxy; 800: people working on the iPhone's camera (how about the app store?); 3.3 million: who knew there were so many Hello Kitty fans?; 26 petabytes: size of League of Legends' data warehouse; 

  • Quotable Quotes:
    • George Torwell: Tor is Peace / Prism is Slavery / Internet is Strength
    • @SciencePorn: Mr Claus will eat 150 BILLION calories and visit 5,556 houses per second this Christmas Eve.
    • @SciencePorn: Blue Whale's heart is so big, a small child can swim through the veins.
    • @BenedictEvans: There are close to 4bn people on earth with a phone (depending on your assumptions). Will go to at least 5bn. So these issues will grow.
    • @JoeSondow: "In real life you won't always have a calculator with you." — math teachers in the 80s
    • James Hamilton: This is all possible due to the latencies we see with EC2 Enhanced networking. Within an availability zone, round-trip times are now tens of microseconds, which make it feasible to propose and commit transactions to multiple resilient nodes in less than a millisecond.
    • Benedict Evans: The mobile ecosystem, now, is heading towards perhaps 10x the scale of the PC industry, and mobile is not just a new thing or a big thing, but that new generation, whose scale makes it the new centre of gravity of the tech industry. Almost everything else will orbit around it. 
    • Ruth Williams: Bacteria growing in an unchanging environment continue to adapt indefinitely.
    • @Raju: Not one venture-backed news aggregator has yet shown a Sustainable Business Model
    • @joeerl: + choose accurate names + favor beauty over performance + design minimal essential API's + document the unobvious
    • @shibuyashadows: There is no such thing as a full-node anymore. Now there are two types: Mining Nodes Economic Nodes. Both sets are now semi-centralized on the network, are heavily inter-dependent and represent the majority of the active Bitcoin users.
    • @TheEconomist: In 1972 a man with a degree aged 25-34 earned 22% more than a man without. Today, it's 70%
    • Dr. David Miller~ We are in the age of Howard Hughes. People make their fortune elsewhere and spend it on space. 
    • Credit for CRISPR: Part of that oversimplification is rooted in the fact that most modern life-science researchers aren’t working to uncover broad biological truths. These days the major discoveries lie waiting in the details
    • @BenedictEvans: Idle observation: Facebook will almost certainly book more revenue in 2015 than the entire internet ad industry made up until to 2000
    • Eric Clemmons: Ultimately, the problem is that by choosing React (and inherently JSX), you’ve unwittingly opted into a confusing nest of build tools, boilerplate, linters, & time-sinks to deal with before you ever get to create anything.
    • Kyle Russell: Why do I need such a powerful PC for VR? Immersive VR experiences are 7x more demanding than PC gaming.
    • @josevalim: The system that manages rate limits for Pinterest written in Elixir with a 90% response time of 800 microseconds.
    • catnaroek: The normal distribution is important because it arises naturally when the preconditions of the central limit theorem hold. But you still have to use your brain - you can't unquestioningly assume that any random variable (or sample or whatever) you will stumble upon will be approximately normally distributed.
    • Dominic Chambers: Now, if you consider the server-side immutable state atom to be a materialized view of the historic events received by a server, you can see that we've already got something very close to a Samza style database, but without the event persistence.
    • Joscha Bach: In my view, the 20th century’s most important addition to understanding the world is not positivist science, computer technology, spaceflight, or the foundational theories of physics. It is the notion of computation. Computation, at its core, and as informally described as possible, is very simple: every observation yields a set of discernible differences.

  • The New Yorker is picking up on the Winner Takes All theme that's been developing, I guess that makes it an official meme. What's missing from their analysis is that users are attracted to the eventual winners because they provide a superior customer experience. Magical algorithms are in support of experience. As long as a product doesn't fail at providing that experience there's little reason to switch after being small networked into a choice. You might think many many products could find purchase along the long tail, but in low friction markets that doesn't seem to be the case. Other choices become invisible and what's invisible starves to death.

  • I wonder how long it took to get to the 1 billionth horse ride? Uber Hits One Billionth Ride in 5.5 years.

  • Let's say you are a frog that has been in a warming pot for the last 15 years, what would you have missed? Robert Scoble has put together quite a list. 15 years ago there was no: Facebook, YouTube, Twitter, Google+, Quora, Uber, Lyft, iPhone, iPads, iPod, Android, HDTV, self driving cars, Waze, Google Maps, Spotify. Soundcloud, WordPress, Wechat, Flipkart, AirBnb, Flipboard, LinkedIn, AngelList, Techcrunch, Google Glass, Y Combinator, Techstars, Geekdom, AWS, OpenStack, Azure, Kindle, Tesla, and a lot more.

  • He who controls the algorithm reaps the rewards. Kansas is now the 5th state where lottery prizes may have been fixed.

  • What Is The Power Grid? A stunning 60% of generated energy is lost before it can be consumed, which is why I like my power grids like my databases: distributed and shared nothing.

Don't miss all that the Internet has to say on Scalability, click below and become eventually consistent with all scalability knowledge (which means this post has many more items to read so please keep on reading)...

Click to read more ...