Stuff The Internet Says On Scalability For May 5th, 2017

Hey, it's HighScalability time:

GPUs and CPUs run hot hot hot. See them in action with thermal imaging. (Tested)If you like this sort of Stuff then please support me on Patreon.

• 25ms: SpaceX satellite latency; 17 million: tax returns received by IRS during week ending April 21; 1.94 billion: Facebook users; 1.2 billion: Lambda requests by Expedia / month; ~$91.5K: Capital One's yearly Serverless TCO; 1.2 billion: Facebook Messenger users; 215 petabytes: storage per gram of DNA; 1/2: households in US are Amazon Prime members; 50.8%: households in US that are mobile phone only; 80 billion: street view images; 3 million: open sourced Instacart orders; $175: RaaS (ransomware-as-a-service); 350,000+: Amazon employees; 


• QuotableQuotes:
  • Paul Barnum: You can have a second computer when you've shown you know how to use the first one
  • @chrisalbon: 2007: “You are the product.”  2017: “You are the training data.”
  • shitloadofbooks: As an Ops guy, I preach Ansible + systemd all day everyday, but so many of our Devs (and Ops) have drunk the containerization Kool-aid.
  • roland-s: Like you, I'm sometimes unsure if this is the right choice. Maybe a monolithic server or traditional VMs + Puppet would be easier, simpler, better? In the end, I think Docker just fit with the way I conceptualized my problem so I went for it.
  • Venki Ramakrishnan: each experiment generates several terabytes of data, which is then massaged, analyzed, and reduced, and finally you get a structure. 
  • @dberkholz: A 19-line sample pulled in 190,000 lines of code in dependencies. Is that what you call a 10000x programmer? #ServerlessConf
  • @asymco: Apple Watch continues to struggle as unit sales more than doubled in six of top 10 markets
  • @pomeranian99: Memory leaks on missiles don't matter, so long as the missile explodes before too much leaks. A 1995 memo: 
  • Paul Johnston: Most of these vendors can cope with what you throw at them so just go for it and stop trying to keep your options open. That way lies madness and mediocrity for your solution (at present).
  • @BrewersStats: 0.3% of the largest breweries make 69.3% of the beer. Conversely, 76.5% of the smallest make 0.7% of the beer.
  • @howardlindzon: Apple is 12.3 billion away from being the first Trillion dollar company
  • @michael_adda: Completely agree with the #serverless async/sync argument "Concurrency within a flow? it needs to move into our infrastructure"
  • resident_ninja: making literally EVERYTHING a stored proc creates a very bad, tight coupling between the app and db, kills scalability, and increases the pain of app and website deployments 
  • Impact Lab: There are about 1,200 malls in America today. In a decade, there might be about 900. That’s not quite the “the death of malls.” But it is decline, and it is inevitable.
  • Joel Frohlich: at that point in history, no other human being had ever experienced a focused beam of radiation at such high energy
  • Shazam: Whenever a user Shazams a song, our algorithm uses GPUs to search that database until it finds a match. This happens successfully over 20 million times per day.
  • Dmitri Zimine: You will rewrite your app, not to move to the other provider but by the progress of your cloud provider. They change existing services and introduce new ones
  • Werner Vogels: Creating added value in an Industry-4.0 environment often happens when B2B companies integrate B2C approaches, in turn sparking change in their own industry.
  • Sivabudh Pac Umpudh: And I hate to wait to discover my typos and type mismatches only after the application runs.
  • @t_crayford: Silly interview question: Which has more impact on global warming, bitcoins or JSON parsing?
  • @trieloff: I processed almost two terabytes of data for this blog post. Today Google sent me the bill: €3.55
  • @mjpt777: Distributed designs based on synchronous request-response may appear easier to reason about but are difficult to compose or make efficient.
  • @sknthla: "Amazon has reduced its total shipping cost by over 50% since 2006" - unnoticed innovation in logistics
  • Margaret Roach: The big shift in horticulture in the next decade will be a shift from thinking about plants as individual objects to communities of interrelated species.
  • @edent: Does anyone have an EASY tutorial for Alexa skills? As in, not 27 steps where step 5 just says "install node" & 26 says "see other tutorial"
  • peterwilli: Scaleway now offers x86 servers. I moved 4 DO instances to 1 bare-metal Scaleway server and it works really well (but it is still slower than dedicated SSD)
  • Joseph Jacks~ I think, the best way to monetize OSS while leveraging the commoditization aspects where appropriate: true category creation based on major shifts. Not just exploiting shifts and framing the offering/value around that. Actually going out and defining, building and setting the stage for a new layer in the stack that needs to exist. 
  • Venki Ramakrishnan: We're at the threshold of a new age of structural biology, where these things that everybody thought were too difficult and would take decades and decades, are all cracking. Now we're coming to pieces of the cell. The real advance is that you're going to be able to look at all these machines and large molecular complexes inside the cell. It will tell you detailed molecular organization of the cell. That's going to be a big leap, to go from molecules to cells and how cells work.
  • James Hamilton: Walmart is still bringing in more revenue than any other company period. $482B. That’s ½ trillion dollars. Looking at the Gross Domestic Product of entire countries (all the finished goods and services produced by the country in a year), Walmart revenue when compared to country GDP would place just slightly behind Poland and well ahead of Belgium.  A truly staggering number.


• Filed under the coolest use of machine learning category. Algorithmic ‘Printed’ Fields Could Make Farms More Productive and Resilient: UK-based designer Benedikt Groß has created algorithmic models that enable him to plant various crops in complex patterns in a field. This improves ecological resilience and diversity through fascinating patterns that are best appreciated from above.


• Fascinating look into the architecture of Hajime, a sophisticated, a flexible and future-proof IoT botnet: The distributed bot network used for command and control and updating is overlaid as a traceless torrent on top of the well-know public BitTorrent peer-to-peer network using dynamic info_hashes that change on a daily basis. All communications through BitTorrent are signed and encrypted using RC4 and private/public keys...The loader stub is dynamically generated using hex encoded strings based on handcrafted assembly programs that are optimized for each supported platform...Hajime does not rashly follow a fixed sequence of credentials; the credentials used during an exploit variate depending on the login banner of the victim. 


• Is Node is the wrong runtime for FaaS? Node is the WRONG runtime for serverless. The idea is your runtime doesn't need to be async because all the concurrency between calls/flows stuff should be pushed to the FaaS platform. Do not build little webservers in your Lambda functions. Can't agree. Functions can run for minutes or even hours. There's no need to increase latency and push all that state maintenance to the platform. Obfuscating code by splatting the logic all over the place isn't a win either. 


• Going from Caches to Cash on AWS using the EC2 Spot Market: our entire platform runs on Spot, which saves us upwards of 50-90% on our EC2 costs. ElastiCache doesn’t support Spot, so we rolled our own service that does.


• The Cloud Native Foundation (Kubernetes) is creating a document defining what CNF should do in the Serverless/FaaS space. Could be important going forward. 


• Intel, did you keep is secret? Yes. Did you keep it safe? Nope. Secret backdoors are not not safe by definition. Remote security exploit in all 2008+ Intel platforms: every Intel platform with AMT, ISM, and SBT from Nehalem in 2008 to Kaby Lake in 2017 has a remotely exploitable security hole in the ME (Management Engine) not CPU firmware. If this isn’t scary enough news, even if your machine doesn’t have SMT, ISM, or SBT provisioned, it is still vulnerable, just not over the network. For the moment. From what SemiAccurate gathers, there is literally no Intel box made in the last 9+ years that isn’t at risk. This is somewhere between nightmarish and apocalyptic.


• As above so below. Building Express Backbone: Facebook’s new long-haul network: one of the insights to making the above happen was following some of the design patterns we use in our data center networks. Keeping the number of network prefixes small (on the order of thousands) in the new network enabled us to leverage commodity network gear. And splitting the physical topology network into four parallel topologies (known as “planes”) was directly based on the idea of our existing “four plane” data center network fabric.


• If you would like to learn more about how deep learning will replace programmers then watch Peter Norvig's talk As We May Program.


• Perhaps if Google can use machine learning to compose automatic answers to emails they can use the same tech to figure out when someone is phishing by impersonating Google? Don’t trust OAuth: Why the “Google Docs” worm was so convincing


• Caching out of Hadoop: How New York Times Embraces New Technology: This is exactly what Rockwell did with Hadoop. Leaving Hadoop in favor of Google’s BigQuery, said Rockwell, resulted in a “Massive cost savings, a huge headache savings, and a giant productivity increase 


• Cell networks are not secure. Don't use SMS for 2FA. After years of warnings, mobile network hackers exploit SS7 flaws to drain bank accounts. 


• Reducing Slack’s memory footprint. Worth reading simply for the diagram showing when a notification is sent. A web application can use between ~130MB (p10) and ~960MB (p99) of RAM. 36% of active Slack users are signed into more than one team in the desktop app. 17% are signed into 3 or more teams, and 5% are signed into 5 or more teams. That's a lot of RAM. Less RAM was used by doing more work on the server and having a thinner client, but the comments are overwhelmingly asking for a more performant native app instead of a web app.


• Storage engine design: Akumuli is based on novel data-structure called “Numeric B+tree”...The data structure itself can be described as B+LSM-tree. Think about LSM-tree but with B+trees instead of SSTables...This data-structure allows Akumuli to maintain separate NB+tree instance for every time-series in the database...It enables queries in column-oriented fashion and, at the same time, it enables really fast parallel writes with very small write amplification. My recent experiment showed that it can write about 16 million data points per second on a c3.8xlarge instance.


• Good description of eight Serverless Design Patterns: Function Chaining, Function Chaining with Rollback (transaction), ASync HTTP (HTTP 202), Fanout (Parallel), Fanout + Fan-in, Long Running Function, Long Running Function with Timeout, Manual interaction with Timeout.


• Making Slack Faster By Being Lazy (part 1, part2). Things that work when you have a few channels don't work when you have many. Lessons: Don’t load all channels’ unread messages right away; Don’t fetch more than needed for channels with unreads; New messages received can present an opportunity to prefetch history; Use “frecency” and user actions to inform and prefetch the user’s next move; Assume the volume of objects in your world (e.g., members, channels, messages) may increase by an order of magnitude; Avoid “loading the world” and enjoy being lazy; Avoid using LocalStorage in general


• Fun list of Node.js War Stories: Debugging Issues in Production. Lessons: Always know your dependencies; Observability is key; Latency in-between your services has a huge impact on user experience; Crypto can take a LOT of time; Always pay attention to CPU bound operations; String operations are CPU-heavy operations. 


• How to do queues for AWS Lambda (part 2). It makes no sense you can't hook SQS directly to Lambda. 


• Brendan Gregg shows how to use BPF (Berkeley Packet Filter) for performance analysis on Linux. Always a treat. 


• Geoff Huston with a detailed IETF 98 Report. Topics include Homenet,  Distributed Mobility Management, Congestion Control, DNSOPS, NTP, Inter-Domain Routing, Multi-Path TCP, QUIC.  


• Considerations for Moving Services to AWS Lambda: Using AWS API Gateway, you will also incur a charge of $3.50/month for the first million requests, resulting in a grand total of $3.71 per month. This price difference may or may not actually be a big deal to you depending on your scale. For some infrastructures, $121/month is still a rounding error. But what’s the tipping point when it makes sense to run on EC2 instead of Lambda; the point where the number of requests outweighs the costs of running a host full time? 


• Cloud GPUs vs baremetal GPUs. The cloud, as usual, wins in a peak vs variable use scenario. Shazam: Why cloud GPUs finally make sense: To meet that demand, we've been maintaining a fleet of GPUs on dedicated bare metal servers that we lease from a managed services provider. Because of the time it takes to source and provision a new physical server, we provision enough to meet peak demand and then run that capacity 24/7, 365 days a year...Thanks to the speed with which we can dial new instances up and down, we maintain GPU infrastructure to handle average use instead of the full capacity for our maximum peak load. Thus far, we’ve migrated about one-third of our infrastructure into Google Cloud...Of course, running on bare metal servers, we also need to provision extra capacity for the inevitable failure scenarios we all experience when operating services at scale. One of the amazing benefits of running in Google is that we can now replace a failed node in just minutes with a brand new one “off the shelf” instead of keeping a pool of nodes around just waiting for failures.


• Videos from Functional Swift Conference 2017 — Brooklyn are now available.


• A high quality series of posts: Let’s Build A Simple Interpreter. Part 1...Let’s Build A Simple Interpreter. Part 13: Semantic Analysis.


• Marcia Villalba with a good review of ServerlessConf Austin. 


• Storing UUID and Generated Columns: Decomposing the UUID is an alternative to storing them in order, but it won’t speed up inserts. It is simpler to execute queries over a range of dates, and look at the row for a particular device, as you will be able to use the MAC (it is recommended to add an index for it). Generated columns give you the possibility to build the UUID back in just one string.


• Where do you train your models? Where do you run inference? Machine learning on mobile: on the device or in the cloud explains all your options. 


• Irony: machine learning courses ranked by humans. Every single Machine Learning course on the internet, ranked by your reviews. 


• pywren/pywren (video):  a transparent distributed execution engine on top of AWS Lambda, which hopefully simplifies many scale-out use cases for data science and computational imaging.  


• trailofbits/manticore: prototyping tool for dynamic binary analysis, with support for symbolic execution, taint analysis, and binary instrumentation.


• improbable-eng/grpc-web: moves the interaction between frontend code and microservices from the sphere of hand-crafted HTTP requests to well-defined user-logic methods.


• github/octodns (article): provides a set of tools & patterns that make it easy to manage your DNS records across multiple providers. The resulting config can live in a repository and be deployed just like the rest of your code, maintaining a clear history and using your existing review & workflow.


• transceptor-technology/siridb-server:  a highly-scalable, robust and super fast time series database. Build from the ground up SiriDB uses a unique mechanism to operate without indexes and allows server resources to be added on the fly. SiriDB's unique query language includes dynamic grouping of time series for easy analysis over large amount's of time series.


• Economies of Density in E-Commerce: A Study of Amazon's Fulfillment Center Network: We find that Amazon saves between $0.17 and $0.47 for every 100 mile reduction in the distance of shipping goods worth $30. In the context of its distribution network expansion, this estimate implies that Amazon has reduced its total shipping cost by over 50% and increased its profit margin by between 5 and 14% since 2006. Separately, we demonstrate that prices on Amazon have fallen by approximately 40% over the same period, suggesting that a significant share of the cost savings have been passed on to consumers.


• Mosaic: Processing a Trillion-Edge Graph on a Single Machine: Our evaluation shows that for smaller graphs, Mosaic consistently outperforms other state-of-the-art out-of-core engines by 3.2-58.6x and shows comparable performance to distributed graph engines. Furthermore, Mosaic can complete one iteration of the Pagerank algorithm on a trillion-edge graph in 21 minutes, outperforming a distributed disk-based engine by 9.2×. Also The impact of fast networks on graph analytics, part 1: Fast 10G networks do help reduce reduce the runtime of parallel computations by significantly more than 2-10%: we've seen speedups up to 3x going from 1G to 10G. 


• GridDB Reliability and Robustness: GridDB is a scale-out database that is both fast and extremely reliable with Hybrid Cluster Management Architecture, Partition Replication, and Client Robustness features that have demonstrated efficacy with the error injection testing performed by Fixstars. Meanwhile, GridDB Autonomous Data Distribution Algorithm ensures that after a failure or upgrade, the data stored in GridDB is stored in a balanced manner.


• EXPLANATION OF BINARY ARITHMETIC by Gottfried Wilheim Leibniz: EXPLANATION OF BINARY ARITHMETIC, WHICH USES ONLY THE CHARACTERS 0 AND 1, WITH SOME REMARKS ON ITS USEFULNESS, AND ON THE LIGHT IT THROWS ON THE ANCIENT CHINESE FIGURES OF FUXI