hot links

Stuff The Internet Says On Scalability For August 18th, 2017

High Scalability

High Scalability

— 23 min read

Sorry about missing last week, but my birthday won out over working:

Ouch! @john_overholt: My actual life is now a science exhibit about the primitive conditions of the past.

If you like this sort of Stuff then please support me on Patreon.

  • 1PB: SSD in 1U chassis; 90%: savings using EC2 Spot for containers; 16: forms of inertia; $2.1B: Alibaba’s profit; 22.6B: app downloads in Q2; 25%: Google generated internet traffic; 20 by 20 micrometers: quantum random number generators; 16: lectures on Convolutional Neural Networks for Visual Recognition; 25,000: digitized gramophone records; 280%: increase in IoT attacks; 6.5%: world's GDP goes to subsidizing fossil fuel; 832 TB: ZFS on Linux;  $250,000: weekly take from breaking slot machines; 30: galatic message exchanges using artificial megastructures in 100,000 years; 

  • Quotable Quotes:
    • @chris__martin: ALIENS: we bring you a gift of reliable computing technol--
      HUMANS: oh no we have that already but JS is easier to hire for
    • @rakyll: "You woman, you like intern." I interned on F-16's flight computer. Even my internship was 100x more legit than any job you will have.
    • @CodeWisdom: "Debugging is like being the detective in a crime movie where you are also the murderer." - Filipe Fortes
    • William Gibson: what I find far more ominous is how seldom, today, we see the phrase “the 22nd century.” Almost never. Compare this with the frequency with which the 21st century was evoked in popular culture during, say, the 1920s.
    • Arador: Amazon EC2, Microsoft Azure and Google Gloud Platform are all seriously screwing their customers over when it comes to bandwidth charges. Every one of the big three has massive buying power yet between them their average bandwidth price is 3.4x higher than colocation facilities.
    • @mattklein123: Good thread: my view: 1) most infra startups will fail. It's an awful business to be in (sorry all, much ❤️).
    • Jean-Louis Gassée: With Services, Apple enjoys the benefits of a virtuous circle: Hardware sales create Services revenue opportunities; Services makes hardware more attractive and “stickier”. Like Apple Stores, Services are part of the ecosystem. Such is the satisfying simplicity and robustness of Apple’s business model.
    • cardine: The price difference between Hetzner and AWS is large enough that it could pay for 4x as much computational power (as much redundancy as you'd ever need), three full time system admins (not that you'd ever need them), and our office lease... with plenty of money left over!
    • Brujo Benavides: Communication is Key: Have you ever watched a movie or a soap opera and thought “If you would’ve just told her that, we would’ve avoided 3 entire episodes, you moron!”. Happens to me all the time. At Inaka we learned that the hard way.
    • @f3ew: Doesn't matter how many layers of stateless services you have in the middle, the interesting ends have state.
    • brianwawok: My cloud cost is less than 5% of my bussiness costs using GCE. Would be foolish to move it to lower my costs 2%.
    • @f3ew: Stateless services are as relevant as routers. Pattern match, compute, push to next layer.
    • Horace Dediu~ when you outsource you're taking knowledge out of your company, which ends up gutting it in terms of the value that is added 
    • Jason Calacanis: Google was the twelfth search engine. Facebook was the tenth social network. iPad was the twentieth tablet. It’s not who gets there first. It’s who gets there first when the market’s ready.
    • puzzle: The B4 paper states multiple times that Google runs links at almost 100% saturation, versus the standard 30-40%. That's accomplished through the use of SDN technology and, even before that, through strict application of QoS.
    • @stu: Serverless has in many ways eclipsed the containers discussion for the hot buzz in the industry
    • @mjpt777: GC is a wonderful thing but I cannot help but feel it leaves the typical developer even less prepared for distributed resource management.
    • joaodlf: Spark works and does a good job, it has many features that I can see us use in the future too. With that said, it's yet another piece of tech that bloats our stack. I would love to reduce our tech debt: We are much more familiar with relational databases like MySQL and Postgres, but we fear they won't answer the analytics problems we have, hence Cassandra and Spark. We use these technologies out of necessity, not love for them.
    • tobyjsullivan: No, dear author. Setting up the AWS billing alarm was the smartest thing you ever did. It probably saved you tens of thousands of dollars (or at least the headache associated with fighting Amazon over the bill). Developers make mistakes. It's part of the job. It's not unusual or bad in any way. A bad developer is one who denies that fact and fails to prepare for it. A great developer is one like the author.
    • Geoff Wozniak: Regardless of whether I find that stored procedures aren't actually that evil or whether I keep using templated SQL, I do know one thing: I won't fall into the "ORMs make it easy" trap.
    • @BenedictEvans: Part of what distinguishes today’s big tech companies is a continual push against complacency. They saw the last 20 years and read the books
    • John Patrick Pullen: In the upcoming fall issue of Porter magazine, the 21-yer-old X-Men: Apocalypse star said, "I auditioned for a project and it was between me and another girl who is a far better actress than I am, far better, but I had the followers, so I got the job," according to The Telegraph. "It’s not right, but it is part of the movie industry now."
    • Rachel Adler: Naturally, faster prints drove up demand for paper, and soon traditional methods of paper production couldn’t keep up. The paper machine, invented in France in 1799 at the Didot family’s paper mill, could make 40 times as much paper per day as the traditional method, which involved pounding rags into pulp by hand using a mortar and pestle.
    • pawelkomarnicki: As a person that can get the product from scratch to production and scale it, I can say I'm a full-stack developer. Can I feel mythical now?
    • Risto: Before integrating any payment flow make sure you understand the whole flow and the different payment states trialing -> active -> unpaid -> cancelled. For Braintree there is a flow chart. For Stripe there is one too. Both payment providers have REST API’s so make sure to play through the payment flows before starting actual coding.
    • Seyi Fabode: I have 3 neighbors in close proximity who also have solar panels on their roofs. And a couple of other neighbors with electric cars. What says we can’t start our own mini-grid system between ourselves?
    • pixl97: Muscles/limbs are only 'vastly' more efficient if you consider they have large numbers of nano scale support systems constantly rebuilding them. Since we don't have nanobots, gears will be better for machines. Also, nature didn't naturally develop a axle.
    • Brave New Greek: I sympathize with the Go team’s desire to keep the overall surface area of the language small and the complexity low, but I have a hard time reconciling this with the existing built-in generics and continued use of interface{} in the standard library.
    • @jeffhollan: Agree to a point. But where does PaaS become “serverless”? Feel should be ‘infinite’ scale of dynamic allocation of resources + micro bill
    • @kcimc: common tempos in 1M songs, 1959-2011: 120 bpm takes over in the late 80s, and bpms at multiples of 10 emerge in the mid 90s
    • How to Map the Circuits That Define Us: If neural circuits can teach one lesson, it is that no network is too small to yield surprises — or to frustrate attempts at comprehension.
    • @orskov: In Q2, The Wall Street Journal had 1,270,000 daily digital-only subscribers, a 34% increase compared to last year
    • Thrust Zone: A panel including tech billionaire Elon Musk is discussing the fact that technology has progressed so much that it may soon destroy us and they have to pass microphones to talk.
    • @damonedwards: When we are all running containers in public clouds, I’m really going to miss datacenter folks one-upping each other on hardware specs.
    • @BenedictEvans: 186 page telecoms report from 1994. 5 pages on ‘videophones’: no mention of internet. 10 pages saying web will lose to VR. Nothing on mobile
    • Thomas Metzinger: The superintelligence concludes that non-existence is in the own best interest of all future self-conscious beings on this planet. Empirically, it knows that naturally evolved biological creatures are unable to realize this fact because of their firmly anchored existence bias. The superintelligence decides to act benevolently.
    • Jeremy Eder: As with all public cloud, you can do whatever you want…for a price.  BurstBalance is the creation of folks who want you to get hooked on great performance (gp2 can run at 3000+ IOPS), but then when you start doing something more than dev/test and run into these weird issues, you’re already hooked and you have no choice but to pay more for a service that is actually usable.
    • Katz and Fan: After all, the important thing for anyone looking to launder money through a casino isn’t to win. It’s to exchange millions of dollars for chips you can swap for cool, untraceable cash at the end of the night.
    • Caitie McCaffrey: Verification in industry generally consists of unit tests, monitoring, and canaries. While this provides some confidence in the system's correctness, it is not sufficient. More exhaustive unit and integration tests should be written. Tools such as random model checkers should be used to test a large subset of the state space. In addition, forcing a system to fail via fault injection should be more widely used. Even simple tests such as running kill −9 on a primary node have found catastrophic bugs.
    • Zupa: FPGAs give you most of the benefits of special-purpose processors, for a fraction of the cost. They are about 10x slower, but that means an FPGA based bitcoin miner is still 100k times faster than a processor based one
    • menge101work: I'm not sure if the implication is that our CPUs will have gate arrays on chip with the generic CPU, that is an interesting idea. But if they are not on chip, the gate array will never be doing anything in a few clock cycles. It'll be more akin to going out to memory, the latency between a real memory load and an L1 or L2 cache hit is huge. (reference)
      Not to say that being able to do complex work on dedicated hardware won't still be fast, but the difference between on-die and off-die is a huge difference in how big of a change this could be.
    • Animats: This article [Why Many Smart Contract Use Cases Are Simply Impossible] outlines the basic problem. If you want smart contracts that do anything off chain, there have to be connections to trusted services that provide information and take actions. If you have trusted services available, you may not need a blockchain.The article points out that you can't construct an ordinary loan on chain, because you have no way to enforce paying it back short of tying up the loaned funds tor the duration of the loan. Useful credit fundamentally requires some way of making debtors pay up later. It's possible to construct various speculative financial products entirely on chain, and that's been done, but it's mostly useful for gambling, broadly defined.
    • curun1r: Your characterization of startup cloud costs is laughably outdated. With credits for startups and the ability to go serverless, I've known startups that didn't pay a dime for hosting their entire first year despite reaching the threshold of hundreds of customers and over $1m ARR. One of my friends actually started doing some ML stuff on AWS because he wanted to use his remaining credits before they expired and his production and staging workloads weren't going to get him there. I'd say it makes no sense to buy your 32gb, 16-core single point of fail, waste $40/mo and half a day setting it up and then have to keep it running yourself when you can easily spin up an API in API Gateway/Lambda that dumps data into dynamo/simpledb and front it with a static site in S3. That setup scales well enough to be mentioned on HN without getting hugged to death and is kept running by someone else. And if it is, literally, free for the first year, how is that not a no brainer?
    • Neil Irwin: In this way of thinking about productivity, inventors and business innovators are always cooking up better ways to do things, but it takes a labor shortage and high wages to coax firms to deploy the investment it takes to actually put those innovations into widespread use. In other words, instead of worrying so much about robots taking away jobs, maybe we should worry more about wages being too low for the robots to even get a chance.

  • The best introduction to Azure Stack—Microsoft's on-prem cloud solution—out there. Datanauts 097: Unpacking Azure Stack For Hybrid Clouds. Not a white box solution, it's all black box baby. You must buy expensive hardware from certified vendors who will do the install for you. It's "your" cloud, yet Azure Stack uses a subscription based model. With Windows licensing in place, spinning up a VM costs $6 per VM's VCPU per month. In Azure you would pay $30-$60 per month for the same VM. Window's licensing costs more if you don't have it. There's still a Window's strategy tax to pay. Suggestion is to buy datacenter licensing. It's closed source, everything is done through an admin portal. It's not even clear if you get admin on the VMs. This is the first release, so the largest configuration is a 12 node system in one region. You get the same API endpoints as you do in Azure, so your tooling works in both environments. You don't get all of Azure. You get VMs, Azure Storage, Azure App Service, Azure Functions, Azure Backup, and Azure Site Recovery. Your don't get: hardware specific VMs, like with GPU accelerators, Azure Data Lake, Azure Data Warehouse, Cognitive services. Base hypervisor runs Server Core 2016 and Hyper-V. Storage platform is Storage Spaces Direct. No centralized SAN, uses Microsoft's SDN to NAT, BGP and segment subnets. Has an app store type of marketplace for buying 3rd party software. Supports IT utilization charge back to tenants. Do not have to change Active Directory.

  • Irony: the automatic update mechanism for distributing software upgrades, the number one hope for quickly propagating security fixes, is also one giant gaping maw of a security risk. Chinese hackers 'built back door hack into software to spy on Britain’s top businesses: A legitimate software update from NetSarang was highjacked by the hackers in a “supply chain” attack...Companies at risk from this latest attack include American weapons firm Lockheed Martin, Russian energy supplier Gazprom and French bank Société Générale.

  • Who would have thunk it, Napoleon was into BigData. Napoleon: A Life: He insisted that his prefects provide him with systematic statistical data, ordering them to make extensive annual tours of their departments to glean first-hand information.

  • Looks like Ford is trying hard to avoid being disrupted. It makes no sense for Ford, as an OEM, not to own core AI technology. It's not a sustaining technology, it's not something you should outsource to your supply chain, it's an existential level bet. An inside look at Ford’s $1 billion bet on Argo AI: Argo is developing self-driving technology that Ford can use to deploy fully autonomous Level 4-capable vehicles for commercial on-demand service. In other words: something like a self-driving taxi service. Level 4 is a designation by SAE International that means the car takes over all of the driving in certain conditions.

  • This is a huge downside, but AWS really should detect infinite loops. Serverless: A lesson learned. The hard way: The actual cost is now $206 and over $1000 forecasted, it makes me think twice about using pay-per-use services in the future. One little mistake can cost a lot of money, the budget notifications were very late so there was very little I could do against it...This is probably the most stupid thing I ever did. One missing return; ended up costing me $206.

  • TimescaleDB vs. Postgres for time-series: 20x higher inserts, 2000x faster deletes, 1.2x-14,000x faster queries: a new, open-source time-series database architected for fast ingest, complex queries, and ease of use. It looks like PostgreSQL to the outside world...users can scale to billions of rows on PostgreSQL, while maintaining high, constant insert rates...TimescaleDB solves this [update the indexes] through its heavily utilization and automation of time-space partitioning, even when running ​on a single machine​. Essentially, all writes to recent time intervals are only to tables that remain in memory. This results in a consistent 20x insert performance improvement over PostgreSQL when inserting data at scale.

  • LinkedIn lost their data-scraping case. So scraping data from websites, seems to be legal...for the moment. UNITED STATES DISTRICT COURT NORTHERN DISTRICT OF CALIFORNIA. LinkedIn must allow a company to scrape users' public profiles and must remove technical barriers within 24 hours.

  • As you scale work needs to move from the center to the edge (and back again :-). That's what Google is doing with routing. Google unveils Espresso for the peering-edge. Espresso makes Google cloud faster, more available and cost effective by extending SDN to the public internet: "Espresso delivers two key pieces of innovation. First, it allows us to dynamically choose from where to serve individual users based on measurements of how end-to-end network connections are performing in real time. Second, we separate the logic and control of traffic management from the confines of individual router 'boxes.'" If you want to understand why edge computing is more than a buzz word, Google makes the point that being at the edge is necessary to provide today and tomorrow's low-latency services: Answering the question “What’s the latest news?” with Google Assistant requires a fast, low-latency connection from a user’s device to the edge of Google’s network, and from the edge of our network to one of our data centers. Once inside a data center, hundreds—or even thousands—of individual servers must consult vast amounts of data to score the mapping of an audio recording to possible phrases in one of many languages and dialects. The resulting phrase is then passed to another cluster to perform a web search, consulting a real-time index of internet content. The results are then gathered, scored and returned to the edge of Google’s network back to the end user. Answering queries in real-time involves coordinating dozens of internet routers and thousands of computers across the globe, often in the space of less than a second! Further, the system must scale to a worldwide audience that generates thousands of queries every second.

  • Move over mobile, AI is the new hotness. Microsoft just officially listed AI as one of its top priorities, replacing mobile. Internet looks on longingly, remembering when it was the new hotness. Microprocessors nod, knowingly. Combustion engines, fart carbon tears. Electricity, is not shocked at all. Guns, want another shot. Wheels say, what comes around goes around. Fire, burns with envy.

  • Automatic database management system tuning through large-scale machine learning: Our results show that OtterTune produces configurations that achieve up to 94% lower latency compared to [the DBMS] default settings or configurations generated by other tuning advisors. We also show that OtterTune generates configurations in under 60 minutes that are comparable to ones created by human experts.

  • Microsoft continues to extend its cloud service offerings. Introducing Azure Event Grid – an event service for modern applications: Azure Event Grid manages all routing of events from any source, to any destination, for any application...makes an event a first-class object...create your own custom events to publish directly to the service...supporting custom web hooks to publish events to any service, even 3rd-party services outside of Azure. It's general eventing model looks to be more flexibile than Lambda's restricted set of event sources. Events can terminate at many different endpoints, like VMs. Not sure how usefule that is. There's also smart filtering on events, not sure how useful that is either. Didn't see anything on rate limiting, batching, etc. How is it different than Service Bus, MSs' enterprise messaging product? Ben Coleman: Service Bus requires polling, this is an asynchronous push to you, rather than you having some code checking in a loop. 

  • @patrickdebois asks Why NOT serverless? And gets some not very convincing answers: @jtopper: Why make yourself a distributed systems problem when you didn’t need to? @samnewman: In sort, hybrid apps especially have challenges, and there are gaps that need to be addressed. @PaulDJohnston: Hiring - fewer people in the pool and no fixed idea of "serverless" means your hiring is harder. @Clausewitz45: The app trace level never going to reach the expected depth. You can't setup an identical environment for testing - latency differs somehow. @jon95016161: Network isolation @sethvargo: Unclear isolation barriers (how do I "promote" something from stage to prod reliably? 

  • A classic Brendan Gregg investigative expose. Linux Load Averages: Solving the Mystery: In 1993, a Linux engineer found a nonintuitive case with load averages, and with a three-line patch changed them forever from "CPU load averages" to what one might call "system load averages." His change included tasks in the uninterruptible state, so that load averages reflected demand for disk resources and not just CPUs. These system load averages count the number of threads working and waiting to work, and are summarized as a triplet of exponentially-damped moving sum averages that use 1, 5, and 15 minutes as constants in an equation. This triplet of numbers lets you see if load is increasing or decreasing, and their greatest value may be for relative comparisons with themselves.

  • Nice summary of patterns. Designing a Microservices Architecture for Failure: apply configuration changes to a subset of your instances gradually; run two production environments; load balancer should skip unhealthy instances; an application can do the necessary steps to recover from a broken state; use two different expiration dates, a shorter that tells how long you can use the cache in a normal situation, and a longer one that says how long can you use the cached data during failure; retry our action as we can expect that the resource will recover after some time; define how many requests can be received or processed by a particular customer or application during a timeframe; hold back lower-priority traffic to give enough resources to critical transactions; using timeouts is an anti-pattern and you should avoid it; applying the bulkheads pattern, we can protect limited resources from being exhausted; a circuit breaker opens when a particular type of error occurs multiple times in a short period; test your system against common issues to make sure that your services can survive various failures

  • An incredible story. Amazing people. Deeply moving. The Loyal Engineers Steering NASA’s Voyager Probes Across the Universe. Doug: Entire generations of consumer telecom have been swept aside while Voyager headed through then out of the Solar system. All the while beaming data back to Earth with the same radio technology that existed when TV stations used to sign off at the end of the broadcast day, and calling your mom actually routed your voice across a chain of copper wires spanning the nation.

  • I'm shocked I tell you. Meet ‘Spoofy’. How a Single entity dominates the price of Bitcoin: The exchange rate of Bitcoin has been heavily manipulated as a result of spoofing, wash trading, and other shenanigans. This is not healthy long term for the price of bitcoin and the possible future collapse of Tether and Bitfinex will cause a dramatic effect on future Bitcoin prices because we will no longer have these false signals coming out from Bitfinex...I believe that a significant amount of the trading on Bitfinex is likely wash trading, and this could be wash trading in both directions up and down. I believe Spoofy is playing on multiple exchanges, but his orders are usually much smaller on the other exchanges.

  • If you are considering Firebase for your BaaS then Why I’m dumping Firebase for Web? rips through a lot of different problems. Good discussion on HackerNews and on Reddit. As always, there's agreement and disagreement. The authors current stack: Google Compute running pm2; otherwise Google App Engine Flexible Env; Monolith Node.js application running Express; Home-brewed REST middleware; MySQL for metadata; In-process (a.k.a RAM) cache for stuff I read a lot; Redis for real-time counters; JWT for user sessions; JSON event/log files rotated directly to BigQuery; Some PWA practices but not SPA; ES5 client-side JavaScript, CDN includes; Some Vue.js.

  • Theadripper reviews are in. The AMD Ryzen Threadripper 1950X and 1920X Review. This Week in Computer Hardware 427. The AMD Ryzen Threadripper 1950X and 1920X Review: CPUs on Steroids. Instructor: TL;DR: clear winner in multithread productivity, intel better in single-thread productivity, around Ryzen 1800X in gaming (in game mode, which tries to pin threads to one die at a time and disables SMT), great TDP (stays under 180w at full load), has some teething issues with the NUMA nature of two-dies, but is overall a great processor.

  • Like something nerdy and interesting to read? This is very well written. Coding Machines. nickpsecurity: The story shows what people typically do if there’s a Karger/Thompson attack.

  • Technology changes how society structures itself. The Stethoscope. Invention of the stethoscope changed the relationship of patients to their doctor. Instead of the doctor listening to the patient to form a diagnosis, patients became a source of data. This shifted the balance of power in the relationship to doctors, who became more distant and authoritarian. Welcome to the future.

  • Uber Mobility: RIB (Router Interactor Builder). Uber spent 6 months looking at MVC, MVVM, VIPER before creating RIB. Their Rider app was boken into 500 RIBs, less 300 lines per class.

  • Jibes with the thought I've always had that programming is explaining something to yourself in code. Learning By Thinking: something about the process of explanation, the process of trying to get an understanding of something, is crucial to how we're able to then predict and intervene in the future...learning by thinking...is the experience of coming to understand something better as a result of explaining it to yourself...you gain new insight or understanding, it isn't driven by that new information that they've provided. In some way, you've rearranged what was already in your head in order to get new insight...The process of trying to explain to yourself is a lot like a thought experiment in science...if you engage in the process of asking yourself why something happened, part of what you're doing is trying to relate the concrete particulars of that story to a more general principle or generalization

  • Serverless Performance Shootout: Google Cloud Functions had the best UI; Microsoft Azure Functions had the largest number of configuration options; AWS Lambda had the most complex configuration setup; Most obscure logs also goes to AWS with its CloudWatch integration; Microsoft clearly comes out the winner here, with a median round trip time that is just about half that of both AWS and Google; Google Cloud Functions has about a 40 millisecond lead time on AWS Lambda when it comes to comparing the machine instance ramp-up time; Microsoft Azure Functions had the best average runtime performance, there was also a higher error rate.

  • Binary is so old school, we're going beyond binary, to quaternary. A radical innovation that could reduce data density by 50%: Their radical solution is to use polymer films containing colored dyes to optically store data. What’s more, binary ones and zeroes—the classic method we use for writing data—should be junked, and we must move over to a space-saving quaternary, or four-symbol code...The four symbols are the absence of color and three colors—fluorescent green, ultramarine and cyan.

  • Apparently when you let a 1000 flowers bloom, most of them wither and die. Worldwide App Downloads Q2 2017: Games Eclipsing Social Media: Another feature worth noting is the fact that all apps in the top 5 fall under the social media and messaging category. However, despite this, the social networking and communication app downloads actually went down compared to Q1 2016. This means users are consolidating around a few select platforms. While the established social media apps are seeing more and more interest, fringe players and upstarts are falling by the wayside.

  • Looks like a giant game of tetris that never completes. Robot army! Based on AI Algorithm, 100+ robots deliver goods at China's largest "smart warehouse" in Guangdong

  • Love this kind of find the bug style detective story. Docker operations slowing down on AWS (this time it’s not DNS). Also love lessons drawn from experience based cynicism: Regardless of cloud provider, you pay a premium for both performance and determinism; If you think you are saving money up front, just wait until the production issues start rolling in which, conveniently, can easily be solved by simply clicking a little button and upgrading to the next tier.

  • Ships fooled in GPS spoofing attack suggest Russian cyberweapon. Nokinside: There are several types of spoofing attacks and it's possible to use spoofing as type of jamming attack against military GPS units from a distance where pure jamming is not effective. Older receivers require lock into C/A code before moving to encrypted P(Y). The new M-code attempts to solve some of these issues. The simplest form of GPS spoofing sends made up signals that misdirects the GPS receiver. Military equipment is protected against these kinds of attacks with authenticated signals. Carry-off attacks (satellite-lock takeover) start with broadcasting perfectly synced repeat of the original signal. This kind of attack can be used to make munitions and missiles to lose their signal lock in critical time. It's also possible that there are multiple weaknesses and bugs in the military receivers that can be exploited. Iranians were somehow able to trick RQ-170 drone to land on Iran. When Iranians captured U.S. Navy patrol boat it was because there was mysterious navigation error.

  • GitHub had a problem releasing new product in production. New services took months to deploy. When that happens you need a better underlying development platform, so GitHub Goes All in on Kubernetes. The article says the SRE team became the bottleneck, which is odd, because at Google SRE involvement is not necessary for a service to go live. They took an evolutionary approach and eventually were able to make a metal cluster pass acceptance tests. They routed test traffic into the cluster, starting with 100 requests-per-second and expanding later to 10 percent of the requests to github.com and api.github.com. They went with multiple clusters to reduce risk. Result: I can already see how the move to Kubernetes is creating an environment at GitHub for more rapid innovation

  • The content wars have begun they have. Disney to End Netflix Deal, Sets Launch of ESPN and Disney-Branded Streaming Services. Everyone wants to be a streaming service with unique content to lock-in users. Disney wants to be Netflix and Netflix wants to be Disney. Even Apple sets $1 billion budget for original TV shows. Though, so far, whatever taste level Apple has in designing nice shiny objects, hasn't rubbed off on their original content. 

  • NVMe: The Change Agent for Next-Gen Data Centers: Non-Volatile Memory Express is a standard protocol and driver for SSDs based on NAND flash memory...NVMe-compliant SSDs can deliver up to 10 times the sequential read performance of SATA-based SSDs...Compared with SATA, the NVMe standard delivers better bandwidth and IOPS performance, plus lower latencies. It also provides scalability to main storage devices without the cost or complexities of battery-backed RAID or HBA cards...Replacing SATA-based SSDs or HDDs with NVMe SSDs can reduce the I/O wait time by 50 percent, enabling both database workloads to run on a single server and requiring only one license.

  • Looks good. Kubernetes: Up and Running: Dive into the Future of Infrastructure. 

  • Alexandr Nikitin: I played with Transparent Hugepages some time ago and I want to share some numbers based on real world high-load applications. We have a JVM application: high-load tcp server based on netty. No clear bottleneck, CPU, memory and network are equally highly loaded. The amount of work depends on request content...As you can see THP performance impact is measurable and too significant to ignore. 4.1 ms vs 3.2 ms 99%% and 100M vs 25M TLB misses. I also used SytemTap to measure few kernel functions like collapse_huge_page, clear_huge_page, split_huge_page. There were no significant spikes using THP. 

  • Well done. Java Blocking Queues Explained!: A computational problem can be decomposed in to pieces. Why? Because we can solve those pieces separately and get the final solution. There are several cores in most of the computer’s processors nowadays so we can share the pieces among the cores.

  • Send this to someone if they are wondering how to start building a web site. How we built our first full-stack JavaScript web app in three weeks: Step 1: Ideate; Step 2: Wireframe UX/UI; Step 3: Choose a data structure and type of database; Step 4: Set up Github and an agile workflow; Step 5: Choose & download a boilerplate; Step 6: Write back-end API routes (or Firebase listeners); Step 7: Build a “Proof Of Concept”; Step 8: Code out the features; Step 9: Choose and code the design scheme; Step 10: Find and squash bugs; Step 11: Deploy the live app.

  • novarto-oss/sane-dbc: A sane approach to interacting with an RDBMS in Java

  • Introducing (n+1)sec – a protocol for distributed multiparty chat encryption:  end-to-end secure, synchronous protocol for group chat developed by eQualit.ie with support from the Open Technology Fund. After 2 years of design, development and testing, we are releasing the (n+1)sec protocol and library for securing group conversations on various messaging systems, like Jabber/XMPP or IRC.

  • Efficient Immutable Collections: In this thesis we introduce data structures that are more performant than comparable state-of-the-art standard library data structures —in terms of operation runtimes and memory footprints— and more general by supporting storage and retrieval of type-heterogeneous data‡ where parametric polymorphism [OW97] on the language level falls short. We specifically target immutable, unordered and hashed data types, such as widely-used hash-sets and hash-maps; we will specify the concepts just mentioned as well as the characteristics of the proposed data structures throughout the remainder of this chapter

  • Computer Security, Privacy, and DNA Sequencing: Compromising Computers with Synthesized DNA, Privacy Leaks, and More: We demonstrate, for the first time, the synthesis of DNA which — when sequenced and processed— gives an attacker arbitrary remote code execution. To study the feasibility of creating and synthesizing a DNA-based exploit, we performed our attack on a modified downstream sequencing utility with a deliberately introduced vulnerability. After sequencing, we observed information leakage in our data due to sample bleeding

  • Papers from Google at ICML 2017. 

  • Engineering Antifragile Systems: A Change In Design Philosophy (article): NASA is working to infuse concepts from Complexity Science into the engineering process. Some of these problems may be solved by a change in design philosophy. Instead of designing systems to meet known requirements that will always lead to fragile systems at some degree, systems should be designed wherever possible to be antifragile: designing cognitive cyber-physical systems that can learn from their experience, adapt to unforeseen events they face in their environment, and grow stronger in the face of adversity. Several examples are presented of on ongoing research efforts to employ this philosophy.

Hey, just letting you know I've written a novella: The Strange Trial of Ciri: The First Sentient AI. It explores the idea of how a sentient AI might arise as ripped from the headlines deep learning techniques are applied to large social networks. Anyway, I like the story. If you do too please consider giving it a review on Amazon. Thanks for your support!

Read more