Friday
Jan112019

Stuff The Internet Says On Scalability For January 11th, 2019

Wake up! It's HighScalability time:

 

The modern day inner sanctum revealed for all to experience. Nausea no extra charge.

 

Do you like this sort of Stuff? Please support me on Patreon. Need cloud? Consume Explain the Cloud Like I'm 10 (35 nearly 5 star reviews).

 

  • 8x: V8 Promiss.all parallel performance improvement;  1.3%: print sales increase; 11%: over 65 shared a hoax; 40%: add jobs after deploying AI; 3%: Eventbot's revenue pledged to open source; 51%: successful Ethereum attack; $308,620: cost of a Bitcoin 51% attack; $30: Apple services revenue per device per year; .3 cents: earnings from selling private data; 2,000: baguettes a day produced on French aircraft carrier; 5.6 nm: future smallest grains on a magnetic disk; 11,000: free books from 1923; 

  • Quotable Quotes:
    • @mekkaokereke: He joined SpaceX as a "founding employee." He designed the Merlin engine. He's CTO of Propulsion. His name is Tom Mueller. Everyone knows Elon Musk. No one knows Tom Mueller, even though Tom is the one currently designing a rocket that will put humans on Mars. 🤷🏿‍♂️
    • Dr. Rachael Tatman: My universal advice is to not get a Ph.D. I even wrote a blog post about it a while ago. The blog’s about linguistics specifically, but most of it applies to machine learning as well. I think that having a Ph.D. can be an advantage when you’re looking for data science jobs, but unless you really want to 1) do research or 2) be a professor there’s no really no benefit to getting a Ph.D. that you can can’t get more quickly doing something else.
    • @unclebobmartin: Unit tests are written by programmers for programmers. Acceptance tests are specified by customers for customers. Integration tests are written by architects for architects.
    • @wrathofgnon: A city can be seen as an attempt to solve the predicament that is human existence. No building can solve all problems, and as soon as we come up with what we think is a solution, we are faced with a new (often far worse) set of problems. Historically cities have been compromises.
    • cdixon: Smartphones are a good example of a broader historical pattern: technologies usually arrive in pairs, a strong form and a weak form...But it’s strong technologies that end up defining new eras...Weak technologies adapt to the world as it currently exists. Strong technologies adapt the world to themselves. Progress depends on strong technologies.
    • This way to more quotes...
Don't miss all that the Internet has to say on Scalability, click below and become eventually consistent with all scalability knowledge (which means this post has many more items to read so please keep on reading)...

Click to read more ...

Tuesday
Jan082019

MySQL High Availability Framework Explained – Part II

In Part I, we introduced a High Availability (HA) framework for MySQL hosting and discussed various components and their functionality. Now in Part II, we will discuss the details of MySQL semisynchronous replication and the related configuration settings that help us ensure redundancy and consistency of the data in our HA setup. Make sure to check back in for Part III where we will review various failure scenarios that could arise and the way the framework responds and recovers from these conditions. What is MySQL Semisynchronous Replication? Simply put, in a MySQL semisynchronous replication configuration, the master commits transactions to the storage engine only after receiving acknowledgement from at least one of the slaves. The slaves would provide acknowledgement only after the events are received and copied to the relay logs and also flushed to the disk. This guarantees that for all transactions committed and returned to the client, the data exists on at least 2 nodes. The term ‘semi’ in semisynchronous (replication) is due to the fact that the master commits the transactions once the events are received and flushed to relay log, but not necessarily committed to the data files on the slave. This is in contrast to fully synchronous replication, where the transaction would have been committed on both the slave and the master before the session returns to the client.

Click to read more ...

Tuesday
Jan082019

Slow MySQL Start Time in GTID mode? Binary Log File Size May Be The Issue

Have you been experiencing slow MySQL startup times in GTID mode? We recently ran into this issue on one of our MySQL hosting deployments and set out to solve the problem. In this blog, we break down the issue that could be slowing down your MySQL restart times, how to debug for your deployment, and what you can do to decrease your start time and improve your understanding of GTID-based replication.

How We Found The Problem

Click to read more ...

Friday
Jan042019

How DevOps Should Use DBaaS (Database-as-a-Service) To Optimize Their Application Development

This post was written by Wendy Dessler of The Blog Frog.

Database-as-a-Service (DBaaS) is quickly gaining in popularity across the tech world. These software platform solutions helps users easily manage their database operations without having to really understand any of the abstractions. This allows developers, DBA’s and DevOps engineers to quickly automate their backups, create new SQL and NoSQL clusters, and monitor the performance of their databases for their application without requiring any internal database expertise.

DBaaS falls under the umbrella of Platform-as-a-Service (PaaS) where the platform itself is actually a database or several databases. This is a great choice for DevOps in particular because it allows for more developer agility, productivity, and also security.

Flexibility and scalability are becoming more important in the world of DevOps and technology in general, and we all know how fast this world moves. Businesses need new ways to keep up with the competition, and developers are looking for an easy, self-service model for managing their databases in order to optimize their app development. Let’s break down the individual benefits so you can decide if DBaaS is right for your DevOps team.

1. Outsourced Security and Administration

Click to read more ...

Friday
Jan042019

Stuff The Internet Says On Scalability For January 4th, 2019

Wake up! It's HighScalability time:

 

Solar system? Nope, the beauty is in your head—neural art.

 

Do you like this sort of Stuff? Please support me on Patreon. Need cloud? Explain the Cloud Like I'm 10 (34 almost 5 star reviews).

 

  • 45%: learned scheduler improves average job completion time; 61%: apps share data with Facebook; 45,037,125: people who watched Bird Box on Netflix in first week; 32,368: color images collected by the Curiosity rover on Mars between August 2012 and November 2018; $36.1B: AI  healthcare market by 2025; 20%: object recognition failure in light rain; 350: pages in Donald Knuth's new book;  $1,000: price needed to deactivate Facebook account; $3B: Epic games profit; 1: bitcoin mined from body heat of 44,000 people; 

  • Quotable Quotes:
    • @Hannah_Chutzpah: What are the technical terms, in your field, for 'dunno'? In medicine there's 'idoeopathic' In archeology/anthropology there's 'ritual purposes' How do you professionally term 'we haven't got a clue'?
      • @peterseibel: In programming we don't have a special term for it, just like fish don't have a special term for 'water'.
    • @robn: everyone seems to want stateless services and stateless protocols but I have three kids that can't remember shit I told them ten seconds ago and believe me it's the worst and I don't know why you'd want that
    • @stranter: "A machine learning agent intended to transform aerial images into street maps and back was found to be cheating by hiding information it would need later in “a nearly imperceptible, high-frequency signal" ||  Sounds like the algo needs some parenting
    • @pmddomingos: Overheard at Stanford: trying to run a university in Silicon Valley is like trying to run a monastery in Gomorrah.
    • @scalzi: So, things I've learned in the last few days: 1. Sometimes 15GB+ updates to video games are in fact necessary and unavoidable 2. Devs mostly hate 'em too but right now there's not much to be done about it 3. I still hate it but UGH OKAY FINE I guess
    • @copyconstruct: "GPUs will increase 1000× in performance by 2025, whereas Moore’s law for CPUs essentially is dead. By replacing branch-heavy algorithms with neural networks, the DBMS can profit from these hardware trends."
    • @hotjamz: he's makin a list / it's kept in plain text / an elf just got phished / you know what is next / santa claus has leaked 2.2 billion user's names, addresses, naughty/nice statuses, and the times they are asleep in their homes
    • So many more quotes...
Don't miss all that the Internet has to say on Scalability, click below and become eventually consistent with all scalability knowledge (which means this post has many more items to read so please keep on reading)...

Click to read more ...

Friday
Dec212018

Stuff The Internet Says On Scalability For December 21st, 2018

Wake up! It's HighScalability time:

 

Have a very scalable Xmas everyone! See you in the New Year.

 

Do you like this sort of Stuff? Please support me on Patreon. I'd really appreciate it. Still looking for that perfect xmas gift? What could be better than a book on the cloud? Explain the Cloud Like I'm 10. And if you know someone with hearing problems they might find Live CC useful.

 

  • 33.5 billion: Pornhub visits; 122 million: miles traveled by Santa; 32,342: government requests to Apple for user data; 10x: faster helicopter design using VR instead of physical models and mockups; 4403: petabytes transferred by Pornhub; 59%: dropped leads on Google AMP; 160: streaming shows now outnumber their traditional-TV counterparts; 80%: machine learning engineers work at Google or Facebook; 25%: adults check phone immediately on waking; 164: iPhone apps made $1 million through in-app subscriptions; 750 petabytes: Backblaze storage; 

  • Quotable Quotes:
    • @ flight radar24: Yesterday was the busiest day of the year in the skies so far and our busiest day ever. 202,157 flights tracked! The first time we've tracked more than 200,000 flights in a single day on 
    • @swardley: X : What's going to happen in cloud in 2019? Me : Nothing special. 1) Enterprise data centres will continue to close. 2) Cloud will decentralise in terms of provision not power i.e. Amazon will "invade" more of those holdouts with AWS Outpost. 3) Serverless will rocket.
    • @odrotbohm: I’ve seen microservice based systems more deranged after 2 years than any 1,5 decades old monolith could ever have been.
    • Jason Kehe: Savings of even a few degrees Celsius can significantly extend the lifespan of electronic components; Microsoft reports that, on the ocean floor 117 feet down, its racks stay 10 degrees cooler than their land-based counterparts. Half a year after deployment, “the equipment is happy,” says Ben Cutler, the project’s manager.
    • @kellabyte: “Open source” infrastructure companies are a giant shit show right now. Whether it’s database or message queues it’s a really weird combo of licenses and features for hostage. Why? Because nobody knows how to make money. Support contracts aren’t enough.
    • Tim Bray: How to talk about [Serverless Latency] · To start with, don’t just say “I need 120ms.” Try some­thing more like “This has to be in Python, the data’s in Cas­san­dra, and I need the P50 down un­der a fifth of a sec­ond, ex­cept I can tol­er­ate 5-second la­ten­cy if it doesn’t hap­pen more than once an hour.” And in most main­stream ap­pli­ca­tion­s, you should be able to get there with server­less. If you plan for it. 
    • Jeremy Daly: Compared to building and maintaining your own systems, cloud computing is ridiculously inexpensive, especially when you’re starting out and haven’t achieved significant scale. Don’t waste your developers’ time trying to shave off nickels and dimes from your bill. Focus on creating more value by delivering and iterating on features faster and worry about cost optimizations later.
    • Zack Kanter: One day, complexity will grow past a breaking point and development velocity will begin to decline irreversibly, and so the ultimate job of the founder is to push that day off as long as humanly possible. The best way to do that is to keep your ball of mud to the minimum possible size— serverless is the most powerful tool ever developed to do exactly that.
    • These are the last quotes of the year. Don't be late...
Don't miss all that the Internet has to say on Scalability, click below and become eventually consistent with all scalability knowledge (which means this post has many more items to read so please keep on reading)...

Click to read more ...

Tuesday
Dec182018

Sponsored Post: pMD, InMemory.Net, Triplebyte, Etleap, Stream, Scalyr

Who's Hiring?

  • pMD is a fast­ growinghighly rated health care technology company that has been recognized as a Best Place to Work by SF Business Times, Modern Healthcare, and Inc. Senior DevOps Engineer: Your engineering work will focus on using your deep knowledge of the web stack including firewalls, web applications, caches and data stores ­ to create innovative infrastructure architectures that are resilient, scalable, and blazingly fast. You will bring best practices for automating the provisioning and patching of servers, replicating and versioning configuration changes, monitoring and alerting, deploying application and data store updates, and high availability / disaster recovery. Please apply here.

  • Triplebyte lets exceptional software engineers skip screening steps at hundreds of top tech companies like Apple, Dropbox, Mixpanel, and Instacart. Make your job search O(1), not O(n). Apply here.

  • Need excellent people? Advertise your job here! 

Fun and Informative Events

  • Advertise your event here!

Cool Products and Services


  • InMemory.Net provides a Dot Net native in memory database for analysing large amounts of data. It runs natively on .Net, and provides a native .Net, COM & ODBC apis for integration. It also has an easy to use language for importing data, and supports standard SQL for querying data. http://InMemory.Net
  • Build, scale and personalize your news feeds and activity streams with getstream.io. Try the API now in this 5 minute interactive tutorialStream is free up to 3 million feed updates so it's easy to get started. Client libraries are available for Node, Ruby, Python, PHP, Go, Java and .NET. Stream is currently also hiring Devops and Python/Go developers in Amsterdam. More than 400 companies rely on Stream for their production feed infrastructure, this includes apps with 30 million users. With your help we'd like to ad a few zeros to that number. Check out the job opening on AngelList.
  • Scalyr is a lightning-fast log management and operational data platform.  It's a tool (actually, multiple tools) that your entire team will love.  Get visibility into your production issues without juggling multiple tabs and different services -- all of your logs, server metrics and alerts are in your browser and at your fingertips. .  Loved and used by teams at Codecademy, ReturnPath, Grab, and InsideSales. Learn more today or see why Scalyr is a great alternative to Splunk.

  • Advertise your product or service here!

If you are interested in a sponsored post for an event, job, or product, please contact us for more information.


pMd is Hiring

pMD is a fast­ growing, highly rated health care technology company that has been recognized as a Best Place to Work by SF Business Times, Modern Healthcare, and Inc. We're profitable, have extremely happy customers, and make up a team of people as talented and passionate as you are. We love what we do and care about doing good in the world.

Senior DevOps Engineer: Your engineering work will focus on using your deep knowledge of the web stack ­ including firewalls, web applications, caches and data stores ­ to create innovative infrastructure architectures that are resilient, scalable, and blazingly fast. You will bring best practices for automating the provisioning and patching of servers, replicating and versioning configuration changes, monitoring and alerting, deploying application and data store updates, and high availability / disaster recovery. You have a vision for how a best­in­the­world web application stack should run, and you're excited to execute on that vision.

At pMD you’ll wear many hats, have a lot of responsibility, and be part of a fun, smart, and creative team. Our team is small and collaborative, so you’ll experience rapid career growth through real mentorship. You'll be a name (or a nickname) on the team and at social events, rather than Engineer #300. You'll get opportunities to be involved in sales, recruiting, customer success, and other aspects of the business. As the company continues to grow, you're ready to start mentoring others and eventually build and lead a team. At pMD you can grow as quickly as you want to.

Where else would you have a chance to architect a new replication topology in the morning, meet with clients for lunch, build a bare metal server in the afternoon, and drink beers with the team after work? Plus, you'll be able to see the impact of your work immediately and you'll hear from our users how your efforts have improved their lives.

Please apply here.


Make Your Job Search O(1) — not O(n)

Triplebyte is unique because they're a team of engineers running their own centralized technical assessment. Companies like Apple, Dropbox, Mixpanel, and Instacart now let Triplebyte-recommended engineers skip their own screening steps.

We found that High Scalability readers are about 80% more likely to be in the top bracket of engineering skill.

Take Triplebyte's multiple-choice quiz (system design and coding questions) to see if they can help you scale your career faster.


The Solution to Your Operational Diagnostics Woes

Scalyr gives you instant visibility of your production systems, helping you turn chaotic logs and system metrics into actionable data at interactive speeds. Don't be limited by the slow and narrow capabilities of traditional log monitoring tools. View and analyze all your logs and system metrics from multiple sources in one place. Get enterprise-grade functionality with sane pricing and insane performance. Learn more today


If you are interested in a sponsored post for an event, job, or product, please contact us for more information.



Friday
Dec142018

Stuff The Internet Says On Scalability For December 14th, 2018

Wake up! It's HighScalability time:

We've come a long way in 50 years. Or have we?

Alan Kay: I believe ARPA spent $ 175,000 of 1968 money for that one demo. That’s probably like a million bucks today.

Bill English: What we did was lease two video circuits from the phone company. They set up a microwave link: two transmitters on the top of the building at SRI, receiver/ transmitters up on Skyline Boulevard on a truck, and two receivers at the Civic Center. Cables of course going down into the room at both ends. That was our video link. Going back we had two dedicated 1,200-baud lines: high-speed lines at the time. Homemade modems.

Doug Engelbart: It was the very first time the world had ever seen a mouse, seen outline processing, seen hypertext, seen mixed text and graphics, seen real-time videoconferencing.

Alan Kay: We could actually see that ideas could be organized in a different way, that they could be filtered in a different way, that what we were looking at was not something that was trying to automate current modes of thought, but that there should be an amplification relationship between us and this new technology.

 

Do you like this sort of Stuff? Please support me on Patreon. I'd really appreciate it. Know anyone looking for a simple book explaining the cloud? Then please recommend my well reviewed (32 reviews on Amazon and 77 on Goodreads!) book: Explain the Cloud Like I'm 10. They'll love it and you'll be their hero forever. And if you know someone with hearing problems they might find Live CC very useful.

 

  • 50,000: images in a National Geographic shoot; 11 billion: Voyager 2 miles traveled; 83%: AI papers originate outside the US; 80%: network partitions lead to catastrophic failures; 97%: large AWS customers use auto scaling; 82%: startup failures due to cashflow problems; $115.7 billion: robot and drone spending in 2019; 

  • Quotable Quotes:
    • @jmarhee: "I went to Kubecon and all I got were these 300 service meshes built by companies that went under by time I got home"
    • @ReformedBroker: “Retention is the new growth.” - CEO of Adobe ($ADBE), which has gained 793% since moving Photoshop etc to a subscription-based service from a one-time software sale in 2011.
    • @mattray: OH: "Cloud native is pretty simple. You just need to know Kubernetes, Prometheus, Fluentd, Jaeger, Envoy, Core DNS, Linkerd, Rook, Vitess, Etcd and Raft."
    • @davidfrum: In order to lose 80% of its value, the bitcoin network expended more electricity than 150 of the world's 195 countries
    • @ChappellTracker: So here's something. Users are testing the limits of Tumblr's new algorithm that flags adult content (aka "censorbot"). This one found that a man's chest was flagged, but a man's chest next a 50% scale owl went unnoticed. 
    • Jimmy Chin: Great editors are brutal.
    • @danbri: intro slide: "Main message is that biology has been computing long before brains evolved. Somatic decision-making and memory are mediated by ancient pre-neural bioelectric networks across all cells. Exploring non-neural Cognition is [an untapped frontier for AI...]"
    • @swardley: X : Large companies are leaving cloud and building internally. If they use it, then it will be hybrid. Serverless is for startups who can't afford their own. 
      Me : Hmmm. The only words of comfort I have are ... if you don't like change, you're going to hate irrelevance.
    • Quotes, get'em while they're hot...
Don't miss all that the Internet has to say on Scalability, click below and become eventually consistent with all scalability knowledge (which means this post has many more items to read so please keep on reading)...

Click to read more ...

Tuesday
Dec112018

Sponsored Post: InMemory.Net, Triplebyte, Etleap, Stream, Scalyr

Who's Hiring? 


  • Triplebyte lets exceptional software engineers skip screening steps at hundreds of top tech companies like Apple, Dropbox, Mixpanel, and Instacart. Make your job search O(1), not O(n). Apply here.

  • Need excellent people? Advertise your job here! 

Fun and Informative Events

  • Advertise your event here!

Cool Products and Services


  • InMemory.Net provides a Dot Net native in memory database for analysing large amounts of data. It runs natively on .Net, and provides a native .Net, COM & ODBC apis for integration. It also has an easy to use language for importing data, and supports standard SQL for querying data. http://InMemory.Net
  • Build, scale and personalize your news feeds and activity streams with getstream.io. Try the API now in this 5 minute interactive tutorialStream is free up to 3 million feed updates so it's easy to get started. Client libraries are available for Node, Ruby, Python, PHP, Go, Java and .NET. Stream is currently also hiring Devops and Python/Go developers in Amsterdam. More than 400 companies rely on Stream for their production feed infrastructure, this includes apps with 30 million users. With your help we'd like to ad a few zeros to that number. Check out the job opening on AngelList.
  • Scalyr is a lightning-fast log management and operational data platform.  It's a tool (actually, multiple tools) that your entire team will love.  Get visibility into your production issues without juggling multiple tabs and different services -- all of your logs, server metrics and alerts are in your browser and at your fingertips. .  Loved and used by teams at Codecademy, ReturnPath, Grab, and InsideSales. Learn more today or see why Scalyr is a great alternative to Splunk.

  • Advertise your product or service here!

If you are interested in a sponsored post for an event, job, or product, please contact us for more information.


Make Your Job Search O(1) — not O(n)

Triplebyte is unique because they're a team of engineers running their own centralized technical assessment. Companies like Apple, Dropbox, Mixpanel, and Instacart now let Triplebyte-recommended engineers skip their own screening steps.

We found that High Scalability readers are about 80% more likely to be in the top bracket of engineering skill.

Take Triplebyte's multiple-choice quiz (system design and coding questions) to see if they can help you scale your career faster.


The Solution to Your Operational Diagnostics Woes

Scalyr gives you instant visibility of your production systems, helping you turn chaotic logs and system metrics into actionable data at interactive speeds. Don't be limited by the slow and narrow capabilities of traditional log monitoring tools. View and analyze all your logs and system metrics from multiple sources in one place. Get enterprise-grade functionality with sane pricing and insane performance. Learn more today


If you are interested in a sponsored post for an event, job, or product, please contact us for more information.

Friday
Dec072018

Stuff The Internet Says On Scalability For December 7th, 2018

Wake up! It's HighScalability time:

Do you like this sort of Stuff? Please support me on Patreon. I'd really appreciate it. Know anyone looking for a simple book explaining the cloud? Then please recommend my well reviewed (31 reviews on Amazon and 72 on Goodreads!) book: Explain the Cloud Like I'm 10. They'll love it and you'll be their hero forever. And if you know someone with hearing problems they might find Live CC very useful.

 

  • $181.5M: top 10 YouTube star earnings; 153 million/1.2 billion/27 billion: Reddit posts/comments/votes;  12%: cut for Epic's new Steam competitor; 2,368: Baidu AI patents; 360: Walmart cleaning robots; 2:1: AMD CPUs outselling Intel; 11157: images in Disguised Faces in the Wild dataset; 7,518: FCC approved SpaceX LEO satellites; $40 million: saved Down Under by Tesla’s giant battery; intense beam of ultra-high energy heavy ions: testing AI chips for space;  $100 million: bot heist; 16TB: HAMR hard drives; 12.8Tbps: new Tofino ASIC; 85%: cloud-hosted TensorFlow workloads run on AWS; 39%: tech workers depressed; 16%: drop in GPU shipments; 4 x 10⁸⁴: photons ever emitted in the universe; 0.00%: blockchain success rate; 

  • Quotable Quotes:
    • @math_rachel: About half of > 200 people seeking visas for @black_in_ai #NeurIPS2018 were denied. This is a huge issue harming our field.
    • @heipei: It's Friday, I've been in a jumpsuit doing manual labor all day (crazy, I know) and weighing my options between passing out on the couch over some Youtube videos, reading the Friday @highscal blog post or writing code. Currently favouring the path of least resistance 😛
    • Paul Tassi: Again, it’s hard to overstate just what a big deal this is, and it’s a massive undertaking for Epic, but one they believe they’re able to handle given how massive Fortnite has scaled to be. Epic famously dodged the Apple store entirely when releasing Fortnite on iOS, and it’s clear those were the first warning shots for something like this.
    • IBM: Our device uses phase-change memory (PCM) for in-memory computing. PCM records synaptic weights in its physical state along a gradient between amorphous and crystalline. The conductance of the material changes along with its physical state and can be modified using electrical pulses. This is how PCM is able to perform calculations. Because the state can be anywhere along the continuum between 0 and 1, it is considered an analog value, as opposed to a digital value, which is either a 0 or a 1, nothing in between.
    • @ivanveram: World R&D leading companies 2017 in US$. No.1: Amazon; No.2: Alphabet (Google); No.3: Samsung. :) That´s the way it is!!!. :)
    • 40acres: Full disclosure, I work for Intel. We have a fabrication plant in Chengdu, it's public knowledge that this fab is helping to manufacture products built on the latest process technology. As I've come to learn more about China's tactics when dealing with foreign companies it's become of great concern to me what this plant means for our future. I don't think it would be far-fetched to assume that some very protected and valuable IP has leeched through our doors and into China's hands. In all honestly I really can't fathom how the American government let this deal occur.
    • So many more quotes...
Don't miss all that the Internet has to say on Scalability, click below and become eventually consistent with all scalability knowledge (which means this post has many more items to read so please keep on reading)...

Click to read more ...