advertise
Friday
Oct182013

Stuff The Internet Says On Scalability For October 18th, 2013

Hey, it's HighScalability time:


Test your sense of scale. Is this image of something microscopic or macroscopic? Find out.
  • $3.5 million: Per Episode Cost of Breaking Bad
  • Quotable Quotes:
    • @GammaCounter: "There are 400 billion trees in the Amazon River basin, close to the number of stars in the Milky Way galaxy." 
    • @rbranson: Virtualization has near-zero overhead, unless the VM spends most of it's time copying between RAM and network… like memcached or haproxy.
    • @HackerNewsOnion: Programming is 1% inspiration, 99% trying to get your environment working.
    • @aneel: "roundtrips, not bandwidth, is now often the bottleneck for most applications"
    • @jamesurquhart: Not to mention the fact that auto-scaling should happen above IaaS layer. Think multi-cloud.
    • Sheref Mansy: A machine keeps sort of chugging away, without worrying about its environment. But a living system has to.
    • V.D. Veksler: it just came to my attention that Javascript v8 is faster than Python. I could not believe it, thought it might just be CPython.
    • Doron Rajwan: For the past 30 years, computer performance has been driven by Moore’s Law; from now on, it will be driven by Amdahl’s Law.
    • Bjarne Stroustrup: There are only two kinds of languages: the ones people complain about and the ones nobody uses.
  • Steve Souders and John Allspaw, the Laurel and Laurel of the DevPerfOps world had a really good interview at Velocity. Some trends...mobile is huge; there's now a big focus on rendering performance; institutionalizing failure - planning and doing something with failure, failure is a friend, not a scary monster; don't panic, when there's a problem figure out what's going on first; Humans and Machines are buds, they are cooperative, not John Henry like adversaries.
  • Forget the history of Kings and peoples. Here's a far more interesting history, the History of Packets. Cool look at how TCP has changed over time, a description of how packets work, and the history of the Internets. Kind of boring as there are no beheadings.

Don't miss all that the Internet has to say on Scalability, click below and become eventually consistent with all scalability knowledge...

Click to read more ...

Wednesday
Oct162013

Interview With Google's Ilya Grigorik On His New Book: High Performance Browser Networking

If you are Google you don't just complain about performance on the web, you do something about it. Doing something about web performance is the job of one Ilya Grigorik, Developer Advocate, Make the Web Fast at Google, and author of a great new book: High Performance Browser Networking: What every web developer should know about networking and web performance

That's a big topic you might be saying to yourself. And it is. The book is 400 plus information packed pages. But never fear. Ilya writes in a very straightforward style. It’s like a man page for the web. Which is a good thing.

In case you are not familiar with Ilya, he's the perfect choice for writing such an ambitious book. For years Ilya has been producing excellent content on his blog and if you search YouTube you'll find presentation after presentation on the topics found in the book. Authority established.

Reading the book I was struck by what a complicated beast or little World Wide Web has become. That's clear from just the chapter titles: Primer on Latency and Bandwidth, Building Blocks of TCP, Building Blocks of UDP, Transport Layer Security, Introduction to Wireless Networks, WiFi, Mobile Networks, Optimizing for Mobile Networks, Brief History of HTTP, Primer on Web Performance, HTTP 1.X, HTTP 2.0, Optimizing Application Delivery, Primer on Browser Networking, XMLLHttpRequest, Server Side Events, WebSocket, and WebRTC. 

I've often imagined there's a white board at Google with "Don't Erase!" written in red at the top. On it is a complicated diagram of every part of the web and Google's master plan for making that part better. The book reads a little like that imagined diagram. The Primer on Latency is a real eye opener. If you wonder why people are always going on about latency then this chapter is for you. The chapter on Mobile Networks is really good, it fills in a lot of details in an excessively complicated space. With HTTP 2.0 on the horizon you can learn about the entire thing here in full on gory detail. I'd never heard of Server Side Events before so that was enlightening. And the WebRTC chapter has an amazing amount of detail on an exciting new browser capability. 

If you've been burned by books that were just reprints of manuals and specification documents, that's not the case with this book. It's full of practical real-world advice. For example, there' one example of how "Apple engineers saw a 300% performance improvement for users on slower networks once they made better reuse of existing TCP connections within iTunes, via HTTP keepalive and pipelining!" Bits like this can be found throughout the book.

And many chapters end with a practical set of things you can do to implement all the stuff you've learned part. For example, the WebRTC chapter ends with a Performance Checklist that begins with a few sections like:

Signaling service
• Use a low-latency transport.
• Provision sufficient capacity.
• Consider using signaling over DataChannel once connection is established.
Firewall and NAT traversal
• Provide a STUN server when initiating RTCPeerConnection.
• Use trickle ICE whenever possible—more signaling, but faster setup.
• Provide a TURN server for relaying failed peer-to-peer connections.

 

Almost every chapter has solid practical advice like this on how to make the web faster for your application.

Here's my email interview with Ilya Grigorik on High Performance Browser Networking. Enjoy.

Please Tell Us Who You Are And What You've Brought To Show And Tell Today?

Click to read more ...

Tuesday
Oct152013

Sponsored Post: Apple, ScaleOut, FreeAgent, CloudStats.me, Intechnica, Couchbase, MongoDB, Stackdriver, BlueStripe, Booking, Rackspace, AiCache, Aerospike, New Relic, LogicMonitor, AppDynamics, ManageEngine, Site24x7

Who's Hiring?

  • Apple is hiring for multiple positions. Imagine what you could do here. At Apple, great ideas have a way of becoming great products, services, and customer experiences very quickly.
    • Sr. Software Engineer. The Identity Management Services team at Apple is in search of a motivated Senior Software Engineer who is self-driven and has a proven track record in design and development of complex, highly available and scalable systems. Please apply here
    • SQE and Operations Manager, iOS Systems. The iOS Systems team is looking for an experienced hands-on manager to lead the Quality Engineering, Build and Release Engineering team. Please apply here
    • Senior Engineer: Emerging Technology. Apple’s Emerging Technology group is looking for a senior engineer passionate about exploring emerging technologies to create paradigm shifting cloud based solutions. Please apply here. 
    • Senior Storage Engineer. Software Engineering Operations (SEO) is seeking an experienced storage engineer to join our team. This role will focus on designing, deploying and maintaining critical SAN and NAS storage solutions. Please apply here

  • UI EngineerAppDynamics, founded in 2008 and lead by proven innovators, is looking for a passionate UI Engineer to design, architect, and develop our their user interface using the latest web and mobile technologies. Make the impossible possible and the hard easy. Apply here.

  • Software Engineer - Infrastructure & Big DataAppDynamics, leader in next generation solutions for managing modern, distributed, and extremely complex applications residing in both the cloud and the data center, is looking for a Software Engineers (All-Levels) to design and develop scalable software written in Java and MySQL for backend component of software that manages application architectures. Apply here.

  • FreeAgent are looking for a talented Operations Engineer to come and work on the FreeAgent app, internal services and supporting infrastructure. You'll be working alongside our Ops team squashing single points of failure, fixing bottlenecks, profiling load and solving interesting scaling and automation problems. Please apply here

  • Intechnica is looking for Performance Architects, Performance Engineers, a Lead Automation Engineer, and a Solution Assurance Analyst. If making super-fast systems is your forte, send your CV with covering letter to careers@intechnica.co.uk.

  • Stackdriver is looking for systems + cloud + dev + ops guru to serve as our liaison within the DevOps community. If you are passionate about monitoring and automation, enjoy working on open source, and are excited by the prospect of sharing your expertise with your peers, get in touch with us today! http://bit.ly/143ARmy

  • We need awesome people @ Booking.com - We want YOU! Come design next generation interfaces, solve critical scalability problems, and hack on one of the largest Perl codebases. Please apply online.

  • LogicMonitor is looking for a Front End developer to have a huge impact, be valued, realize their dreams, and help us realize ours. We are looking for someone to own the code that delivers the design and usability of LogicMonitor's enterprise SaaS application(s). Please apply online

  • New Relic is looking for a Java Instrumentation Engineer, Java Scalability Engineer,  Distributed Systems Engineer and Android app engineer in Portland, OR. Ready to scale a web service with more incoming bits/second than Twitter? 

Fun and Informative Events

  • Your event here.

Cool Products and Services

  • Rapidly Develop Hadoop MapReduce Code. With ScaleOut hServer™ you can use a subset of your Hadoop data and run your MapReduce code in seconds for fast code development and you don’t need to load and manage the Hadoop software  stack, it's a self-contained Hadoop MapReduce execution environment. To learn more check out www.scaleoutsoftware.com/prototypehadoop/

  • CloudStats.me - Monitor all your VPS, Dedicated and Cloud servers from one place. Whether you have only one server or hundreds of them, you will be able to check their status in seconds from the dashboard. Try server monitoring now for free.

  • The leading technology companies use Couchbase as their NoSQL database. Download the free open-source version of Couchbase Server and make something awesome today.

  • MongoDB Management Service (MMS) is a cloud-based suite of services for managing MongoDB deployments. In addition to monitoring and alerting, now you can seamlessly back up your MongoDB deployment to the cloud using using MMS. To get started with monitoring and backup, visit mms.mongodb.com.

  • BlueStripe FactFinder Express is the ultimate tool for server monitoring and solving performance problems. Monitor URL response times and see if the problem is the application, a back-end call, a disk, or OS resources.

  • NEW! Aerospike 3 - Download FREE. Introducing the new Aerospike 3 database that builds off of Aerospike's legacy of speed, scale, and reliability, adding an extensible data model that supports complex data types, large data types, queries using secondary indexes, user defined functions (UDFs) and distributed aggregations using Stream UDFs for real-time data.

  • The Rackspace Cloud Application Programming Interface (API)  has changed the game allowing customers to easily modify their cloud configuration with just a few lines of code. The API is a powerful tool and something everyone should know about, regardless of your level of technical ability.

  • aiScaler, aiProtect, aiMobile integrated solutions for Dynamic Site Acceleration, Denial of Service Protection and Simplifying Mobile Content. Free instant trial, no sign-up required . http://aicache.com/

  • LogicMonitor - Hosted monitoring of your entire technology stack. Dashboards, trending graphs, alerting. Try it free and be up and running in just 15 minutes.

  • ManageEngine Applications Manager : Monitor physical, virtual and Cloud Applications.

  • www.site24x7.com : Monitor End User Experience from a global monitoring network.

If any of these items interest you there's a full description of each sponsor below. Please click to read more...

Click to read more ...

Sunday
Oct132013

AIDA: Badoo’s journey into Continuous Integration

It’s hardly news to anyone that product development and testing involve a lot of boring routine work, which can lead to human error. To avoid complications stemming from this, we use AIDA.

AIDA (Automated Interactive Deploy Assistant) is a utility that automatically performs many of the processes in Git, TeamCity and JIRA.

In this post, we focus on how through using AIDA we were able to automate multiple workflows and create a scheme of continuous integration.

We’ll start by looking at the version control system (VCS) we use here at Badoo, specifically how Git is used to automate creation of release branches, and their subsequent merging. Then we’ll discuss AIDA’s major contribution to both JIRA integration and TeamCity.

Git flow

Click to read more ...

Friday
Oct112013

Stuff The Internet Says On Scalability For October 11th, 2013

Hey, it's HighScalability time:


In honor of Twitter's Cha-Ching moment, here's Twitter By the Numbers
  • Quotable Quotes:
    • @BrandonBloom: Mutable data structures are only faster b/c they have fewer features: ie no persistence. You must manually recover that feature w/ copying.
    • @hayesdrumwright: Scale breaks hardware Speed breaks software Scale and speed breaks everything Adrian Cockcroft - Netflix #TechSummit
    • Gladwell, Malcolm: Saul thinks of power in terms of physical might. He doesn’t appreciate that power can come in other forms as well— in breaking rules, in substituting speed and surprise for strength.
  • Now here's an irony. The East India Company established a major trading post at Bantam in Java. They called their trading posts "factories." Get it? Java. Factories. That's stranger than fiction.

Don't miss all that the Internet has to say on Scalability, click below and become eventually consistent with all scalability knowledge...

Click to read more ...

Tuesday
Oct082013

F1 and Spanner Holistically Compared

This aricle, F1: A Distributed SQL Database That Scales by Srihari Srinivasan, is republished with permission from a blog you really should follow: Systems We Make - Curating Complex Distributed Systems.

With both the F1 and Spanner papers out its now possible to understand their interplay a bit holistically. So lets start by revisiting the key goals of both systems.

Key Goals of F1′s design
  • System must be able to scale up by adding resources
  • Ability to re-shard and rebalance data without application changes
  • ACID consistency for transactions
  • Full SQL support, support for indexes
Spanner’s objectives

Click to read more ...

Monday
Oct072013

Ask HS: Is Microsoft the Right Technology for a Scalable Web-based System?

This question was asked over email and I thought a larger audience might want to take a whack at it.

I have a problem I’d like to have your view on. I’ve looked around a lot, and I haven’t found a definite answer. The question is this:

Is it true that for a scalable web-based system targeting millions of users (hopefully), using Microsoft technology(.Net/SQL Server) over open source technologies like python/ruby/php and mysql (mariadb) / postgresql will cost you more? Is there any justification for paying up for Microsoft licenses(OS, SQL Server, …)?

I am in charge of selecting the technology toolbox for a startup which is going to build a scalable public web platform. I’ve worked as a developer and database developer/admin (mainly as a DBA) using different platforms and technologies, but my main focus is on Microsoft technology. I’ve considered all other important factors for this decision, and at the end, I always come back to the question of money. When I finish developing the first stage of the system, and present it to possible investors to raise money and expand it, will it be a negative point(or even a deal breaker) to have a system developed on top of Microsoft technology stack?

Every time I decide to go with Microsoft, I ask myself “why no other major web-based system (other than stackoverflow) is built on Microsoft technology?”, and I’m back to square one.

Thanks for your time.

On HackerNewsOn Reddit

Friday
Oct042013

Stuff The Internet Says On Scalability For October 4th, 2013

Hey, it's HighScalability time:


(Consumption as a Service: Amazon has 80 1.2 million square foot warehouses.)
  • 100 megapixel cameras, taking 40 million pictures a second, creating of 1 petabyte of data every second:  Large Hadron Collider in Higgs search

  • Quotable Quotes:
    • Tim Bell: You can't do transformation in transitional stages
    • @lleung: Also, we could really use a “high scalability” like blog for Enterprise IT to see the interesting stuff
    • xkcd: Functional programming combines the flexibility and power of abstract mathematics with the intuitive clarity of abstract mathematics.
    • @pbailis: That said, just because you can now build distributed transactions doesn't mean they'll be fast or work well during failures.

  • Ever wonder how you can get those shoes and that tube of miracle cream from Amazon at the same time in the same box? Here's a fascinating look at The Amazon Warehouses chaotic storage system: Chaotic storage is like organized confusion. It’s an organic shelving system without permanent areas or sections. That means there is no area just for books, or a place just for televisions (like you might expect in a retail store layout).

Don't miss all that the Internet has to say on Scalability, click below and become eventually consistent with all scalability knowledge...

Click to read more ...

Wednesday
Oct022013

RFC 1925 - The Twelve (Timeless) Networking Truths

The Twelve Networking Truths is one of a long series of timeless truths documented in sacred April Fools' RFCs. Though issued in 1996, it's no less true today.

It's hard to pick a favorite because they are all good. But if I had to pick, it would be:

Some things in life can never be fully appreciated nor understood unless experienced firsthand.

As we grow comfortable behind garden walls, clutching gadgets like lifelines and ideologies like shields, empathy is the true social network.

 

Network Working Group                                  R. Callon, Editor
Request for Comments: 1925                                          IOOF
Category: Informational                                     1 April 1996


                      The Twelve Networking Truths

Click to read more ...

Tuesday
Oct012013

Sponsored Post: Apple, Intechnica, Couchbase, MongoDB, Stackdriver, BlueStripe, Surge, Booking, Rackspace, AiCache, Aerospike, New Relic, LogicMonitor, AppDynamics, ManageEngine, Site24x7

Who's Hiring?

  • Intechnica is looking for Performance Architects, Performance Engineers, a Lead Automation Engineer, and a Solution Assurance Analyst. If making super-fast systems is your forte, send your CV with covering letter to careers@intechnica.co.uk.

  • Apple is hiring a Senior Storage Engineer. Software Engineering Operations (SEO) is seeking an experienced storage engineer to join our team. This role will focus on designing, deploying and maintaining critical SAN and NAS storage solutions. Please apply here

  • Stackdriver is looking for systems + cloud + dev + ops guru to serve as our liaison within the DevOps community. If you are passionate about monitoring and automation, enjoy working on open source, and are excited by the prospect of sharing your expertise with your peers, get in touch with us today! http://bit.ly/143ARmy

  • We need awesome people @ Booking.com - We want YOU! Come design next generation interfaces, solve critical scalability problems, and hack on one of the largest Perl codebases. Please apply online.

  • Apple Applications Architect. Apple's Customer Systems group, within the global Information Systems and Technology (IS&T) organization, helps design and implement all of the critical infrastructure that allows Apple to provide the best support in the world. Imagine what you could do here. Please apply here.

  • Apple Software Engineer, Researcher. The OS X Analytics group is looking for a skilled software engineer. In this position you will develop tools to collect, process, and analyze data from millions of systems. Imagine what you could do here. Please apply here.

  • LogicMonitor is looking for a Front End developer to have a huge impact, be valued, realize their dreams, and help us realize ours. We are looking for someone to own the code that delivers the design and usability of LogicMonitor's enterprise SaaS application(s). Please apply online

  • New Relic is looking for a Java Instrumentation Engineer, Java Scalability Engineer,  Distributed Systems Engineer and Android app engineer in Portland, OR. Ready to scale a web service with more incoming bits/second than Twitter? 

Fun and Informative Events

  • Surge - The Scalability & Performance Conference, presented by OmniTI, Sept. 12th-13th, features speakers from Joyent, Fastly, Dyn, Netflix, Linkedin and Amazon. Special, High Scalability Reader Rate: $50 off registration--through Sept. 10! Book hotel and get $50 off, from OmniTI. 

Cool Products and Services

  • The leading technology companies use Couchbase as their NoSQL database. Download the free open-source version of Couchbase Server and make something awesome today.

  • MongoDB Management Service (MMS) is a cloud-based suite of services for managing MongoDB deployments. In addition to monitoring and alerting, now you can seamlessly back up your MongoDB deployment to the cloud using using MMS. To get started with monitoring and backup, visit mms.mongodb.com.

  • BlueStripe FactFinder Express is the ultimate tool for server monitoring and solving performance problems. Monitor URL response times and see if the problem is the application, a back-end call, a disk, or OS resources.

  • AppDynamics is an easy-to-use application performance management solution that offers code-level insight into Java, .NET and PHP applications. Get the free trial.

  • NEW! Aerospike 3 - Download FREE. Introducing the new Aerospike 3 database that builds off of Aerospike's legacy of speed, scale, and reliability, adding an extensible data model that supports complex data types, large data types, queries using secondary indexes, user defined functions (UDFs) and distributed aggregations using Stream UDFs for real-time data.

  • The Rackspace Cloud Application Programming Interface (API)  has changed the game allowing customers to easily modify their cloud configuration with just a few lines of code. The API is a powerful tool and something everyone should know about, regardless of your level of technical ability.

  • aiScaler, aiProtect, aiMobile integrated solutions for Dynamic Site Acceleration, Denial of Service Protection and Simplifying Mobile Content. Free instant trial, no sign-up required . http://aicache.com/

  • LogicMonitor - Hosted monitoring of your entire technology stack. Dashboards, trending graphs, alerting. Try it free and be up and running in just 15 minutes.

  • AppDynamics is the very first free product designed for troubleshooting Java performance while getting full visibility in production environments. Visit  http://www.appdynamics.com/freetrial.

  • ManageEngine Applications Manager : Monitor physical, virtual and Cloud Applications.

  • www.site24x7.com : Monitor End User Experience from a global monitoring network.

If any of these items interest you there's a full description of each sponsor below. Please click to read more...

Click to read more ...