advertise
Tuesday
May312011

Sponsored Post: Animoto, deviantART, Hadapt, Clustrix, Percona, Mathworks, AppDynamics, ScaleOut, Cloudkick, Membase, CloudSigma, ManageEngine, Site24x7

Who's Hiring?

  • Animoto is building a Systems (DevOps) Team. Please apply here
  • Clustrix Inc. - Learn more about Clustrix's interpretation of NewSQL. Please apply here.
  • deviantART is looking for Network and Systems Operations Engineer. Please apply here.
  • Hadapt brings high-performance SQL to Hadoop, and is looking for a systems engineer to join this fast-growing company. Please apply at http://www.hadapt.com/jobs.
  • MathWorks Looking for Multiple, Full-time Scaling Experts. Apply now: http://matlab.my/lVmunb 

Fun and Informative Events

  • Percona is running an intensive one-day MySQL conference in New York City on May 26th.  High Scalability readers save $50 with the code PLNY-HiSc. Learn more and register at percona.com/live/.
  • CouchDB Developer Training coming to Washington, D.C., Portland, San Francisco and Chicago! Membase Server Ops Training coming to New York City and San Francisco!

Cool Products and Services

For a longer description of each sponsor, please read more below...

Click to read more ...

Friday
May272011

Stuff The Internet Says On Scalability For May 27, 2011

Submitted for your scaling pleasure: 

Read much more of Stuff the Internet Says by clicking the down below...

Click to read more ...

Wednesday
May252011

Stuff to Watch from Surge 2010

Surge is a conference put on by OmniTI targeting practical Scalability matters. OmniTI specializes in helping people solve their scalability problems, as is only natural, as it was founded by Theo Schlossnagle, author of the canonical Scalable Internet Architectures

Now that Surge 2011 is on the horizon, they've generously made available nearly all the videos from the Surge 2010 conference.  A pattern hopefully every conference will follow (only don't wait a year please). We lose a lot of collective wisdom from events not being available online in a timely manner.

In truth, nearly all the talks are on topic and are worth watching, but here are a few that seem especially relevant:

Click to read more ...

Monday
May232011

Evernote Architecture - 9 Million Users and 150 Million Requests a Day

The folks at Evernote were kind enough to write up an overview of their architecture in a post titled Architectural Digest. Dave Engberg describes their approach to networking, sharding, user storage, search, and some other custom services.

Evernote is a cool application, partially realizing Vannevar Bush's amazing vision of a memex. Wikipedia describes Evernote's features succinctly: 

Evernote is a suite of software and services designed for notetaking and archiving. A "note" can be a piece of formattable text, a full webpage or webpage excerpt, a photograph, a voice memo, or a handwritten "ink" note. Notes can also have file attachments. Notes can then be sorted into folders, tagged, annotated, edited, given comments, and searched. Evernote supports a number of operating system platforms (including Android, Mac OS X, iOS, Microsoft Windows and WebOS), and also offers online synchronization and backup services.

Key here is that Evernote stores a lot of data, that must be searched, and synced through their cloud to any device you use. 

Another key is the effect of Evernote's business model and cost structure. Evernote is notable for their pioneering of the freemium model, based on the idea from their CEO: The easiest way to get 1 million people paying is to get 1 billion people using. Evernote is designed to become profitable at a 1% conversion rate. The free online service limits users to a hefty 60 MB/month while premium users pay $45 per year for 1,000 MB/month. To be profitable they most store a lot of data without spending a lot of money. There's not a lot of room for extras, which accounts for the simple practicality of their architecture. 

The article is short and succinct, so definitely read it for details. Some takeaways:  

Click to read more ...

Friday
May202011

Stuff The Internet Says On Scalability For May 20, 2011

Submitted for your reading pleasure on this beautiful morning: 

 

  • Group Decision Making in Honey Bee Swarms. In distributed computing systems nodes reach a quorum when deciding what to do as a group. It turns out bees also use quorum logic when deciding on where to nest! Bees do it a bit differently of course:  A scout bee votes for a site by spending time at it, somehow the scouts act and interact so that their numbers rise faster at superior sites, and somehow the bees at each site monitor their numbers there so that they know whether they've reached the threshold number (quorum) and can proceed to initiating the swarm's move to this site. Ants use similar mechanisms to control foraging. Distributed systems may share common mechanisms based on their nature as being a distributed system,  the components may not matter that much.
  • Fire! Fire!  Brent Chapman shows how to put that IT fire out in Incident Command for IT: What We Can Learn from the Fire Department
  • Scale Fail (part 1). Josh Berkus warns against hopping on the trendy train: Scaling an application is all about management of resources and administrative repeatability. Use data so that you work on real unknowns instead of unknown unknowns. And blocking processes, just don't do it.
  • Quotable quotes:
    • @Sri_few_words: Every 600 phones, means a new server in data center" - Cloud Computing being driven strongly by smartphones; tablets
There's a lot more Stuff the Internet says, don't be left out, read more below...

Click to read more ...

Thursday
May192011

Zynga's Z Cloud - Scale Fast or Fail Fast by Merging Private and Public Clouds

Release early and often. A/B testing. Creating a landing page and buying ads on AdSense. All are ways of providing quick feedback in order to validate an idea. If you are like Zynga, with 250 million active users a month, how do you cost effectively prove out a game that could flop or get 90 million users (like CityVille) in an instant?

Zynga handles this problem inlle an innovative way, by inverting the typical cloud burst scenario that has excess traffic flowing from a datacenter to a cloud, to having a game start in the cloud and then moving to the datacenter once the game has proved popular enough to keep.

This process is nicely described by Charles Babcock in Lessons From FarmVille: How Zynga Uses The Cloud, in an interview with Allan Leinwand, CTO of infrastructure engineering at Zynga.

When paired down to its essence, Zynga's strategy goes something like this:

Click to read more ...

Tuesday
May172011

Facebook: An Example Canonical Architecture for Scaling Billions of Messages

What should the architecture of your scalable, real-time, highly available service look like? There are as many options as there are developers, but if you are looking for a general template, this architecture as described by Prashant Malik, Facebook's lead for the Messages back end team, in Scaling the Messages Application Back End, is a very good example to consider. 

Although Messages is tasked with handling 135+ billion messages a month, from email, IM, SMS,  text messages, and Facebook messages, you may think this is an example of BigArchitecture and doesn't apply to smaller sites. Not so. It's a good, well thought out example of a non-cloud architecture exhibiting many qualities any mom would be proud of:

Click to read more ...

Tuesday
May172011

Sponsored Post: Animoto, deviantART, Hadapt, Clustrix, Percona, Mathworks, AppDynamics, ScaleOut, Cloudkick, Membase, CloudSigma, ManageEngine, Site24x7

Who's Hiring?

  • Animoto is building a Systems (DevOps) Team. Please apply here
  • Clustrix Inc. - Learn more about Clustrix's interpretation of NewSQL. Please apply here.
  • deviantART is looking for Network and Systems Operations Engineer. Please apply here.
  • Hadapt brings high-performance SQL to Hadoop, and is looking for a systems engineer to join this fast-growing company. Please apply at http://www.hadapt.com/jobs.
  • MathWorks Looking for Multiple, Full-time Scaling Experts. Apply now: http://matlab.my/lVmunb 

Fun and Informative Events

  • Percona is running an intensive one-day MySQL conference in New York City on May 26th.  High Scalability readers save $50 with the code PLNY-HiSc. Learn more and register at percona.com/live/.
  • CouchDB Developer Training coming to Washington, D.C., Portland, San Francisco and Chicago! 
    Membase Server Ops Training coming to New York City and San Francisco!

Cool Products and Services

For a longer description of each sponsor please read more below...

Click to read more ...

Sunday
May152011

Building a Database remote availability site

The AWS East Region outage showed all of us the importance of running our apps and databases across multiple Amazon regions (or multiple cloud providers). In this post, I’ll try to explain how to build a MySQL (or Amazon RDS) redundant site. For simplicity, we create a passive redundant site. This means that the site is not used during normal operation and only comes into action when the primary site crashes. There are many reasons for choosing such an architecture – it’s easy to configure, simple to understand, and minimizes the risk of data collision. The downside is that you have hardware just sitting around doing nothing. Still, it’s a common enough scenario. So what do we need to do to make it work?

Click to read more ...

Friday
May132011

Stuff The Internet Says On Scalability For May 13, 2011

Submitted for your reading pleasure on this beautiful blue sky, birds chirping Friday morning: