Scalability Worst Practices

Brian Zimmer, architect at travel startup Yapta, highlights some worst practices jeopardizing the growth and scalability of a system:
* The Golden Hammer. Forcing a particular technology to work in ways it was not intended is sometimes counter-productive. Using a database to store key-value pairs is one example. Another example is using threads to program for concurrency.
* Resource Abuse. Manage the availability of shared resources because when they fail, by definition, their failure is experienced pervasively rather than in isolation. For example, connection management to the database through a thread pool.
* Big Ball of Mud. Failure to manage dependencies inhibits agility and scalability.
* Everything or Something. In both code and application dependency management, the worst practice is not understanding the relationships and formulating a model to facilitate their management. Failure to enforce diligent control is a contributing scalability inhibiter.
* Forgetting to check the time. To properly scale a system it is imperative to manage the time alloted for requests to be handled.
* Hero Pattern. One popular solution to the operation issue is a Hero who can and often will manage the bulk of the operational needs. For a large system of many components this approach does not scale, yet it is one of the most frequently-deployed solutions.
* Not automating. A system too dependent on human intervention, frequently the result of having a Hero, is dangerously exposed to issues of reproducibility and hit-by-a-bus syndrome.
* Monitoring. Monitoring, like testing, is often one of the first items sacrificed when time is tight.

Comments

Re: Scalability Worst Practices

I've found monitoring gets developed better if it (alongside tests) are the way the developers test the system. If the test runs include the monitor info, and that is how you have to work out what went wrong, you add good diagnostics.

Re: Scalability Worst Practices

"The Golden Hammer. Forcing a particular technology to work in ways it was not intended is sometimes counter-productive. Using a database to store key-value pairs is one example."

So what is the best-practice way of persistently storing key-value pairs? Don't say memcached as that isn't persistent.

Re: Scalability Worst Practices

> So what is the best-practice way of persistently storing key-value pairs? Don't say memcached as that isn't persistent.

berkely db's are a good candidate for key/value pair lookup:

http://en.wikipedia.org/wiki/Berkeley_DB

Todd Hoff's picture

Re: Scalability Worst Practices

Take a look at memcachedb too. Next week I'll have something on how Digg is using memcachedb for their some high write tasks.

Re: Scalability Worst Practices

The growing IT sector of Bangalore is no longer news. So much so that the number of business hotels in Bangalore have tremendously increased after the entry and subsequent flourish of the IT sector.

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

Post new comment

The content of this field is kept private and will not be shown publicly.
  • Web page addresses and e-mail addresses turn into links automatically.
  • Allowed HTML tags: <a> <em> <strong> <cite> <code> <ul> <ol> <li> <dl> <dt> <dd><div ?=?><p ?=?> <img ?=?><h1 ?=?><h2 ?=?><h3 ?=?>
  • Lines and paragraphs break automatically.
  • Glossary terms will be automatically marked with links to their descriptions
  • You may link to webpages through the weblinks registry

More information about formatting options

To combat spam, please enter the code in the image.