Strategy: Solve Only 80 Percent of the Problem
Solve only 80% of a problem. That's usually good enough and you'll not only get done faster, you'll actually have a chance of getting done at all.
This strategy is given by Amix in HOW TWITTER (AND FACEBOOK) SOLVE PROBLEMS PARTIALLY. The idea is solving 100% of a complex problem can be so hard and so expensive that you'll end up wasting all your bullets on a problem that could have been satisfactoraly solved in a much simpler way.
The example given is for Twitter's real-time search. Real-time search almost by definition is focussed on recent events. So in the design should you be able to search historically back from the beginning of time or should you just be able to search for recent time periods? A complete historical search is the 100% solution. The recent data only search is the 80% solution. Which should you choose?
The 100% solution is dramatically more difficult to solve. It requires searching disk in real-time which is a killer. So it makes more sense to work on the 80% problem because it will satisfy most of your users and is much more doable.
By reducing the amount of data you need to search it's possible to make some simplifying design choices, like using fixed sized buffers that reside completely in memory. With that architecture your streaming searches can be blisteringly fast while returning the most relevant data. Users are happy and you are happy.
It's not a 100% solution, but it's a good enough solution that works. Sometimes as programmers we are blinded by the glory of the challenge of solving the 100% solution when there's a more reasonable, rational alternative that's almost as good. Something to keep in mind when you are wondering how you'll possibly get it all done. Don't even try.
Amix has a very good discussion of Twitter and this strategy on his blog.
Worse is Better
A Hacker News post discussing this article brought up that this strategy is the same as Richard Gabriel's famous Worse-is-Better paradox which holds: The right thing is frequently a monolithic piece of software, but for no reason other than that the right thing is often designed monolithically. That is, this characteristic is a happenstance. The lesson to be learned from this is that it is often undesirable to go for the right thing first. It is better to get half of the right thing available so that it spreads like a virus. Once people are hooked on it, take the time to improve it to 90% of the right thing.
Unix, C, C++, Twitter and almost every product that has experienced wide adoption has followed this philosophy.
Worse-is-Better solutions have the following characteristics:
In my gut I think Worse-is-Better is different than "Solve Only 80 Percent of the Problem" primarily because Worse-is-Better is more about product adoption curves and 80% is more a design heuristic. After some cogitating this seems a false distinction so I have to concluded I'm wrong and have added Worse-is-Better to this post.
