How did FarmVille scale a web application to handle 75 million players a month? Fortunately FarmVille's Luke Rajlich has agreed to let us in on a few their challenges and secrets. Here's what Luke has to say...

Related: High Availability Principle : Concurrency Control (http://saasinterrupted.com/2010/02/05/high-availability-principle-concurrency-control/) by Ashish Soni

1. Voldemort gets rebalancing. It's one thing to shard data to scale, it's a completely different level of functionality to manage those shards intelligently. Voldemort has stepped up by adding advanced rebalancing functionality: Dynamic addition of new nodes to the cluster; Deletion of nodes from cluster; Load balancing of data inside a cluster.
2. Microsoft Finally Opens Azure for Business. Out of the blue Microsoft opens up their platform as a service service. Good to have more competition and we'll keep an eye out for experience reports.
3. New details on LinkedIn architecture by Greg Linden. LinkedIn appears to only use caching minimally, preferring to spend their efforts and machine resources on making sure they can recompute computations quickly than on hiding poor performance behind caching layers.
4. The end of SQL and relational databases?  by David Intersimone. For new projects, I believe, we have genuine non-relational alternatives on the table (pun intended).
5. HipHop for PHP: Move Fast. When you make millions of widgets saving pennies per widget quickly adds up to real money. Facebook released HipHop, a PHP compiler, aimed at shaving off cycle of CPU and bytes of memory in production of their social widgets. 

This all made perfect sense. Usually we just want stuff to work and going with what you know is often the best path to that goal. And what we have known is MySQL. All the different pros and cons of this approach are covered wonderfully in the post.

But the world has changed.

Read more on BigDataMatters.com

If datacenters are the new castles, then what will be the new gunpowder? As soon as gunpowder came on the scene, castles, which are defensive structures, quickly became the future's cold, drafty hotels. Gunpowder fueled cannon balls make short work of castle walls.

There's a long history of "gunpowder" type inventions in the tech industry. PCs took out the timeshare model. The cloud is taking out the PC model. There must be something that will take out the cloud.

Right now it's hard to believe the cloud will one day be no more. They seem so much the future, but something will transcend the cloud.]]>http://highscalability.com/blog/rss-comments-entry-6518043.xmlhot linksTodd HoffThu, 28 Jan 2010 05:52:51 +0000http://highscalability.com/blog/2010/1/27/hot-scalability-links-for-january-28-2010.html426227:4867632:6450749

From the NoSQL Archive the summary on HyperGraphDB is: API: Java (and Java Langs), Written in:Java,  Query Method: Java or P2P, Replication: P2P, Concurrency: STM, Misc: Open-Source, Especially for AI and Semantic Web.

So it has some interesting features, like software transactional memory and P2P  for data distribution, but I found that my first and most obvious question was not answered: what the heck is a hypergraph and why do I care? Buried in the tutorial was:

A HyperGraphDB database is a generalized graph of entities. The generalization is two-fold:

1. Links/edges "point to" an arbitrary number of elements instead of just two as in regular graphs 
2. Links can be pointed to by other links as well.

OK, but I wish there was some explanation of why this is valuable. What can I do with it that I can't do with normal graphs? Given that there have been concerns over the complexity of the API this would seem a natural topic to cover. I assume it's cool, it sounds cool, but I would like to know why :-)

In any case it looks like an interesting product to take a look at. Database options are expanding fast.

There's a pattern, already begun, that has accelerated by the need for applications to scale and increase complexity, the end result of which will be that applications give up their independence and enter a kind of feudal relationship with their platform provider.

To understand how this process works, like a glacier slowly and inevitably carving out a deep river valley, here's the type of question I get quite a lot:

I've learned PHP and MySQL and I've built a web app that I HOPE will receive traffic comparable to eBay's with a similar database structure. I see all these different opinions and different techniques and languages being recommended and it's so confusing. All I want is perhaps one book or one website that focuses on PHP and MySQL and building a large database web app like eBay's. Does something like this exist?

I'm always at a loss for words with these questions. What can I possibly say?]]>http://highscalability.com/blog/rss-comments-entry-6425082.xml