High Scalability Building bigger, faster, more reliable websites.

• Google AppEngine - A Second Look (275)

It's been a few days now since GAE (Google App Engine) was released and we had our First Look. It's high time for a retrospective. Too soon? Hey, this is Internet time baby. So how is GAE doing? I did get an invite so hopefully I'll have a more experience grounded take a little later. I don't know Python and being the more methodical type it may take me a while. To perform our retrospective we'll take a look at the three sources of information available to us: actual applications in the AppGallery, blogspew, and developer issues in the forum.

The result: a cautious thumbs up. The biggest issue so far seems to be the change in mindset needed by developers to use GAE. BigTable is not MySQL. The runtime environment is not a VM. A service based approach is not the same as using libraries. A scalable architecture is not the same as one based on optimizing speed. A different approach is needed, but as of yet Google doesn't give you all the tools you need to fully embrace the red pill vision.

I think this quote by Brandon Smith in a thread on how to best implement sessions in GAE nicely sums up the new perspective:

Consider the lack of your daddy's sessions a feature. It's what will make your app scale on Google's infrastructure. 

In other words: when in Rome. But how do we know what the Romans do when the Romans do what they do?

Brett Morgan expands our cultural education in a thread on slow GAE databases performance when he talks about why MySQL thinking won't work on BigTable:

It might look almost look like a sql db when you squint, but it's
optimized for a totally different goal. If you think that each
different entity you retrieve could be retrieving a different disk
block from a different machine in the cluster, then suddenly things
start to make sense. avg() over a column in a sql server makes sense,
because the disk accesses are pulling blocks in a row from the same
disk (hopefully), or even better, all from the same ram on the one
computer. With DataStore, which is built on top of BigTable, which is
built on top of GFS, there ain't no such promise. Each entity in
DataStore is quite possibly a different file in gfs.

So if you build things such that web requests are only ever pulling a
single entity from DataStore - by always precomputing everything -
then your app will fly on all the read requests. In fact, if that
single entity gets hot - is highly utilized across the cluster - then
it will be replicated across the cluster.

Yes, this means that everything that we think we know about building
web applications is suddenly wrong. But this is actually a good thing.
Having been on the wrong side of trying to scale up web app code, I
can honestly say it is better to push the requirements of scaling into
the face of us developers so that we do the right thing from the
beginning. It's easier to solve the issues at the start, than try and
retrofit hacks at the end of the development cycle. 

A truly excellent explanation of the differences between MySQL thinking and GAE thinking.

Now, if you can't use MySQL's avg feature, how can an average be calculated using BigTable? Brett advises:

Instead of calculating the results at query time, calculate them when
you are adding the records. This means that displaying the results is
just a lookup, and that the calculation costs are amortized over each
record addition. 

Clearly this is more work for the programmer and at first blush doesn't seem worth the effort, especially when you are used to the convenience of MySQL. That's why in the same thread Barry Hunter insightfully comments that GAE may not be for everyone:

This might be a very naive observation, but I perhaps wonder then if
GAE is the tool for you.

As I see it the App Engine is for applications that are meant to
scale, scale and really scale. Sounds like an application with a few
hundred hits daily could easily run on traditional hosting platforms.

It's a completely different mindset. 
...
Again maybe I am missing something, but the DataStore isn't designed to
be super fast at the small scale, but rather handle large amounts of
data, and be distributed (and because its distributed it can appear
very fast at large scale).

So you break down your database access into very simple processes.
Assume your database access is VERY slow, and rethink how you do
things. (Of course the piece in the puzzle 'we' are missing is
MapReduce! - the 'processing' part of the BigTable mindset) 

Before developers can take full advantage of GAE these types of lessons need to be extracted and popularized with the same ferocity the multi-tier RDBMS framework has been marketed. It will be a long difficult transition.

Interestingly, many lessons from AWS are not transferable to GAE. AWS has a VM model whereas GAE has an application centric model. They are inverses of each other.

In AWS you have a bag of lowish level components out of which you architect your application. You can write all the fine low level implementations bits you desire. A service layer is then put in front of everything to hide the components. In GAE you have a high level application component and you build out your application using services. You can't build any low level components in GAE. In AWS the goal is to drive load to the CPU because CPU and bandwidth are plentiful. In GAE you get very limitted CPU, certainly none to burn on useless activities like summing up an average over a whole slice of data returned from SimpleDB. And in GAE the amount of data returnable from the database is small so your architecture needs to be very smart about how data is stored and accessed.

Very different approaches that lead to very different applications.

Applications

The number of applications has exploded. I am always amazed at how enthusiastic and productive people can be when they are actually interested in what they are doing. It happens so rarely. True, most applications aren't even up to Facebook standards yet, but it's early days. What's impressive is how fast they were created and deployed. That speaks volumes about the efficacy of the application centric development model.Will it be as effective delivering "real" apps? That's a question I'm not sure about.

So far application performance is acceptable. Certainly nothing spectacular. What can you do about it? Nada.

I like the sketch application because people immediately and quite predictably drew lewd depictions of various body parts. I also like this early incarnation of a forum app. A forum is one of the ideas I thought might work well on AppEngine because the scalable storage problem is solved. I do wonder how the performance will be with a fine tuned caching layer? Vorby is a movie quote site showing a more realistic level of complexity. It has tabs, long lists of text, some graphical elements, some more complex screens, and ratings. It shows you can make applications you wouldn't mind people using.

An option I'd like to see in the App Gallery is a view source link. Developers could indicate when adding an application if others can view their application source. Then when browsing the gallery we could all learn by looking at real working code. This is how html spread so quickly. Anyone could view the source for any page, copy paste, and you're on your way! With an application centric model the view source viral spread approach would also work.

Blogspew

As expected there's lots of blog activity on GAE: