Use Instance Caches to Save Money: Latency == $$$
Tuesday, September 27, 2011 at 9:57AM
Todd Hoff in Strategy

In the post Using memcache might be free, but it's costing you money with instance pricing! Use instance caches if possible made on the Google App Engine group, Santiago Lema brings up an oldie but a goody of an idea that was once used to improve performance, but now it's used to save money:

Where have we seen this before?

This is a variant of the old Sticky Session idea where web sessions are stored in RAM in an application server and all further interaction with a user on that session are routed back to that same server. This approach has gone out fashion in favor of storing session state in the database, or in memcache, or by not having a sense of session state at all.

It's not completely out fashion however. The StackExchange folks, for example, use Sticky Sessions for speed reasons and to take a load off the network. All those cache requests put a lot stress on the network, so it's better to avoid them when possible.

There's also a parallel to the idea of replicating staticish tables across servers so that joins can be local, avoiding the expense of remote access. VoltDB uses this strategy to great effect.

It sounds like this technique may be making a comeback. All the usual cache consistency and memory limitation issues apply, but when latency is so strongly linked to cost, serving data out of local RAM in the web tier is as fast and cheap as it gets. 

Article originally appeared on (
See website for complete article licensing information.