Drop ACID and Think About Data
Tuesday, May 5, 2009 at 5:08AM
Todd Hoff in key-value store

The abstract for the talk given by Bob Ippolito, co-founder and CTO of Mochi Media, Inc:

Building large systems on top of a traditional single-master RDBMS data storage layer is no longer good enough. This talk explores the landscape of new technologies available today to augment your data layer to improve performance and reliability. Is your application a good fit for caches, bloom filters, bitmap indexes, column stores, distributed key/value stores, or document databases? Learn how they work (in theory and practice) and decide for yourself.

Bob does an excellent job highlighting different products and the key concepts to understand when pondering the wide variety of new database offerings. It's unlikely you'll be able to say oh, this is the database for me after watching the presentation, but you will be much better informed on your options. And I imagine slightly confused as to what to do :-)

An interesting observation in the talk is that the more robust products are internal to large companies like Amazon and Google or are commercial. A lot of the open source products aren't yet considered ready for prime-time and Bob encourages developers to join a project and make patches rather than start yet another half finished key-value store clone. From my monitoring of the interwebs this does seem to be happening and existing products are starting to mature.

From all the choices discussed the column database Vertica seems closest to Bob's heart and it's the product they use. It supports clustering, column storage, compression, bitmapped indexes, bloom filters, grids, and lots of other useful features. And most importantly: it works, which is always a plus :-)

Here's a summary of some of the points talked about in the presentation: Video Presentation of Drop ACID and Think About Data.



If you can't have all of the ACID guarantees it turns out you can have two of the following three characteristics:


The types of large systems based on CAP aren't ACID they are BASE (har har):

Everyone who builds big applications builds them on CAP and BASE: Google, Yahoo, Facebook, Amazon, eBay, etc.

Google's BigTable

Amazon's Dynamo

Facebook's Cassandra

Distributed Database Musings

Simple Key-Value Store

Memcached Key-Value Stores

Tokyo Cabinet/Tyrant Key-Value Store

Redis Data Structure Store

Document Databases

CouchDB Document Database

MongoDB Document Database

Column Databases

MonetDB Column Database

LucidDB Column Database

Vertica Column Database

Related Articles

Article originally appeared on (http://highscalability.com/).
See website for complete article licensing information.