OpenStack - The Answer to: How do We Compete with Amazon?

The Silicon Valley Cloud Computing Group had a meetup Wednesday on OpenStack, whose tag line is the open source, open standards cloud. I was shocked at the large turnout. 287 people registered and it looked like a large percentage of them actually showed up. I wonder, was it the gourmet pizza, the free t-shirts, or are people really that interested in OpenStack? And if they are really interested, why are they that interested? On the surface an open cloud doesn't seem all that sexy a topic, but with contributions from NASA, from Rackspace, and from a very avid user community, a lot of interest there seems to be.

The brief intro blurb to OpenStack is:

OpenStack allows any organization to create and offer cloud computing capabilities using open source software running on standard hardware. OpenStack Compute is software for automatically creating and managing large groups of virtual private servers. OpenStack Storage is software for creating redundant, scalable object storage using clusters of commodity servers to store terabytes or even petabytes of data.

This meeting seemed eerily familiar because oh so long ago, like just a few months ago, NASA presented on their internal cloud platform Nebula. And in that short a time everything changed. OpenStack was born, an active development community formed, and OpenStack is near its first official release. How quickly they grow up.

At the time I remember thinking about Nebula: wow, the government is really screwed up, good on these guys for trying to fix it, what a great job they are doing, not sure it would work for anyone else; going with an Amazon API compatibility strategy makes sense; and what a hodgepodge of different components, I bet it will be an operations nightmare.

I learned yesterday that the biggest change is that they are dumping Amazon compatibility mode. Since Amazon hasn't said it's OK to use their APIs it's risky legally to clone them. But I suspect the most important reason is the entrance of Rackspace into the mix. Instead of Amazon APIs they are moving to Rackspace APIs. This makes perfect sense and argues for the result actually working in a real-life cloud since Rackspace is a real-life cloud vendor. Rackspace is also very active in the open source movement, so this is a great match and should solidify adoption.

I imagine what Rackspace gets out of this is, that if successful, they will at least have some sort of leverage with Amazon. Amazon is a machine. Amazon executes to perfection and they release new features at a relentless pace with no signs of slowing down. They don't leave much of a door open for others to get into the game. With a real open cloud alternative it might allow a lot of people to play in the cloud space that would have been squeezed out before.

What Amazon can't match is the open cloud's capability of simultaneous supporting applications that can run seamlessly in a private cloud hosted in a corporate datacenter, in local development and test clouds, and in a full featured public cloud.

What I've heard is that this is a move by public cloud vendors to commoditize the cloud infrastructure, which then allows the concentration on selling managed services. This strategy can only work if the infrastructure is a true commodity. Amazon has the ability to keep adding high-end features that can potentially nullify the commodity argument. Can an open source cloud move fast enough to compete?

Every framework and every API is a form of lock-in, but this feels a lot less like a strangle-hold. Amazon is concerned enough at least to bother condemning private clouds as unworkable and completely unnecessary with Amazon around. It at least means they care.

Eucalyptus, an Amazon compatible clone, was once part of Nebula but was dropped in favor of building their own software and using Rackspace's work. Their reasoning was they had scaling problems with Eucalyptus and source code changes weren't being merged into the main release so they had to maintain a separate code base, which isn't a viable working relationship. So they opted for more control and are going their own way. Eucalyptus hasn't stayed single long. They are forming a new self-service private cloud along with newScale and rPath.

A key design point emphasized in the meeting is that the design is componentized. They want you to be able swap in and out different implementations so it can be customized for your needs. It should be able to work on your network topology and on your disks. If you want a different authentication system then that should be possible. It's still early though and their philosophy is make it work and then make it good, so it may be a while before they reach component nirvana, but that's the vision. Part of the win of partnering with Rackspace is they've been through these wars before and understand how to build this sort of architecture.

The other point they really really wanted you to take home from the meeting is that this is the most openest open source project you've ever seen. An Apache style license is used so the entire source base is open for use and abuse. They rejected an open core model, so all the source is available and all the features are available. You are without limits.

The other sense that the project is open is that it has a very active development community. Have you been frustrated by other open source projects that give you a big rejection notice when you try to contribute code because you haven't yet attained inner sanctum level 32? OpenStack promises to be different. They will actually merge your code in. How long this can last once real releases start happening, who knows, but it's something.

Sebastion Stadl, founder of both the cloud meetup and Scalr, said he will talk to the OpenStack folks about making Scalr the RightScale equivalent for OpenStack. This would be great. The feeling I got from the talk is that OpenStack will primarily work on the framework, not operations, monitoring, etc., which is a mistake IMHO, but having Scalr on the job would help patch that hole in the offering.

The vibe from the meeting was excited and enthusiastic, like something was happening. There wasn't a lot of technical detail in the presentations. This was much more of a coming-out party, the purpose of which is to inform society that there's a new force in the market. We'll see how many hearts get broken.

Related Articles

• Nova - A NASA project which is the compute part of OpenStack.
• Swift - a Rackspace project that is the storage part of OpenStack. Currently used for CloudFiles at Rackspace. It's product quality code than can handle 100-petabyte clusters and 100,000 requests per second.
• OpenStack iPad app - uses the OpenStack compute and storage APIs to manage compute and storage resources. It can: View RSS system status feeds; Ping compute nodes from several locations around the world; Email files from OpenStack Object Storage; Integrate with Chef and the Opscode Platform.
• OpenStack on Twitter
• Barton George interviews Brett Piatt of Rackspace on who has Joined OpenStack
• irc.freenode.com / #openstack
• The Technology Behind the OpenStack Cloud Computing Project by Jacek Furmankiewicz.