Cloud Bursting between AWS and Rackspace

Cloud bursting is an application deployment model in which an application runs in a private cloud or data center and bursts into a public cloud when the demand for computing capacity spikes. The advantage of such a hybrid cloud deployment is that an organization only pays for extra compute resources when they are needed. (Source: SearchCloudComputing)

Neal Sample the former CTO - X.commerce at eBay gave an interesting talk last year on the economic benefit of Cloud Bursting. Neal pointed out eBay traffic statistics and showed some real numbers of the business impact of bursting peak load activities using on-demand cloud resources as presented in the diagram from his talk.

In order to use cloud bursting effectively we need to address the following set of challenges:

• Workload migration - specifically the ability to clone our application environment in a consistent way across sites in an on demand fashion.
• Data Synchronization - The ability to maintain real time copy of the data between the two sites.
• Network connectivity - The ability to enable flow of netwrok traffic between between two sites.

A Step By Step example: Migrating Workload and Data from AWS to Rackspace

To make things simpler we piked Amazon and Rackspace as the two target sites. We have tried the same example with a combination of HP Cloud Services and a flavor of a Private cloud.

The example demonstrates a simple web application with global load-balancer (Rackspace), and Web Application (Petclinic) based on Tomcat as the Web front end and MySQL as the database.

On both ends we used GigaSpaces XAP Transactional WAN replication as a replication channel between the two instances of MySQL and Cloudify to handle the workload migration between the sites.

The Goal: Cloud Bursting and Fail-Over with no change to the target application

The goal that we set to ourself is that all that would be done seamlessly without any change to the target application or database. We achieved that by plugging the replication service to the existing instances of MySQL. The replicating service listen to the MySQL events and replicate every change to its peer MySQL instance - Cloudify enabled us to clone the same application in in both Amazon and Racksapce while maintaining consistent configuration setup as well as consistent scaling and fail-over SLA's. Cloudify does that by abstracting all this inforamtion through portable recipe definition. Cloudify wraps the application instances with its management and control services based on the defintion that is provided in the recipie. This enabled us to clone the environment as well as add elastic scaling without changing the target application (Petclinic in this specific case).

You can read the full details including code references on Github on Dotan Horvitz blog post Bursting into the Clouds – Experimenting with Cloud Bursting