advertise
« Stuff The Internet Says On Scalability For September 21, 2012 | Main | The 4 Building Blocks of Architecting Systems for Scale »
Thursday
Sep202012

How Vimeo Saves 50% on EC2 by Playing a Smarter Game

Nothing shows how much software architectures have changed than the intelligent scheduling of computation over differently priced compute resources. This isn't just a false economy either. Vimeo saves up to 50% on their video transcoding bill by intelligently playing the spot, reserved, and on-demand markets. If you are ready for some advanced reindeer games then take a look at  Vimeo EC2 transcoding where they explain their thinking. Even if you don't like their rules, it's the strategy that matters. This presentation was from 2011, so it would be interesting to see if the new reserved instance market has made a difference in their strategy. 

Here's Vimeo's approach for minimizing costs using spot, reserved, and on-demand instances:

  • Never bid more than threshold. It is currently set to 80% of on-demand price.
  • Not more than 10 open spot requests at any time.
  • Bid 10% more than the average price over last hour
  • Buy reserve instance capacity to meet non-peak hour loads.
  • Use spots for low priority and less time critical jobs
  • Never kill spots. Let Amazon do it.
  • Have more retries for jobs running on spots.
  • Watch out for open spot requests. Add expiry to your requests.
  • Long running jobs, bid higher or use on-demand
  • Failover to on-demand when spot marked is saturated

Related Articles

Reader Comments (1)

The talk is @ 01:19:00
http://psav.mediasite.com/Mediasite/Play/dc1c4baff4434151b64df74a2f971b721d

September 22, 2012 | Unregistered Commenteraws

PostPost a New Comment

Enter your information below to add a new comment.
Author Email (optional):
Author URL (optional):
Post:
 
Some HTML allowed: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <code> <em> <i> <strike> <strong>