Stuff The Internet Says On Scalability For May 13, 2011
Submitted for your reading pleasure on this beautiful blue sky, birds chirping Friday morning:
- Unlocked Achievements: Twitter: 900,000 Apps, 600,000 Developers And 13 Billion API Requests Per Day; Rackspace Now Has 70,000 Servers
- Quotable Quotes for the speed of light Alex:
- @LusciousPear According to everyone I talk to, in aggregate, every single #nosql company is irrelevant, doing great, making tons of money, and broke.
- XIX Give a man a GO compiler and he eats for a day. Give a man a C compiler and he feeds upon php/python/jscript/lua/etc for the rest of his life.
- @movingsideways This tweet features a robust architecture and high scalability.
- @serenestudios Human touch trumps code 'scalability'? People matter more than code - we agree.
- @SunGardFS At current growth rates, server farms will consume more energy than all air travel by 2020
- @marcloney After the hype of NoSQL, are we just going to end up with a bunch of enterprise-class products with no database support?
- Choosing NoSQL For The Right Reason. Markus Winand says Performance is—almost always—the wrong reason for NoSQL and that Git is so cool it also is NoSQL. Hacker News thread.
- Amazon is giving a Architecture Webinar Series on best practices to build highly scalable and reliable applications in the AWS Cloud. Looks good so far.
- Lots happening in Google App Engine world. With the new 1.5 release GAE is all grown up, it's a first class product now. New features include Backends, which are essentially VPSs; improved task queue features; support for the Go language in addition to Python and Java; Java is now multi-threaded. What being all grown up really means though is there's no more living for free at home and that means big changes in the GAE pricing model. Prices are way up depending on how you look at it. GAE is now charging by instance hour instead of by CPU hour. On Java this is mitigated somewhat by applications being allowed to be multi-threaded, but Python is still single-threaded. We'll see how this shakes out, but this definitely seems to level the playing field. Also: Google I/O 2011: Scaling App Engine Applications.
- OpenFlow specification emerges: Rise of the software-defined network. Greg Ferro explains software-defined networking and its benefits. For programmers it means more abstraction and being able to say what hardware? Also: OpenFlow – Why It Can Cross the Adoption Gap
- Using Scalr to avoid future Amazon problems: Surviving Region outages. Part of a Scalr series on how their service provides safe harbor in the sometimes stormy Amazon seas. Good introduction to the problems you need to take care of regardless of the solution.
- Ec2 Stress Results thread on the NoSQL Databases group. Experiences testing different EC2 configurations.
- Mike Kavis with Cloud computing lessons learned and your data center. Ask What problem are you trying to solve? Shift your thought process away from “to cloud or not to cloud” and think about “how can I provide the same competitive advantages that companies like Amazon are providing to their customers?
- On Quora: How does Heroku work? Nice pull of information on Heroku by Tom Robinson.
- Mixpanel reveals their Sharding techniques . They process billions of API transactions each month with 100 req/s spikes by horizontally distributing data through sharding. Please, no sharding by the first letter in the first name.
- The Elephant, The Blind Men And Fusion IO by Howard Marks. Fusion IO is the ultimate in scale-up for storage technology. Is it a good idea?
- Basho says It's Time to Drop the "F" Bomb - or "Lies, Damn Lies, and NoSQL." Another volley in the CAP wars. Write loads spread across multiple data centers is not easy. What our technology generation is attempting is really hard. There is no easy button. You can’t play fast and loose with the laws of physics or hand-wave around critical durability issues. You can sell this stuff to your venture capitalist, but we’re not buying it. Hacker News thread.
- Developers have more options and S3 has some more competition with Google Storage. Early reports have GS a more sophisticated product, slightly more expensive at low volumes, more expensive at high volumes. Hacker News thread.
- Fast processors without fast networks are like being lactose intolerant in a cheese shop. Sebastian Anthony writes about how that could change: Graphene-Powered Optical Networks Could Lead to Petabit and Exabit Transmission Speeds. One-atom-thick layers of graphene can switch light on and off incredibly quickly. Thank you nanotechnology.
- How Garbage Collection differs in the three big JVMs. Garbage collection pauses are one of the most common problems in JVM deployments. Michael Kopp explains how all JVMs are not the same and along the way provides a great explanation of GC in general.
- Max Schireson challenges the relational experts in A data modeling challenge to the relational-is-always-best-set to say why the relational model is better for a tracking orders for customer service use case. Good discussion, not surprisingly, minds unchanged. Hacker News thread.
- James Gosling Extols the Virtues of Hash Tables and RAM. There is a time and place for this technology versus normal SQL. But there is no hammer for every nail. It's all a matter of picking the right technology for the problem.
- Making our MongoDB Code Run Faster. Karl squeezes out more performance by: Reducing Index Memory; Rename fields; Removing an Index. Also: Practical NoSQL - Solving a Real Problem with MongoDB and Redis. Hacker News thread.
- Kenneth Falck with a good slide deck on Scaling Django Apps with AWS.
- Video: Cabling A Rack at SoftLayer. That's a lot of cabling. Definitely an art to the whole thing.
- How do I generate a histogram? NoSQL Databases on Google Groups. The fun of new technology is figuring out how to do all the old stuff all over again.
- Red Hat Announces NoSQL Inspired Distributed Data Cache. A cloud-ready, highly scalable distributed data cache.
- Leveldb. LevelDB is a library that implements a fast persistent key-value store.