Example

Google+ is Built Using Tools You Can Use Too: Closure, Java Servlets, JavaScript, BigTable, Colossus, Quick Turnaround

High Scalability

High Scalability

— 4 min read

Joseph Smarr, former CTO of Plaxo (which explains why I recognized his picture), in I'm a technical lead on the Google+ team. Ask me anything, reveals the stack used for building Google+:

Our stack is pretty standard fare for Google apps these days: we use Java servlets for our server code and JavaScript for the browser-side of the UI, largely built with the (open-source) Closure framework, including Closure's JavaScript compiler and template system. A couple nifty tricks we do: we use the HTML5 History API to maintain pretty-looking URLs even though it's an AJAX app (falling back on hash-fragments for older browsers); and we often render our Closure templates server-side so the page renders before any JavaScript is loaded, then the JavaScript finds the right DOM nodes and hooks up event handlers, etc. to make it responsive (as a result, if you're on a slow connection and you click on stuff really fast, you may notice a lag before it does anything, but luckily most people don't run into this in practice). Our backends are built mostly on top of BigTable and Colossus/GFS, and we use a lot of other common Google technologies such as MapReduce (again, like many other Google apps do).

At first I read Clojure, which would have been a real surprise, but it's Closure, a suite of JavaScript tools consisting of a library, compiler, and templates. The compiler is a true compiler for JavaScript for making JavaScript download and run faster.  The library is modular and cross-browser JavaScript library. Templates is a server-side templating system that helps you dynamically build reusable HTML and UI elements. It's all open source so you can use it too.

While you don't have Google's stack available to you, you do have some open source options. HBase is replacement for BigTable. Then there's Hadoop MapReduce. Colossus is Google's next generation file system, a replacement for GFS. Since we don't know much about Colossus, it's hard to say what a suitable replacement would be. There's the Hadoop distributed file system HDFS. And if you are looking for some of the cloud like infrastructure glue there's OpenStack (which also has storage system).

Google probably uses a custom Java servlet container, but the choice here doesn't matter that much. Most of the work will be spawned in parallel and performed on other servers implemented in C++, Java, or Python.

Whereas most communication with Google is non-existent, the Google+ development team is noticeably more responsive, turning out visible improvements quickly and consistently. Joseph tells us why: We put extra emphasis on engineering speed/agility--we try to release code updates on a daily basis while still keeping quality/stability/latency as high as you'd expect from google. This helps us move fast and respond quickly to user feedback. We try to do a full push (almost) every day, and we sometimes sneak in patch releases too if needed. But there are humans in the loop, it's not a "auto-push if all tests pass" situation or anything like that.

For Google+'s most innovative feature, video conferencing with Hangouts, GigaOM has a good article on that stack, which is based on Announcing Google+ Hangouts, written by Tech Lead Justin Uberti. Unlike Skype, which runs on a cost effective P2P model, Hangouts is completely hosted by Google. This must cost a staggering amount of money. You are on your own here. Nobody can replace the bandwidth being donated by the Google fairy.

That's Google in a box. Then again, an ex-Googler thinks you can do better using MessagePack, JSON, Hadoop, jQuery, and MongoDB. If you can do better for a world wide base of a billion users is a completely different question.

  • Closure Library / Closure Templates / Closure Compiler
  • VMware exposes its plans to be the OS for the cloud - another infrastructure option.
  • Colossus of Rhodes - wonder of the world.
  • Colossus Computer - electronic computing device used by British codebreakers to help read encrypted German messages during World War II. They used vacuum tubes (thermionic valves) to perform the calculations.
  • Colossus: The Forbin Project - SF film about a massive American defense computer, named Colossus, becoming sentient and deciding to assume control of the world.
  • This post On Hacker News
  • This post On Reddit
  • OpenGSE Released - Something that may be of interest to Open Source enthusiasts is the recent release of the Open Google Servlet Engine (OpenGSE). Once upon a time engineers within Google developed their own fast, lightweight servlet engine which we inside Google refer to as simply the "Google Servlet Engine" or GSE.
  • On Quora: Is Google's software infrastructure obsolete? TLDR: Nope. Hector Yee has a good summary: The Google infrastructure is far more stable and well documented than the open source variants. They work together as a coherent whole. It's much easier to have a stable, massively scalable system built in the Google framework than with the equivalent open source tools. On the other hand, it's easy to chafe at the restrictions imposed by a big company with a defined way of doing things. Both view points can be true.
  • nostrademons on JQuery vs Closure: Closure is used in basically every Google product. There's no comparison - in terms of pageviews, edge cases, and general real-world exposure, Closure is more widely used than probably every other JS library. I actually think they fill very different niches. JQuery is for sites that are primarily HTML but require some additional interactivity layered on via progressive enhancement. Closure is meant for JavaScript apps, where the whole client is basically written in JavaScript.
  • RyanDScott on why GWT isn't used internally at Google:  Closure is just JavaScript. No waiting for the GWT team to build the features you need, no having to build those features yourself. And in my experience, when I need cutting edge with GWT, I end up writing a bunch of wrappers for JavaScript functions, which makes me wonder why I'm not writing everything in JavaScript. (Like I said, the GWT library is very compelling and when planning a large project, Java often feels more comfortable for some.)

Read more