Building bigger, faster, more reliable websites.
hot links
Hey, it's HighScalability time: (Still not a transporter: Looping at 685 mph) * 898 exabytes: US storage, 1/3 global total; 1 Kb/s: data transmit rate from harvestable energy from human motion * Create your own trust nobody point-to-point private cloud. Dan Brown shows how step-by-step in How I
software
Someone reading my now ancient C++ coding standard recommendation for using doxygen to automatically generate documentation from source code, asked a great question: I've often considered using doxygen, I always ask myself - is this really useful? Would I use it if I were new to a project?
James Hamilton in Counting Servers is Hard has an awesome breakdown of what one million plus servers really means in terms of resource usage. The summary from his calculations are eye popping: * Facilities: 15 to 30 large datacenters * Capital expense: $4.25 Billion * Total power: 300MW * Power Consumption: 2.6TWh
AskHighScalability
Can anyone convincingly explain why properties sporting traffic statistics that may seem in-line with with the capabilities of a single big-iron machine need so many machines in their architecture? This is a common reaction to architecture profiles on High Scalability: I could do all that on a few machines so
hot links
Hey, it's HighScalability time: (The Shard Scaling 6) * 3500+: number of species found underneath miles of ice; 360 TB/disc data capacity: 5D optical memory in glass; 50% Of Internet Traffic: Comes From Only 35 Sites/Services; 60 billion: potential alien planets that could give Snowden asylum * Quotable
sponsored post
Who's Hiring? * We need awesome people @ Booking.com - We want YOU! Come design next generation interfaces, solve critical scalability problems, and hack on one of the largest Perl codebases. Please apply online. * An exciting opportunity for a Software Engineer to join Apple's Messaging Services team.
Example
Toy solutions solving Twitter’s “problems” are a favorite scalability trope. Everybody has this idea that Twitter is easy. With a little architectural hand waving we have a scalable Twitter, just that simple. Well, it’s not that simple as Raffi Krikorian, VP of Engineering at Twitter, describes in his
hot links
Hey, it's HighScalability time: (Dolls nerds can nest with) * Quotable Quotes: * @Carnage4Life: "Google uses Bayesian filtering the way Microsoft uses the if statement" - http://www.joelonsoftware.com/items/2005/10/17.html … <= finally at the point where I get this * @etherealmind: You can dramatically
PRISM
This is a guest post by BugSense Founder/CTO Jon Vlachogiannis and Head of Infrastructure at BugSense Panagiotis Papadomitsos. There has been a lot of speculation and assumptions around whether PRISM exists and if it is cost effective. I don't know whether it exists or not, but I
hot links
Hey, it's HighScalability time: (Leandro Erlich's super cool scaling illusion) * Who am I? I have 50 petabytes of data stored in Hadoop and Teradata, 400 million items for sale, 250 million queries a day, 100,000 pages served per second, 112 million active users, $75 billions
Paper
Erasure codes are one of those seemingly magical mathematical creations that with the developments described in the paper XORing Elephants: Novel Erasure Codes for Big Data, are set to replace triple replication as the data storage protection mechanism of choice. The result says Robin Harris (StorageMojo) in an excellent article,
Example
This is a guest post by Yelp's Jim Blomo. Jim manages a growing data mining team that uses Hadoop, mrjob, and oddjob to process TBs of data. Before Yelp, he built infrastructure for startups and Amazon. Check out his upcoming talk at OSCON 2013 on Building a Cloud