Alexey Radul in his fascinating 174 page dissertation Propagation Networks: A Flexible and Expressive Substrate for Computation, offers to help us break free of the tyranny of linear time by arranging computation as a network of autonomous but interconnected machines. We can do this by organizing computation as a network of interconnected machines of some kind, each of which is free to run when it pleases, propagating  information around the network as proves possible. The consequence of this freedom is that the structure of the aggregate does not impose an order of time. The abstract from his thesis is:
- Velocity Web Performance and Operations Conference - Fast by Default
- Social Developer Summit (HighScalability.com readers save 15% with discount code SDSHS)
Twitter has a big hairy audacious goal of reaching one billion users by 2013. Three forces stand against Twitter. The world will end in 2012. But let's be optimistic and assume we'll make it. Next is Facebook. Currently Facebook is the user leader with over 400 million users. Will Facebook stumble or will they rocket to one billion users before Twitter? And lastly, there's Twitter's "low" starting point and "slow" growth rate. Twitter currently has 106 million registered users and adds about 300,000 new users a day. That doesn't add up to a billion in three years. Twitter needs to triple the number of registered users they add per day. How will Twitter reach its goal of over one billion users served?
Isn't the secret to fast, scalable websites to cache everything? Caching, if not the secret sauce of many a website, is it at least a popular condiment. But not so fast says Peter Zaitsev in Beyond great cache hit ratio. The point Peter makes is that we read about websites like Amazon and Facebook that can literally make hundreds of calls to satisfy a user request. Even if you have an awesome cache hit ratio, pages can still be slow because making and processing all those requests takes time. The solution is to remove requests all together. You do this by caching larger blocks so you have to make fewer requests.
The post has a lot of good advice worth reading: 1) Make non cacheable blocks as small as possible, 2) Maximize amount of uses of the cache item, 3) Control invalidation, 4) Multi-Get.
Google made a right move by adding web-speed to the search engine ranking. With this change site latency does't just impact the user experience but it will determine where you will be placed in google search results.
This move could be a real game changer as it bring site latency and scalability to the front stage. Sites will not only compete on content but on thier performance. It is now clear that site performance have much more direct impact on our business (as appose to indirect impact resulted in user expereince) then ever before.
In this post i try to provide some architecture guide line on how to control and improve site latency under scale based on a discussion with a leading eCommerce site the Netherland.
See detailed story here
Get Your High Scalability Fix at Digg
Interested in working on cutting-edge high-scale infrastructure at Digg? We're making a big investment in scaling and have committed to the NoSQL (Not only SQL) path with Cassandra. We're using other open-source infrastructure to help us scale including Hadoop, RabbitMQ, Zookeeper, Thrift, HDFS and Lucene. We're rewriting Digg from the ground up and we need amazing developers to join our world-class team. If you think you are up for the challenge, or you know someone who might be, take a look at our jobs page for more information.
In my last post, I outlined considerations that need to be taken into account when choosing between a centralized and federated security model. So, how do we implement the chosen model? Based on a real-world case study, I will outline a Kerberos architecture that enables cutting-edge collaborative research through federated sharing of resources.
Read more on BigDataMatters.com
Cloud computing promises a number of advantages for the deployment of data-intensive applications. Most prominently, these include reducing cost with a pay-as-you-go business model and (virtually) unlimited throughput by adding servers if the workload increases. At the Systems Group, ETH Zurich, we did an extensive end-to-end performance study to compare the major cloud offerings regarding their ability to fulfill these promises and their implied cost.
The idea came up in this Hacker News thread, commenting on a 37signals interview, that having three system administrators is the minimum optimal number of admins. Everyone wants to lower their costs by having each admin administer a lot of machines. The problem is when you have fewer than three admins you can never get a break from the constant corrosive pressure of always being on call. When every moment of your life you are dreading the next emergency, it eats at you. Having three admins solves that problem. With three admins you can:
- Go on a real vacation. The two remaining admins can switch off being on call.
- Not be on call all the time.
A larger shop will naturally have more admins so it's not as big an issue, but at smaller shops trying to minimize head count, carrying three admins (or people in those roles) might be something to consider.