High Scalability - Comments

Al comments on Big List of 20 Common Bottlenecks

Al — Fri, 01 Jun 2012 19:13:25 +0000

Good article...

Paul comments on Pinterest Architecture Update - 18 Million Visitors, 10x Growth,12 Employees, 410 TB of Data

Paul — Fri, 01 Jun 2012 09:45:05 +0000

90% of that 410TB likely infringes copyrights. They have absolutely no right to copy fullsize photos to their servers.

Josiah C. comments on Strategy: Get Servers for Free and Make Users Happy by Turning on Compression

Josiah C. — Thu, 31 May 2012 01:57:02 +0000

Compress/decompress, not encrypt/decrypt.

Gyi Tsakalakis comments on The Anatomy of Search Technology: Crawling using Combinators

Gyi Tsakalakis — Wed, 30 May 2012 22:11:32 +0000

The link to the chapter on crawling is broken. Here's the correct link: http://nlp.stanford.edu/IR-book/html/htmledition/web-crawling-and-indexes-1.html (look like a hyphen might be in the wrong place).

Justin Rovang comments on Strategy: Get Servers for Free and Make Users Happy by Turning on Compression

Justin Rovang — Wed, 30 May 2012 21:34:42 +0000

Hey, there's nothing in there about getting servers for free =[

Mani comments on Strategy: Get Servers for Free and Make Users Happy by Turning on Compression

Mani — Wed, 30 May 2012 21:22:43 +0000

More data or index keys fit in RAM?

PBR comments on Scaling an AWS infrastructure - Tools and Patterns

PBR — Tue, 29 May 2012 16:16:09 +0000

I disagree on the choice of not using NOOP at Amazon or any VM instance.

Regardless if the "guest OS" IO requests are properly queued or not, the underlying "host" will re-arrange the queue based on the requests coming from all "guests OS". Putting NOOP at the VM level reduces CPU cycles and leaves the underlying OS to set the queue the best way.

Ref : http://www.monperrus.net/martin/IO+scheduling+for+san+and+virtualization

Aaron Binns comments on The Anatomy of Search Technology: Crawling using Combinators

Aaron Binns — Tue, 29 May 2012 15:49:49 +0000

There's some brokenness around the links for Heritrix and the Common Crawl project.

Nati Shalom comments on Build your own twitter like real time analytics - a step by step guide

Nati Shalom — Mon, 28 May 2012 04:46:21 +0000

@dev fixed, Thanks for spotting this!
@tobi - deployment for public cloud means that you could provision the entire stack on Amazon, Openstack (HP, Rackspace), Azure using cloudify and get the entire environment seup through a single install command. By environment i'm also referring to management and monitoring and post deployment aspects such as fail-over and scaling.

You can see how this setup works in this specific example here

For more information on cloudify and how it works with different cloud environments i'd suggest that you'll look into cloudifysource.org - you can see some of the live-demos

krogon comments on DynamoDB Talk Notes and the SSD Hot S3 Cold Pattern

krogon — Mon, 28 May 2012 02:35:15 +0000

Changing limits is very weird. Limitations:
- You can only update one table at the same time.
- You can only increase throughput by 10% - 100% at one time.
- You can decrease throughput once a day for each table.
It is very hard to auto-scale parameters depending on the traffic you've got.