MySpace Architecture

Thu, 07/12/2007 - 06:12 — Todd Hoff

MySpace Architecture (2121)

MySpace.com is one of the fastest growing site on the Internet with 65 million subscribers and 260,000 new users registering each day. Often criticized for poor performance, MySpace has had to tackle scalability issues few other sites have faced. How did they do it?

Site: http://myspace.com

Information Sources

Inside MySpace.com

Platform

ASP.NET 2.0

Windows

IIS

SQL Server

What's Inside?

MySpace processes 1.5 Billion page views per day and handles 2.3 million concurrent users during the day

Membership Milestones:
- 500,000 Users: A Simple Architecture Stumbles
- 1 Million Users:Vertical Partitioning Solves Scalability Woes
- 3 Million Users: Scale-Out Wins Over Scale-Up
- 9 Million Users: Site Migrates to ASP.NET, Adds Virtual Storage
- 26 Million Users: MySpace Embraces 64-Bit Technology

500,000 accounts was too much load for two web servers and a single database.

At 1-2 Million Accounts
- They used a database architecture built around the concept of vertical partitioning, with separate databases for parts of the website that served different functions such as the log-in screen, user profiles and blogs.
- The vertical partitioning scheme helped divide up the workload for database reads and writes alike, and when users demanded a new feature, MySpace would put a new database online to support it.
- MySpace switched from using storage devices directly attached to its database servers to a storage area network (SAN), in which a pool of disk storage devices are tied together by a high-speed, specialized network, and the databases connect to the SAN. The change to a SAN boosted performance, uptime and reliability.

At 3 Million Accounts
- the vertical partitioning solution didn't last because they replicated some horizontal information like user accounts across all vertical slices. With so many replications one would fail and slow down the system.
- individual applications like blogs on sub-sections of the Web site would grow too large for a single database server
- Reorganized all the core data to be logically organized into one database
- split its user base into chunks of 1 million accounts and put all the data keyed to those accounts in a separate instance of SQL Server

9 Million–17 Million Accounts
- Moved to ASP.NET which used less resources than their previous architecture. 150 servers running the new code were able to do the same work that had previously required 246.
- Saw storage bottlenecks again. Implementing a SAN had solved some early performance problems, but now the Web site's demands were starting to periodically overwhelm the SAN's I/O capacity—the speed with which it could read and write data to and from disk storage.
- Hit limits with the 1 million-accounts-per-database division approach as these limits were exceeded.
- Moved to a virtualized storage architecture where the entire SAN is treated as one big pool of storage capacity, without requiring that specific disks be dedicated to serving specific applications. MySpace now standardized on equipment from a relatively new SAN vendor, 3PARdata

Added a caching tier—a layer of servers placed between the Web servers and the database servers whose sole job was to capture copies of frequently accessed data objects in memory and serve them to the Web application without the need for a database lookup.

26 Million Accounts
- Moved to 64-bit SQL server to work around their memory bottleneck issues. Their standard database server configuration uses 64 GB of RAM.

Lessons Learned

You can build big websites using Microsoft tech.

A cache should have been used from the beginning.

The cache is a better place to store transitory data that doesn't need to be recorded in a database, such as temporary files created to track a particular user's session on the Web site.

Built in OS features to detect denial of service attacks can cause inexplicable failures.

Distribute your data to geographically diverse data centers to handle power failures.

Consider using virtualized storage/clustered file systems from the start. It allows you to massively parallelize IO access while being able to add disk as needed without any reorganization needed.

Visit MySpace Architecture
19446 reads

Comments

Thu, 07/26/2007 - 05:02 — Pradeep (not verified)

Resourceful web site!

I have never seen such a web site which is so much focussed on web site architecture. I am enlightened by the bountiful information it contains!

Tue, 08/07/2007 - 18:47 — Chandraprakash Loonker (not verified)

Windows vs linux

64bit hardware is the key here. All the bad name windows got for poor performance because of the use of 32 bit hardware. Most linux/unix hardware migrated to 64 bit hardware much earlier on. Additional RAM supported by 64 bit machines make the whole difference in performance of a machine.

Wed, 08/08/2007 - 08:15 — Anonymous (not verified)

"You can build big websites

"You can build big websites using Microsoft tech."

It seems that you still can't, speaking from personal Myspace experience. It doesn't work as it should with errors popping all over the place. Looks like "scalability" through dropping every second or third request, thus reducing load on the system.

Thu, 08/16/2007 - 10:07 — Arthur (not verified)

Cheers,

Tue, 12/11/2007 - 20:37 — cbmeeks

Re: MySpace Architecture

150 servers for 26 million accounts?

That's 173,333 accounts per server.

Is that really a good benchmark? I don't know either way. How many servers does Facebook use?

http://codershangout.com
A place for coders to hangout!

Sun, 05/18/2008 - 10:02 — dizi izle (not verified)

Wed, 06/25/2008 - 00:06 — youtube (not verified)

Fri, 07/25/2008 - 16:18 — مسجات (not verified)

GHJDGJH

THANK YOU

VERY VERY GOOOOOOOOD

VERY NIC

Fri, 07/25/2008 - 16:23 — برامج (not verified)

ykl

The subject of a very wonderful and distinct
I thank you for continuing excellence
Thank you

High Scalability Building bigger, faster, more reliable websites.

Navigation

Who's online

User login

Syndicate