Example

TypePad Architecture

Todd Hoff

Aug 20, 2007 — 1 min read

TypePad is considered the largest paid blogging service in the world. After experience problems because of their meteoric growth, they eventually transitioned to an architecture patterned after their sister company, LiveJournal.

Site: http://www.typepad.com/

The Platform

MySQL

Memcached

Perl

MogileFS

Apache

Linux

The Stats

As of 2005 TypePad sends 250mbps of traffic using multiple network pipes for 3TB of traffic a day. They were growing by 10-20% each month. I was unable to find more recent statistics.

The Architecture

Original Architecture:
- Single server running Linux, Apache, Postgres, Perl, mod_perl
- Storage was NFS on a filer.

A Devastating Crash Caused a New Direction
- A RAID controller failed and spewed data across all RAID disks.
- The database was corrupted and the backups were corrupted.
- Their redundant filers suffered from "split brain" syndrome.

They move to LiveJournal Architecture type architecture which isn't surprising since TypePad and LiveJounral are both owned by Six Apart.
- Replicated MySQL clusters partitioned by ID.
- A global DB generated globally unique sequence numbers and mapped users to partitions.
- Other data was mapped by role.

Highly Available Database Configuration:
- A master-master MySQL replication model is used.
- The Linux clustering heartbeat was used to failover using virtual IP addresses.

MogileFS is used to serve images.

Perlbal is used as reverse proxy and to load balance requests.

A reliable, asynchronous job dispatch system called TheSchwartz is used to support moblogging, adding comments, future publishing, cache invalidation, and publishing.

Memcached is used to store counts, sets, stats, and heavyweight data.

Migration from the old architecture to the new architecture was tricky:
- All users were migrated over without service interruption.
- Postgres was removed.
- During the migration images were served from NFS and MogileFS.

Benefits of their new architecture:
- Can easily add new machines and adjust workload.
- More highly available and is cheaply scalable

Lessons Learned

Small details are important.

Every mistake is a learning experience.

Success requires coordination and cooperation.

LiveJournal Architecture.

Linux High Availability.

Capturing A Billion Emo(j)i-ons

This blog post was written by Dedeepya Bonthu. This is a repost from her Medium article, approved by the author. In stadiums, sports fans love to express themselves by cheering for their favorite teams, holding up placards and team logos. Emoji’s allow fans at home to rapidly express themselves,

Brief History of Scaling Uber

This blog post was written by Josh Clemm, Senior Director of Engineering at Uber Eats. This is a repost from his LinkedIn article, approved by the author. On a cold evening in Paris in 2008, Travis Kalanick and Garrett Camp couldn't get a cab. That's when

Behind AWS S3’s Massive Scale

This is a guest article by Stanislav Kozlovski, an Apache Kafka Committer. If you would like to connect with Stanislav, you can do so on Twitter and LinkedIn. AWS S3 is a service every engineer is familiar with. It’s the service that popularized the notion of cold-storage to the

The Swedbank Outage shows that Change Controls don't work

This week I’ve been reading through the recent judgment from the Swedish FSA on the Swedbank outage. If you’re unfamiliar with this story, Swedbank had a major outage in April 2022 that was caused by an unapproved change to their IT systems. It temporarily left nearly a million