Example

TypePad Architecture

Todd Hoff

— 1 min read

TypePad is considered the largest paid blogging service in the world. After experience problems because of their meteoric growth, they eventually transitioned to an architecture patterned after their sister company, LiveJournal.

Site: http://www.typepad.com/

The Platform

  • MySQL
  • Memcached
  • Perl
  • MogileFS
  • Apache
  • Linux

    The Stats

  • As of 2005 TypePad sends 250mbps of traffic using multiple network pipes for 3TB of traffic a day. They were growing by 10-20% each month. I was unable to find more recent statistics.

    The Architecture

  • Original Architecture:
    - Single server running Linux, Apache, Postgres, Perl, mod_perl
    - Storage was NFS on a filer.
  • A Devastating Crash Caused a New Direction
    - A RAID controller failed and spewed data across all RAID disks.
    - The database was corrupted and the backups were corrupted.
    - Their redundant filers suffered from "split brain" syndrome.
  • They move to LiveJournal Architecture type architecture which isn't surprising since TypePad and LiveJounral are both owned by Six Apart.
    - Replicated MySQL clusters partitioned by ID.
    - A global DB generated globally unique sequence numbers and mapped users to partitions.
    - Other data was mapped by role.
  • Highly Available Database Configuration:
    - A master-master MySQL replication model is used.
    - The Linux clustering heartbeat was used to failover using virtual IP addresses.
  • MogileFS is used to serve images.
  • Perlbal is used as reverse proxy and to load balance requests.
  • A reliable, asynchronous job dispatch system called TheSchwartz is used to support moblogging, adding comments, future publishing, cache invalidation, and publishing.
  • Memcached is used to store counts, sets, stats, and heavyweight data.
  • Migration from the old architecture to the new architecture was tricky:
    - All users were migrated over without service interruption.
    - Postgres was removed.
    - During the migration images were served from NFS and MogileFS.
  • Benefits of their new architecture:
    - Can easily add new machines and adjust workload.
    - More highly available and is cheaply scalable

    Lessons Learned

  • Small details are important.
  • Every mistake is a learning experience.
  • Success requires coordination and cooperation.

    Related Articles

  • LiveJournal Architecture.
  • Linux High Availability.

    • Read more