A Super Short on the Youporn Stack - 300K QPS and 100 Million Page Views Per Day
Eric Pickup from Youporn.com posted on a news group that Youporn is now 100% Redis based and will soon be revealing more about their architecture at the ConFoo conference. Some stunning, but not surprising numbers were revealed:
- 100 million page views per day
- A cluster of Redis slaves are handling over 300k queries per second.
Some additional nuggets:
- Additional Redis nodes were added because the network cards couldn't keep up with Redis.
- Impressed with Redis' performance.
- All reads come from Redis; we are maintaining MySQL just to allow us to build new sorted sets as our requirement change
- Most data is found in hashes with ordered sets used to know what data to show.
- A typical lookup would be an zInterStore on: videos:filters:released, Videos:filters:orientation:straight,Videos:filters:categories:{category_id}, Videos:ordering:rating
- Then perform a zRange to get the pages we want and get the list of video_ids back.
- Then start a pipeline and get all the videos from hashes.
- Do use some key/value lookups and some lists, but the majority of our operations are using the above pattern.
Not much to see yet, but hopefully we'll learn more after their talk.
