advertise
« Scalability Strategies Primer: Database Sharding | Main | Facebook is Hiring »
Tuesday
Dec162008

[ANN] New Open Source Cache System

The SHOP.COM Cache System is now available at http://code.google.com/p/sccache/

The SHOP.COM Cache System is an object cache system that...
* is an in-process cache and external, shared Cache
* is horizontally scalable
* stores cached objects to disk
* supports associative keys
* is non-transactional
* can have any size key and any size data
* does auto-GC based on TTL
* is container and platform neutral

It was built in-house at SHOP.COM (by me) and has powered our website for years. We are open-sourcing it in the hope that it will be useful to others and to get some help in its maintenance.

This is our first open source attempt and we'd appreciate any help and comments.

Reader Comments (13)

How does this compare with memcached?

November 29, 1990 | Unregistered CommenterTim

Interesting. What programming language is it written in?

November 29, 1990 | Unregistered CommenterOwkaye Go

With the horizontal scalability, is there any replication? Ie if one cache server dies, can the others take over?

November 29, 1990 | Unregistered CommenterBill

Thanks for opening the code! Opensource is the way to go!

November 29, 1990 | Unregistered CommenterAntonio Ognio

This looks pretty good, one question though, can I interact with it using PHP?

From what I am seeing, my application would need to be in Java to interact with it easily. Is there anyway for there to be some kind of front end wrapper so that we don't have to mix languages on our servers? I'd like to continue doing everything in PHP and maybe interact with this system via some kind of PHP to JAVA wrapper over HTTP.

November 29, 1990 | Unregistered CommenterJoel Strellner

I've got a bare-bones comparison table started at http://code.google.com/p/sccache/wiki/Comparison

November 29, 1990 | Unregistered CommenterRandgalt

>Interesting. What programming language is it written in?
100% Java

November 29, 1990 | Unregistered CommenterRandgalt

>With the horizontal scalability, is there any replication?
>Ie if one cache server dies, can the others take over?

You can mark some objects as "important" and they will be written to two cache server instances. So, if one goes down the object will still be available. However, sccache is not transactional and is not meant to replace a database.

November 29, 1990 | Unregistered CommenterRandgalt

>Is there anyway for there to be some kind of front end
>wrapper so that we don't have to mix languages on our servers?
When I get time I can do it. But, maybe some intrepid programmer can add it in :)

November 29, 1990 | Unregistered CommenterRandgalt

One other thing, is the cache timeout configurable? In our environment your solution would work perfectly (except for the java part, but we can get around that), but I'd like for the objects to never expire, except if we overwrite them.

It looks like it is set at 12 hours by default. Can we set it to forever?

November 29, 1990 | Unregistered CommenterJoel Strellner

Would have been nice if they stated that it was Java in the article somewhere. I had to click through to the project to see that it was Java.

November 29, 1990 | Unregistered CommenterJon

>It looks like it is set at 12 hours by default. Can we set it to forever?
You could, but it affects how garbage collection works. The disk storage portion of the system does garbage collection based on maximum TTL. You could set it a huge number, but the cache files might grow very large.

How many objects would you be caching? Another option is to create two cache farms. One for the standard objects and another for the infinite objects. sccache is totally pluggable so you could plug a different disk storage implementation for the infinite objects.

November 29, 1990 | Unregistered CommenterRandgalt
November 29, 1990 | Unregistered CommenterRandgalt

PostPost a New Comment

Enter your information below to add a new comment.
Author Email (optional):
Author URL (optional):
Post:
 
Some HTML allowed: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <code> <em> <i> <strike> <strong>