« Running Hadoop MapReduce on Amazon EC2 and Amazon S3 | Main | Product: Mashery »

Scaling IMAP and POP3

Just thought I'd drop a brief suggestion to anyone building a large mail system. Our solution for scaling mail pickup was to develop a sharded architecture whereby accounts are spread across a cluster of servers, each with imap/pop3 capability. Then we use a cluster of reverse proxies (Perdition) speaking to the backend imap/pop3 servers . The benefit of this approach is you can use simply use round-robin or HA loadbalancing on the perdition servers that end users connect to (e.g. admins can easily move accounts around on the backend storage servers without affecting end users). Perdition manages routing users to the appropriate backend servers and has MySQL support. What we also liked about this approach was that it had no dependency on a distributed or networked filesystem, so less chance of corruption or data consistency issues. When an individual server reaches capacity, we just off load users to a less used server. If any server goes offline, it only affects the fraction of users assigned to that server.


Erik Osterman

Reader Comments (4)

As a load balancer Perdition sounds nice, however, how do you handle availability issues? E.g. if one of your "storage" mail servers explodes. Perhaps users will complain if they've lost their mailboxes.

November 29, 1990 | Unregistered CommenterSerge!

I have scaled POP3/IMAP by using IPVS and a cluster of servers running a POP3/IMAP server.
User accounts are on a LDAP server and storage is on a NFS.
IPVS takes care about load balancing (all servers receive same number of connections) and is fault-tolerant.

Its easy to scale because servers are Xen paravirtualized machines, and setting up a server are as easy as deploy a new disk image.

But perdition is a good product with useful features too :)

November 29, 1990 | Unregistered CommenterPedro

I am trying use Nginx as a proxy POP3 server.
My goal is use it as follows
Pop3/web client <-> nginx_proxy_pop3_server <-> Actual POP3 server.

Can somebody please provide a example proxy POP3 configuration for Nginx.

I really appreciate any help in this regard.

Thanks and regards


November 29, 1990 | Unregistered CommenterU K Achary

PostPost a New Comment

Enter your information below to add a new comment.
Author Email (optional):
Author URL (optional):
Some HTML allowed: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <code> <em> <i> <strike> <strong>