Performance and Scalability

SurgeMail has been designed from scratch with performance in mind. This means you can almost literally host "an unlimited number of users and virtual domains on a single system". For very large systems though you may want to use various of the proxy options below, first lets talk about hardware an OS.

Technical info on clustering

Hardware suggestions

Number of active users	Rough hardware outline
1-1000	Anything will work fine. We recommend you use a mirrored drive for the mail spool though.
1000-5000	Dual core Pentium SATA or SSD disk 2gig ram Install a SurgeMail/mirror system for failover.
5000-100000	4-8 processor core, I7 or Phenom II 8gig RAM Internal RAID 10 disk array (or SSD disk array) Seperate disk drive for operating system. Move 'SurgeWeb' to a seperate system. Install a SurgeMail/mirror system for failover.

Operating system options

Here we recommend you choose the operating system you are most familiar with from the choices of Linux or Windows , there is no significant performance difference between the two as far as SurgeMail is concerned so the key factor is which system you are best at doing the major common tasks on, e.g. OS install, configure, partition disks etc.

Scaling a system up from 100,000 to 100,000,000 users

However we also had in mind ridiculously large systems with 1-100 million users, in these situations there are two methods you can use.

Proxy Servers (huge systems)
NFS / Shared Networked drives (huge systems)
Simple splitting up of function (medium sized systems)

Proxy Servers (huge systems)

This systems allows both infinite scaling, and 3 layer security. The incoming POP/SMTP connections arrive at one of several front end 'proxy' servers (running SurgeMail in proxy mode) these servers then lookup the user in the networked user database (via LDAP or our own TCPAuth module) and along with the normal response an extra response code of 'tohost=backend.host.name' is returned, the proxy then redirects the user to the appropriate back end system. Note: This mechanism doesn't currently support some web based user features (friends,exception rules etc)

So you might run 4 back end systems, each with 100,000 users, and 2 front end systems. To add more users you just add as many front end and back end servers as needed to cope with the load.

Each user is only on one of the back end systems, the only piece in the system that has to handle all the users is the user database, which is a relatively trivial task as the quantity of data per entry is so small. We recommend the use of NWAuth or LDAPAuth but any of the database back end authent modules would be suitable.

See here for technical details

Note: 3 Layer Security: This model is called '3 layer security' as the front and back end systems can be separated by another fire wall. And in the case of 'WebMail' the user web interface can also be separated from the front end systems by a fire wall, hence '3 layer' :-)

To implement this system set on the proxy system the setting g_proxy true, and in the authent module add the 'tohost=xxx' field. For existing user accounts you can define g_proxy_default host.name so that user records with no 'tohost' entry are correctly sent to the existing back end system. In this way a non proxy based system can be instantly turned into a proxy based system.

Proxy config info

NFS / Shared Drives, Clustering Support

In this mode you simply define your main drop path for a domain as a networked drive and setup multiple systems using that same drop path. As SurgeMail uses a 'maildir' directory format there are almost no locking issues even with bad NFS implementations (and most NFS implementations are a little dodgy :-).