Large scale email server clusters
hey i'm just feeling quite bored today to consider the implementation of such a system to compete with Yxxx! Mail and Hxxxxx, i guess i won't be able to successfully construct a WORKING & highly avialable system like this but still want to hear some of your valueable input
here we GO!!
============================
A) The concepts:
the whole system will be broken down into 2 main clusters for the ease of eventually seperating them as 2 standalone services(but still dependent on each other) for commercial purposes
i) the FIRST cluster (I name it "cluster MX") being focusing on server-to-server email delievery(incoming) as well as spam/virus scanning, custom rules filtering, it also provides backup mx/redundancy for the SECOND cluster
ii) the SECOND cluster (I name it "cluster HOST")will be for end-users/reseller use, like everyone.net, it accepts already-processed incoming emails from the FIRST cluster conditionally/unconditionally and provide storage & retrieval for them. it also processes outgoing emails
==============================
B) Dividing cluster MX into layers
i) multiple identical clusters will be spread across physically (i.e US cluster & EU cluster) for high availability & fallover of an entire datacenter (very rare), each node in a cluster is connected via vLAN (or preferably theyre in the same rack). nodes can be further divided into 5 "layers", the weight of how many nodes in each layer is by the ratio:
2 : 1 : 1 : 2 : 1
and the numbers are highly scalable:
4 : 1 : 2 : 4 : 2
the above can handle double load technically. Different layers work on differnet ports by default, so magically 5 layers can be put in a single vps server forming a single-node cluster if ppl with limited budget wants to try to have fun with it
ii) 1st layer: front-end servers - servers which accept connections to port 25, with addressese where the MX records point to
iii) 2nd layer: "hints" servers - servers which hold hints to which node in 3rd layer to ask for info if requested by nodes in other layers
iv) 3rd layer: user-data servers - each node in this layer holds data of completely different users, such data include email addresses, custom filtering rules, destined email server the emails will finally be delievered to..etc. Data of users prefix A-C will be hold in (3rd layer) node A, prefix with D-G in node B...... /[Y-Z0-9]/ in node T ..... imagine there are 1,000,000 users with 10,000,000 email adressess and 100,000,000 custom rules registered, not a database resulotion can accomplish such write-heavy in just one single database, and no affordable and reliable mean of repulication associated with such databasse vendors i can think of
v) 4th layer: filtering servers - guess you already know what it is for by name?
vi) 5th layer: outgoing / tmp backup servers - complete the function of cluster MX by either forward emails to destined email servers or tmp storage as long as a destined email server is down
=================================
C) Flow in cluster MX
1st layer verifies external inputs by pulling "hints" from the 2nd layer nodes, using those "hints" to retrieve custom rules set by users from the 3rd layer. After verifications (here i suppose it passes), dump the msg incl. attactchment (if any) to the 4th layer to do hardcore spam/virus scanning (here i also suppose its clean), finally dump the whole thing (along with flags indicating additional info) to the 5th nodes to do either forwarding to the destined email servers or tmp storage and re-delievery later if it is failed at the time
===================================
E) Failover / High availability inside a cluster
there's already failover mechanisium inside a single cluster - if there's another node in the same layer a failing node reside. i.e. in a ratio like this
4 : 1 : 2 : 4 : 2
if a node in layer 3 fails the remaining one will take over
=====================================
F) Failover / High availability across clusters
as said in the start of this post, multiple identical clusters are formed, so if the only node in layer 2 in cluster ONE fails, all traffic originally within the cluster to that node will be redirected to the same node in cluster TWO at a cost of wasting some bandwidth and causing lags in the flow. you can totally avoid this from happening or at the least reduce the loss in such disaster by increasing nodes in a cluster
=================================
G) Extra servers the cluster MX
servers mentioned here don't interfere directly with email flow, but rather are add-ons, monitering or ensuring the health and efficiency of those layers in the clusters, they do not neccesarily reside physically aside with any cluster
i) master/control server - doing all the dirty and frequent monitoring of the layers
ii) XML/web services server - its to support the reseller functionality by providing an API, it also has the right to modify settings in clusters
iii) Wev server - its to host the user/reseller CP and communicate with the XML server to do admin work
================================
H) Who needs cluster MX?
obviously cluster MX is a so-called backup/second MX with little more functions for very large enterprises and resellers (obviously one reseller being cluster HOST ^0^), it empowers the efficiency of cluster HOST which will be discussed later...........
.
.
.
.
Sorry for my very poor english and not going very in-depth (i really wish to but limited by my fluency of written eng) about programming the system, if you really read everything up to this point you may find that it can even be done with PHP ( surely i'll avoid at all cost)
feel free to blame or help correect some misunderstandings underestimates of this whole project, i really want to hear inputs from other programmers
i'll continue writting my cluster HOST doc if anyone's interested
