Exim sender_verify (on by default in cPanel) - good or bad?
1. The default config of exim on cpanel boxes is
sender_verify = true
2. The exim documentaion, AFAIK, is not very explicit in exactly what is checked. When sender_verify fails, the exim_rejectlog logs that "Host Lookup" failed. Assuming that the name servers for the domain of the sender's email address are up, does the Host Lookup search for a valid A or CNAME for the sender's domain or just a valid MX record? It would seem that it is looking for a valid A or CNAME from my experience. Why would it need to find a valid A or CNAME for the sender's domain? Why not just a valid MX record?
This is what the exim specification says:
sender_verify
Type: boolean
Default: false
If this option is true, envelope sender addresses on incoming SMTP messages are checked to ensure that they are valid. Messages with invalid envelope senders are rejected with a permanent error code if sender_verify_reject is set (the default). Otherwise a warning is logged. See section 45.2 for details of the rejection, which can happen at three different points in the SMTP dialogue. If a sender cannot immediately be verified, a temporary error code is returned after reading the data (so the headers can be logged). The sender_try_verify option is less severe: it operates in exactly the same way as sender_verify except that if an address cannot immediately be verified, it is accepted instead of being temporarily rejected.
Type: boolean
Default: false
If this option is true, envelope sender addresses on incoming SMTP messages are checked to ensure that they are valid. Messages with invalid envelope senders are rejected with a permanent error code if sender_verify_reject is set (the default). Otherwise a warning is logged. See section 45.2 for details of the rejection, which can happen at three different points in the SMTP dialogue. If a sender cannot immediately be verified, a temporary error code is returned after reading the data (so the headers can be logged). The sender_try_verify option is less severe: it operates in exactly the same way as sender_verify except that if an address cannot immediately be verified, it is accepted instead of being temporarily rejected.
45.2 Sender verification
When sender_verify is set, Exim checks the senders of incoming SMTP messages, that is, the addresses given in the SMTP MAIL commands. This does not apply to batch SMTP input by default, but sender_verify_batch can be set true if it is required. The check is performed by running the same verification code as is used when Exim is called with the -bvs option, that is, by running the directors and routers in verify mode.
A dilemma arises when a local address is expanded by aliasing or forwarding: should verification continue with the generated addresses, or should the successful expansion of the original address be enough to verify it? Exim (since release 3.20) takes the following pragmatic approach:
When an incoming address is aliased to just one child address, in an aliasfile or a smartuser director (but not for forwardfile), verification continues with the child address, and if that fails to verify, the original verification also fails.
This seems the most reasonable behaviour for the common use of aliasing as a way of directing different local parts to the same mailbox. It means, for example, that a pair of alias entries of the form
A.Wol: aw123
aw123: :fail: Gone away, no forwarding address
work as expected, with both local parts causing verification failure. On the other hand, when an alias generates more than one address, the behaviour is more like a mailing list, where the existence of the alias itself is sufficient for verification.
The sender verification check is performed when the MAIL command is received. If the address cannot immediately be verified (typically because of DNS timeouts), a temporary failure error response (code 451) is given after the data for the message has been received. It is delayed until this time so that the message's headers can be logged. However, if sender_try_verify is set, the sender is accepted with a warning message after a temporary verification failure.
Exim remembers temporary sender verification errors in a hints database. Subsequent temporary errors for the same address from the same host within 24 hours cause a 451 error after MAIL instead of after the data. This reduces the data on the reject log and also the amount repeatedly transferred over the net.
If sender_verify_max_retry_rate is set greater than zero, and the rate of temporary rejection of a specific incoming sender address from a specific host, in units of rejections per hour, exceeds it, the temporary error is converted into a permanent verification error. This should help in stopping hosts hammering too frequently with temporarily failing sender addresses. The default value of the option is 12, which means that a sender address that has a temporary verification error more than once every 5 minutes will soon get permanently rejected. Once permanent rejection has been triggered, subsequent temporary failures will all cause permanent errors, until there has been an interval of at least 24 hours since the last failure. After 24 hours, the hint expires.
What happens if verification fails with a permanent error depends on the setting of the sender_verify_reject option. If it is set (the default) then the message is rejected. Otherwise a warning message is logged, and processing continues.
Because remote postmasters always want to see the message headers when there is a problem, Exim does not give an error response immediately a sender address fails, but instead it reads the data for the message first. The headers of rejected messages are written to the reject log, for use in tracking down the problem or tracing mail abusers. Up to three envelope recipients are also logged with the headers.
Unfortunately, there are a number of mailers in use that treat any SMTP error response given after the data has been transmitted as a temporary failure. Exim sends code 550 when it rejects a message because of a bad sender, and RFC 821 is quite clear in stating that all codes starting with 5 are always `permanent negative completion' replies. However, it does not give any guidance as to what should be done on receiving such replies, and some mailers persist in trying to send messages when they receive such a code at the end of the data.
To get round this, Exim keeps a database in which it remembers the bad sender address and host name when it rejects a message. If the same host sends the same bad sender address within 24 hours, Exim rejects the message at the MAIL command, before it reads the data for the message. This should prevent the sender from trying to send the message again, but there seem to be plenty of broken mailers out there that do keep on trying, sometimes for days on end.
In an attempt to shut such MTAs up, if the same host sends the same bad sender for a third time within 24 hours, MAIL is accepted, but all subsequent RCPT commands are rejected with a 550 error code. This means `unknown user' and if a remote mailer doesn't treat that as a hard error, it is very seriously broken.
The sender_verify_hosts option can be used to restrict hosts and RFC 1413 idents for which sender verification is not applied. If a cluster of hosts all check incoming external messages, there is no need to waste effort checking mail sent between them. For example:
sender_verify_hosts = ! *.ref.book : ! exim@mailer.fict.book
When sender_verify is set, Exim checks the senders of incoming SMTP messages, that is, the addresses given in the SMTP MAIL commands. This does not apply to batch SMTP input by default, but sender_verify_batch can be set true if it is required. The check is performed by running the same verification code as is used when Exim is called with the -bvs option, that is, by running the directors and routers in verify mode.
A dilemma arises when a local address is expanded by aliasing or forwarding: should verification continue with the generated addresses, or should the successful expansion of the original address be enough to verify it? Exim (since release 3.20) takes the following pragmatic approach:
When an incoming address is aliased to just one child address, in an aliasfile or a smartuser director (but not for forwardfile), verification continues with the child address, and if that fails to verify, the original verification also fails.
This seems the most reasonable behaviour for the common use of aliasing as a way of directing different local parts to the same mailbox. It means, for example, that a pair of alias entries of the form
A.Wol: aw123
aw123: :fail: Gone away, no forwarding address
work as expected, with both local parts causing verification failure. On the other hand, when an alias generates more than one address, the behaviour is more like a mailing list, where the existence of the alias itself is sufficient for verification.
The sender verification check is performed when the MAIL command is received. If the address cannot immediately be verified (typically because of DNS timeouts), a temporary failure error response (code 451) is given after the data for the message has been received. It is delayed until this time so that the message's headers can be logged. However, if sender_try_verify is set, the sender is accepted with a warning message after a temporary verification failure.
Exim remembers temporary sender verification errors in a hints database. Subsequent temporary errors for the same address from the same host within 24 hours cause a 451 error after MAIL instead of after the data. This reduces the data on the reject log and also the amount repeatedly transferred over the net.
If sender_verify_max_retry_rate is set greater than zero, and the rate of temporary rejection of a specific incoming sender address from a specific host, in units of rejections per hour, exceeds it, the temporary error is converted into a permanent verification error. This should help in stopping hosts hammering too frequently with temporarily failing sender addresses. The default value of the option is 12, which means that a sender address that has a temporary verification error more than once every 5 minutes will soon get permanently rejected. Once permanent rejection has been triggered, subsequent temporary failures will all cause permanent errors, until there has been an interval of at least 24 hours since the last failure. After 24 hours, the hint expires.
What happens if verification fails with a permanent error depends on the setting of the sender_verify_reject option. If it is set (the default) then the message is rejected. Otherwise a warning message is logged, and processing continues.
Because remote postmasters always want to see the message headers when there is a problem, Exim does not give an error response immediately a sender address fails, but instead it reads the data for the message first. The headers of rejected messages are written to the reject log, for use in tracking down the problem or tracing mail abusers. Up to three envelope recipients are also logged with the headers.
Unfortunately, there are a number of mailers in use that treat any SMTP error response given after the data has been transmitted as a temporary failure. Exim sends code 550 when it rejects a message because of a bad sender, and RFC 821 is quite clear in stating that all codes starting with 5 are always `permanent negative completion' replies. However, it does not give any guidance as to what should be done on receiving such replies, and some mailers persist in trying to send messages when they receive such a code at the end of the data.
To get round this, Exim keeps a database in which it remembers the bad sender address and host name when it rejects a message. If the same host sends the same bad sender address within 24 hours, Exim rejects the message at the MAIL command, before it reads the data for the message. This should prevent the sender from trying to send the message again, but there seem to be plenty of broken mailers out there that do keep on trying, sometimes for days on end.
In an attempt to shut such MTAs up, if the same host sends the same bad sender for a third time within 24 hours, MAIL is accepted, but all subsequent RCPT commands are rejected with a 550 error code. This means `unknown user' and if a remote mailer doesn't treat that as a hard error, it is very seriously broken.
The sender_verify_hosts option can be used to restrict hosts and RFC 1413 idents for which sender verification is not applied. If a cluster of hosts all check incoming external messages, there is no need to waste effort checking mail sent between them. For example:
sender_verify_hosts = ! *.ref.book : ! exim@mailer.fict.book
3. Here is my question. Do you think that sender_verify should be On as it is in the default cpanel config (the exim documentation states that the default is off)? Will turning it off result in a lot more spam, etc?
I've just noticed in the documentation listed above that I can turn it off for specific hosts wit hsender_verify_hosts = ! ... At least that will solve the problem for those few customers who report problems receiving mail from some senders who's domain does not have a valid A of CNAME record.
cheers
