Network Working Group G. Lindberg Request for Comments: 2505 Chalmers University of Technology BCP: 30 February 1999 Category: Best Current Practice Anti-Spam Recommendations for SMTP MTAs Status of this Memo This document specifies an Internet Best Current Practices for the Internet Community, and requests discussion and suggestions for improvements. Distribution of this memo is unlimited. Copyright Notice Copyright (C) The Internet Society (1999). All Rights Reserved.
AbstractThis memo gives a number of implementation recommendations for SMTP, , MTAs (Mail Transfer Agents, e.g. sendmail, ) to make them more capable of reducing the impact of spam(*). The intent is that these recommendations will help clean up the spam situation, if applied on enough SMTP MTAs on the Internet, and that they should be used as guidelines for the various MTA vendors. We are fully aware that this is not the final solution, but if these recommendations were included, and used, on all Internet SMTP MTAs, things would improve considerably and give time to design a more long term solution. The Future Work section suggests some ideas that may be part of such a long term solution. It might, though, very well be the case that the ultimate solution is social, political, or legal, rather than technical in nature. The implementor should be aware of the increased risk of denial of service attacks that several of the proposed methods might lead to. For example, increased number of queries to DNS servers and increased size of logfiles might both lead to overloaded systems and system crashes during an attack. A brief summary of this memo is: o Stop unauthorized mail relaying. o Spammers then have to operate in the open; deal with them. o Design a mail system that can handle spam.
requested" when you never asked for it, and generally do everything they can without regard to honesty or ethics, to try to get a few more people to look at their message. In fact some of the spam-programs take a pride in adding false info that will "make the ISPs scratch their heads". It is usually the case that people who send in protests (often according to the instructions in the mail) find their mail addresses added to more lists and sold to other parties. o It is quite common practice to make use of third party hosts as relays to get the spam mail sent out to the receivers. This theft of service is illegal in most - if not all - countries (at least in the US spammers have been successfully sued). However, with the original sender in the US, the (innocent) relay in Sweden and the list of receivers back in the US, the legal process of getting damages from the spammers becomes extremely difficult.
RFC2119, : o "MUST" This word or the adjective "REQUIRED" means that the item is an absolute requirement. o "SHOULD" This word or the adjective "RECOMMENDED" means that there may exist valid reasons in particular circumstances to ignore this item, but the full implications should be understood and the case carefully weighed before choosing a different course. o "MAY" This word or the adjective "OPTIONAL" means that this item is truly optional. One vendor may choose to include the item because a particular marketplace requires it or because it enhances the product, for example; another vendor may omit the same item. RFC1034, , and RFC1035, . When we suggest use of FQDNs rather than IP addresses this is because FQDNs are intuitively much easier to use. However, all such usage depends heavily on DNS and .IN-ADDR.ARPA (PTR) information. Since it is fairly easy to forge that, either by false cache information injected in DNS servers or spammers running their own DNS with false information in them, host and domain names must be used with care, e.g. verified so that the translation address->name corresponds to name->address. With Secure DNS, RFC2065, , things will improve, since spoofing of .IN-ADDR.ARPA will no longer be possible. One of the recommendations is about verifying "MAIL From:" (envelope originator) domains with the DNS (assure that appropriate DNS information exists for the domain). When making use of this capability there are a few things to consider:
(1) One must not forget the increased amount of DNS queries which might result in problems for the DNS server itself to cope with the load. This itself can result in a denial of service attack against the DNS server just by sending email to a site. (2) It should be noted that with negative caching in the DNS, forged DNS responses can be used to mount denial of service attacks. For example, if a site is known to implement a FQDN validity check on addresses in SMTP "MAIL From:" commands, an attacker may be able to use negative DNS responses to effectively block acceptance of mail from one or more origins. Because of this, one should carefully check the DNS server in use, e.g. how it verifies that incoming responses correspond to outstanding queries, to minimize the risk for such attacks. (3) For early versions of spam software FQDN checks provide quite some relief, since that software generates mail with completely bogus "MAIL From:" that will never get into the system if verified with the DNS. This is in active use at many installations today and it does reduce spam. On the other hand, sites with weak DNS connectivity may find their legitimate mail having problems reaching destinations due to DNS timeouts when the receivers verify their "MAIL From:". However, since DNS information is handled asynchronously and is cached even though the initial requester has given up, chances are high that the necessary information is there at a later attempt. For later versions of spam software, a check of "MAIL From:" is much less likely to help, since spam software evolves too and will start using existing mail addresses (whether or not that is legal is beyond the scope of this memo). But, at least the Internet will benefit from the side effect that this test stops typos and misconfigured UAs.
1] for a discussion: o 5xx is a Permanent Negative Completion reply (Fatal Error) and results in the mail transfer being terminated and the mail returned to sender. o 4xx is a Transient Negative Completion reply (Temporary Error) and results in the mail transfer being put back on queue again and a new attempt being made later. o 2xx is a Positive Completion reply and indicates that the MTA now has taken responsibility for the delivery of the mail. When making use of the options/"knobs" described in this memo there are a few things to consider: For some events, like "Denied - you're on the spammer's list", 5xx may be the correct Return Code, since it terminates the session at once and we are done with it (assuming that the spammer plays by the SMTP rules, which he may decide not to do - in fact he can put the mail back on queue or turn to some other host, regardless of Return Code). However, a 5xx mistake in a configuration may cause legitimate mail to bounce, which may be quite unfortunate. Therefore, we suggest 4xx as the Return Code for most cases. Since that is a non fatal error, the mail gets re-queued at the sender and we have at least some time to discover and correct configuration errors, rather than have mail bounce (typically this is when we refuse to Relay for domains that we actually should accept since we are on their MX list). A 4xx response also makes the spammer's host re-queue the mail and if it really is his own host who gets to do this it is probably a good thing - fill up his disks with his own spam. If, on the other hand, he is using someone else as Relay Host, all the spam mail being queued is a fairly strong evidence that something bad is going on and should cause attention at that Relay Host. However, 4xx Temporary Errors may have unexpected interaction with MX-records. If the receiving domain has several MX records and the lowest preference MX-host refuses to receive mail with a "451" Response Code, the sending host may choose to - and often will - use the next host on the MX list.
If that next MX host does not have the same refuse-list, it will of course accept the mail, only to find that the final host still refuses to receive that piece of mail ("MAIL From:"). Our intent was to make the offending mail stay at the offending sender's host and fill up his mqueue disk, but it all ended up at our friend, the next lowest preference MX-host. Finally, it has been suggested that one may use a 2xx Return Code but nevertheless decide not to forward or receive the spam mail; typical alternatives are to store it elsewhere (e.g. /dev/null). This clearly violates the intent of RFC821 and should not be done without careful consideration - instead of blindly dropping the mail one could re- queue it and manually (or automagically) inspect whether it is spam or legitimate mail and whether it should be dropped or forwarded.
7a) SHOULD be able to refuse mail from a specific "MAIL From:" user, <firstname.lastname@example.org>. 7b) SHOULD be able to refuse mail from an entire "MAIL From:" domain <.*@domain.example>. 8) SHOULD be able to limit ("Rate Control") mail flow. 9) SHOULD be able to verify "MAIL From:" domain (using DNS or other means). 10) SHOULD be able to verify <local-part> in outgoing mail. 11) SHOULD be able to control SMTP VRFY and EXPN. 12) SHOULD be able to control SMTP ETRN. 13) MUST be able to configure to provide different Return Codes for different rules (e.g. 451 Temp Fail vs 550 Fatal Error). The discussion below often ends up with a need to do various forms of pattern matching on domain/host names and IP addresses/subnets. It is RECOMMENDED that the data/template for doing so may be supplied outside of the MTA, e.g. that the pattern matching rules be included in the MTA but that the actual patterns may be in an external file. It is also RECOMMENDED that the pattern matching rules (external file) may contain regular expressions, to give maximum flexibility. Of course string matching on domain/host names MUST NOT be case sensitive. Since <local-part> may be case sensitive it may be natural to keep that here. However, since <sPAmMeR@domain.example> and <email@example.com> is most probably the same user and since the string compares are used to refuse his messages, we suggest that <local-part> comparisons be case insensitive too. The interpretation that should apply to all these recommendations is flexibility - regardless of how well we design anti-spam rules today, spammers will find ways around them and a well designed MTA should be flexible enough to meet those new threats.
In an SMTP session we have 4 elements, each with a varying degree of trust: 1) "HELO Hostname" Easily and often forged. 2) "MAIL From:" Easily and often forged. 3) "RCPT To:" Correct, or at least intended. 4) SMTP_Caller (host) IP.src addr OK, FQDN may be OK. Since 1) and 2) are so easily and often forged, we cannot depend on them at all to authorize usage of our host as Mail Relay. Instead, the MTA MUST be able to authorize Mail Relay usage based on a combination of: o "RCPT To:" address (domain). o SMTP_Caller FQDN hostname. o SMTP_Caller IP address. The suggested algorithm is: a) If "RCPT To:" is one of "our" domains, local or a domain that we accept to forward to (alternate MX), then accept to Relay. b) If SMTP_Caller is authorized, either its IP.src or its FQDN (depending on if you trust the DNS), then accept to Relay. c) Else refuse to Relay. When doing a) you have to make sure all kinds of SMTP source routing (both the official [@a,@b:u@c], the '%' hack and uucp-style '!' path) is either removed completely before the test, or at least is taken into account. A site implementing this requirement must also be aware that they might block correctly addressed messages, especially such originating or terminating in a gateway to a different mail system than SMTP. Before implementing such a policy, a careful inventory should be done to make sure all routing algorithms used, either by other mail systems or ad-hoc, are known. Each one of such systems must be taken care of on a case-by-case basis. Examples of such mail systems, and their addressing schemes are X.400 with an address of the type: "/c=us/admd= /prmd=xyz/dd.rfc-822=user(a)final/"@x400-gateway Another example involves DECnet MAIL-11, which can have addresses in the form:
"gateway::smtp%\"user@final\""@mail-11-gateway In all cases the configuration MUST support wild cards for FQDNs and classful IP addresses and SHOULD support "address/mask" for classless IP addresses, e.g. domain.example and *.domain.example; 10.11.*.*, 192.168.1.*, 192.168.2.*, 10.0.0.0/13, 192.168.1.0/23. The configuration SHOULD allow for the decision/template data to be supplied by an external source, e.g. text file or dbm database. The decision/template SHOULD be allowed to contain regular expressions. RFC822, , and required in RFC1123, ). This "Received:" line MUST contain enough information to make it possible to trace the mail path back to the sender. We have two cases: RFC822, , pp 18. It SHOULD contain: o The FQDN corresponding to the callers IP address. o The argument given in the "HELO" statement. o Authentication information, if an authenticated connection was used for the transmission / submission. It is suggested that most other "Received:" fields described in RFC822 be included in the "Received:" lines. Basically, any information that can help tracing the message can and should be added to the "Received:" line. It is true even when the initial submission is non-SMTP, for example submission via a web-based
mail client where http is used between the web client and server, a "Received:" line can be used to identify that connection stating what IP-address was used when connecting to the http server where the mail was created. These recommendations are deliberately stronger than RFC1123, , and are there to assure that mail sent directly from a spammer's host to a recipient can be traced with enough accuracy; a typical example is when a spammer uses a dialup account and the ISP needs to have his IP address at the 'date-time' to be able to take action against him. RFC1123, ). By doing so they take on the full responsibility to trace spammers that send from inside their organization or they accept to be held responsible for those spammers' activities. It is REQUIRED that the information provided in their outgoing mail is sufficient for them to perform any necessary traces. In the case of incoming mail to an organization, the "Received:" lines MUST be kept intact to make sure that users receiving mail on the inside can give information needed to trace incoming messages to their origin. Generally, a gateway SHOULD NOT change or delete "Received:" lines unless it is a security requirement to do so. Changing the content of existing "Received:" lines to make sure they "make sense" when passing a mail gateway of some kind most often destroys and deletes information needed to make a message traceable. Care must be taken to preserve the information in "Received:" lines, either in the message itself, the mail that the receiver gets, or if that is impossible, in logfiles.
accept host.domain.example refuse *.domain.example accept 10.11.12.13 accept 192.168.1.0/24 refuse 10.0.0.0/8 The list is searched until first match and the accept/refuse action is based on that. IP-address/length is RECOMMENDED. However, implementations with wild cards, e.g. 10.11.12.* (classful networks on byte boundaries only) are of course much better than those without. To improve filtering even more, the MTA MAY provide complete regular expressions to be used for hostnames; possibly also for IP addresses.
However, the MTA MAY throttle down the TCP connection ("read()" frequency) if there are more than one "RCPT To:" and that way slow down spammers using "MAIL From: <>". RFC1123, , gives a clear requirement that "MAIL From:" for mail from a mailing list should reflect the owner of the list, rather than the individual sender. Because of this fact, and the fact that the owner of the list might not be in the same domain as the list (list host) itself, mail may arrive to the list owner's domain (mail host) from a foreign domain (from a host serving a foreign domain) with the list owner's local domain in the "Mail From:" command. If "MAIL From: <firstname.lastname@example.org>" is rejected, both these cases will result in failure to deliver legitimate mail.
MUST NOT be refused (see above), except when other policies block the connection, for example when the SMTP_Caller IP address of the peer belongs to a network which is deliberately refused.
MAIL From: <email@example.com> and/or malicious users MAIL From: <I.firstname.lastname@example.org> As always this can be overcome by spammers really wanting to do so, but with more strict rules for relaying it becomes harder and harder. In fact, catching "typos" at the initial (and official) mail relay is in itself enough motivation for this recommendation. RFC821, . The response can, though, be "252 Argument not checked" to represent "off" or blocked via an access list. This should be the default. Default for the "EXPN" command should be "off".
that the selection of exactly what error code to use is very subtle and that many software implementations do check more than the first digit (2, 4 or 5) in the return code. However, when the response is the result of a DNS lookup and the DNS system returned TempFail, a temporary error, the MTA MUST reflect this and provide a 4xx return code. If the DNS response is an Authoritative NXdomain (host or domain unknown) the MTA MAY reflect this by a 5xx Return Code. Please refer to the previous discussion on SMTP Return Codes for additional information.
The correct way to handle this situation, though, is by some other mail-sending protocol between the UA and the MTA. Although a separate submission protocol does not exist, a profile of SMTP for this, the "Message Submission" specification, , has recently been defined. Or, we could note that when the SMTP Authentication work, , is all in place, it will allow for Authenticated SMTP to serve as The Protocol between the UAs and the home MTA (whether that should be considered a new protocol or "the same old SMTP" is irrelevant here). This adds one item to the suggested Relay algorithm in section 2.1: + If "SMTP Authenticated" then accept to Relay.
C RCPT To: <email@example.com> S 250 <firstname.lastname@example.org>... Recipient ok C RCPT To: <email@example.com> S 451 <firstname.lastname@example.org>... Denied due to spam list C RCPT To: <email@example.com> S 550 <firstname.lastname@example.org>... Denied due to spam list Of course one could decide on either "250 OK" or "550 Denied" with no other alternatives for the individual user, but this too has to be explained enough that an ordinary user understands the implications of "Refuse 'MAIL From: <.*@spam.example>'" and that it can do away with, or block out, mail he actually wanted. 10], has already been mentioned as a method to authorize Mail Relaying, but of course there is much more to it than that. When that infrastructure and functionality is all in place, spammers will have a much harder time forging addresses and hiding.
receiver hosts have their disk flooded, the mail being returned may also fail, i.e. the email service may become less trustworthy than before. o Hosts used as unauthorized Mail Relays become overloaded. Besides the technical implications, this too requires a lot of human resources, cleaning up mail queues and taking care of furious external users that were spammed through the Relay. o The fight against spammers includes blocking their hosts (which is described in this memo). However, there is a great risk that Mail Relay hosts may be blocked too, even though they are also victims. In the long run, this may cause Internet mail service to deteriorate. o The common use of forged "MAIL From:" and "From:" addresses puts the blame on innocent persons/hosts/organizations. This is bad for reputations and may affect business relations. Several of the methods described in this document increases the load on several support systems to the email system itself. Those support systems can be DNS, logging, databases with lists of local users, authentication mechanisms and others. Implementing the methods described in this document will, because of that, increase the risk of a denial of service attack against the support system by sending spam to a site. Logging facilities must for example be able to handle more logging (what happens when the logfiles fills the disk?). DNS servers and authentication mechanisms must be able to stand the load of more lookups etc. The functionality of the support systems during high load should be carefully studied before implementing the methods described in this document. The mail system should be carefully studied, e.g. how it behaves when one or more of the support systems needed for a specific method fails. A mail server MUST NOT respond with "Permanent Failure" (5xx) if there is a temporary problem with one of its support systems.
We want to acknowledge valuable input and suggestions from Andras Salamon, John Myers, Bob Flandrena, Dave Presotto, Dave Kristol, Donald Eastlake, Ned Freed, Keith Moore and Paul Hoffman. Finally many thanks to Harald Alvestrand and Patrik Faltstrom, both for useful comments and for their support and guidance through the IETF process.  Postel, J., "Simple Mail Transfer Protocol", STD 10, RFC 821, August 1982.  Crocker, D., "Standard for the format of ARPA Internet text messages", STD 11, RFC 822, August 1982.  Braden, R., "Requirements for Internet hosts - application and support", STD 3, RFC 1123, October 1989.  Bradner, S., "Key words for use in RFCs to Indicate Requirement Level", BCP 14, RFC 2119, March 1997.  Mockapetris, P., "Domain Names - Concepts and Facilities", STD 13, RFC 1034, November 1987.  Mockapetris, P., "Domain Names - Implementation and Specifications", STD 13, RFC 1035, November 1987.  Eastlake, D. and C. Kaufman, "Domain Name System Security Extensions", RFC 2065, January 1997.  sendmail Home Page. http://www.sendmail.org  Gellens, R. and J. Klensin "Message Submission", RFC 2476, September 1998.  Myers, J., "SMTP Service Extension for Authentication", Work in Progress. * Spam (R) (capitalized) is a registered trademark of a meat product made by Hormel. Use of the term spam (uncapitalized) in the Internet community comes from a Monty Python sketch and is almost Internet folklore. The term spam is usually pejorative, however this is not in any way intended to describe the Hormel product.
Full Copyright Statement Copyright (C) The Internet Society (1999). All Rights Reserved. This document and translations of it may be copied and furnished to others, and derivative works that comment on or otherwise explain it or assist in its implementation may be prepared, copied, published and distributed, in whole or in part, without restriction of any kind, provided that the above copyright notice and this paragraph are included on all such copies and derivative works. However, this document itself may not be modified in any way, such as by removing the copyright notice or references to the Internet Society or other Internet organizations, except as needed for the purpose of developing Internet standards in which case the procedures for copyrights defined in the Internet Standards process must be followed, or as required to translate it into languages other than English. The limited permissions granted above are perpetual and will not be revoked by the Internet Society or its successors or assigns. This document and the information contained herein is provided on an "AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.