4.2 TRIVIAL FILE TRANSFER PROTOCOL -- TFTP 4.2.1 INTRODUCTION The Trivial File Transfer Protocol TFTP is defined in RFC-783 [TFTP:1]. TFTP provides its own reliable delivery with UDP as its transport protocol, using a simple stop-and-wait acknowledgment system. Since TFTP has an effective window of only one 512 octet segment, it can provide good performance only over paths that have a small delay*bandwidth product. The TFTP file interface is very simple, providing no access control or security. TFTP's most important application is bootstrapping a host over a local network, since it is simple and small enough to be easily implemented in EPROM [BOOT:1, BOOT:2]. Vendors are urged to support TFTP for booting. 4.2.2 PROTOCOL WALK-THROUGH The TFTP specification [TFTP:1] is written in an open style, and does not fully specify many parts of the protocol. 18.104.22.168 Transfer Modes: RFC-783, Page 3 The transfer mode "mail" SHOULD NOT be supported. 22.214.171.124 UDP Header: RFC-783, Page 17 The Length field of a UDP header is incorrectly defined; it includes the UDP header length (8). 4.2.3 SPECIFIC ISSUES 126.96.36.199 Sorcerer's Apprentice Syndrome There is a serious bug, known as the "Sorcerer's Apprentice Syndrome," in the protocol specification. While it does not cause incorrect operation of the transfer (the file will always be transferred correctly if the transfer completes), this bug may cause excessive retransmission, which may cause the transfer to time out. Implementations MUST contain the fix for this problem: the sender (i.e., the side originating the DATA packets) must never resend the current DATA packet on receipt of a
duplicate ACK. DISCUSSION: The bug is caused by the protocol rule that either side, on receiving an old duplicate datagram, may resend the current datagram. If a packet is delayed in the network but later successfully delivered after either side has timed out and retransmitted a packet, a duplicate copy of the response may be generated. If the other side responds to this duplicate with a duplicate of its own, then every datagram will be sent in duplicate for the remainder of the transfer (unless a datagram is lost, breaking the repetition). Worse yet, since the delay is often caused by congestion, this duplicate transmission will usually causes more congestion, leading to more delayed packets, etc. The following example may help to clarify this problem. TFTP A TFTP B (1) Receive ACK X-1 Send DATA X (2) Receive DATA X Send ACK X (ACK X is delayed in network, and A times out): (3) Retransmit DATA X (4) Receive DATA X again Send ACK X again (5) Receive (delayed) ACK X Send DATA X+1 (6) Receive DATA X+1 Send ACK X+1 (7) Receive ACK X again Send DATA X+1 again (8) Receive DATA X+1 again Send ACK X+1 again (9) Receive ACK X+1 Send DATA X+2 (10) Receive DATA X+2 Send ACK X+3 (11) Receive ACK X+1 again Send DATA X+2 again (12) Receive DATA X+2 again Send ACK X+3 again
Notice that once the delayed ACK arrives, the protocol settles down to duplicate all further packets (sequences 5-8 and 9-12). The problem is caused not by either side timing out, but by both sides retransmitting the current packet when they receive a duplicate. The fix is to break the retransmission loop, as indicated above. This is analogous to the behavior of TCP. It is then possible to remove the retransmission timer on the receiver, since the resent ACK will never cause any action; this is a useful simplification where TFTP is used in a bootstrap program. It is OK to allow the timer to remain, and it may be helpful if the retransmitted ACK replaces one that was genuinely lost in the network. The sender still requires a retransmit timer, of course. 188.8.131.52 Timeout Algorithms A TFTP implementation MUST use an adaptive timeout. IMPLEMENTATION: TCP retransmission algorithms provide a useful base to work from. At least an exponential backoff of retransmission timeout is necessary. 184.108.40.206 Extensions A variety of non-standard extensions have been made to TFTP, including additional transfer modes and a secure operation mode (with passwords). None of these have been standardized. 220.127.116.11 Access Control A server TFTP implementation SHOULD include some configurable access control over what pathnames are allowed in TFTP operations. 18.104.22.168 Broadcast Request A TFTP request directed to a broadcast address SHOULD be silently ignored. DISCUSSION: Due to the weak access control capability of TFTP, directed broadcasts of TFTP requests to random networks
could create a significant security hole. 4.2.4 TFTP REQUIREMENTS SUMMARY | | | | |S| | | | | | |H| |F | | | | |O|M|o | | |S| |U|U|o | | |H| |L|S|t | |M|O| |D|T|n | |U|U|M| | |o | |S|L|A|N|N|t | |T|D|Y|O|O|t FEATURE |SECTION | | | |T|T|e -------------------------------------------------|--------|-|-|-|-|-|-- Fix Sorcerer's Apprentice Syndrome |22.214.171.124 |x| | | | | Transfer modes: | | | | | | | netascii |RFC-783 |x| | | | | octet |RFC-783 |x| | | | | mail |126.96.36.199 | | | |x| | extensions |188.8.131.52 | | |x| | | Use adaptive timeout |184.108.40.206 |x| | | | | Configurable access control |220.127.116.11 | |x| | | | Silently ignore broadcast request |18.104.22.168 | |x| | | | -------------------------------------------------|--------|-|-|-|-|-|-- -------------------------------------------------|--------|-|-|-|-|-|--
5. ELECTRONIC MAIL -- SMTP and RFC-822 5.1 INTRODUCTION In the TCP/IP protocol suite, electronic mail in a format specified in RFC-822 [SMTP:2] is transmitted using the Simple Mail Transfer Protocol (SMTP) defined in RFC-821 [SMTP:1]. While SMTP has remained unchanged over the years, the Internet community has made several changes in the way SMTP is used. In particular, the conversion to the Domain Name System (DNS) has caused changes in address formats and in mail routing. In this section, we assume familiarity with the concepts and terminology of the DNS, whose requirements are given in Section 6.1. RFC-822 specifies the Internet standard format for electronic mail messages. RFC-822 supercedes an older standard, RFC-733, that may still be in use in a few places, although it is obsolete. The two formats are sometimes referred to simply by number ("822" and "733"). RFC-822 is used in some non-Internet mail environments with different mail transfer protocols than SMTP, and SMTP has also been adapted for use in some non-Internet environments. Note that this document presents the rules for the use of SMTP and RFC-822 for the Internet environment only; other mail environments that use these protocols may be expected to have their own rules. 5.2 PROTOCOL WALK-THROUGH This section covers both RFC-821 and RFC-822. The SMTP specification in RFC-821 is clear and contains numerous examples, so implementors should not find it difficult to understand. This section simply updates or annotates portions of RFC-821 to conform with current usage. RFC-822 is a long and dense document, defining a rich syntax. Unfortunately, incomplete or defective implementations of RFC-822 are common. In fact, nearly all of the many formats of RFC-822 are actually used, so an implementation generally needs to recognize and correctly interpret all of the RFC-822 syntax. 5.2.1 The SMTP Model: RFC-821 Section 2 DISCUSSION: Mail is sent by a series of request/response transactions between a client, the "sender-SMTP," and a server, the
"receiver-SMTP". These transactions pass (1) the message proper, which is composed of header and body, and (2) SMTP source and destination addresses, referred to as the "envelope". The SMTP programs are analogous to Message Transfer Agents (MTAs) of X.400. There will be another level of protocol software, closer to the end user, that is responsible for composing and analyzing RFC-822 message headers; this component is known as the "User Agent" in X.400, and we use that term in this document. There is a clear logical distinction between the User Agent and the SMTP implementation, since they operate on different levels of protocol. Note, however, that this distinction is may not be exactly reflected the structure of typical implementations of Internet mail. Often there is a program known as the "mailer" that implements SMTP and also some of the User Agent functions; the rest of the User Agent functions are included in a user interface used for entering and reading mail. The SMTP envelope is constructed at the originating site, typically by the User Agent when the message is first queued for the Sender-SMTP program. The envelope addresses may be derived from information in the message header, supplied by the user interface (e.g., to implement a bcc: request), or derived from local configuration information (e.g., expansion of a mailing list). The SMTP envelope cannot in general be re-derived from the header at a later stage in message delivery, so the envelope is transmitted separately from the message itself using the MAIL and RCPT commands of SMTP. The text of RFC-821 suggests that mail is to be delivered to an individual user at a host. With the advent of the domain system and of mail routing using mail-exchange (MX) resource records, implementors should now think of delivering mail to a user at a domain, which may or may not be a particular host. This DOES NOT change the fact that SMTP is a host-to-host mail exchange protocol. 5.2.2 Canonicalization: RFC-821 Section 3.1 The domain names that a Sender-SMTP sends in MAIL and RCPT commands MUST have been "canonicalized," i.e., they must be fully-qualified principal names or domain literals, not nicknames or domain abbreviations. A canonicalized name either identifies a host directly or is an MX name; it cannot be a
CNAME. 5.2.3 VRFY and EXPN Commands: RFC-821 Section 3.3 A receiver-SMTP MUST implement VRFY and SHOULD implement EXPN (this requirement overrides RFC-821). However, there MAY be configuration information to disable VRFY and EXPN in a particular installation; this might even allow EXPN to be disabled for selected lists. A new reply code is defined for the VRFY command: 252 Cannot VRFY user (e.g., info is not local), but will take message for this user and attempt delivery. DISCUSSION: SMTP users and administrators make regular use of these commands for diagnosing mail delivery problems. With the increasing use of multi-level mailing list expansion (sometimes more than two levels), EXPN has been increasingly important for diagnosing inadvertent mail loops. On the other hand, some feel that EXPN represents a significant privacy, and perhaps even a security, exposure. 5.2.4 SEND, SOML, and SAML Commands: RFC-821 Section 3.4 An SMTP MAY implement the commands to send a message to a user's terminal: SEND, SOML, and SAML. DISCUSSION: It has been suggested that the use of mail relaying through an MX record is inconsistent with the intent of SEND to deliver a message immediately and directly to a user's terminal. However, an SMTP receiver that is unable to write directly to the user terminal can return a "251 User Not Local" reply to the RCPT following a SEND, to inform the originator of possibly deferred delivery. 5.2.5 HELO Command: RFC-821 Section 3.5 The sender-SMTP MUST ensure that the <domain> parameter in a HELO command is a valid principal host domain name for the client host. As a result, the receiver-SMTP will not have to perform MX resolution on this name in order to validate the HELO parameter. The HELO receiver MAY verify that the HELO parameter really
corresponds to the IP address of the sender. However, the receiver MUST NOT refuse to accept a message, even if the sender's HELO command fails verification. DISCUSSION: Verifying the HELO parameter requires a domain name lookup and may therefore take considerable time. An alternative tool for tracking bogus mail sources is suggested below (see "DATA Command"). Note also that the HELO argument is still required to have valid <domain> syntax, since it will appear in a Received: line; otherwise, a 501 error is to be sent. IMPLEMENTATION: When HELO parameter validation fails, a suggested procedure is to insert a note about the unknown authenticity of the sender into the message header (e.g., in the "Received:" line). 5.2.6 Mail Relay: RFC-821 Section 3.6 We distinguish three types of mail (store-and-) forwarding: (1) A simple forwarder or "mail exchanger" forwards a message using private knowledge about the recipient; see section 3.2 of RFC-821. (2) An SMTP mail "relay" forwards a message within an SMTP mail environment as the result of an explicit source route (as defined in section 3.6 of RFC-821). The SMTP relay function uses the "@...:" form of source route from RFC- 822 (see Section 5.2.19 below). (3) A mail "gateway" passes a message between different environments. The rules for mail gateways are discussed below in Section 5.3.7. An Internet host that is forwarding a message but is not a gateway to a different mail environment (i.e., it falls under (1) or (2)) SHOULD NOT alter any existing header fields, although the host will add an appropriate Received: line as required in Section 5.2.8. A Sender-SMTP SHOULD NOT send a RCPT TO: command containing an explicit source route using the "@...:" address form. Thus, the relay function defined in section 3.6 of RFC-821 should not be used.
DISCUSSION: The intent is to discourage all source routing and to abolish explicit source routing for mail delivery within the Internet environment. Source-routing is unnecessary; the simple target address "user@domain" should always suffice. This is the result of an explicit architectural decision to use universal naming rather than source routing for mail. Thus, SMTP provides end-to-end connectivity, and the DNS provides globally-unique, location-independent names. MX records handle the major case where source routing might otherwise be needed. A receiver-SMTP MUST accept the explicit source route syntax in the envelope, but it MAY implement the relay function as defined in section 3.6 of RFC-821. If it does not implement the relay function, it SHOULD attempt to deliver the message directly to the host to the right of the right-most "@" sign. DISCUSSION: For example, suppose a host that does not implement the relay function receives a message with the SMTP command: "RCPT TO:<@ALPHA,@BETA:joe@GAMMA>", where ALPHA, BETA, and GAMMA represent domain names. Rather than immediately refusing the message with a 550 error reply as suggested on page 20 of RFC-821, the host should try to forward the message to GAMMA directly, using: "RCPT TO:<joe@GAMMA>". Since this host does not support relaying, it is not required to update the reverse path. Some have suggested that source routing may be needed occasionally for manually routing mail around failures; however, the reality and importance of this need is controversial. The use of explicit SMTP mail relaying for this purpose is discouraged, and in fact it may not be successful, as many host systems do not support it. Some have used the "%-hack" (see Section 5.2.16) for this purpose. 5.2.7 RCPT Command: RFC-821 Section 4.1.1 A host that supports a receiver-SMTP MUST support the reserved mailbox "Postmaster". The receiver-SMTP MAY verify RCPT parameters as they arrive; however, RCPT responses MUST NOT be delayed beyond a reasonable time (see Section 5.3.2). Therefore, a "250 OK" response to a RCPT does not necessarily
imply that the delivery address(es) are valid. Errors found after message acceptance will be reported by mailing a notification message to an appropriate address (see Section 5.3.3). DISCUSSION: The set of conditions under which a RCPT parameter can be validated immediately is an engineering design choice. Reporting destination mailbox errors to the Sender-SMTP before mail is transferred is generally desirable to save time and network bandwidth, but this advantage is lost if RCPT verification is lengthy. For example, the receiver can verify immediately any simple local reference, such as a single locally- registered mailbox. On the other hand, the "reasonable time" limitation generally implies deferring verification of a mailing list until after the message has been transferred and accepted, since verifying a large mailing list can take a very long time. An implementation might or might not choose to defer validation of addresses that are non-local and therefore require a DNS lookup. If a DNS lookup is performed but a soft domain system error (e.g., timeout) occurs, validity must be assumed. 5.2.8 DATA Command: RFC-821 Section 4.1.1 Every receiver-SMTP (not just one that "accepts a message for relaying or for final delivery" [SMTP:1]) MUST insert a "Received:" line at the beginning of a message. In this line, called a "time stamp line" in RFC-821: * The FROM field SHOULD contain both (1) the name of the source host as presented in the HELO command and (2) a domain literal containing the IP address of the source, determined from the TCP connection. * The ID field MAY contain an "@" as suggested in RFC-822, but this is not required. * The FOR field MAY contain a list of <path> entries when multiple RCPT commands have been given. An Internet mail program MUST NOT change a Received: line that was previously added to the message header.
DISCUSSION: Including both the source host and the IP source address in the Received: line may provide enough information for tracking illicit mail sources and eliminate a need to explicitly verify the HELO parameter. Received: lines are primarily intended for humans tracing mail routes, primarily of diagnosis of faults. See also the discussion under 5.3.7. When the receiver-SMTP makes "final delivery" of a message, then it MUST pass the MAIL FROM: address from the SMTP envelope with the message, for use if an error notification message must be sent later (see Section 5.3.3). There is an analogous requirement when gatewaying from the Internet into a different mail environment; see Section 5.3.7. DISCUSSION: Note that the final reply to the DATA command depends only upon the successful transfer and storage of the message. Any problem with the destination address(es) must either (1) have been reported in an SMTP error reply to the RCPT command(s), or (2) be reported in a later error message mailed to the originator. IMPLEMENTATION: The MAIL FROM: information may be passed as a parameter or in a Return-Path: line inserted at the beginning of the message. 5.2.9 Command Syntax: RFC-821 Section 4.1.2 The syntax shown in RFC-821 for the MAIL FROM: command omits the case of an empty path: "MAIL FROM: <>" (see RFC-821 Page 15). An empty reverse path MUST be supported. 5.2.10 SMTP Replies: RFC-821 Section 4.2 A receiver-SMTP SHOULD send only the reply codes listed in section 4.2.2 of RFC-821 or in this document. A receiver-SMTP SHOULD use the text shown in examples in RFC-821 whenever appropriate. A sender-SMTP MUST determine its actions only by the reply code, not by the text (except for 251 and 551 replies); any text, including no text at all, must be acceptable. The space (blank) following the reply code is considered part of the text. Whenever possible, a sender-SMTP SHOULD test only the
first digit of the reply code, as specified in Appendix E of RFC-821. DISCUSSION: Interoperability problems have arisen with SMTP systems using reply codes that are not listed explicitly in RFC- 821 Section 4.3 but are legal according to the theory of reply codes explained in Appendix E. 5.2.11 Transparency: RFC-821 Section 4.5.2 Implementors MUST be sure that their mail systems always add and delete periods to ensure message transparency. 5.2.12 WKS Use in MX Processing: RFC-974, p. 5 RFC-974 [SMTP:3] recommended that the domain system be queried for WKS ("Well-Known Service") records, to verify that each proposed mail target does support SMTP. Later experience has shown that WKS is not widely supported, so the WKS step in MX processing SHOULD NOT be used. The following are notes on RFC-822, organized by section of that document. 5.2.13 RFC-822 Message Specification: RFC-822 Section 4 The syntax shown for the Return-path line omits the possibility of a null return path, which is used to prevent looping of error notifications (see Section 5.3.3). The complete syntax is: return = "Return-path" ":" route-addr / "Return-path" ":" "<" ">" The set of optional header fields is hereby expanded to include the Content-Type field defined in RFC-1049 [SMTP:7]. This field "allows mail reading systems to automatically identify the type of a structured message body and to process it for display accordingly". [SMTP:7] A User Agent MAY support this field. 5.2.14 RFC-822 Date and Time Specification: RFC-822 Section 5 The syntax for the date is hereby changed to: date = 1*2DIGIT month 2*4DIGIT
All mail software SHOULD use 4-digit years in dates, to ease the transition to the next century. There is a strong trend towards the use of numeric timezone indicators, and implementations SHOULD use numeric timezones instead of timezone names. However, all implementations MUST accept either notation. If timezone names are used, they MUST be exactly as defined in RFC-822. The military time zones are specified incorrectly in RFC-822: they count the wrong way from UT (the signs are reversed). As a result, military time zones in RFC-822 headers carry no information. Finally, note that there is a typo in the definition of "zone" in the syntax summary of appendix D; the correct definition occurs in Section 3 of RFC-822. 5.2.15 RFC-822 Syntax Change: RFC-822 Section 6.1 The syntactic definition of "mailbox" in RFC-822 is hereby changed to: mailbox = addr-spec ; simple address / [phrase] route-addr ; name & addr-spec That is, the phrase preceding a route address is now OPTIONAL. This change makes the following header field legal, for example: From: <email@example.com> 5.2.16 RFC-822 Local-part: RFC-822 Section 6.2 The basic mailbox address specification has the form: "local- part@domain". Here "local-part", sometimes called the "left- hand side" of the address, is domain-dependent. A host that is forwarding the message but is not the destination host implied by the right-hand side "domain" MUST NOT interpret or modify the "local-part" of the address. When mail is to be gatewayed from the Internet mail environment into a foreign mail environment (see Section 5.3.7), routing information for that foreign environment MAY be embedded within the "local-part" of the address. The gateway will then interpret this local part appropriately for the foreign mail environment.
DISCUSSION: Although source routes are discouraged within the Internet (see Section 5.2.6), there are non-Internet mail environments whose delivery mechanisms do depend upon source routes. Source routes for extra-Internet environments can generally be buried in the "local-part" of the address (see Section 5.2.16) while mail traverses the Internet. When the mail reaches the appropriate Internet mail gateway, the gateway will interpret the local-part and build the necessary address or route for the target mail environment. For example, an Internet host might send mail to: "a!b!c!user@gateway-domain". The complex local part "a!b!c!user" would be uninterpreted within the Internet domain, but could be parsed and understood by the specified mail gateway. An embedded source route is sometimes encoded in the "local-part" using "%" as a right-binding routing operator. For example, in: user%domain%relay3%relay2@relay1 the "%" convention implies that the mail is to be routed from "relay1" through "relay2", "relay3", and finally to "user" at "domain". This is commonly known as the "%- hack". It is suggested that "%" have lower precedence than any other routing operator (e.g., "!") hidden in the local-part; for example, "a!b%c" would be interpreted as "(a!b)%c". Only the target host (in this case, "relay1") is permitted to analyze the local-part "user%domain%relay3%relay2". 5.2.17 Domain Literals: RFC-822 Section 6.2.3 A mailer MUST be able to accept and parse an Internet domain literal whose content ("dtext"; see RFC-822) is a dotted- decimal host address. This satisfies the requirement of Section 2.1 for the case of mail. An SMTP MUST accept and recognize a domain literal for any of its own IP addresses.
5.2.18 Common Address Formatting Errors: RFC-822 Section 6.1 Errors in formatting or parsing 822 addresses are unfortunately common. This section mentions only the most common errors. A User Agent MUST accept all valid RFC-822 address formats, and MUST NOT generate illegal address syntax. o A common error is to leave out the semicolon after a group identifier. o Some systems fail to fully-qualify domain names in messages they generate. The right-hand side of an "@" sign in a header address field MUST be a fully-qualified domain name. For example, some systems fail to fully-qualify the From: address; this prevents a "reply" command in the user interface from automatically constructing a return address. DISCUSSION: Although RFC-822 allows the local use of abbreviated domain names within a domain, the application of RFC-822 in Internet mail does not allow this. The intent is that an Internet host must not send an SMTP message header containing an abbreviated domain name in an address field. This allows the address fields of the header to be passed without alteration across the Internet, as required in Section 5.2.6. o Some systems mis-parse multiple-hop explicit source routes such as: @relay1,@relay2,@relay3:user@domain. o Some systems over-qualify domain names by adding a trailing dot to some or all domain names in addresses or message-ids. This violates RFC-822 syntax. 5.2.19 Explicit Source Routes: RFC-822 Section 6.2.7 Internet host software SHOULD NOT create an RFC-822 header containing an address with an explicit source route, but MUST accept such headers for compatibility with earlier systems. DISCUSSION:
In an understatement, RFC-822 says "The use of explicit source routing is discouraged". Many hosts implemented RFC-822 source routes incorrectly, so the syntax cannot be used unambiguously in practice. Many users feel the syntax is ugly. Explicit source routes are not needed in the mail envelope for delivery; see Section 5.2.6. For all these reasons, explicit source routes using the RFC- 822 notations are not to be used in Internet mail headers. As stated in Section 5.2.16, it is necessary to allow an explicit source route to be buried in the local-part of an address, e.g., using the "%-hack", in order to allow mail to be gatewayed into another environment in which explicit source routing is necessary. The vigilant will observe that there is no way for a User Agent to detect and prevent the use of such implicit source routing when the destination is within the Internet. We can only discourage source routing of any kind within the Internet, as unnecessary and undesirable. 5.3 SPECIFIC ISSUES 5.3.1 SMTP Queueing Strategies The common structure of a host SMTP implementation includes user mailboxes, one or more areas for queueing messages in transit, and one or more daemon processes for sending and receiving mail. The exact structure will vary depending on the needs of the users on the host and the number and size of mailing lists supported by the host. We describe several optimizations that have proved helpful, particularly for mailers supporting high traffic levels. Any queueing strategy MUST include: o Timeouts on all activities. See Section 5.3.2. o Never sending error messages in response to error messages. 22.214.171.124 Sending Strategy The general model of a sender-SMTP is one or more processes that periodically attempt to transmit outgoing mail. In a typical system, the program that composes a message has some method for requesting immediate attention for a new piece of outgoing mail, while mail that cannot be transmitted
immediately MUST be queued and periodically retried by the sender. A mail queue entry will include not only the message itself but also the envelope information. The sender MUST delay retrying a particular destination after one attempt has failed. In general, the retry interval SHOULD be at least 30 minutes; however, more sophisticated and variable strategies will be beneficial when the sender-SMTP can determine the reason for non- delivery. Retries continue until the message is transmitted or the sender gives up; the give-up time generally needs to be at least 4-5 days. The parameters to the retry algorithm MUST be configurable. A sender SHOULD keep a list of hosts it cannot reach and corresponding timeouts, rather than just retrying queued mail items. DISCUSSION: Experience suggests that failures are typically transient (the target system has crashed), favoring a policy of two connection attempts in the first hour the message is in the queue, and then backing off to once every two or three hours. The sender-SMTP can shorten the queueing delay by cooperation with the receiver-SMTP. In particular, if mail is received from a particular address, it is good evidence that any mail queued for that host can now be sent. The strategy may be further modified as a result of multiple addresses per host (see Section 5.3.4), to optimize delivery time vs. resource usage. A sender-SMTP may have a large queue of messages for each unavailable destination host, and if it retried all these messages in every retry cycle, there would be excessive Internet overhead and the daemon would be blocked for a long period. Note that an SMTP can generally determine that a delivery attempt has failed only after a timeout of a minute or more; a one minute timeout per connection will result in a very large delay if it is repeated for dozens or even hundreds of queued messages.
When the same message is to be delivered to several users on the same host, only one copy of the message SHOULD be transmitted. That is, the sender-SMTP should use the command sequence: RCPT, RCPT,... RCPT, DATA instead of the sequence: RCPT, DATA, RCPT, DATA,... RCPT, DATA. Implementation of this efficiency feature is strongly urged. Similarly, the sender-SMTP MAY support multiple concurrent outgoing mail transactions to achieve timely delivery. However, some limit SHOULD be imposed to protect the host from devoting all its resources to mail. The use of the different addresses of a multihomed host is discussed below. 126.96.36.199 Receiving strategy The receiver-SMTP SHOULD attempt to keep a pending listen on the SMTP port at all times. This will require the support of multiple incoming TCP connections for SMTP. Some limit MAY be imposed. IMPLEMENTATION: When the receiver-SMTP receives mail from a particular host address, it could notify the sender-SMTP to retry any mail pending for that host address. 5.3.2 Timeouts in SMTP There are two approaches to timeouts in the sender-SMTP: (a) limit the time for each SMTP command separately, or (b) limit the time for the entire SMTP dialogue for a single mail message. A sender-SMTP SHOULD use option (a), per-command timeouts. Timeouts SHOULD be easily reconfigurable, preferably without recompiling the SMTP code. DISCUSSION: Timeouts are an essential feature of an SMTP implementation. If the timeouts are too long (or worse, there are no timeouts), Internet communication failures or software bugs in receiver-SMTP programs can tie up SMTP processes indefinitely. If the timeouts are too short, resources will be wasted with attempts that time out part way through message delivery. If option (b) is used, the timeout has to be very large, e.g., an hour, to allow time to expand very large mailing lists. The timeout may also need to increase linearly
with the size of the message, to account for the time to transmit a very large message. A large fixed timeout leads to two problems: a failure can still tie up the sender for a very long time, and very large messages may still spuriously time out (which is a wasteful failure!). Using the recommended option (a), a timer is set for each SMTP command and for each buffer of the data transfer. The latter means that the overall timeout is inherently proportional to the size of the message. Based on extensive experience with busy mail-relay hosts, the minimum per-command timeout values SHOULD be as follows: o Initial 220 Message: 5 minutes A Sender-SMTP process needs to distinguish between a failed TCP connection and a delay in receiving the initial 220 greeting message. Many receiver-SMTPs will accept a TCP connection but delay delivery of the 220 message until their system load will permit more mail to be processed. o MAIL Command: 5 minutes o RCPT Command: 5 minutes A longer timeout would be required if processing of mailing lists and aliases were not deferred until after the message was accepted. o DATA Initiation: 2 minutes This is while awaiting the "354 Start Input" reply to a DATA command. o Data Block: 3 minutes This is while awaiting the completion of each TCP SEND call transmitting a chunk of data. o DATA Termination: 10 minutes. This is while awaiting the "250 OK" reply. When the receiver gets the final period terminating the message data, it typically performs processing to deliver the message to a user mailbox. A spurious timeout at this point would be very wasteful, since the message has been
successfully sent. A receiver-SMTP SHOULD have a timeout of at least 5 minutes while it is awaiting the next command from the sender. 5.3.3 Reliable Mail Receipt When the receiver-SMTP accepts a piece of mail (by sending a "250 OK" message in response to DATA), it is accepting responsibility for delivering or relaying the message. It must take this responsibility seriously, i.e., it MUST NOT lose the message for frivolous reasons, e.g., because the host later crashes or because of a predictable resource shortage. If there is a delivery failure after acceptance of a message, the receiver-SMTP MUST formulate and mail a notification message. This notification MUST be sent using a null ("<>") reverse path in the envelope; see Section 3.6 of RFC-821. The recipient of this notification SHOULD be the address from the envelope return path (or the Return-Path: line). However, if this address is null ("<>"), the receiver-SMTP MUST NOT send a notification. If the address is an explicit source route, it SHOULD be stripped down to its final hop. DISCUSSION: For example, suppose that an error notification must be sent for a message that arrived with: "MAIL FROM:<@a,@b:user@d>". The notification message should be sent to: "RCPT TO:<user@d>". Some delivery failures after the message is accepted by SMTP will be unavoidable. For example, it may be impossible for the receiver-SMTP to validate all the delivery addresses in RCPT command(s) due to a "soft" domain system error or because the target is a mailing list (see earlier discussion of RCPT). To avoid receiving duplicate messages as the result of timeouts, a receiver-SMTP MUST seek to minimize the time required to respond to the final "." that ends a message transfer. See RFC-1047 [SMTP:4] for a discussion of this problem. 5.3.4 Reliable Mail Transmission To transmit a message, a sender-SMTP determines the IP address of the target host from the destination address in the envelope. Specifically, it maps the string to the right of the
"@" sign into an IP address. This mapping or the transfer itself may fail with a soft error, in which case the sender- SMTP will requeue the outgoing mail for a later retry, as required in Section 188.8.131.52. When it succeeds, the mapping can result in a list of alternative delivery addresses rather than a single address, because of (a) multiple MX records, (b) multihoming, or both. To provide reliable mail transmission, the sender-SMTP MUST be able to try (and retry) each of the addresses in this list in order, until a delivery attempt succeeds. However, there MAY also be a configurable limit on the number of alternate addresses that can be tried. In any case, a host SHOULD try at least two addresses. The following information is to be used to rank the host addresses: (1) Multiple MX Records -- these contain a preference indication that should be used in sorting. If there are multiple destinations with the same preference and there is no clear reason to favor one (e.g., by address preference), then the sender-SMTP SHOULD pick one at random to spread the load across multiple mail exchanges for a specific organization; note that this is a refinement of the procedure in [DNS:3]. (2) Multihomed host -- The destination host (perhaps taken from the preferred MX record) may be multihomed, in which case the domain name resolver will return a list of alternative IP addresses. It is the responsibility of the domain name resolver interface (see Section 184.108.40.206 below) to have ordered this list by decreasing preference, and SMTP MUST try them in the order presented. DISCUSSION: Although the capability to try multiple alternative addresses is required, there may be circumstances where specific installations want to limit or disable the use of alternative addresses. The question of whether a sender should attempt retries using the different addresses of a multihomed host has been controversial. The main argument for using the multiple addresses is that it maximizes the probability of timely delivery, and indeed sometimes the probability of any delivery; the counter argument is that it may result in unnecessary resource use. Note that resource use is also strongly determined by the
sending strategy discussed in Section 5.3.1. 5.3.5 Domain Name Support SMTP implementations MUST use the mechanism defined in Section 6.1 for mapping between domain names and IP addresses. This means that every Internet SMTP MUST include support for the Internet DNS. In particular, a sender-SMTP MUST support the MX record scheme [SMTP:3]. See also Section 7.4 of [DNS:2] for information on domain name support for SMTP. 5.3.6 Mailing Lists and Aliases An SMTP-capable host SHOULD support both the alias and the list form of address expansion for multiple delivery. When a message is delivered or forwarded to each address of an expanded list form, the return address in the envelope ("MAIL FROM:") MUST be changed to be the address of a person who administers the list, but the message header MUST be left unchanged; in particular, the "From" field of the message is unaffected. DISCUSSION: An important mail facility is a mechanism for multi- destination delivery of a single message, by transforming or "expanding" a pseudo-mailbox address into a list of destination mailbox addresses. When a message is sent to such a pseudo-mailbox (sometimes called an "exploder"), copies are forwarded or redistributed to each mailbox in the expanded list. We classify such a pseudo-mailbox as an "alias" or a "list", depending upon the expansion rules: (a) Alias To expand an alias, the recipient mailer simply replaces the pseudo-mailbox address in the envelope with each of the expanded addresses in turn; the rest of the envelope and the message body are left unchanged. The message is then delivered or forwarded to each expanded address. (b) List A mailing list may be said to operate by "redistribution" rather than by "forwarding". To
expand a list, the recipient mailer replaces the pseudo-mailbox address in the envelope with each of the expanded addresses in turn. The return address in the envelope is changed so that all error messages generated by the final deliveries will be returned to a list administrator, not to the message originator, who generally has no control over the contents of the list and will typically find error messages annoying. 5.3.7 Mail Gatewaying Gatewaying mail between different mail environments, i.e., different mail formats and protocols, is complex and does not easily yield to standardization. See for example [SMTP:5a], [SMTP:5b]. However, some general requirements may be given for a gateway between the Internet and another mail environment. (A) Header fields MAY be rewritten when necessary as messages are gatewayed across mail environment boundaries. DISCUSSION: This may involve interpreting the local-part of the destination address, as suggested in Section 5.2.16. The other mail systems gatewayed to the Internet generally use a subset of RFC-822 headers, but some of them do not have an equivalent to the SMTP envelope. Therefore, when a message leaves the Internet environment, it may be necessary to fold the SMTP envelope information into the message header. A possible solution would be to create new header fields to carry the envelope information (e.g., "X- SMTP-MAIL:" and "X-SMTP-RCPT:"); however, this would require changes in mail programs in the foreign environment. (B) When forwarding a message into or out of the Internet environment, a gateway MUST prepend a Received: line, but it MUST NOT alter in any way a Received: line that is already in the header. DISCUSSION: This requirement is a subset of the general "Received:" line requirement of Section 5.2.8; it is restated here for emphasis. Received: fields of messages originating from other
environments may not conform exactly to RFC822. However, the most important use of Received: lines is for debugging mail faults, and this debugging can be severely hampered by well-meaning gateways that try to "fix" a Received: line. The gateway is strongly encouraged to indicate the environment and protocol in the "via" clauses of Received field(s) that it supplies. (C) From the Internet side, the gateway SHOULD accept all valid address formats in SMTP commands and in RFC-822 headers, and all valid RFC-822 messages. Although a gateway must accept an RFC-822 explicit source route ("@...:" format) in either the RFC-822 header or in the envelope, it MAY or may not act on the source route; see Sections 5.2.6 and 5.2.19. DISCUSSION: It is often tempting to restrict the range of addresses accepted at the mail gateway to simplify the translation into addresses for the remote environment. This practice is based on the assumption that mail users have control over the addresses their mailers send to the mail gateway. In practice, however, users have little control over the addresses that are finally sent; their mailers are free to change addresses into any legal RFC-822 format. (D) The gateway MUST ensure that all header fields of a message that it forwards into the Internet meet the requirements for Internet mail. In particular, all addresses in "From:", "To:", "Cc:", etc., fields must be transformed (if necessary) to satisfy RFC-822 syntax, and they must be effective and useful for sending replies. (E) The translation algorithm used to convert mail from the Internet protocols to another environment's protocol SHOULD try to ensure that error messages from the foreign mail environment are delivered to the return path from the SMTP envelope, not to the sender listed in the "From:" field of the RFC-822 message. DISCUSSION: Internet mail lists usually place the address of the mail list maintainer in the envelope but leave the
original message header intact (with the "From:" field containing the original sender). This yields the behavior the average recipient expects: a reply to the header gets sent to the original sender, not to a mail list maintainer; however, errors get sent to the maintainer (who can fix the problem) and not the sender (who probably cannot). (F) Similarly, when forwarding a message from another environment into the Internet, the gateway SHOULD set the envelope return path in accordance with an error message return address, if any, supplied by the foreign environment. 5.3.8 Maximum Message Size Mailer software MUST be able to send and receive messages of at least 64K bytes in length (including header), and a much larger maximum size is highly desirable. DISCUSSION: Although SMTP does not define the maximum size of a message, many systems impose implementation limits. The current de facto minimum limit in the Internet is 64K bytes. However, electronic mail is used for a variety of purposes that create much larger messages. For example, mail is often used instead of FTP for transmitting ASCII files, and in particular to transmit entire documents. As a result, messages can be 1 megabyte or even larger. We note that the present document together with its lower- layer companion contains 0.5 megabytes.
5.4 SMTP REQUIREMENTS SUMMARY | | | | |S| | | | | | |H| |F | | | | |O|M|o | | |S| |U|U|o | | |H| |L|S|t | |M|O| |D|T|n | |U|U|M| | |o | |S|L|A|N|N|t | |T|D|Y|O|O|t FEATURE |SECTION | | | |T|T|e -----------------------------------------------|----------|-|-|-|-|-|-- | | | | | | | RECEIVER-SMTP: | | | | | | | Implement VRFY |5.2.3 |x| | | | | Implement EXPN |5.2.3 | |x| | | | EXPN, VRFY configurable |5.2.3 | | |x| | | Implement SEND, SOML, SAML |5.2.4 | | |x| | | Verify HELO parameter |5.2.5 | | |x| | | Refuse message with bad HELO |5.2.5 | | | | |x| Accept explicit src-route syntax in env. |5.2.6 |x| | | | | Support "postmaster" |5.2.7 |x| | | | | Process RCPT when received (except lists) |5.2.7 | | |x| | | Long delay of RCPT responses |5.2.7 | | | | |x| | | | | | | | Add Received: line |5.2.8 |x| | | | | Received: line include domain literal |5.2.8 | |x| | | | Change previous Received: line |5.2.8 | | | | |x| Pass Return-Path info (final deliv/gwy) |5.2.8 |x| | | | | Support empty reverse path |5.2.9 |x| | | | | Send only official reply codes |5.2.10 | |x| | | | Send text from RFC-821 when appropriate |5.2.10 | |x| | | | Delete "." for transparency |5.2.11 |x| | | | | Accept and recognize self domain literal(s) |5.2.17 |x| | | | | | | | | | | | Error message about error message |5.3.1 | | | | |x| Keep pending listen on SMTP port |220.127.116.11 | |x| | | | Provide limit on recv concurrency |18.104.22.168 | | |x| | | Wait at least 5 mins for next sender cmd |5.3.2 | |x| | | | Avoidable delivery failure after "250 OK" |5.3.3 | | | | |x| Send error notification msg after accept |5.3.3 |x| | | | | Send using null return path |5.3.3 |x| | | | | Send to envelope return path |5.3.3 | |x| | | | Send to null address |5.3.3 | | | | |x| Strip off explicit src route |5.3.3 | |x| | | | Minimize acceptance delay (RFC-1047) |5.3.3 |x| | | | | -----------------------------------------------|----------|-|-|-|-|-|--
| | | | | | | SENDER-SMTP: | | | | | | | Canonicalized domain names in MAIL, RCPT |5.2.2 |x| | | | | Implement SEND, SOML, SAML |5.2.4 | | |x| | | Send valid principal host name in HELO |5.2.5 |x| | | | | Send explicit source route in RCPT TO: |5.2.6 | | | |x| | Use only reply code to determine action |5.2.10 |x| | | | | Use only high digit of reply code when poss. |5.2.10 | |x| | | | Add "." for transparency |5.2.11 |x| | | | | | | | | | | | Retry messages after soft failure |22.214.171.124 |x| | | | | Delay before retry |126.96.36.199 |x| | | | | Configurable retry parameters |188.8.131.52 |x| | | | | Retry once per each queued dest host |184.108.40.206 | |x| | | | Multiple RCPT's for same DATA |220.127.116.11 | |x| | | | Support multiple concurrent transactions |18.104.22.168 | | |x| | | Provide limit on concurrency |22.214.171.124 | |x| | | | | | | | | | | Timeouts on all activities |5.3.1 |x| | | | | Per-command timeouts |5.3.2 | |x| | | | Timeouts easily reconfigurable |5.3.2 | |x| | | | Recommended times |5.3.2 | |x| | | | Try alternate addr's in order |5.3.4 |x| | | | | Configurable limit on alternate tries |5.3.4 | | |x| | | Try at least two alternates |5.3.4 | |x| | | | Load-split across equal MX alternates |5.3.4 | |x| | | | Use the Domain Name System |5.3.5 |x| | | | | Support MX records |5.3.5 |x| | | | | Use WKS records in MX processing |5.2.12 | | | |x| | -----------------------------------------------|----------|-|-|-|-|-|-- | | | | | | | MAIL FORWARDING: | | | | | | | Alter existing header field(s) |5.2.6 | | | |x| | Implement relay function: 821/section 3.6 |5.2.6 | | |x| | | If not, deliver to RHS domain |5.2.6 | |x| | | | Interpret 'local-part' of addr |5.2.16 | | | | |x| | | | | | | | MAILING LISTS AND ALIASES | | | | | | | Support both |5.3.6 | |x| | | | Report mail list error to local admin. |5.3.6 |x| | | | | | | | | | | | MAIL GATEWAYS: | | | | | | | Embed foreign mail route in local-part |5.2.16 | | |x| | | Rewrite header fields when necessary |5.3.7 | | |x| | | Prepend Received: line |5.3.7 |x| | | | | Change existing Received: line |5.3.7 | | | | |x| Accept full RFC-822 on Internet side |5.3.7 | |x| | | | Act on RFC-822 explicit source route |5.3.7 | | |x| | |
Send only valid RFC-822 on Internet side |5.3.7 |x| | | | | Deliver error msgs to envelope addr |5.3.7 | |x| | | | Set env return path from err return addr |5.3.7 | |x| | | | | | | | | | | USER AGENT -- RFC-822 | | | | | | | Allow user to enter <route> address |5.2.6 | | | |x| | Support RFC-1049 Content Type field |5.2.13 | | |x| | | Use 4-digit years |5.2.14 | |x| | | | Generate numeric timezones |5.2.14 | |x| | | | Accept all timezones |5.2.14 |x| | | | | Use non-num timezones from RFC-822 |5.2.14 |x| | | | | Omit phrase before route-addr |5.2.15 | | |x| | | Accept and parse dot.dec. domain literals |5.2.17 |x| | | | | Accept all RFC-822 address formats |5.2.18 |x| | | | | Generate invalid RFC-822 address format |5.2.18 | | | | |x| Fully-qualified domain names in header |5.2.18 |x| | | | | Create explicit src route in header |5.2.19 | | | |x| | Accept explicit src route in header |5.2.19 |x| | | | | | | | | | | | Send/recv at least 64KB messages |5.3.8 |x| | | | |