RFC 1123
Requirements for Internet Hosts - Application and Support
Pages: 98
Internet Standard: 3
→ Errata
STD 3 is also: 1122
Updates: 0822 0952
Updated by: 1349 2181 5321 5966 7766 9210
Internet Standard: 3
→ Errata
STD 3 is also: 1122
Updates: 0822 0952
Updated by: 1349 2181 5321 5966 7766 9210
Part 3 of 4 – Pages 44 to 71
4.2 TRIVIAL FILE TRANSFER PROTOCOL -- TFTP 4.2.1 INTRODUCTION The Trivial File Transfer Protocol TFTP is defined in RFC-783 [TFTP:1]. TFTP provides its own reliable delivery with UDP as its transport protocol, using a simple stop-and-wait acknowledgment system. Since TFTP has an effective window of only one 512 octet segment, it can provide good performance only over paths that have a small delay*bandwidth product. The TFTP file interface is very simple, providing no access control or security. TFTP's most important application is bootstrapping a host over a local network, since it is simple and small enough to be easily implemented in EPROM [BOOT:1, BOOT:2]. Vendors are urged to support TFTP for booting. 4.2.2 PROTOCOL WALK-THROUGH The TFTP specification [TFTP:1] is written in an open style, and does not fully specify many parts of the protocol. 4.2.2.1 Transfer Modes: RFC-783, Page 3 The transfer mode "mail" SHOULD NOT be supported. 4.2.2.2 UDP Header: RFC-783, Page 17 The Length field of a UDP header is incorrectly defined; it includes the UDP header length (8). 4.2.3 SPECIFIC ISSUES 4.2.3.1 Sorcerer's Apprentice Syndrome There is a serious bug, known as the "Sorcerer's Apprentice Syndrome," in the protocol specification. While it does not cause incorrect operation of the transfer (the file will always be transferred correctly if the transfer completes), this bug may cause excessive retransmission, which may cause the transfer to time out. Implementations MUST contain the fix for this problem: the sender (i.e., the side originating the DATA packets) must never resend the current DATA packet on receipt of a
duplicate ACK.
DISCUSSION:
The bug is caused by the protocol rule that either
side, on receiving an old duplicate datagram, may
resend the current datagram. If a packet is delayed in
the network but later successfully delivered after
either side has timed out and retransmitted a packet, a
duplicate copy of the response may be generated. If
the other side responds to this duplicate with a
duplicate of its own, then every datagram will be sent
in duplicate for the remainder of the transfer (unless
a datagram is lost, breaking the repetition). Worse
yet, since the delay is often caused by congestion,
this duplicate transmission will usually causes more
congestion, leading to more delayed packets, etc.
The following example may help to clarify this problem.
TFTP A TFTP B
(1) Receive ACK X-1
Send DATA X
(2) Receive DATA X
Send ACK X
(ACK X is delayed in network,
and A times out):
(3) Retransmit DATA X
(4) Receive DATA X again
Send ACK X again
(5) Receive (delayed) ACK X
Send DATA X+1
(6) Receive DATA X+1
Send ACK X+1
(7) Receive ACK X again
Send DATA X+1 again
(8) Receive DATA X+1 again
Send ACK X+1 again
(9) Receive ACK X+1
Send DATA X+2
(10) Receive DATA X+2
Send ACK X+3
(11) Receive ACK X+1 again
Send DATA X+2 again
(12) Receive DATA X+2 again
Send ACK X+3 again
Notice that once the delayed ACK arrives, the protocol
settles down to duplicate all further packets
(sequences 5-8 and 9-12). The problem is caused not by
either side timing out, but by both sides
retransmitting the current packet when they receive a
duplicate.
The fix is to break the retransmission loop, as
indicated above. This is analogous to the behavior of
TCP. It is then possible to remove the retransmission
timer on the receiver, since the resent ACK will never
cause any action; this is a useful simplification where
TFTP is used in a bootstrap program. It is OK to allow
the timer to remain, and it may be helpful if the
retransmitted ACK replaces one that was genuinely lost
in the network. The sender still requires a retransmit
timer, of course.
4.2.3.2 Timeout Algorithms
A TFTP implementation MUST use an adaptive timeout.
IMPLEMENTATION:
TCP retransmission algorithms provide a useful base to
work from. At least an exponential backoff of
retransmission timeout is necessary.
4.2.3.3 Extensions
A variety of non-standard extensions have been made to TFTP,
including additional transfer modes and a secure operation
mode (with passwords). None of these have been
standardized.
4.2.3.4 Access Control
A server TFTP implementation SHOULD include some
configurable access control over what pathnames are allowed
in TFTP operations.
4.2.3.5 Broadcast Request
A TFTP request directed to a broadcast address SHOULD be
silently ignored.
DISCUSSION:
Due to the weak access control capability of TFTP,
directed broadcasts of TFTP requests to random networks
could create a significant security hole.
4.2.4 TFTP REQUIREMENTS SUMMARY
| | | | |S| |
| | | | |H| |F
| | | | |O|M|o
| | |S| |U|U|o
| | |H| |L|S|t
| |M|O| |D|T|n
| |U|U|M| | |o
| |S|L|A|N|N|t
| |T|D|Y|O|O|t
FEATURE |SECTION | | | |T|T|e
-------------------------------------------------|--------|-|-|-|-|-|--
Fix Sorcerer's Apprentice Syndrome |4.2.3.1 |x| | | | |
Transfer modes: | | | | | | |
netascii |RFC-783 |x| | | | |
octet |RFC-783 |x| | | | |
mail |4.2.2.1 | | | |x| |
extensions |4.2.3.3 | | |x| | |
Use adaptive timeout |4.2.3.2 |x| | | | |
Configurable access control |4.2.3.4 | |x| | | |
Silently ignore broadcast request |4.2.3.5 | |x| | | |
-------------------------------------------------|--------|-|-|-|-|-|--
-------------------------------------------------|--------|-|-|-|-|-|--
5. ELECTRONIC MAIL -- SMTP and RFC-822 5.1 INTRODUCTION In the TCP/IP protocol suite, electronic mail in a format specified in RFC-822 [SMTP:2] is transmitted using the Simple Mail Transfer Protocol (SMTP) defined in RFC-821 [SMTP:1]. While SMTP has remained unchanged over the years, the Internet community has made several changes in the way SMTP is used. In particular, the conversion to the Domain Name System (DNS) has caused changes in address formats and in mail routing. In this section, we assume familiarity with the concepts and terminology of the DNS, whose requirements are given in Section 6.1. RFC-822 specifies the Internet standard format for electronic mail messages. RFC-822 supercedes an older standard, RFC-733, that may still be in use in a few places, although it is obsolete. The two formats are sometimes referred to simply by number ("822" and "733"). RFC-822 is used in some non-Internet mail environments with different mail transfer protocols than SMTP, and SMTP has also been adapted for use in some non-Internet environments. Note that this document presents the rules for the use of SMTP and RFC-822 for the Internet environment only; other mail environments that use these protocols may be expected to have their own rules. 5.2 PROTOCOL WALK-THROUGH This section covers both RFC-821 and RFC-822. The SMTP specification in RFC-821 is clear and contains numerous examples, so implementors should not find it difficult to understand. This section simply updates or annotates portions of RFC-821 to conform with current usage. RFC-822 is a long and dense document, defining a rich syntax. Unfortunately, incomplete or defective implementations of RFC-822 are common. In fact, nearly all of the many formats of RFC-822 are actually used, so an implementation generally needs to recognize and correctly interpret all of the RFC-822 syntax. 5.2.1 The SMTP Model: RFC-821 Section 2 DISCUSSION: Mail is sent by a series of request/response transactions between a client, the "sender-SMTP," and a server, the
"receiver-SMTP". These transactions pass (1) the message
proper, which is composed of header and body, and (2) SMTP
source and destination addresses, referred to as the
"envelope".
The SMTP programs are analogous to Message Transfer Agents
(MTAs) of X.400. There will be another level of protocol
software, closer to the end user, that is responsible for
composing and analyzing RFC-822 message headers; this
component is known as the "User Agent" in X.400, and we
use that term in this document. There is a clear logical
distinction between the User Agent and the SMTP
implementation, since they operate on different levels of
protocol. Note, however, that this distinction is may not
be exactly reflected the structure of typical
implementations of Internet mail. Often there is a
program known as the "mailer" that implements SMTP and
also some of the User Agent functions; the rest of the
User Agent functions are included in a user interface used
for entering and reading mail.
The SMTP envelope is constructed at the originating site,
typically by the User Agent when the message is first
queued for the Sender-SMTP program. The envelope
addresses may be derived from information in the message
header, supplied by the user interface (e.g., to implement
a bcc: request), or derived from local configuration
information (e.g., expansion of a mailing list). The SMTP
envelope cannot in general be re-derived from the header
at a later stage in message delivery, so the envelope is
transmitted separately from the message itself using the
MAIL and RCPT commands of SMTP.
The text of RFC-821 suggests that mail is to be delivered
to an individual user at a host. With the advent of the
domain system and of mail routing using mail-exchange (MX)
resource records, implementors should now think of
delivering mail to a user at a domain, which may or may
not be a particular host. This DOES NOT change the fact
that SMTP is a host-to-host mail exchange protocol.
5.2.2 Canonicalization: RFC-821 Section 3.1
The domain names that a Sender-SMTP sends in MAIL and RCPT
commands MUST have been "canonicalized," i.e., they must be
fully-qualified principal names or domain literals, not
nicknames or domain abbreviations. A canonicalized name either
identifies a host directly or is an MX name; it cannot be a
CNAME.
5.2.3 VRFY and EXPN Commands: RFC-821 Section 3.3
A receiver-SMTP MUST implement VRFY and SHOULD implement EXPN
(this requirement overrides RFC-821). However, there MAY be
configuration information to disable VRFY and EXPN in a
particular installation; this might even allow EXPN to be
disabled for selected lists.
A new reply code is defined for the VRFY command:
252 Cannot VRFY user (e.g., info is not local), but will
take message for this user and attempt delivery.
DISCUSSION:
SMTP users and administrators make regular use of these
commands for diagnosing mail delivery problems. With the
increasing use of multi-level mailing list expansion
(sometimes more than two levels), EXPN has been
increasingly important for diagnosing inadvertent mail
loops. On the other hand, some feel that EXPN represents
a significant privacy, and perhaps even a security,
exposure.
5.2.4 SEND, SOML, and SAML Commands: RFC-821 Section 3.4
An SMTP MAY implement the commands to send a message to a
user's terminal: SEND, SOML, and SAML.
DISCUSSION:
It has been suggested that the use of mail relaying
through an MX record is inconsistent with the intent of
SEND to deliver a message immediately and directly to a
user's terminal. However, an SMTP receiver that is unable
to write directly to the user terminal can return a "251
User Not Local" reply to the RCPT following a SEND, to
inform the originator of possibly deferred delivery.
5.2.5 HELO Command: RFC-821 Section 3.5
The sender-SMTP MUST ensure that the <domain> parameter in a
HELO command is a valid principal host domain name for the
client host. As a result, the receiver-SMTP will not have to
perform MX resolution on this name in order to validate the
HELO parameter.
The HELO receiver MAY verify that the HELO parameter really
corresponds to the IP address of the sender. However, the
receiver MUST NOT refuse to accept a message, even if the
sender's HELO command fails verification.
DISCUSSION:
Verifying the HELO parameter requires a domain name lookup
and may therefore take considerable time. An alternative
tool for tracking bogus mail sources is suggested below
(see "DATA Command").
Note also that the HELO argument is still required to have
valid <domain> syntax, since it will appear in a Received:
line; otherwise, a 501 error is to be sent.
IMPLEMENTATION:
When HELO parameter validation fails, a suggested
procedure is to insert a note about the unknown
authenticity of the sender into the message header (e.g.,
in the "Received:" line).
5.2.6 Mail Relay: RFC-821 Section 3.6
We distinguish three types of mail (store-and-) forwarding:
(1) A simple forwarder or "mail exchanger" forwards a message
using private knowledge about the recipient; see section
3.2 of RFC-821.
(2) An SMTP mail "relay" forwards a message within an SMTP
mail environment as the result of an explicit source route
(as defined in section 3.6 of RFC-821). The SMTP relay
function uses the "@...:" form of source route from RFC-
822 (see Section 5.2.19 below).
(3) A mail "gateway" passes a message between different
environments. The rules for mail gateways are discussed
below in Section 5.3.7.
An Internet host that is forwarding a message but is not a
gateway to a different mail environment (i.e., it falls under
(1) or (2)) SHOULD NOT alter any existing header fields,
although the host will add an appropriate Received: line as
required in Section 5.2.8.
A Sender-SMTP SHOULD NOT send a RCPT TO: command containing an
explicit source route using the "@...:" address form. Thus,
the relay function defined in section 3.6 of RFC-821 should
not be used.
DISCUSSION:
The intent is to discourage all source routing and to
abolish explicit source routing for mail delivery within
the Internet environment. Source-routing is unnecessary;
the simple target address "user@domain" should always
suffice. This is the result of an explicit architectural
decision to use universal naming rather than source
routing for mail. Thus, SMTP provides end-to-end
connectivity, and the DNS provides globally-unique,
location-independent names. MX records handle the major
case where source routing might otherwise be needed.
A receiver-SMTP MUST accept the explicit source route syntax in
the envelope, but it MAY implement the relay function as
defined in section 3.6 of RFC-821. If it does not implement
the relay function, it SHOULD attempt to deliver the message
directly to the host to the right of the right-most "@" sign.
DISCUSSION:
For example, suppose a host that does not implement the
relay function receives a message with the SMTP command:
"RCPT TO:<@ALPHA,@BETA:joe@GAMMA>", where ALPHA, BETA, and
GAMMA represent domain names. Rather than immediately
refusing the message with a 550 error reply as suggested
on page 20 of RFC-821, the host should try to forward the
message to GAMMA directly, using: "RCPT TO:<joe@GAMMA>".
Since this host does not support relaying, it is not
required to update the reverse path.
Some have suggested that source routing may be needed
occasionally for manually routing mail around failures;
however, the reality and importance of this need is
controversial. The use of explicit SMTP mail relaying for
this purpose is discouraged, and in fact it may not be
successful, as many host systems do not support it. Some
have used the "%-hack" (see Section 5.2.16) for this
purpose.
5.2.7 RCPT Command: RFC-821 Section 4.1.1
A host that supports a receiver-SMTP MUST support the reserved
mailbox "Postmaster".
The receiver-SMTP MAY verify RCPT parameters as they arrive;
however, RCPT responses MUST NOT be delayed beyond a reasonable
time (see Section 5.3.2).
Therefore, a "250 OK" response to a RCPT does not necessarily
imply that the delivery address(es) are valid. Errors found
after message acceptance will be reported by mailing a
notification message to an appropriate address (see Section
5.3.3).
DISCUSSION:
The set of conditions under which a RCPT parameter can be
validated immediately is an engineering design choice.
Reporting destination mailbox errors to the Sender-SMTP
before mail is transferred is generally desirable to save
time and network bandwidth, but this advantage is lost if
RCPT verification is lengthy.
For example, the receiver can verify immediately any
simple local reference, such as a single locally-
registered mailbox. On the other hand, the "reasonable
time" limitation generally implies deferring verification
of a mailing list until after the message has been
transferred and accepted, since verifying a large mailing
list can take a very long time. An implementation might
or might not choose to defer validation of addresses that
are non-local and therefore require a DNS lookup. If a
DNS lookup is performed but a soft domain system error
(e.g., timeout) occurs, validity must be assumed.
5.2.8 DATA Command: RFC-821 Section 4.1.1
Every receiver-SMTP (not just one that "accepts a message for
relaying or for final delivery" [SMTP:1]) MUST insert a
"Received:" line at the beginning of a message. In this line,
called a "time stamp line" in RFC-821:
* The FROM field SHOULD contain both (1) the name of the
source host as presented in the HELO command and (2) a
domain literal containing the IP address of the source,
determined from the TCP connection.
* The ID field MAY contain an "@" as suggested in RFC-822,
but this is not required.
* The FOR field MAY contain a list of <path> entries when
multiple RCPT commands have been given.
An Internet mail program MUST NOT change a Received: line that
was previously added to the message header.
DISCUSSION:
Including both the source host and the IP source address
in the Received: line may provide enough information for
tracking illicit mail sources and eliminate a need to
explicitly verify the HELO parameter.
Received: lines are primarily intended for humans tracing
mail routes, primarily of diagnosis of faults. See also
the discussion under 5.3.7.
When the receiver-SMTP makes "final delivery" of a message,
then it MUST pass the MAIL FROM: address from the SMTP envelope
with the message, for use if an error notification message must
be sent later (see Section 5.3.3). There is an analogous
requirement when gatewaying from the Internet into a different
mail environment; see Section 5.3.7.
DISCUSSION:
Note that the final reply to the DATA command depends only
upon the successful transfer and storage of the message.
Any problem with the destination address(es) must either
(1) have been reported in an SMTP error reply to the RCPT
command(s), or (2) be reported in a later error message
mailed to the originator.
IMPLEMENTATION:
The MAIL FROM: information may be passed as a parameter or
in a Return-Path: line inserted at the beginning of the
message.
5.2.9 Command Syntax: RFC-821 Section 4.1.2
The syntax shown in RFC-821 for the MAIL FROM: command omits
the case of an empty path: "MAIL FROM: <>" (see RFC-821 Page
15). An empty reverse path MUST be supported.
5.2.10 SMTP Replies: RFC-821 Section 4.2
A receiver-SMTP SHOULD send only the reply codes listed in
section 4.2.2 of RFC-821 or in this document. A receiver-SMTP
SHOULD use the text shown in examples in RFC-821 whenever
appropriate.
A sender-SMTP MUST determine its actions only by the reply
code, not by the text (except for 251 and 551 replies); any
text, including no text at all, must be acceptable. The space
(blank) following the reply code is considered part of the
text. Whenever possible, a sender-SMTP SHOULD test only the
first digit of the reply code, as specified in Appendix E of
RFC-821.
DISCUSSION:
Interoperability problems have arisen with SMTP systems
using reply codes that are not listed explicitly in RFC-
821 Section 4.3 but are legal according to the theory of
reply codes explained in Appendix E.
5.2.11 Transparency: RFC-821 Section 4.5.2
Implementors MUST be sure that their mail systems always add
and delete periods to ensure message transparency.
5.2.12 WKS Use in MX Processing: RFC-974, p. 5
RFC-974 [SMTP:3] recommended that the domain system be queried
for WKS ("Well-Known Service") records, to verify that each
proposed mail target does support SMTP. Later experience has
shown that WKS is not widely supported, so the WKS step in MX
processing SHOULD NOT be used.
The following are notes on RFC-822, organized by section of that
document.
5.2.13 RFC-822 Message Specification: RFC-822 Section 4
The syntax shown for the Return-path line omits the possibility
of a null return path, which is used to prevent looping of
error notifications (see Section 5.3.3). The complete syntax
is:
return = "Return-path" ":" route-addr
/ "Return-path" ":" "<" ">"
The set of optional header fields is hereby expanded to include
the Content-Type field defined in RFC-1049 [SMTP:7]. This
field "allows mail reading systems to automatically identify
the type of a structured message body and to process it for
display accordingly". [SMTP:7] A User Agent MAY support this
field.
5.2.14 RFC-822 Date and Time Specification: RFC-822 Section 5
The syntax for the date is hereby changed to:
date = 1*2DIGIT month 2*4DIGIT
All mail software SHOULD use 4-digit years in dates, to ease
the transition to the next century.
There is a strong trend towards the use of numeric timezone
indicators, and implementations SHOULD use numeric timezones
instead of timezone names. However, all implementations MUST
accept either notation. If timezone names are used, they MUST
be exactly as defined in RFC-822.
The military time zones are specified incorrectly in RFC-822:
they count the wrong way from UT (the signs are reversed). As
a result, military time zones in RFC-822 headers carry no
information.
Finally, note that there is a typo in the definition of "zone"
in the syntax summary of appendix D; the correct definition
occurs in Section 3 of RFC-822.
5.2.15 RFC-822 Syntax Change: RFC-822 Section 6.1
The syntactic definition of "mailbox" in RFC-822 is hereby
changed to:
mailbox = addr-spec ; simple address
/ [phrase] route-addr ; name & addr-spec
That is, the phrase preceding a route address is now OPTIONAL.
This change makes the following header field legal, for
example:
From: <craig@nnsc.nsf.net>
5.2.16 RFC-822 Local-part: RFC-822 Section 6.2
The basic mailbox address specification has the form: "local-
part@domain". Here "local-part", sometimes called the "left-
hand side" of the address, is domain-dependent.
A host that is forwarding the message but is not the
destination host implied by the right-hand side "domain" MUST
NOT interpret or modify the "local-part" of the address.
When mail is to be gatewayed from the Internet mail environment
into a foreign mail environment (see Section 5.3.7), routing
information for that foreign environment MAY be embedded within
the "local-part" of the address. The gateway will then
interpret this local part appropriately for the foreign mail
environment.
DISCUSSION:
Although source routes are discouraged within the Internet
(see Section 5.2.6), there are non-Internet mail
environments whose delivery mechanisms do depend upon
source routes. Source routes for extra-Internet
environments can generally be buried in the "local-part"
of the address (see Section 5.2.16) while mail traverses
the Internet. When the mail reaches the appropriate
Internet mail gateway, the gateway will interpret the
local-part and build the necessary address or route for
the target mail environment.
For example, an Internet host might send mail to:
"a!b!c!user@gateway-domain". The complex local part
"a!b!c!user" would be uninterpreted within the Internet
domain, but could be parsed and understood by the
specified mail gateway.
An embedded source route is sometimes encoded in the
"local-part" using "%" as a right-binding routing
operator. For example, in:
user%domain%relay3%relay2@relay1
the "%" convention implies that the mail is to be routed
from "relay1" through "relay2", "relay3", and finally to
"user" at "domain". This is commonly known as the "%-
hack". It is suggested that "%" have lower precedence
than any other routing operator (e.g., "!") hidden in the
local-part; for example, "a!b%c" would be interpreted as
"(a!b)%c".
Only the target host (in this case, "relay1") is permitted
to analyze the local-part "user%domain%relay3%relay2".
5.2.17 Domain Literals: RFC-822 Section 6.2.3
A mailer MUST be able to accept and parse an Internet domain
literal whose content ("dtext"; see RFC-822) is a dotted-
decimal host address. This satisfies the requirement of
Section 2.1 for the case of mail.
An SMTP MUST accept and recognize a domain literal for any of
its own IP addresses.
5.2.18 Common Address Formatting Errors: RFC-822 Section 6.1 Errors in formatting or parsing 822 addresses are unfortunately common. This section mentions only the most common errors. A User Agent MUST accept all valid RFC-822 address formats, and MUST NOT generate illegal address syntax. o A common error is to leave out the semicolon after a group identifier. o Some systems fail to fully-qualify domain names in messages they generate. The right-hand side of an "@" sign in a header address field MUST be a fully-qualified domain name. For example, some systems fail to fully-qualify the From: address; this prevents a "reply" command in the user interface from automatically constructing a return address. DISCUSSION: Although RFC-822 allows the local use of abbreviated domain names within a domain, the application of RFC-822 in Internet mail does not allow this. The intent is that an Internet host must not send an SMTP message header containing an abbreviated domain name in an address field. This allows the address fields of the header to be passed without alteration across the Internet, as required in Section 5.2.6. o Some systems mis-parse multiple-hop explicit source routes such as: @relay1,@relay2,@relay3:user@domain. o Some systems over-qualify domain names by adding a trailing dot to some or all domain names in addresses or message-ids. This violates RFC-822 syntax. 5.2.19 Explicit Source Routes: RFC-822 Section 6.2.7 Internet host software SHOULD NOT create an RFC-822 header containing an address with an explicit source route, but MUST accept such headers for compatibility with earlier systems. DISCUSSION:
In an understatement, RFC-822 says "The use of explicit
source routing is discouraged". Many hosts implemented
RFC-822 source routes incorrectly, so the syntax cannot be
used unambiguously in practice. Many users feel the
syntax is ugly. Explicit source routes are not needed in
the mail envelope for delivery; see Section 5.2.6. For
all these reasons, explicit source routes using the RFC-
822 notations are not to be used in Internet mail headers.
As stated in Section 5.2.16, it is necessary to allow an
explicit source route to be buried in the local-part of an
address, e.g., using the "%-hack", in order to allow mail
to be gatewayed into another environment in which explicit
source routing is necessary. The vigilant will observe
that there is no way for a User Agent to detect and
prevent the use of such implicit source routing when the
destination is within the Internet. We can only
discourage source routing of any kind within the Internet,
as unnecessary and undesirable.
5.3 SPECIFIC ISSUES
5.3.1 SMTP Queueing Strategies
The common structure of a host SMTP implementation includes
user mailboxes, one or more areas for queueing messages in
transit, and one or more daemon processes for sending and
receiving mail. The exact structure will vary depending on the
needs of the users on the host and the number and size of
mailing lists supported by the host. We describe several
optimizations that have proved helpful, particularly for
mailers supporting high traffic levels.
Any queueing strategy MUST include:
o Timeouts on all activities. See Section 5.3.2.
o Never sending error messages in response to error
messages.
5.3.1.1 Sending Strategy
The general model of a sender-SMTP is one or more processes
that periodically attempt to transmit outgoing mail. In a
typical system, the program that composes a message has some
method for requesting immediate attention for a new piece of
outgoing mail, while mail that cannot be transmitted
immediately MUST be queued and periodically retried by the
sender. A mail queue entry will include not only the
message itself but also the envelope information.
The sender MUST delay retrying a particular destination
after one attempt has failed. In general, the retry
interval SHOULD be at least 30 minutes; however, more
sophisticated and variable strategies will be beneficial
when the sender-SMTP can determine the reason for non-
delivery.
Retries continue until the message is transmitted or the
sender gives up; the give-up time generally needs to be at
least 4-5 days. The parameters to the retry algorithm MUST
be configurable.
A sender SHOULD keep a list of hosts it cannot reach and
corresponding timeouts, rather than just retrying queued
mail items.
DISCUSSION:
Experience suggests that failures are typically
transient (the target system has crashed), favoring a
policy of two connection attempts in the first hour the
message is in the queue, and then backing off to once
every two or three hours.
The sender-SMTP can shorten the queueing delay by
cooperation with the receiver-SMTP. In particular, if
mail is received from a particular address, it is good
evidence that any mail queued for that host can now be
sent.
The strategy may be further modified as a result of
multiple addresses per host (see Section 5.3.4), to
optimize delivery time vs. resource usage.
A sender-SMTP may have a large queue of messages for
each unavailable destination host, and if it retried
all these messages in every retry cycle, there would be
excessive Internet overhead and the daemon would be
blocked for a long period. Note that an SMTP can
generally determine that a delivery attempt has failed
only after a timeout of a minute or more; a one minute
timeout per connection will result in a very large
delay if it is repeated for dozens or even hundreds of
queued messages.
When the same message is to be delivered to several users on
the same host, only one copy of the message SHOULD be
transmitted. That is, the sender-SMTP should use the
command sequence: RCPT, RCPT,... RCPT, DATA instead of the
sequence: RCPT, DATA, RCPT, DATA,... RCPT, DATA.
Implementation of this efficiency feature is strongly urged.
Similarly, the sender-SMTP MAY support multiple concurrent
outgoing mail transactions to achieve timely delivery.
However, some limit SHOULD be imposed to protect the host
from devoting all its resources to mail.
The use of the different addresses of a multihomed host is
discussed below.
5.3.1.2 Receiving strategy
The receiver-SMTP SHOULD attempt to keep a pending listen on
the SMTP port at all times. This will require the support
of multiple incoming TCP connections for SMTP. Some limit
MAY be imposed.
IMPLEMENTATION:
When the receiver-SMTP receives mail from a particular
host address, it could notify the sender-SMTP to retry
any mail pending for that host address.
5.3.2 Timeouts in SMTP
There are two approaches to timeouts in the sender-SMTP: (a)
limit the time for each SMTP command separately, or (b) limit
the time for the entire SMTP dialogue for a single mail
message. A sender-SMTP SHOULD use option (a), per-command
timeouts. Timeouts SHOULD be easily reconfigurable, preferably
without recompiling the SMTP code.
DISCUSSION:
Timeouts are an essential feature of an SMTP
implementation. If the timeouts are too long (or worse,
there are no timeouts), Internet communication failures or
software bugs in receiver-SMTP programs can tie up SMTP
processes indefinitely. If the timeouts are too short,
resources will be wasted with attempts that time out part
way through message delivery.
If option (b) is used, the timeout has to be very large,
e.g., an hour, to allow time to expand very large mailing
lists. The timeout may also need to increase linearly
with the size of the message, to account for the time to
transmit a very large message. A large fixed timeout
leads to two problems: a failure can still tie up the
sender for a very long time, and very large messages may
still spuriously time out (which is a wasteful failure!).
Using the recommended option (a), a timer is set for each
SMTP command and for each buffer of the data transfer.
The latter means that the overall timeout is inherently
proportional to the size of the message.
Based on extensive experience with busy mail-relay hosts, the
minimum per-command timeout values SHOULD be as follows:
o Initial 220 Message: 5 minutes
A Sender-SMTP process needs to distinguish between a
failed TCP connection and a delay in receiving the initial
220 greeting message. Many receiver-SMTPs will accept a
TCP connection but delay delivery of the 220 message until
their system load will permit more mail to be processed.
o MAIL Command: 5 minutes
o RCPT Command: 5 minutes
A longer timeout would be required if processing of
mailing lists and aliases were not deferred until after
the message was accepted.
o DATA Initiation: 2 minutes
This is while awaiting the "354 Start Input" reply to a
DATA command.
o Data Block: 3 minutes
This is while awaiting the completion of each TCP SEND
call transmitting a chunk of data.
o DATA Termination: 10 minutes.
This is while awaiting the "250 OK" reply. When the
receiver gets the final period terminating the message
data, it typically performs processing to deliver the
message to a user mailbox. A spurious timeout at this
point would be very wasteful, since the message has been
successfully sent.
A receiver-SMTP SHOULD have a timeout of at least 5 minutes
while it is awaiting the next command from the sender.
5.3.3 Reliable Mail Receipt
When the receiver-SMTP accepts a piece of mail (by sending a
"250 OK" message in response to DATA), it is accepting
responsibility for delivering or relaying the message. It must
take this responsibility seriously, i.e., it MUST NOT lose the
message for frivolous reasons, e.g., because the host later
crashes or because of a predictable resource shortage.
If there is a delivery failure after acceptance of a message,
the receiver-SMTP MUST formulate and mail a notification
message. This notification MUST be sent using a null ("<>")
reverse path in the envelope; see Section 3.6 of RFC-821. The
recipient of this notification SHOULD be the address from the
envelope return path (or the Return-Path: line). However, if
this address is null ("<>"), the receiver-SMTP MUST NOT send a
notification. If the address is an explicit source route, it
SHOULD be stripped down to its final hop.
DISCUSSION:
For example, suppose that an error notification must be
sent for a message that arrived with:
"MAIL FROM:<@a,@b:user@d>". The notification message
should be sent to: "RCPT TO:<user@d>".
Some delivery failures after the message is accepted by
SMTP will be unavoidable. For example, it may be
impossible for the receiver-SMTP to validate all the
delivery addresses in RCPT command(s) due to a "soft"
domain system error or because the target is a mailing
list (see earlier discussion of RCPT).
To avoid receiving duplicate messages as the result of
timeouts, a receiver-SMTP MUST seek to minimize the time
required to respond to the final "." that ends a message
transfer. See RFC-1047 [SMTP:4] for a discussion of this
problem.
5.3.4 Reliable Mail Transmission
To transmit a message, a sender-SMTP determines the IP address
of the target host from the destination address in the
envelope. Specifically, it maps the string to the right of the
"@" sign into an IP address. This mapping or the transfer
itself may fail with a soft error, in which case the sender-
SMTP will requeue the outgoing mail for a later retry, as
required in Section 5.3.1.1.
When it succeeds, the mapping can result in a list of
alternative delivery addresses rather than a single address,
because of (a) multiple MX records, (b) multihoming, or both.
To provide reliable mail transmission, the sender-SMTP MUST be
able to try (and retry) each of the addresses in this list in
order, until a delivery attempt succeeds. However, there MAY
also be a configurable limit on the number of alternate
addresses that can be tried. In any case, a host SHOULD try at
least two addresses.
The following information is to be used to rank the host
addresses:
(1) Multiple MX Records -- these contain a preference
indication that should be used in sorting. If there are
multiple destinations with the same preference and there
is no clear reason to favor one (e.g., by address
preference), then the sender-SMTP SHOULD pick one at
random to spread the load across multiple mail exchanges
for a specific organization; note that this is a
refinement of the procedure in [DNS:3].
(2) Multihomed host -- The destination host (perhaps taken
from the preferred MX record) may be multihomed, in which
case the domain name resolver will return a list of
alternative IP addresses. It is the responsibility of the
domain name resolver interface (see Section 6.1.3.4 below)
to have ordered this list by decreasing preference, and
SMTP MUST try them in the order presented.
DISCUSSION:
Although the capability to try multiple alternative
addresses is required, there may be circumstances where
specific installations want to limit or disable the use of
alternative addresses. The question of whether a sender
should attempt retries using the different addresses of a
multihomed host has been controversial. The main argument
for using the multiple addresses is that it maximizes the
probability of timely delivery, and indeed sometimes the
probability of any delivery; the counter argument is that
it may result in unnecessary resource use.
Note that resource use is also strongly determined by the
sending strategy discussed in Section 5.3.1.
5.3.5 Domain Name Support
SMTP implementations MUST use the mechanism defined in Section
6.1 for mapping between domain names and IP addresses. This
means that every Internet SMTP MUST include support for the
Internet DNS.
In particular, a sender-SMTP MUST support the MX record scheme
[SMTP:3]. See also Section 7.4 of [DNS:2] for information on
domain name support for SMTP.
5.3.6 Mailing Lists and Aliases
An SMTP-capable host SHOULD support both the alias and the list
form of address expansion for multiple delivery. When a
message is delivered or forwarded to each address of an
expanded list form, the return address in the envelope
("MAIL FROM:") MUST be changed to be the address of a person
who administers the list, but the message header MUST be left
unchanged; in particular, the "From" field of the message is
unaffected.
DISCUSSION:
An important mail facility is a mechanism for multi-
destination delivery of a single message, by transforming
or "expanding" a pseudo-mailbox address into a list of
destination mailbox addresses. When a message is sent to
such a pseudo-mailbox (sometimes called an "exploder"),
copies are forwarded or redistributed to each mailbox in
the expanded list. We classify such a pseudo-mailbox as
an "alias" or a "list", depending upon the expansion
rules:
(a) Alias
To expand an alias, the recipient mailer simply
replaces the pseudo-mailbox address in the envelope
with each of the expanded addresses in turn; the rest
of the envelope and the message body are left
unchanged. The message is then delivered or
forwarded to each expanded address.
(b) List
A mailing list may be said to operate by
"redistribution" rather than by "forwarding". To
expand a list, the recipient mailer replaces the
pseudo-mailbox address in the envelope with each of
the expanded addresses in turn. The return address in
the envelope is changed so that all error messages
generated by the final deliveries will be returned to
a list administrator, not to the message originator,
who generally has no control over the contents of the
list and will typically find error messages annoying.
5.3.7 Mail Gatewaying
Gatewaying mail between different mail environments, i.e.,
different mail formats and protocols, is complex and does not
easily yield to standardization. See for example [SMTP:5a],
[SMTP:5b]. However, some general requirements may be given for
a gateway between the Internet and another mail environment.
(A) Header fields MAY be rewritten when necessary as messages
are gatewayed across mail environment boundaries.
DISCUSSION:
This may involve interpreting the local-part of the
destination address, as suggested in Section 5.2.16.
The other mail systems gatewayed to the Internet
generally use a subset of RFC-822 headers, but some
of them do not have an equivalent to the SMTP
envelope. Therefore, when a message leaves the
Internet environment, it may be necessary to fold the
SMTP envelope information into the message header. A
possible solution would be to create new header
fields to carry the envelope information (e.g., "X-
SMTP-MAIL:" and "X-SMTP-RCPT:"); however, this would
require changes in mail programs in the foreign
environment.
(B) When forwarding a message into or out of the Internet
environment, a gateway MUST prepend a Received: line, but
it MUST NOT alter in any way a Received: line that is
already in the header.
DISCUSSION:
This requirement is a subset of the general
"Received:" line requirement of Section 5.2.8; it is
restated here for emphasis.
Received: fields of messages originating from other
environments may not conform exactly to RFC822.
However, the most important use of Received: lines is
for debugging mail faults, and this debugging can be
severely hampered by well-meaning gateways that try
to "fix" a Received: line.
The gateway is strongly encouraged to indicate the
environment and protocol in the "via" clauses of
Received field(s) that it supplies.
(C) From the Internet side, the gateway SHOULD accept all
valid address formats in SMTP commands and in RFC-822
headers, and all valid RFC-822 messages. Although a
gateway must accept an RFC-822 explicit source route
("@...:" format) in either the RFC-822 header or in the
envelope, it MAY or may not act on the source route; see
Sections 5.2.6 and 5.2.19.
DISCUSSION:
It is often tempting to restrict the range of
addresses accepted at the mail gateway to simplify
the translation into addresses for the remote
environment. This practice is based on the
assumption that mail users have control over the
addresses their mailers send to the mail gateway. In
practice, however, users have little control over the
addresses that are finally sent; their mailers are
free to change addresses into any legal RFC-822
format.
(D) The gateway MUST ensure that all header fields of a
message that it forwards into the Internet meet the
requirements for Internet mail. In particular, all
addresses in "From:", "To:", "Cc:", etc., fields must be
transformed (if necessary) to satisfy RFC-822 syntax, and
they must be effective and useful for sending replies.
(E) The translation algorithm used to convert mail from the
Internet protocols to another environment's protocol
SHOULD try to ensure that error messages from the foreign
mail environment are delivered to the return path from the
SMTP envelope, not to the sender listed in the "From:"
field of the RFC-822 message.
DISCUSSION:
Internet mail lists usually place the address of the
mail list maintainer in the envelope but leave the
original message header intact (with the "From:"
field containing the original sender). This yields
the behavior the average recipient expects: a reply
to the header gets sent to the original sender, not
to a mail list maintainer; however, errors get sent
to the maintainer (who can fix the problem) and not
the sender (who probably cannot).
(F) Similarly, when forwarding a message from another
environment into the Internet, the gateway SHOULD set the
envelope return path in accordance with an error message
return address, if any, supplied by the foreign
environment.
5.3.8 Maximum Message Size
Mailer software MUST be able to send and receive messages of at
least 64K bytes in length (including header), and a much larger
maximum size is highly desirable.
DISCUSSION:
Although SMTP does not define the maximum size of a
message, many systems impose implementation limits.
The current de facto minimum limit in the Internet is 64K
bytes. However, electronic mail is used for a variety of
purposes that create much larger messages. For example,
mail is often used instead of FTP for transmitting ASCII
files, and in particular to transmit entire documents. As
a result, messages can be 1 megabyte or even larger. We
note that the present document together with its lower-
layer companion contains 0.5 megabytes.
5.4 SMTP REQUIREMENTS SUMMARY | | | | |S| | | | | | |H| |F | | | | |O|M|o | | |S| |U|U|o | | |H| |L|S|t | |M|O| |D|T|n | |U|U|M| | |o | |S|L|A|N|N|t | |T|D|Y|O|O|t FEATURE |SECTION | | | |T|T|e -----------------------------------------------|----------|-|-|-|-|-|-- | | | | | | | RECEIVER-SMTP: | | | | | | | Implement VRFY |5.2.3 |x| | | | | Implement EXPN |5.2.3 | |x| | | | EXPN, VRFY configurable |5.2.3 | | |x| | | Implement SEND, SOML, SAML |5.2.4 | | |x| | | Verify HELO parameter |5.2.5 | | |x| | | Refuse message with bad HELO |5.2.5 | | | | |x| Accept explicit src-route syntax in env. |5.2.6 |x| | | | | Support "postmaster" |5.2.7 |x| | | | | Process RCPT when received (except lists) |5.2.7 | | |x| | | Long delay of RCPT responses |5.2.7 | | | | |x| | | | | | | | Add Received: line |5.2.8 |x| | | | | Received: line include domain literal |5.2.8 | |x| | | | Change previous Received: line |5.2.8 | | | | |x| Pass Return-Path info (final deliv/gwy) |5.2.8 |x| | | | | Support empty reverse path |5.2.9 |x| | | | | Send only official reply codes |5.2.10 | |x| | | | Send text from RFC-821 when appropriate |5.2.10 | |x| | | | Delete "." for transparency |5.2.11 |x| | | | | Accept and recognize self domain literal(s) |5.2.17 |x| | | | | | | | | | | | Error message about error message |5.3.1 | | | | |x| Keep pending listen on SMTP port |5.3.1.2 | |x| | | | Provide limit on recv concurrency |5.3.1.2 | | |x| | | Wait at least 5 mins for next sender cmd |5.3.2 | |x| | | | Avoidable delivery failure after "250 OK" |5.3.3 | | | | |x| Send error notification msg after accept |5.3.3 |x| | | | | Send using null return path |5.3.3 |x| | | | | Send to envelope return path |5.3.3 | |x| | | | Send to null address |5.3.3 | | | | |x| Strip off explicit src route |5.3.3 | |x| | | | Minimize acceptance delay (RFC-1047) |5.3.3 |x| | | | | -----------------------------------------------|----------|-|-|-|-|-|--
| | | | | | |
SENDER-SMTP: | | | | | | |
Canonicalized domain names in MAIL, RCPT |5.2.2 |x| | | | |
Implement SEND, SOML, SAML |5.2.4 | | |x| | |
Send valid principal host name in HELO |5.2.5 |x| | | | |
Send explicit source route in RCPT TO: |5.2.6 | | | |x| |
Use only reply code to determine action |5.2.10 |x| | | | |
Use only high digit of reply code when poss. |5.2.10 | |x| | | |
Add "." for transparency |5.2.11 |x| | | | |
| | | | | | |
Retry messages after soft failure |5.3.1.1 |x| | | | |
Delay before retry |5.3.1.1 |x| | | | |
Configurable retry parameters |5.3.1.1 |x| | | | |
Retry once per each queued dest host |5.3.1.1 | |x| | | |
Multiple RCPT's for same DATA |5.3.1.1 | |x| | | |
Support multiple concurrent transactions |5.3.1.1 | | |x| | |
Provide limit on concurrency |5.3.1.1 | |x| | | |
| | | | | | |
Timeouts on all activities |5.3.1 |x| | | | |
Per-command timeouts |5.3.2 | |x| | | |
Timeouts easily reconfigurable |5.3.2 | |x| | | |
Recommended times |5.3.2 | |x| | | |
Try alternate addr's in order |5.3.4 |x| | | | |
Configurable limit on alternate tries |5.3.4 | | |x| | |
Try at least two alternates |5.3.4 | |x| | | |
Load-split across equal MX alternates |5.3.4 | |x| | | |
Use the Domain Name System |5.3.5 |x| | | | |
Support MX records |5.3.5 |x| | | | |
Use WKS records in MX processing |5.2.12 | | | |x| |
-----------------------------------------------|----------|-|-|-|-|-|--
| | | | | | |
MAIL FORWARDING: | | | | | | |
Alter existing header field(s) |5.2.6 | | | |x| |
Implement relay function: 821/section 3.6 |5.2.6 | | |x| | |
If not, deliver to RHS domain |5.2.6 | |x| | | |
Interpret 'local-part' of addr |5.2.16 | | | | |x|
| | | | | | |
MAILING LISTS AND ALIASES | | | | | | |
Support both |5.3.6 | |x| | | |
Report mail list error to local admin. |5.3.6 |x| | | | |
| | | | | | |
MAIL GATEWAYS: | | | | | | |
Embed foreign mail route in local-part |5.2.16 | | |x| | |
Rewrite header fields when necessary |5.3.7 | | |x| | |
Prepend Received: line |5.3.7 |x| | | | |
Change existing Received: line |5.3.7 | | | | |x|
Accept full RFC-822 on Internet side |5.3.7 | |x| | | |
Act on RFC-822 explicit source route |5.3.7 | | |x| | |
Send only valid RFC-822 on Internet side |5.3.7 |x| | | | | Deliver error msgs to envelope addr |5.3.7 | |x| | | | Set env return path from err return addr |5.3.7 | |x| | | | | | | | | | | USER AGENT -- RFC-822 | | | | | | | Allow user to enter <route> address |5.2.6 | | | |x| | Support RFC-1049 Content Type field |5.2.13 | | |x| | | Use 4-digit years |5.2.14 | |x| | | | Generate numeric timezones |5.2.14 | |x| | | | Accept all timezones |5.2.14 |x| | | | | Use non-num timezones from RFC-822 |5.2.14 |x| | | | | Omit phrase before route-addr |5.2.15 | | |x| | | Accept and parse dot.dec. domain literals |5.2.17 |x| | | | | Accept all RFC-822 address formats |5.2.18 |x| | | | | Generate invalid RFC-822 address format |5.2.18 | | | | |x| Fully-qualified domain names in header |5.2.18 |x| | | | | Create explicit src route in header |5.2.19 | | | |x| | Accept explicit src route in header |5.2.19 |x| | | | | | | | | | | | Send/recv at least 64KB messages |5.3.8 |x| | | | |
(next page on part 4)