Tech-invite3GPPspaceIETF RFCsSIP
in Index   Prev   Next

RFC 5537

Netnews Architecture and Protocols

Pages: 48
Proposed Standard
Obsoletes:  10361849
Updated by:  8315
Part 1 of 2 – Pages 1 to 23
None   None   Next

Top   ToC   RFC5537 - Page 1
Network Working Group                                    R. Allbery, Ed.
Request for Comments: 5537                           Stanford University
Obsoletes: 1036                                               C. Lindsey
Category: Standards Track                                  November 2009

                   Netnews Architecture and Protocols


This document defines the architecture of Netnews systems and specifies the correct manipulation and interpretation of Netnews articles by software that originates, distributes, stores, and displays them. It also specifies the requirements that must be met by any protocol used to transport and serve Netnews articles. Status of This Memo This document specifies an Internet standards track protocol for the Internet community, and requests discussion and suggestions for improvements. Please refer to the current edition of the "Internet Official Protocol Standards" (STD 1) for the standardization state and status of this protocol. Distribution of this memo is unlimited. Copyright Notice Copyright (c) 2009 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents ( in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the BSD License.

Table of Contents

1. Introduction ....................................................3 1.1. Basic Concepts .............................................3 1.2. Scope ......................................................3 1.3. Requirements Notation ......................................3 1.4. Syntax Notation ............................................3 1.5. Definitions ................................................4 2. Transport .......................................................5
Top   ToC   RFC5537 - Page 2
   3. Duties of Agents ................................................6
      3.1. General Principles .........................................6
      3.2. The Path Header Field ......................................7
           3.2.1. Constructing the Path Header Field ..................8
           3.2.2. Path Header Field Example ...........................9
      3.3. Article History and Duplicate Suppression .................10
      3.4. Duties of a Posting Agent .................................11
           3.4.1. Proto-Articles .....................................12
           3.4.2. Multiple Injection of Articles .....................13
           3.4.3. Followups ..........................................14
           3.4.4. Construction of the References Header Field ........15
      3.5. Duties of an Injecting Agent ..............................15
           3.5.1. Forwarding Messages to a Moderator .................18
      3.6. Duties of a Relaying Agent ................................19
      3.7. Duties of a Serving Agent .................................21
      3.8. Duties of a Reading Agent .................................22
      3.9. Duties of a Moderator .....................................22
      3.10. Duties of a Gateway ......................................24
           3.10.1. Duties of an Outgoing Gateway .....................25
           3.10.2. Duties of an Incoming Gateway .....................25
           3.10.3. Original-Sender Header Field ......................27
           3.10.4. Gateway Example ...................................28
   4. Media Types ....................................................29
      4.1. application/news-transmission .............................30
      4.2. application/news-groupinfo ................................31
      4.3. application/news-checkgroups ..............................33
   5. Control Messages ...............................................35
      5.1. Authentication and Authorization ..........................35
      5.2. Group Control Messages ....................................36
           5.2.1. The newgroup Control Message .......................36
         newgroup Control Message Example ..........37
           5.2.2. The rmgroup Control Message ........................38
           5.2.3. The checkgroups Control Message ....................38
      5.3. The cancel Control Message ................................40
      5.4. The Supersedes Header Field ...............................40
      5.5. The ihave and sendme Control Messages .....................41
      5.6. Obsolete Control Messages .................................42
   6. Security Considerations ........................................42
      6.1. Compromise of System Integrity ............................42
      6.2. Denial of Service .........................................44
      6.3. Leakage ...................................................44
   7. IANA Considerations ............................................45
   8. References .....................................................45
      8.1. Normative References ......................................45
      8.2. Informative References ....................................46
   Appendix A.  Changes to the Existing Protocols ....................47
   Appendix B.  Acknowledgements .....................................48
Top   ToC   RFC5537 - Page 3

1. Introduction

1.1. Basic Concepts

"Netnews" is a set of protocols for generating, storing, and retrieving news "articles" whose format is defined in [RFC5536], and for exchanging them amongst a readership that is potentially widely distributed. It is organized around "newsgroups", with the expectation that each reader will be able to see all articles posted to each newsgroup in which he participates. These protocols most commonly use a flooding algorithm that propagates copies throughout a network of participating servers. Typically, only one copy is stored per server, and each server makes it available on demand to readers able to access that server. "Usenet" is a particular worldwide, publicly accessible network based on the Netnews protocols. It is only one such possible network; there are deployments of the Netnews protocols other than Usenet (such as ones internal to particular organizations). This document discusses the more general Netnews architecture and protocols.

1.2. Scope

This document defines the architecture of Netnews systems and specifies the correct manipulation and interpretation of Netnews articles by software that originates, distributes, stores, and displays them. It addresses protocol issues that are independent of transport protocols such as the Network News Transfer Protocol (NNTP) [RFC3977], and specifies the requirements Netnews places on those underlying transport protocols. It also specifies the handling of control messages. The format and syntax of Netnews articles are specified in [RFC5536], which should be read in conjunction with this document.

1.3. Requirements Notation

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in [RFC2119].

1.4. Syntax Notation

Syntax defined in this document uses the Augmented Backus-Naur Form (ABNF) notation (including the Core Rules) defined in [RFC5234] and constructs defined in [RFC5536] and [RFC5322].
Top   ToC   RFC5537 - Page 4
   The ABNF rules defined elsewhere and used in this document are:

         CRLF                = <see [RFC5234] Appendix B.1>
         DIGIT               = <see [RFC5234] Appendix B.1>
         HTAB                = <see [RFC5234] Appendix B.1>
         SP                  = <see [RFC5234] Appendix B.1>
         WSP                 = <see [RFC5234] Appendix B.1>
         VCHAR               = <see [RFC5234] Appendix B.1>

         argument            = <see [RFC5536] Section 3.2.3>
         article-locator     = <see [RFC5536] Section 3.2.14>
         component           = <see [RFC5536] Section 3.1.4>
         control-command     = <see [RFC5536] Section 3.2.3>
         diag-keyword        = <see [RFC5536] Section 3.1.5>
         diag-match          = <see [RFC5536] Section 3.1.5>
         diag-other          = <see [RFC5536] Section 3.1.5>
         dist-name           = <see [RFC5536] Section 3.2.4>
         msg-id              = <see [RFC5536] Section 3.1.3>
         newsgroup-name      = <see [RFC5536] Section 3.1.4>
         path-diagnostic     = <see [RFC5536] Section 3.1.5>
         path-identity       = <see [RFC5536] Section 3.1.5>
         path-nodot          = <see [RFC5536] Section 3.1.5>
         tail-entry          = <see [RFC5536] Section 3.1.5>
         verb                = <see [RFC5536] Section 3.2.3>

         display-name        = <see [RFC5322] Section 3.4>
         local-part          = <see [RFC5322] Section 3.4.1>
         mailbox             = <see [RFC5322] Section 3.4>

1.5. Definitions

Any term used in this document that is defined in Section 1.5 of [RFC5536] is used with the definition given there. In addition, the following terms will be used: A "hierarchy" is the set of all newsgroups whose names share a first <component> (as defined in Section 3.1.4 of [RFC5536]). A "sub- hierarchy" is the set of all newsgroups whose names share several initial components. A "news server" is further distinguished into the roles of "injecting agent", "relaying agent", and "serving agent". An "injecting agent" accepts a proto-article with the goal of distributing it to relaying and serving agents and hence to readers. A "relaying agent" accepts articles from other relaying agents or injecting agents and distributes them to other relaying agents or serving agents. A "serving agent" receives an article from a relaying agent or injecting agent and makes it available to readers.
Top   ToC   RFC5537 - Page 5
   A "user agent" is further distinguished into the roles of "posting
   agent" and "reading agent".  A "posting agent" is software that
   assists in the preparation of a proto-article and then passes it to
   an injecting agent.  A "reading agent" is software that retrieves
   articles from a serving agent for presentation to a reader.

   "Injecting" an article is the processing of a proto-article by an
   injecting agent.  Normally, this action is done once and only once
   for a given article.  "Multiple injection" is passing the same
   article to multiple injecting agents, either serially or in parallel,
   by one or several posting agents.

   A "gateway" is software that receives news articles and converts them
   to messages of some other kind (such as [RFC5322] mail messages),
   receives messages of some other kind and converts them to news
   articles, or conveys articles between two separate Netnews networks.

2. Transport

The exact means used to transmit articles from one agent to another is not specified. NNTP [RFC3977] is the most common transport mechanism for Netnews networks. Other methods in use include the Unix-to-Unix Copy Protocol [UUCP] (extensively used in the early days of Usenet) and physically delivered magnetic and optical media. Any mechanism may be used in conjunction with this protocol provided that it can meet the requirements specified here. Transports for Netnews articles MUST treat news articles as uninterpreted sequences of octets, excluding the values %d00 (which may not occur in Netnews articles), %d13, and %d10 (which MUST only appear in Netnews articles as a pair in that order and which, together, denote a line separator). These octets are the US-ASCII [ASCII] characters NUL, CR, and LF respectively. NOTE: This corresponds to the range of octets permitted in MIME 8bit data [RFC2045]. Transports for Netnews are not required to support transmission of MIME binary data. In particular, transports MUST convey all header fields unmodified (including header fields within message/rfc822 objects in article bodies), even if they contain octets in the range of 128 to 255. Furthermore, transports for relaying and serving agents MUST, and transports for other agents SHOULD, convey lines even if they exceed 998 characters in length, especially in article bodies. (This requirement is stricter than MIME 8bit data.) These requirements include the transport paths between posting agents, injecting agents, serving agents, and reading agents.
Top   ToC   RFC5537 - Page 6

3. Duties of Agents

The following section specifies the duties of the agents involved in the creation, relaying, and serving of Netnews articles. This protocol is described by following the life of a typical Usenet article: it is prepared by a posting agent, given to an injecting agent, transferred through one or more relaying agents, accepted by a serving agent, and finally retrieved by a reading agent. Articles submitted to moderated groups go through an additional process, which is described separately (see Section 3.5.1 and Step 7 of Section 3.5). Finally, the additional duties and requirements of a gateway are discussed. At each step, each agent has a set of checks and transformations of the article that it is required to perform. These are described as sequences of steps to be followed, but it should be understood that it is the effect of these sequences that is important, and implementations may use any method that produces the same effect. Many news servers combine the functions of injecting agent, relaying agent, and serving agent in a single software package. For the purposes of this specification, such combined agents should conceptually be treated as an injecting agent that sends articles to a serving agent and, optionally, to a relaying agent. The requirements of all three agents MUST still be met when the news server is performing the functions of those agents. On news servers that accept them, control messages may have additional effects than those described below. Those effects are described in Section 5.

3.1. General Principles

There are two important principles that news implementors and administrators need to keep in mind. The first is the well-known Internet Robustness Principle: Be liberal in what you accept, and conservative in what you send. As applied to Netnews, this primarily means that unwanted or non- compliant articles SHOULD be rejected as early as possible, but once they are in general circulation, relaying and serving agents may wish to accept them where possible rather than lose information. Posting agents and injecting agents SHOULD therefore be maximally strict in their application of both this protocol and [RFC5536], and reading agents SHOULD be robust in the presence of violations of the Netnews article format where possible.
Top   ToC   RFC5537 - Page 7
   In the case of Netnews, there is an even more important principle,
   derived from a much older code of practice, the Hippocratic Oath (we
   may thus call this the Hippocratic Principle):

      First, do no harm.

   It is vital to realize that decisions that might be merely suboptimal
   in a smaller context can become devastating mistakes when amplified
   by the actions of thousands of hosts within a few minutes.

   No Netnews agent is ever required to accept any article.  It is
   common for injecting, relaying, and serving agents to reject well-
   formed articles for reasons of local policy (such as not wishing to
   carry a particular newsgroup or attempting to filter out unwanted
   articles).  This document specifies how articles are to be treated if
   they are accepted and specifies some cases where they must be
   rejected, but an agent MAY always reject any article for other
   reasons than those stated here.

   A primary goal of the Netnews protocol is to ensure that all readers
   receiving a particular article (as uniquely identified by the content
   of its Message-ID header field) see the identical article, apart from
   allowable divergence in trace headers and local metadata.
   Accordingly, agents (other than moderators) MUST NOT modify articles
   in ways other than described here.  Unacceptable articles MUST be
   rejected rather than corrected.

3.2. The Path Header Field

All news server components (injecting agents, relaying agents, and serving agents) MUST identify themselves, when processing an article, by prepending their <path-identity> (defined in Section 3.1.5 of [RFC5536]) to the Path header field. Injecting agents MUST also use the same identity in Injection-Info header fields that they add, and serving and relaying agents SHOULD use the same identity in any Xref header fields they add. The <path-identity> used by an agent may be chosen via one of the following methods (in decreasing order of preference): 1. The fully qualified domain name (FQDN) of the system on which the agent is running. 2. A fully qualified domain name (FQDN) within a domain affiliated with the administrators of the agent and guaranteed to be unique by the administrators of that domain. For example, the
Top   ToC   RFC5537 - Page 8
       uniqueness of could be guaranteed by the
       administrator of even if there is no DNS record for itself.

   3.  Some other (arbitrary) name in the form of a <path-nodot>,
       believed to be unique and registered at least with all the other
       news servers to which that relaying agent or injecting agent
       sends articles.  This option SHOULD NOT be used unless the
       earlier options are unavailable or unless the name is of
       longstanding usage.

   Some existing implementations treat <path-identity> as case-
   sensitive, some as case-insensitive.  The <path-identity> therefore
   SHOULD be all lowercase and implementations SHOULD compare identities

3.2.1. Constructing the Path Header Field

If a relaying or serving agent receives an article from an injecting or serving agent that is part of the same news server, it MAY leave the Path header field of the article unchanged. Otherwise, every injecting, relaying, or serving agent that accepts an article MUST update the Path header field as follows. Note that the Path header field content is constructed from right to left by prepending elements. 1. The agent MUST prepend "!" to the Path header field content. 2. An injecting agent SHOULD prepend the <path-diagnostic> "!.POSTED", optionally followed by "." and the FQDN or IP address of the source, to the Path header field content. 3. A relaying or serving agent SHOULD prepend a <path-diagnostic> to the Path header field content, where the <path-diagnostic> is chosen as follows: * If the expected <path-identity> of the source of the article matches the leftmost <path-identity> of the Path header field's content, use "!" (<diag-match>), resulting in two consecutive "!"s. * If the expected <path-identity> of the source of the article does not match, use "!.MISMATCH." followed by the expected <path-identity> of the source or its IP address. * If the relaying or serving agent is not willing or able to check the <path-identity>, use "!.SEEN." followed by the FQDN, IP address, or expected <path-identity> of the source.
Top   ToC   RFC5537 - Page 9
       The "expected <path-identity> of the source of the article" is a
       <path-identity> for the injecting or relaying agent that passed
       the article to this relaying or serving agent, determined by
       properties of the connection via which the article was received
       (for example, an authentication identity or a peer IP address).
       Be aware that [RFC1036] did not include <path-diagnostic>.
       Implementations that predate this specification will add only
       single "!" characters between <path-identity> strings.

   4.  The agent MAY then prepend to the Path header field content "!"
       or "!!" followed by an additional <path-identity> for itself
       other than its primary one.  Using "!!", and thereby adding a
       <diag-match> since the <path-identity> clearly is verified, is
       RECOMMENDED.  This step may be repeated any number of times.
       This is permitted for agents that have multiple <path-identity>s
       (such as during a transition from one to another).  Each of these
       <path-identity>s MUST meet the requirements set out in
       Section 3.2.

   5.  Finally, the agent MUST prepend its primary <path-identity> to
       the Path header field content.  The primary <path-identity> is
       the <path-identity> it normally advertises to its peers for their
       use in generating <path-diagnostic>s as described above.

   Any agent that modifies the Path header field MAY fold it by
   inserting FWS (folding white space) immediately after any <path-
   identity> or <diag-other> it added (see Section 3.1.5 of [RFC5536]
   for allowable locations for FWS).

3.2.2. Path Header Field Example

Here is an example of a Path header field created by following the rules for injecting and relaying agents. Path: foo.isp.example!.SEEN.isp.example!foo-news !.MISMATCH.2001:DB8:0:0:8:800:200C:417A!bar.isp.example !!!barbaz!!baz.isp.example !.POSTED.dialup123.baz.isp.example!not-for-mail This article was injected by baz.isp.example as indicated by the <diag-keyword> "POSTED". The injector has recorded that it received the article from dialup123.baz.isp.example. "not-for-mail" is a common <tail-entry>.
Top   ToC   RFC5537 - Page 10
   The article was relayed to the relaying agent known, at least to, as "barbaz".  That relaying agent confirmed to its
   satisfaction that "baz.isp.example" was an expected <path-identity>
   for the source of the article and therefore used <diag-match> ("!")
   for its <path-diagnostic>.

   barbaz relayed it to, which does not support <diag-
   keyword> and therefore used the old "!" delimiter.  This indicates
   that the identity of "barbaz" was not verified and may have been
   forged. relayed it to a news server using the <path-
   identity> of bar.isp.example and claiming (by using the "!" <path-
   diagnostic>) to have verified that it came from

   bar.isp.example relayed it to foo-news, which, not being convinced
   that it truly came from bar.isp.example, inserted the <diag-keyword>
   "MISMATCH" and then stated that it received the article from the IPv6
   address [2001:DB8:0:0:8:800:200C:417A].  (This is not to say that
   bar.isp.example was not a correct <path-identity> for that source but
   simply that the identity did not match the expectations of foo-news.)

   foo-news then passed the article to foo.isp.example, which declined
   to validate its <path-identity> and instead appended the <diag-
   keyword> "SEEN" to indicate it knows the source of the article as
   isp.example.  This may be either an expected <path-identity> or the
   FQDN of the system from which it received the article.  Presumably,
   foo.isp.example is a serving agent that then delivered the article to
   a reading agent.

   baz.isp.example, bar.isp.example, and foo-news folded the Path header

3.3. Article History and Duplicate Suppression

Netnews normally uses a flood-fill algorithm for propagation of articles in which each news server offers the articles it accepts to multiple peers, and each news server may be offered the same article from multiple other news servers. Accordingly, duplicate suppression is key; if a news server accepted every article it was offered, it may needlessly accept (and then potentially retransmit) dozens of copies of every article. Relaying and serving agents therefore MUST keep a record of articles they have already seen and use that record to reject additional offers of the same article. This record is called the "history" file or database.
Top   ToC   RFC5537 - Page 11
   Each article is uniquely identified by its message identifier, so a
   relaying or serving agent could satisfy this requirement by storing a
   record of every message identifier that agent has ever seen.  Such a
   history database would grow without bound, however, so it is common
   and permitted to optimize based on the Injection-Date or Date header
   field of an article as follows.  (In the following discussion, the
   "date" of an article is defined to be the date represented by its
   Injection-Date header field, if present; otherwise, by its Date
   header field.)

   o  Agents MAY select a cutoff interval and reject any article with a
      date farther in the past than that cutoff interval.  If this
      interval is shorter than the time it takes for an article to
      propagate through the network, the agent might reject an article
      it had not yet seen, so it ought not to be aggressively short.
      For Usenet, for example, a cutoff interval of no less than seven
      days is conventional.

   o  Agents that enforce such a cutoff MAY then drop records of
      articles that had dates older than the cutoff from their history
      databases.  If such an article were offered to the agent again, it
      would be rejected due to the cutoff date, so the history record is
      no longer required to suppress the duplicate.

   o  Alternatively, agents MAY drop history records according to the
      date when the article was first seen by that agent rather than the
      date of the article.  In this case, the history retention interval
      MUST be at least 24 hours longer than the cutoff interval to allow
      for articles dated in the future.  This interval matches the
      allowable error in the date of the article (see Section 3.5).

   These are just two implementation strategies for article history,
   albeit the most common ones.  Relaying and serving agents are not
   required to use these strategies, only to meet the requirement of not
   accepting an article more than once.  However, these strategies are
   safe and widely deployed, and implementors are encouraged to use one
   of them, especially if they do not have extensive experience with
   Netnews and the subtle effects of its flood-fill algorithm.

3.4. Duties of a Posting Agent

A posting agent is the component of a user agent that assists a poster in creating a valid proto-article and forwarding it to an injecting agent.
Top   ToC   RFC5537 - Page 12
   Posting agents SHOULD ensure that proto-articles they create are
   valid according to [RFC5536] and any other applicable policies.  They
   MUST NOT create any Injection-Info header field; this header field
   may only be added by the injecting agent.

   If the proto-article already contains both Message-ID and Date header
   fields, posting agents MAY add an Injection-Date header field to that
   proto-article immediately before passing that proto-article to an
   injection agent.  They SHOULD do so if the Date header field
   (representing the composition time of the proto-article) is more than
   a day in the past at the time of injection.  They MUST do so if the
   proto-article is being submitted to more than one injecting agent;
   see Section 3.4.2.

   The Injection-Date header field is new in this revision of the
   Netnews protocol and is designed to allow the Date header field to
   hold the composition date (as recommended in Section 3.6.1 of
   [RFC5322]), even if the proto-article is not to be injected for some
   time after its composition.  However, note that all implementations
   predating this specification ignore the Injection-Date header field
   and use the Date header field in its stead for rejecting articles
   older than their cutoff (see Section 3.3), and injecting agents
   predating this specification do not add an Injection-Date header.
   Articles with a Date header field substantially in the past will
   still be rejected by implementations predating this specification,
   regardless of the Injection-Date header field, and hence may suffer
   poorer propagation.

   Contrary to [RFC5322], which implies that the mailbox or mailboxes in
   the From header field should be that of the poster or posters, a
   poster who does not, for whatever reason, wish to use his own mailbox
   MAY use any mailbox ending in the top-level domain ".invalid"

   Posting agents meant for use by ordinary posters SHOULD reject any
   attempt to post an article that cancels or supersedes (via the
   Supersedes header field) another article of which the poster is not
   the author or sender.

3.4.1. Proto-Articles

A proto-article is an article in the format used by a posting agent when offering that article to an injecting agent. It may omit certain header fields that can be better supplied by the injecting agent and will not contain header fields that are added by the injecting agent. A proto-article is only for transmission to an injecting agent and SHOULD NOT be transmitted to any other agent.
Top   ToC   RFC5537 - Page 13
   A proto-article has the same format as a normal article except that
   the Injection-Info and Xref header fields MUST NOT be present, the
   Path header field SHOULD NOT contain a "POSTED" <diag-keyword>, and
   any of the following mandatory header fields MAY be omitted:
   Message-ID, Date, and Path.  In all other respects, a proto-article
   MUST be a valid Netnews article.  In particular, the header fields
   that may be omitted MUST NOT be present with invalid content.

   If a posting agent intends to offer the same proto-article to
   multiple injecting agents, the header fields Message-ID, Date, and
   Injection-Date MUST be present and identical in all copies of the
   proto-article.  See Section 3.4.2.

3.4.2. Multiple Injection of Articles

Under some circumstances (for example, when posting to multiple, supposedly disjoint, networks, when using injecting agents with spotty connectivity, or when desiring additional redundancy), a posting agent may wish to offer the same article to multiple injecting agents. In this unusual case, the goal is not to create multiple independent articles but rather to inject the same article at multiple points and let the normal duplicate suppression facility of Netnews (see Section 3.3) ensure that any given agent accepts the article only once, even if supposedly disjoint networks have unexpected links. Whenever possible, multiple injection SHOULD be done by offering the same proto-article to multiple injecting agents. The posting agent MUST supply the Message-ID, Date, and Injection-Date header fields, and the proto-article as offered to each injecting agent MUST be identical. In some cases, offering the same proto-article to all injecting agents may not be possible (such as when gatewaying, after injection, articles found on one Netnews network to another supposedly unconnected one). In this case, the posting agent MUST remove any Xref header field and rename or remove any Injection-Info, Path, and other trace header fields before passing it to another injecting agent. (This converts the article back into a proto-article.) It MUST retain unmodified the Message-ID, Date, and Injection-Date header fields. It MUST NOT add an Injection-Date header field if it is missing from the existing article. NOTE: Multiple injection inherently risks duplicating articles. Multiple injection after injection, by converting an article back to a proto-article and injecting it again, additionally risks loops, loss of trace information, unintended repeat injection into the same network, and other problems. It should be done with care
Top   ToC   RFC5537 - Page 14
      and only when there is no alternative.  The requirement to retain
      Message-ID, Date, and Injection-Date header fields minimizes the
      possibility of a loop and ensures that the newly injected article
      is not treated as a new, separate article.

   Multiple injection of an article that lists one or more moderated
   newsgroups in its Newsgroups header field SHOULD only be done by a
   moderator and MUST only be done after the proto-article has been
   approved for all moderated groups to which it is to be posted and
   after an Approved header field has been added (see Section 3.9).
   Multiple injection of an unapproved article intended for moderated
   newsgroups will normally only result in the moderator receiving
   multiple copies, and if the newsgroup status is not consistent across
   all injecting agents, may result in duplication of the article or
   other problems.

3.4.3. Followups

A followup is an article that contains a response to the contents of an earlier article, its precursor. In addition to its normal duties, a posting agent preparing a followup is also subject to the following requirements. Wherever in the following it is stated that, by default, a header field is said to be inherited from one of those header fields in the precursor, it means that its initial content is to be a copy of the content of that precursor header field (with changes in folding permitted). However, posters MAY then override that default before posting. Despite the historic practice of some posting agents, the Keywords header field SHOULD NOT be inherited by default from the precursor article. 1. If the Followup-To header field of the precursor article consists of "poster", the followup MUST NOT be posted by default but, by default, is to be emailed to the address given in the precursor's Reply-To or From header field following the rules for an email reply [RFC5322]. This action MAY be overridden by the poster, in which case the posting agent should continue as if the Followup-To header field in the precursor did not exist. 2. The Newsgroups header field SHOULD, by default, be inherited from the precursor's Followup-To header field if present; otherwise, it is inherited from the precursor's Newsgroups header field.
Top   ToC   RFC5537 - Page 15
   3.  The Subject header field SHOULD, by default, be inherited from
       that of the precursor.  The case-sensitive string "Re: "
       (including the space after the colon) MAY be prepended to the
       content of its Subject header field unless it already begins with
       that string.

          NOTE: Prepending "Re: " serves no protocol function and hence
          is not required, but it is widely expected and not doing so
          would be surprising.

   4.  The Distribution header field SHOULD, by default, be inherited
       from the precursor's Distribution header field, if present.

   5.  The followup MUST have a References header field referring to its
       precursor, constructed in accordance with Section 3.4.4.

3.4.4. Construction of the References Header Field

The following procedure is to be used whenever some previous article (the "parent") is to be referred to in the References header field of a new article, whether because the new article is a followup and the parent is its precursor or for some other reason. The content of the new article's References header field MUST be formed from the content of the parent's References header field if present, followed by the content of the Message-ID header field of the parent. If the parent had a References header, FWS as defined in [RFC5536] MUST be added between its content and the Message-ID header field content. If the resulting References header field would, after unfolding, exceed 998 characters in length (including its field name but not the final CRLF), it MUST be trimmed (and otherwise MAY be trimmed). Trimming means removing any number of message identifiers from its content, except that the first message identifier and the last two MUST NOT be removed. An essential property of the References header field, guaranteed by the above procedure and REQUIRED to be maintained by any extensions to this protocol, is that an article MUST NOT precede one of its parents.

3.5. Duties of an Injecting Agent

An injecting agent takes a proto-article from a posting agent and either forwards it to a moderator or passes it to a relaying or serving agent or agents. An injecting agent bears the primary responsibility for ensuring that any article it injects conforms with
Top   ToC   RFC5537 - Page 16
   the rules of the Netnews standards.  The administrator of an
   injecting agent is also expected to bear some responsibility towards
   the rest of the Netnews network to which it is connected for the
   articles the injecting agent accepts.

   Injecting agents, when rejecting articles, are encouraged to
   communicate the reason for rejection to the posting agent by using
   whatever facility is provided by the underlying transport.  The
   injecting agent is in a unique position to communicate the reason for
   rejection; relaying agents and serving agents normally have to reject
   messages silently.  The injecting agent therefore bears much of the
   burden of diagnosing broken posting agents or communicating policy
   violations to posters.

   An injecting agent MUST have available a list (possibly empty) of
   moderated groups for which it accepts articles and the corresponding
   submission addresses.  It SHOULD have available a list of valid
   newsgroups to catch articles not posted to a valid newsgroup and
   therefore likely to be silently discarded by relaying and serving
   agents.  Usually, an injecting agent is deployed in conjunction with
   a serving agent and maintains these lists based on control messages
   received by the serving agent.

   An injecting agent processes proto-articles as follows:

   1.   It SHOULD verify that the article is from a trusted source (for
        example, by relying on the authorization capability of the
        underlying transport used to talk to the posting agent).

   2.   It MUST reject any proto-article that does not have the proper
        mandatory header fields for a proto-article, that has Injection-
        Info or Xref header fields, that has a Path header field
        containing the "POSTED" <diag-keyword>, or that is not
        syntactically valid as defined by [RFC5536].  It SHOULD reject
        any proto-article that contains a header field deprecated for
        Netnews (see, for example, [RFC3798]).  It MAY reject any proto-
        article that contains trace header fields (e.g., NNTP-Posting-
        Host) indicating that it was already injected by an injecting
        agent that did not add Injection-Info or Injection-Date.

   3.   It SHOULD reject any article whose Injection-Date or Date header
        field is more than 24 hours into the future (and MAY use a
        margin less than 24 hours).  It SHOULD reject any article whose
        Injection-Date header field is too far in the past (older than
        the cutoff interval of a relaying agent that the injecting agent
        is using, for example).  It SHOULD similarly reject any article
        whose Date header field is too far in the past, since not all
        news servers support Injection-Date and only the injecting agent
Top   ToC   RFC5537 - Page 17
        can provide a useful error message to the posting agent.  In
        either case, this interval SHOULD NOT be any shorter than 72
        hours into the past.

   4.   It SHOULD reject any proto-article whose Newsgroups header field
        does not contain at least one <newsgroup-name> for a valid
        group, or that contains a <newsgroup-name> reserved for specific
        purposes by Section 3.1.4 of [RFC5536] unless that specific
        purpose or local agreement applies to the proto-article being
        processed.  Crossposting to unknown newsgroups is not precluded
        provided that at least one of the newsgroups in the Newsgroups
        header is valid.

   5.   The Message-ID and Date header fields with appropriate contents
        MUST be added when not present in the proto-article.

   6.   The injecting agent MUST NOT alter the body of the article in
        any way (including any change of Content-Transfer-Encoding).  It
        MAY add other header fields not already provided by the poster,
        but injecting agents are encouraged to use the Injection-Info
        header for such information and to minimize the addition of
        other headers.  It SHOULD NOT alter, delete, or reorder any
        existing header field except the Path header field.  It MUST NOT
        alter or delete any existing Message-ID header field.

   7.   If the Newsgroups header contains one or more moderated groups
        and the proto-article does not contain an Approved header field,
        the injecting agent MUST either forward it to a moderator as
        specified in Section 3.5.1 or, if that is not possible, reject
        it.  This forwarding MUST be done after adding the Message-ID
        and Date headers if required, and before adding the Injection-
        Info and Injection-Date headers.

   8.   Otherwise, a Path header field with a <tail-entry> MUST be added
        if not already present.

   9.   The injecting agent MUST then update the Path header field as
        described in Section 3.2.1.

   10.  An Injection-Info header field SHOULD be added that identifies
        the source of the article and possibly other trace information
        as described in Section 3.2.8 of [RFC5536].

   11.  If the proto-article already had an Injection-Date header field,
        it MUST NOT be modified or replaced.  If the proto-article had
        both a Message-ID header field and a Date header field, an
        Injection-Date header field MUST NOT be added, since the proto-
        article may have been multiply injected by a posting agent that
Top   ToC   RFC5537 - Page 18
        predates this standard.  Otherwise, the injecting agent MUST add
        an Injection-Date header field containing the current date and

   12.  Finally, the injecting agent forwards the article to one or more
        relaying agents, and the injection process is complete.

3.5.1. Forwarding Messages to a Moderator

An injecting agent MUST forward the proto-article to the moderator of the leftmost moderated group listed in the Newsgroups header field, customarily via email. There are two standard ways in which it may do this: 1. The complete proto-article is encapsulated, header fields and all, within the email. This SHOULD be done by creating an email message with a Content-Type of application/news-transmission with the usage parameter set to "moderate". The body SHOULD NOT contain any content other than the message. This method has the advantage of removing any possible conflict between Netnews and email header fields and any changes to those fields during transport through email. 2. The proto-article is sent as an email with the addition of any header fields required for an email as defined in [RFC5322], and possibly with the addition of other header fields conventional in email, such as To and Received. The existing Message-ID header field SHOULD be retained. Although both of these methods have been used in the past and the first has clear technical advantages, the second is in more common use and many moderators are not prepared to deal with messages in the first format. Accordingly, the first method SHOULD NOT be used unless the moderator to which it is being forwarded is known to be able to handle this method. NOTE: Deriving the email address of the moderator of a group is outside the scope of this document. It is worth mentioning, however, that a common method is to use a forwarding service that handles submissions for many moderated groups. For maximum compatibility with existing news servers, such forwarding services generally form the submission address for a moderated group by replacing each "." in the <newsgroup-name> with "-" and then using that value as the <local-part> of a <mailbox> formed by appending a set domain.
Top   ToC   RFC5537 - Page 19
   Forwarding proto-articles to moderators via email is the most general
   method and the most common in large Netnews networks such as Usenet,
   but any means of forwarding the article that preserves it without
   injecting it MAY be used.  For example, if the injecting agent has
   access to a database used by the moderator to store proto-articles
   awaiting processing, it may place the proto-article directly into
   that database.  Such methods may be more appropriate for smaller
   Netnews networks.

3.6. Duties of a Relaying Agent

A relaying agent accepts injected articles from injecting and other relaying agents and passes them on to relaying or serving agents. To avoid bypass of injecting agent policies and forgery of Path and Injection-Info headers, relaying agents SHOULD accept articles only from trusted agents. An article SHOULD NOT be relayed unless the sending agent has been configured to supply, and the receiving agent to receive, at least one of the <newsgroup-name>s in its Newsgroups header field and at least one of the <dist-name>s in its Distribution header field (if present). Exceptionally, control messages creating or removing newsgroups (newgroup or rmgroup control messages, for example) SHOULD be relayed if the affected group appears in its Newsgroups header field and both the sending and receiving relaying agents are configured to relay a newsgroup of that name (whether or not such a newsgroup exists). In order to avoid unnecessary relaying attempts, an article SHOULD NOT be relayed if the <path-identity> of the receiving agent (or some known alias thereof) appears as a <path-identity> (excluding within the <tail-entry> or following a "POSTED" <diag-keyword>) in its Path header field. A relaying agent processes an article as follows: 1. It MUST reject any article without a Newsgroups header field or Message-ID header field, or without either an Injection-Date or Date header field. 2. It MUST examine the Injection-Date header field or, if absent, the Date header field, and reject the article if that date is more than 24 hours into the future. It MAY reject articles with dates in the future with a smaller margin than 24 hours.
Top   ToC   RFC5537 - Page 20
   3.  It MUST reject any article that has already been accepted.  If it
       implements one of the mechanisms described in Section 3.3, this
       means that it MUST reject any article whose date falls outside
       the cutoff interval since it won't know whether or not such
       articles had been accepted previously.

   4.  It SHOULD reject any article that does not include all the
       mandatory header fields.  It MAY reject any article that contains
       header fields that do not have valid contents.

   5.  It SHOULD reject any article that matches an already-received
       cancel control message or the contents of the Supersedes header
       field of an accepted article, provided that the relaying agent
       has chosen (on the basis of local site policy) to honor that
       cancel control message or Supersedes header field.

   6.  It MAY reject any article without an Approved header field posted
       to a newsgroup known to be moderated.  This practice is strongly
       encouraged, but the information necessary to do so is not
       required to be maintained by a relaying agent.

   7.  It MUST update the Path header field as described in
       Section 3.2.1.

   8.  It MAY delete any Xref header field already present.  It MAY add
       a new Xref header field for its own use (but recall that
       [RFC5536] permits at most one such header field).

   9.  Finally, it passes the article on to other relaying and serving
       agents to which it is configured to send articles.

   Relaying agents SHOULD, where possible in the underlying transport,
   inform the agent that passed the article to the relaying agent if the
   article was rejected.  Relaying agents MUST NOT inform any other
   external entity of the rejection of an article unless that external
   entity has explicitly requested that it be informed of such errors.

   Relaying agents MUST NOT alter, delete, or rearrange any part of an
   article except for the Path and Xref header fields.  They MUST NOT
   modify the body of articles in any way.  If an article is not
   acceptable as is, the article MUST be rejected rather than modified.
Top   ToC   RFC5537 - Page 21

3.7. Duties of a Serving Agent

A serving agent accepts articles from a relaying agent or injecting agent, stores them, and makes them available to reading agents. Articles are normally indexed by newsgroup and <article-locator> (Section 3.2.14 of [RFC5536]), usually in the form of a decimal number. If the serving agent stores articles by newsgroup, control messages SHOULD NOT be stored in the newsgroups listed in the control message's Newsgroups header field. Instead, they SHOULD be stored in a newsgroup in the hierarchy "control", which is reserved for this purpose. Conventionally, control messages are stored in newsgroups named for the type of control message (such as "control.cancel" for cancel control messages). A serving agent MUST have available a list (possibly empty) of moderated groups for which it accepts articles so that it can reject unapproved articles posted to moderated groups. Frequently, a serving agent is deployed in combination with an injecting agent and can use the same list as the injecting agent. A serving agent processes articles as follows: 1. It MUST reject any article that does not include all the mandatory header fields or any article that contains header fields that do not have valid contents. 2. It MUST examine the Injection-Date header field or, if absent, the Date header field, and reject the article if that date is more than 24 hours into the future. It MAY reject articles with dates in the future with a smaller margin than 24 hours. 3. It MUST reject any article that has already been accepted. If it implements one of the mechanisms described in Section 3.3, this means that it MUST reject any article whose date falls outside the cutoff interval since it won't know whether or not such articles had been accepted previously. 4. It SHOULD reject any article that matches an already-received and honored cancel message or Supersedes header field, following the same rules as a relaying agent (Section 3.6). 5. It MUST reject any article without an Approved header field posted to any newsgroup listed as moderated. 6. It MUST update the Path header field as described in Section 3.2.1.
Top   ToC   RFC5537 - Page 22
   7.  It MUST remove any Xref header field from each article (except
       when specially configured to preserve the <article-locator>s set
       by the sending site).  It then MAY (and usually will) add a new
       Xref header field (but recall that [RFC5536] permits at most one
       such header field).

   8.  Finally, it stores the article and makes it available for reading

   Serving agents MUST NOT create new newsgroups simply because an
   unrecognized <newsgroup-name> occurs in a Newsgroups header field.
   Newsgroups are normally created via control messages (Section 5.2.1).

   Serving agents MUST NOT alter, delete, or rearrange any part of an
   article except for the Path and Xref header fields.  They MUST NOT
   modify the body of the articles in any way.  If an article is not
   acceptable as is, the article MUST be rejected rather than modified.

3.8. Duties of a Reading Agent

Since a reading agent is only a passive participant in a Netnews network, there are no specific protocol requirements placed on it. See [USEAGE] for best-practice recommendations.

3.9. Duties of a Moderator

A moderator receives news articles, customarily by email, decides whether to approve them and, if so, either passes them to an injecting agent or forwards them to further moderators. Articles are normally received by the moderator in email, either encapsulated as an object of Content-Type application/ news-transmission (or possibly encapsulated but without an explicit Content-Type header field) or else directly as an email already containing all the header fields appropriate for a Netnews article (see Section 3.5.1). Moderators who may receive articles via email SHOULD be prepared to accept articles in either format. There are no protocol restrictions on what criteria are used for accepting or rejecting messages or on what modifications a moderator may make to a message (both header fields and body) before injecting it. Recommended best practice, however, is to make the minimal required changes. Moderators need to be aware that modifications made to articles may invalidate signatures created by the poster or previous moderators. See [USEAGE] for further best-practice recommendations.
Top   ToC   RFC5537 - Page 23
   Moderators process articles as follows:

   1.  They decide whether to approve or reject a proto-article and, if
       approved, prepare the proto-article for injection.  If the proto-
       article was received as an unencapsulated email message, this
       will require converting it back to a valid Netnews proto-article.
       If the article is rejected, it is normally rejected for all
       newsgroups to which it was posted and nothing further is done.
       If it is approved, the moderator proceeds with the following

   2.  If the Newsgroups header field contains further moderated
       newsgroups for which approval has not already been given, they
       may either reach some agreement with the other moderators on the
       disposition of the article or, more generally, add an indication
       (identifying both the moderator and the name of the newsgroup)
       that they approve the article and then forward it to the
       moderator of the leftmost unapproved newsgroup.  This forwarding
       SHOULD be done following the procedure in Section 3.5.1.  It MAY
       be done by rotating the <newsgroup-name>s in the Newsgroups
       header field so that the leftmost unapproved newsgroup is the
       leftmost moderated newsgroup in that field and then posting it,
       letting the injecting agent do the forwarding.  However, when
       using this mechanism, they MUST first ensure that the article
       contains no Approved header field.

   3.  If the Newsgroups header field contains no further unapproved
       moderated groups, they add an Approved header field (see Section
       3.2.1 of [RFC5536]) identifying the moderator and, insofar as is
       possible, all the other moderators who have approved the article.
       The moderator who takes this step assumes responsibility for
       ensuring that the article was approved by the moderators of all
       moderated newsgroups to which it was posted.

   4.  Moderators are encouraged to retain the Message-ID header field
       unless it is invalid or the article was significantly changed
       from its original form.  Moderators are also encouraged to retain
       the Date header field unless it appears to be stale (72 hours or
       more in the past) for reasons understood by the moderator (such
       as delays in the moderation process), in which case they MAY
       substitute the current date.  Any Injection-Date, Injection-Info,
       or Xref header fields already present MUST be removed.

   5.  Any Path header field MUST either be removed or truncated to only
       those entries following its "POSTED" <diag-keyword>, if any.

   6.  The moderator then passes the article to an injecting agent,
       having first observed all the duties of a posting agent.

(next page on part 2)

Next Section