Tech-invite3GPPspaceIETF RFCsSIP
929190898887868584838281807978777675747372717069686766656463626160595857565554535251504948474645444342414039383736353433323130292827262524232221201918171615141312111009080706050403020100
in Index   Prev   Next

RFC 7011

Specification of the IP Flow Information Export (IPFIX) Protocol for the Exchange of Flow Information

Pages: 76
Internet Standard: 77
Errata
Obsoletes:  5101
Part 3 of 4 – Pages 38 to 61
First   Prev   Next

Top   ToC   RFC7011 - Page 38   prevText

8. Template Management

This section describes the management of Templates and Options Templates at the Exporting and Collecting Processes. The goal of Template management is to ensure, to the extent possible, that the Exporting Process and Collecting Process have a consistent view of the Templates and Options Templates used to encode and decode the Records sent from the Exporting Process to the Collecting Process.
Top   ToC   RFC7011 - Page 39
   Achieving this goal is complicated somewhat by two factors: 1) the
   need to support the reuse of Template IDs within a Transport Session
   and 2) the need to support unreliable transmission for Templates when
   UDP is used as the transport protocol for IPFIX Messages.

   The Template Management mechanisms defined in this section apply to
   the export of IPFIX Messages on SCTP, TCP, or UDP.  Additional
   considerations specific to SCTP and UDP transport are given in
   Sections 8.3 and 8.4, respectively.

   The Exporting Process assigns and maintains Template IDs per
   Transport Session and Observation Domain.  A newly created Template
   Record is assigned an unused Template ID by the Exporting Process.
   The Collecting Process MUST store all received Template Record
   information for the duration of each Transport Session until reuse or
   withdrawal as described in Section 8.1, or expiry over UDP as
   described in Section 8.4, so that it can interpret the corresponding
   Data Records.

   The Collecting Process MUST NOT assume that the Template IDs from a
   given Exporting Process refer to the same Templates as they did in
   previous Transport Sessions from the same Exporting Process; a
   Collecting Process MUST NOT use Templates from one Transport Session
   to decode Data Sets in a subsequent Transport Session.

   If a specific Information Element is required by a Template but is
   not present in observed packets, the Exporting Process MAY choose to
   export Flow Records without this Information Element in a Data Record
   described by a new Template.

   If an Information Element is required more than once in a Template,
   the different occurrences of this Information Element SHOULD follow
   the logical order of their treatments by the Metering Process.  For
   example, if a selected packet goes through two hash functions, and if
   the two hash values are sent within a single Template, the first
   occurrence of the hash value should belong to the first hash function
   in the Metering Process.  For example, when exporting the two source
   IP addresses of an IPv4-in-IPv4 packet, the first sourceIPv4Address
   Information Element occurrence should be the IPv4 address of the
   outer header, while the second occurrence should be the address of
   the inner header.  Collecting Processes MUST properly handle
   Templates with multiple identical Information Elements.

   The Exporting Process SHOULD transmit the Template Set and Options
   Template Set in advance of any Data Sets that use that (Options)
   Template ID, to help ensure that the Collector has the Template
   Record before receiving the first Data Record.  Data Records that
   correspond to a Template Record MAY appear in the same and/or
Top   ToC   RFC7011 - Page 40
   subsequent IPFIX Message(s).  However, a Collecting Process MUST NOT
   assume that the Data Set and the associated Template Set (or Options
   Template Set) are exported in the same IPFIX Message.

   Though a Collecting Process normally receives Template Records from
   the Exporting Process before receiving Data Records, this is not
   always the case, e.g., in the case of reordering or Collecting
   Process restart over UDP.  In these cases, the Collecting Process MAY
   buffer Data Records for which it has no Templates, to wait for
   Template Records describing them; however, note that in the presence
   of Template withdrawal and redefinition (Section 8.1) this may lead
   to incorrect interpretation of Data Records.

   Different Observation Domains within a Transport Session MAY use the
   same Template ID value to refer to different Templates; Collecting
   Processes MUST properly handle this case.

   Options Templates and Templates that are related or interdependent
   (e.g., by sharing common properties as described in [RFC5473]) SHOULD
   be sent together in the same IPFIX Message.

8.1. Template Withdrawal and Redefinition

Templates that will not be used further by an Exporting Process MAY be withdrawn by sending a Template Withdrawal. After receiving a Template Withdrawal, a Collecting Process MUST stop using the Template to interpret subsequently exported Data Sets. Note that this mechanism does not apply when UDP is used to transport IPFIX Messages; for that case, see Section 8.4. A Template Withdrawal consists of a Template Record for the Template ID to be withdrawn, with a Field Count of 0. The format of a Template Withdrawal is shown in Figure T. 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Set ID = (2 or 3) | Length = 16 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Template ID N | Field Count = 0 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Template ID ... | Field Count = 0 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Template ID M | Field Count = 0 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Figure T: Template Withdrawal Format
Top   ToC   RFC7011 - Page 41
   The Set ID field MUST contain the value 2 for Template Set Withdrawal
   or the value 3 for Options Template Set Withdrawal.  Multiple
   Template IDs MAY be withdrawn with a single Template Withdrawal; in
   that case, padding MAY be used.

   Template Withdrawals MAY appear interleaved with Template Sets,
   Options Template Sets, and Data Sets within an IPFIX Message.  In
   this case, the Templates and Template Withdrawals shall be
   interpreted as taking effect in the order in which they appear in the
   IPFIX Message.  An Exporting Process SHOULD NOT send a Template
   Withdrawal until sufficient time has elapsed to allow receipt and
   processing of any Data Records described by the withdrawn Templates;
   see Section 8.2 for details regarding the sequencing of Template
   management actions.

   The end of a Transport Session implicitly withdraws all the Templates
   used within the Transport Session, and Templates must be resent
   during subsequent Transport Sessions between an Exporting Process and
   Collecting Process.  This applies to SCTP and TCP only; see
   Sections 8.4 and 10.3.4 for discussions of Transport Session and
   Template lifetime over UDP.

   All Templates for a given Observation Domain MAY also be withdrawn
   using an All Templates Withdrawal, as shown in Figure U.  All Options
   Templates for a given Observation Domain MAY likewise be withdrawn
   using an All Options Templates Withdrawal, as shown in Figure V.

      0                   1                   2                   3
      0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
     |             Set ID = 2        |          Length = 8           |
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
     |         Template ID = 2       |        Field Count = 0        |
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

               Figure U: All Templates Withdrawal Set Format

      0                   1                   2                   3
      0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
     |             Set ID = 3        |          Length = 8           |
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
     |         Template ID = 3       |        Field Count = 0        |
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

           Figure V: All Options Templates Withdrawal Set Format
Top   ToC   RFC7011 - Page 42
   Template IDs MAY be reused for new Templates by sending a new
   Template Record or Options Template Record for a given Template ID
   after withdrawing the Template.

   If a Collecting Process receives a Template Withdrawal for a Template
   or Options Template it does not presently have stored, this indicates
   a malfunctioning or improperly implemented Exporting Process.  The
   continued receipt and interpretation of Data Records are still
   possible, but the Collecting Process MUST ignore the Template
   Withdrawal and SHOULD log the error.

   If a Collecting Process receives a new Template Record or Options
   Template Record for an already-allocated Template ID, and that
   Template or Options Template is identical to the already-received
   Template or Options Template, it SHOULD log the retransmission;
   however, this is not an error condition, as it does not affect the
   interpretation of Data Records.

   If a Collecting Process receives a new Template Record or Options
   Template Record for an already-allocated Template ID, and that
   Template or Options Template is different from the already-received
   Template or Options Template, this indicates a malfunctioning or
   improperly implemented Exporting Process.  The continued receipt and
   unambiguous interpretation of Data Records for this Template ID are
   no longer possible, and the Collecting Process SHOULD log the error.
   Further Collecting Process actions are out of scope for this
   specification.

8.2. Sequencing Template Management Actions

Since there is no guarantee of the ordering of exported IPFIX Messages across SCTP Streams or over UDP, an Exporting Process MUST sequence all Template management actions (i.e., Template Records defining new Templates and Template Withdrawals withdrawing them) using the Export Time field in the IPFIX Message Header. An Exporting Process MUST NOT export a Data Set described by a new Template in an IPFIX Message with an Export Time before the Export Time of the IPFIX Message containing that Template. If a new Template and a Data Set described by it appear in the same IPFIX Message, the Template Set containing the Template MUST appear before the Data Set in the Message. An Exporting Process MUST NOT export any Data Sets described by a withdrawn Template in IPFIX Messages with an Export Time after the Export Time of the IPFIX Message containing the Template Withdrawal withdrawing that Template.
Top   ToC   RFC7011 - Page 43
   Put another way, a Template describes Data Records contained in IPFIX
   Messages when the Export Time of such messages is between a specific
   start and end time, inclusive.  The start time is the Export Time of
   the IPFIX Message containing the Template Record.  The end time is
   one of two times: if the template is withdrawn during the session,
   then it is the Export Time of the IPFIX Message containing the
   Template Withdrawal for the template; otherwise, it is the end of the
   Transport Session.

   Even if sent in order, IPFIX Messages containing Template management
   actions could arrive at the Collecting Process out of order, i.e., if
   sent via UDP or via different SCTP Streams.  Given this, Template
   Withdrawals and subsequent reuse of Template IDs can significantly
   complicate the problem of determining Template lifetimes at the
   Collecting Process.  A Collecting Process MAY implement a buffer and
   use Export Time information to disambiguate the order of Template
   management actions.  This buffer, if implemented, SHOULD be
   configurable to impart a delay on the order of the maximum reordering
   delay experienced at the Collecting Process.  Note, in this case,
   that the Collecting Process's clock is irrelevant: it is only
   comparing the Export Times of Messages to each other.

8.3. Additional Considerations for Template Management over SCTP

The specifications in this section apply only to SCTP; in cases of contradiction with specifications in Section 8 or Section 8.1, this section takes precedence. Template Sets and Options Template Sets MAY be sent on any SCTP Stream. Data Sets sent on a given SCTP Stream MAY be represented by Template Records exported on any SCTP Stream. Template Sets and Options Template Sets MUST be sent reliably, using SCTP ordered delivery. Template Withdrawals MAY be sent on any SCTP Stream. Template Withdrawals MUST be sent reliably, using SCTP ordered delivery. Template IDs MAY be reused by sending a Template Withdrawal and/or a new Template Record on a different SCTP Stream than the stream on which the original Template was sent. Additional Template Management considerations are provided in [RFC6526], which specifies an extension to explicitly link Templates with SCTP Streams. In exchange for more restrictive rules on the assignment of Template Records to SCTP Streams, this extension allows fast, reliable reuse of Template IDs and estimation of Data Record loss per Template.
Top   ToC   RFC7011 - Page 44

8.4. Additional Considerations for Template Management over UDP

The specifications in this section apply only to UDP; in cases of contradiction with specifications in Section 8 or Section 8.1, this section takes precedence. Since UDP provides no method for reliable transmission of Templates, Exporting Processes using UDP as the transport protocol MUST periodically retransmit each active Template at regular intervals. The Template retransmission interval MUST be configurable via, for example, the templateRefreshTimeout and optionsTemplateRefreshTimeout parameters as defined in [RFC6728]. Default settings for these values are deployment- and application-specific. Before exporting any Data Records described by a given Template Record or Options Template Record, especially in the case of Template ID reuse as described in Section 8.1, the Exporting Process SHOULD send multiple copies of the Template Record in a separate IPFIX Message, in order to help ensure that the Collecting Process has received it. In order to minimize resource requirements for Templates that are no longer being used by the Exporting Process, the Collecting Process MAY associate a lifetime with each Template received in a Transport Session. Templates not refreshed by the Exporting Process within the lifetime can then be discarded by the Collecting Process. The Template lifetime at the Collecting Process MAY be exposed by a configuration parameter or MAY be derived from observation of the interval of periodic Template retransmissions from the Exporting Process. In this latter case, the Template lifetime SHOULD default to at least 3 times the observed retransmission rate. Template Withdrawals (Section 8.1) MUST NOT be sent by Exporting Processes exporting via UDP and MUST be ignored by Collecting Processes collecting via UDP. Template IDs MAY be reused by Exporting Processes by exporting a new Template for the Template ID after waiting at least 3 times the retransmission delay. Note that Template ID reuse may lead to incorrect interpretation of Data Records if the retransmission and lifetime are not properly configured. When a Collecting Process receives a new Template Record or Options Template Record via UDP for an already-allocated Template ID, and that Template or Options Template is identical to the already- received Template or Options Template, it SHOULD NOT log the retransmission, as this is the normal operation of Template refresh over UDP.
Top   ToC   RFC7011 - Page 45
   When a Collecting Process receives a new Template Record or Options
   Template Record for an already-allocated Template ID, and that
   Template or Options Template is different from the already-received
   Template or Options Template, the Collecting Process MUST replace the
   Template or Options Template for that Template ID with the newly
   received Template or Options Template.  This is the normal operation
   of Template ID reuse over UDP.

   As Template IDs are unique per UDP session and per Observation
   Domain, at any given time, the Collecting Process SHOULD maintain the
   following for all the current Template Records and Options Template
   Records: <IPFIX Device, Exporter source UDP port, Collector IP
   address, Collector destination UDP port, Observation Domain ID,
   Template ID, Template Definition, Last Received>.

9. The Collecting Process's Side

This section describes the handling of the IPFIX protocol at the Collecting Process common to all transport protocols. Additional considerations for SCTP and UDP are provided in Sections 9.2 and 9.3, respectively. Template management at Collecting Processes is covered in Section 8. The Collecting Process MUST listen for association requests / connections to start new Transport Sessions from the Exporting Process. The Collecting Process MUST note the Information Element identifier of any Information Element that it does not understand and MAY discard that Information Element from received Data Records. The Collecting Process MUST accept padding in Data Records and Template Records. The padding size is the Set Length minus the size of the Set Header (4 octets for the Set ID and the Set Length), modulo the minimum Record size deduced from the Template Record. The IPFIX protocol has a Sequence Number field in the Export header that increases with the number of IPFIX Data Records in the IPFIX Message. A Collector can detect out-of-sequence, dropped, or duplicate IPFIX Messages by tracking the Sequence Number. A Collector SHOULD provide a logging mechanism for tracking out-of- sequence IPFIX Messages. Such out-of-sequence IPFIX Messages may be due to Exporter resource exhaustion where it cannot transmit messages at their creation rate, an Exporting Process reset, congestion on the network link between the Exporter and Collector, Collector resource exhaustion where it cannot process the IPFIX Messages at their arrival rate, out-of-order packet reception, duplicate packet reception, or an attacker injecting false messages.
Top   ToC   RFC7011 - Page 46

9.1. Collecting Process Handling of Malformed IPFIX Messages

If the Collecting Process receives a malformed IPFIX Message, it MUST discard the IPFIX Message and SHOULD log the error. A malformed IPFIX Message is one that cannot be interpreted due to nonsensical length values (e.g., a variable-length Information Element longer than its enclosing Set, a Set longer than its enclosing IPFIX Message, or an IPFIX Message shorter than an IPFIX Message Header) or a reserved Version value (which may indicate that a future version of IPFIX is being used for export but in practice occurs most often when non-IPFIX data is sent to an IPFIX Collecting Process). Note that non-zero Set padding does not constitute a malformed IPFIX Message. As the most likely cause of malformed IPFIX Messages is a poorly implemented Exporting Process, or the sending of non-IPFIX data to an IPFIX Collecting Process, human intervention is likely necessary to correct the issue. In the meantime, the Collecting Process MAY attempt to rectify the situation any way it sees fit, including: - terminating the TCP connection or SCTP connection - using the receiver window to reduce network load from the malfunctioning Exporting Process - buffering and saving malformed IPFIX Message(s) to assist in diagnosis - attempting to resynchronize the stream, e.g., as described in Section 10.3 of [RFC5655] Resynchronization should only be attempted if the Collecting Process has reason to believe that the error is transient. On the other hand, the Collecting Process SHOULD stop processing IPFIX Messages from clearly malfunctioning Exporting Processes (e.g., those from which the last few IPFIX Messages have been malformed).

9.2. Additional Considerations for SCTP Collecting Processes

As an Exporting Process may request and support more than one stream per SCTP association, the Collecting Process MUST support the opening of multiple SCTP Streams.

9.3. Additional Considerations for UDP Collecting Processes

A Transport Session for IPFIX Messages transported over UDP is defined from the point of view of the Exporting Process and roughly corresponds to the time during which a given Exporting Process sends IPFIX Messages over UDP to a given Collecting Process. Since this is
Top   ToC   RFC7011 - Page 47
   difficult to detect at the Collecting Process, the Collecting Process
   MAY discard all Transport Session state after no IPFIX Messages are
   received from a given Exporting Process within a given Transport
   Session during a configurable idle timeout.

   The Collecting Process SHOULD accept Data Records without the
   associated Template Record (or other definitions such as Common
   Properties) required to decode the Data Record.  If the Template
   Records or other definitions have not been received at the time Data
   Records are received, the Collecting Process MAY store the Data
   Records for a short period of time and decode them after the Template
   Records or other definitions are received, comparing Export Times of
   IPFIX Messages containing the Template Records with those containing
   the Data Records as discussed in Section 8.2.  Note that this
   mechanism may lead to incorrectly interpreted records in the presence
   of Template ID reuse or other identifiers with limited lifetimes.

10. Transport Protocol

The IPFIX Protocol Specification has been designed to be transport protocol independent. Note that the Exporter can export to multiple Collecting Processes using independent transport protocols. The IPFIX Message Header 16-bit Length field limits the length of an IPFIX Message to 65535 octets, including the header. A Collecting Process MUST be able to handle IPFIX Message lengths of up to 65535 octets. While an Exporting Process or Collecting Process may support multiple transport protocols, Transport Sessions are bound to a transport protocol. Transport Session state MUST NOT be migrated by an Exporting Process or Collecting Process among Transport Sessions using different transport protocols between the same Exporting Process and Collecting Process pair. In other words, an Exporting Process supporting multiple transport protocols is conceptually multiple Exporting Processes, one per supported transport protocol. Likewise, a Collecting Process supporting multiple transport protocols is conceptually multiple Collecting Processes, one per supported transport protocol.

10.1. Transport Compliance and Transport Usage

SCTP [RFC4960] using the Partially Reliable SCTP (PR-SCTP) extension as specified in [RFC3758] MUST be implemented by all compliant implementations. UDP [UDP] MAY also be implemented by compliant implementations. TCP [TCP] MAY also be implemented by compliant implementations.
Top   ToC   RFC7011 - Page 48
   SCTP should be used in deployments where Exporters and Collectors are
   communicating over links that are susceptible to congestion.  SCTP is
   capable of providing any required degree of reliability when used
   with the PR-SCTP extension.

   TCP may be used in deployments where Exporters and Collectors
   communicate over links that are susceptible to congestion, but SCTP
   is preferred, due to its ability to limit back pressure on Exporters
   and its message-versus-stream orientation.

   UDP may be used, although it is not a congestion-aware protocol.
   However, in this case the IPFIX traffic between the Exporter and
   Collector must be separately contained or provisioned to minimize the
   risk of congestion-related loss.

   By default, the Collecting Process listens for connections on SCTP,
   TCP, and/or UDP port 4739.  By default, the Collecting Process
   listens for secure connections on SCTP, TCP, and/or UDP port 4740
   (refer to the Security Considerations section).  By default, the
   Exporting Process attempts to connect to one of these ports.  It MUST
   be possible to configure both the Exporting and Collecting Processes
   to use different ports than the default.

10.2. SCTP

This section describes how IPFIX is transported over SCTP [RFC4960] using the PR-SCTP [RFC3758] extension.

10.2.1. Congestion Avoidance

SCTP provides the required level of congestion avoidance by design. SCTP detects congestion in the end-to-end path between the IPFIX Exporting Process and the IPFIX Collecting Process, and limits the transfer rate accordingly. When an IPFIX Exporting Process has records to export but detects that transmission by SCTP is temporarily impossible, it can either wait until sending is possible again or decide to drop the record. In the latter case, the dropped export data SHOULD be accounted for, so that the amount of dropped export data can be reported using the mechanism described in Section 4.3.
Top   ToC   RFC7011 - Page 49

10.2.2. Reliability

The SCTP transport protocol is by default reliable but has the capability to deliver messages with partial reliability [RFC3758]. Using reliable SCTP messages for IPFIX export is not in itself a guarantee that all Data Records will be delivered. If there is congestion on the link from the Exporting Process to the Collecting Process, or if a significant number of retransmissions are required, the send queues on the Exporting Process may fill up; the Exporting Process MAY either suspend, export, or discard the IPFIX Messages. If Data Records are discarded, the IPFIX Sequence Numbers used for export MUST reflect the loss of data.

10.2.3. MTU

SCTP provides the required IPFIX Message fragmentation service based on Path MTU (PMTU) discovery.

10.2.4. Association Establishment and Shutdown

The IPFIX Exporting Process initiates an SCTP association with the IPFIX Collecting Process. The Exporting Process MAY establish more than one association (connection "bundle" in SCTP terminology) to the Collecting Process. An Exporting Process MAY support more than one active association to different Collecting Processes (including the case of different Collecting Processes on the same host). When an Exporting Process is shut down, it SHOULD shut down the SCTP association. When a Collecting Process no longer wants to receive IPFIX Messages, it SHOULD shut down its end of the association. The Collecting Process SHOULD continue to receive and process IPFIX Messages until the Exporting Process has closed its end of the association. When a Collecting Process detects that the SCTP association has been abnormally terminated, it MUST continue to listen for a new association establishment. When an Exporting Process detects that the SCTP association to the Collecting Process is abnormally terminated, it SHOULD try to re-establish the association. Association timeouts SHOULD be configurable.
Top   ToC   RFC7011 - Page 50

10.2.5. Failover

If the Collecting Process does not acknowledge an attempt by the Exporting Process to establish an association, SCTP will automatically retry association establishment using exponential backoff. The Exporter MAY log an alarm if the underlying SCTP association establishment times out; this timeout should be configurable on the Exporter. The Exporting Process MAY open a backup SCTP association to a Collecting Process in advance, if it supports Collecting Process failover.

10.2.6. Streams

An Exporting Process MAY request more than one SCTP Stream per association. Each of these streams may be used for the transmission of IPFIX Messages containing Data Sets, Template Sets, and/or Options Template Sets. Depending on the requirements of the application, the Exporting Process may send Data Sets with full or partial reliability, using ordered or out-of-order delivery, over any SCTP Stream established during SCTP association setup. An IPFIX Exporting Process MAY use any PR-SCTP service definition as per Section 4 of the PR-SCTP specification [RFC3758] when using partial reliability to transmit IPFIX Messages containing only Data Sets. However, Exporting Processes SHOULD mark such IPFIX Messages for retransmission for as long as resource or other constraints allow.

10.3. UDP

This section describes how IPFIX is transported over UDP [UDP].

10.3.1. Congestion Avoidance

UDP has no integral congestion-avoidance mechanism. Its use over congestion-sensitive network paths is therefore not recommended. UDP MAY be used in deployments where Exporters and Collectors always communicate over dedicated links that are not susceptible to congestion, i.e., links that are over-provisioned compared to the maximum export rate from the Exporters.
Top   ToC   RFC7011 - Page 51

10.3.2. Reliability

UDP is not a reliable transport protocol and cannot guarantee delivery of messages. IPFIX Messages sent from the Exporting Process to the Collecting Process using UDP may therefore be lost. UDP MUST NOT be used unless the application can tolerate some loss of IPFIX Messages. The Collecting Process SHOULD deduce the loss and reordering of IPFIX Data Records by looking at the discontinuities in the IPFIX Sequence Number. In the case of UDP, the IPFIX Sequence Number contains the total number of IPFIX Data Records sent for the Transport Session prior to the receipt of this IPFIX Message, modulo 2^32. A Collector SHOULD detect out-of-sequence, dropped, or duplicate IPFIX Messages by tracking the Sequence Number. Exporting Processes exporting IPFIX Messages via UDP MUST include a valid UDP checksum [UDP] in UDP datagrams including IPFIX Messages.

10.3.3. MTU

The maximum size of exported messages MUST be configured such that the total packet size does not exceed the PMTU. If the PMTU is unknown, a maximum packet size of 512 octets SHOULD be used.

10.3.4. Session Establishment and Shutdown

As UDP is a connectionless protocol, there is no real session establishment or shutdown for IPFIX over UDP. An Exporting Process starts sending IPFIX Messages to a Collecting Process at one point in time and stops sending them at another point in time. This can lead to some complications in Template management, as outlined in Section 8.4 above.

10.3.5. Failover and Session Duplication

Because UDP is not a connection-oriented protocol, the Exporting Process is unable to determine from the transport protocol that the Collecting Process is no longer able to receive the IPFIX Messages. Therefore, it cannot invoke a failover mechanism. However, the Exporting Process MAY duplicate the IPFIX Message to several Collecting Processes.
Top   ToC   RFC7011 - Page 52

10.4. TCP

This section describes how IPFIX is transported over TCP [TCP].

10.4.1. Congestion Avoidance

TCP controls the rate at which data can be sent from the Exporting Process to the Collecting Process, using a mechanism that takes into account both congestion in the network and the capabilities of the receiver. Therefore, an IPFIX Exporting Process may not be able to send IPFIX Messages at the rate that the Metering Process generates them, either because of congestion in the network or because the Collecting Process cannot handle IPFIX Messages fast enough. As long as congestion is transient, the Exporting Process can buffer IPFIX Messages for transmission. But such buffering is necessarily limited, both because of resource limitations and because of timeliness requirements, so ongoing and/or severe congestion may lead to a situation where the Exporting Process is blocked. When an Exporting Process has Data Records to export but the transmission buffer is full, and it wants to avoid blocking, it can decide to drop some Data Records. The dropped Data Records MUST be accounted for, so that the number of lost records can later be reported as described in Section 4.3.

10.4.2. Reliability

TCP ensures reliable delivery of data from the Exporting Process to the Collecting Process.

10.4.3. MTU

As TCP offers a stream service instead of a datagram or sequential packet service, IPFIX Messages transported over TCP are instead separated using the Length field in the IPFIX Message Header. The Exporting Process can choose any valid length for exported IPFIX Messages, as TCP handles segmentation. Exporting Processes may choose IPFIX Message lengths lower than the maximum in order to ensure timely export of Data Records.
Top   ToC   RFC7011 - Page 53

10.4.4. Connection Establishment and Shutdown

The IPFIX Exporting Process initiates a TCP connection to the Collecting Process. An Exporting Process MAY support more than one active connection to different Collecting Processes (including the case of different Collecting Processes on the same host). An Exporting Process MAY support more than one active connection to the same Collecting Process to avoid head-of-line blocking across Observation Domains. The Exporter MAY log an alarm if the underlying TCP connection establishment times out; this timeout should be configurable on the Exporter. When an Exporting Process is shut down, it SHOULD shut down the TCP connection. When a Collecting Process no longer wants to receive IPFIX Messages, it SHOULD close its end of the connection. The Collecting Process SHOULD continue to read IPFIX Messages until the Exporting Process has closed its end. When a Collecting Process detects that the TCP connection to the Exporting Process has terminated abnormally, it MUST continue to listen for a new connection. When an Exporting Process detects that the TCP connection to the Collecting Process has terminated abnormally, it SHOULD try to re-establish the connection. Connection timeouts and retry schedules SHOULD be configurable. In the default configuration, an Exporting Process MUST NOT attempt to establish a connection more frequently than once per minute.

10.4.5. Failover

If the Collecting Process does not acknowledge an attempt by the Exporting Process to establish a connection, TCP will automatically retry connection establishment using exponential backoff. The Exporter MAY log an alarm if the underlying TCP connection establishment times out; this timeout should be configurable on the Exporter. The Exporting Process MAY open a backup TCP connection to a Collecting Process in advance, if it supports Collecting Process failover.
Top   ToC   RFC7011 - Page 54

11. Security Considerations

The security considerations for the IPFIX protocol have been derived from an analysis of potential security threats, as discussed in the Security Considerations section of the IPFIX requirements document [RFC3917]. The requirements for IPFIX security are as follows: 1. IPFIX must provide a mechanism to ensure the confidentiality of IPFIX data transferred from an Exporting Process to a Collecting Process, in order to prevent disclosure of Flow Records transported via IPFIX. 2. IPFIX must provide a mechanism to ensure the integrity of IPFIX data transferred from an Exporting Process to a Collecting Process, in order to prevent the injection of incorrect data or control information (e.g., Templates), or the duplication of Messages, in an IPFIX Message stream. 3. IPFIX must provide a mechanism to authenticate IPFIX Collecting and Exporting Processes, to prevent the collection of data from an unauthorized Exporting Process or the export of data to an unauthorized Collecting Process. Because IPFIX can be used to collect information for network forensics and billing purposes, attacks designed to confuse, disable, or take information from an IPFIX collection system may be seen as a prime objective during a sophisticated network attack. An attacker in a position to inject false messages into an IPFIX Message stream can affect either the application using IPFIX (by falsifying data) or the IPFIX Collecting Process itself (by modifying or revoking Templates, or changing options); for this reason, IPFIX Message integrity is important. The IPFIX Messages themselves may also contain information of value to an attacker, including information about the configuration of the network as well as end-user traffic and payload data, so care must be taken to confine their visibility to authorized users. When an Information Element containing end-user payload information is exported, it SHOULD be transmitted to the Collecting Process using a means that secures its contents against eavesdropping. Suitable mechanisms include the use of either a direct point-to-point connection assumed to be unavailable to attackers, or the use of an encryption mechanism. It is the responsibility of the Collecting Process to provide a satisfactory degree of security for this collected data, including, if necessary, encryption and/or anonymization of any reported data; see Section 11.8.
Top   ToC   RFC7011 - Page 55

11.1. Applicability of TLS and DTLS

Transport Layer Security (TLS) [RFC5246] and Datagram Transport Layer Security (DTLS) [RFC6347] were designed to provide the confidentiality, integrity, and authentication assurances required by the IPFIX protocol, without the need for pre-shared keys. IPFIX Exporting Processes and Collecting Processes using TCP MUST support TLS version 1.1 and SHOULD support TLS version 1.2 [RFC5246], including the mandatory ciphersuite(s) specified in each version. IPFIX Exporting Processes and Collecting Processes using UDP or SCTP MUST support DTLS version 1.0 and SHOULD support DTLS version 1.2 [RFC6347], including the mandatory ciphersuite(s) specified in each version. Note that DTLS is selected as the security mechanism for SCTP. Though TLS bindings to SCTP are defined in [RFC3436], they require that all communication be over reliable, bidirectional streams, and they also require one TLS connection per stream. This arrangement is not compatible with the rationale behind the choice of SCTP as an IPFIX transport protocol. Note that using DTLS has a man-in-the-middle vulnerability not present in TLS, allowing a message to be removed from the stream without the knowledge of either the sender or receiver. Additionally, when using DTLS over SCTP, an attacker could inject SCTP control information and shut down the SCTP association, causing a loss of IPFIX Messages if those messages are buffered outside of the SCTP association. Techniques such as those described in [RFC6083] could be used to overcome these vulnerabilities. When using DTLS over SCTP, the Exporting Process MUST ensure that each IPFIX Message is sent over the same SCTP Stream that would be used when sending the same IPFIX Message directly over SCTP. Note that DTLS may send its own control messages on stream 0 with full reliability; however, this will not interfere with the processing of stream 0 IPFIX Messages at the Collecting Process, because DTLS consumes its own control messages before passing IPFIX Messages up to the application layer. When using DTLS over SCTP or UDP, the Heartbeat Extension [RFC6520] SHOULD be used, especially on long-lived Transport Sessions, to ensure that the association remains active. Exporting and Collecting Processes MUST NOT request, offer, or use any version of the Secure Socket Layer (SSL), or any version of TLS prior to 1.1, due to known security vulnerabilities in prior versions of TLS; see Appendix E of [RFC5246] for more information.
Top   ToC   RFC7011 - Page 56

11.2. Usage

The IPFIX Exporting Process initiates the communication to the IPFIX Collecting Process and acts as a TLS or DTLS client according to [RFC5246] and [RFC6347], while the IPFIX Collecting Process acts as a TLS or DTLS server. The DTLS client opens a secure connection on SCTP port 4740 of the DTLS server if SCTP is selected as the transport protocol. The TLS client opens a secure connection on TCP port 4740 of the TLS server if TCP is selected as the transport protocol. The DTLS client opens a secure connection on UDP port 4740 of the DTLS server if UDP is selected as the transport protocol.

11.3. Mutual Authentication

When using TLS or DTLS, IPFIX Exporting Processes and IPFIX Collecting Processes SHOULD be identified by a certificate containing the DNS-ID as discussed in Section 6.4 of [RFC6125]; the inclusion of Common Names (CN-IDs) in certificates identifying IPFIX Exporting Processes or Collecting Processes is NOT RECOMMENDED. To prevent man-in-the-middle attacks from impostor Exporting or Collecting Processes, the acceptance of data from an unauthorized Exporting Process, or the export of data to an unauthorized Collecting Process, mutual authentication MUST be used for both TLS and DTLS. Exporting Processes MUST verify the reference identifiers of the Collecting Processes to which they are exporting IPFIX Messages against those stored in the certificates. Likewise, Collecting Processes MUST verify the reference identifiers of the Exporting Processes from which they are receiving IPFIX Messages against those stored in the certificates. Exporting Processes MUST NOT export to non-verified Collecting Processes, and Collecting Processes MUST NOT accept IPFIX Messages from non-verified Exporting Processes. Exporting Processes and Collecting Processes MUST support the verification of certificates against an explicitly authorized list of peer certificates identified by Common Name and SHOULD support the verification of reference identifiers by matching the DNS-ID or CN-ID with a DNS lookup of the peer. IPFIX Exporting Processes and Collecting Processes MUST use non-NULL ciphersuites for authentication, integrity, and confidentiality.
Top   ToC   RFC7011 - Page 57

11.4. Protection against DoS Attacks

An attacker may mount a denial-of-service (DoS) attack against an IPFIX collection system either directly, by sending large amounts of traffic to a Collecting Process, or indirectly, by generating large amounts of traffic to be measured by a Metering Process. Direct DoS attacks can also involve state exhaustion, whether at the transport layer (e.g., by creating a large number of pending connections) or within the IPFIX Collecting Process itself (e.g., by sending Flow Records pending Template or scope information, or a large amount of Options Template Records, etc.). SCTP mandates a cookie-exchange mechanism designed to defend against SCTP state exhaustion DoS attacks. Similarly, TCP provides the "SYN cookie" mechanism to mitigate state exhaustion; SYN cookies SHOULD be used by any Collecting Process accepting TCP connections. DTLS also provides cookie exchange to protect against DTLS server state exhaustion. The reader should note that there is no way to prevent fake IPFIX Message processing (and state creation) for UDP and SCTP communication. The use of TLS and DTLS can obviously prevent the creation of fake states, but they are themselves prone to state exhaustion attacks. Therefore, Collector rate limiting SHOULD be used to protect TLS and DTLS (like limiting the number of new TLS or DTLS sessions per second to a sensible number). IPFIX state exhaustion attacks can be mitigated by limiting the rate at which new connections or associations will be opened by the Collecting Process; limiting the rate at which IPFIX Messages will be accepted by the Collecting Process; and adaptively limiting the amount of state kept, particularly for records waiting for Templates. These rate and state limits MAY be provided by a Collecting Process, and if provided, the limits SHOULD be user configurable. Additionally, an IPFIX Collecting Process can eliminate the risk of state exhaustion attacks from untrusted nodes by requiring TLS or DTLS mutual authentication, causing the Collecting Process to accept IPFIX Messages only from trusted sources. With respect to indirect denial of service, the behavior of IPFIX under overload conditions depends on the transport protocol in use. For IPFIX over TCP, TCP congestion control would cause the flow of IPFIX Messages to back off and eventually stall, blinding the IPFIX system. SCTP improves upon this situation somewhat, as some IPFIX Messages would continue to be received by the Collecting Process due to the avoidance of head-of-line blocking by SCTP's multiple streams
Top   ToC   RFC7011 - Page 58
   and partial reliability features, possibly affording some visibility
   of the attack.  The situation is similar with UDP, as some datagrams
   may continue to be received at the Collecting Process, effectively
   applying sampling to the IPFIX Message stream and implying that some
   information about the attack will be available.

   To minimize IPFIX Message loss under overload conditions, some
   mechanism for service differentiation could be used to prioritize
   IPFIX traffic over other traffic on the same link.  Alternatively,
   IPFIX Messages can be transported over a dedicated network.  In this
   case, care must be taken to ensure that the dedicated network can
   handle the expected peak IPFIX Message traffic.

11.5. When DTLS or TLS Is Not an Option

The use of DTLS or TLS might not be possible in some cases, due to performance issues or other operational concerns. Without TLS or DTLS mutual authentication, IPFIX Exporting Processes and Collecting Processes can fall back on using IP source addresses to authenticate their peers. A policy of allocating Exporting Process and Collecting Process IP addresses from specified address ranges, and using ingress filtering to prevent spoofing, can improve the usefulness of this approach. Again, completely segregating IPFIX traffic on a dedicated network, where possible, can improve security even further. In any case, the use of open Collecting Processes (those that will accept IPFIX Messages from any Exporting Process regardless of IP address or identity) is discouraged. Modern TCP and SCTP implementations are resistant to blind insertion attacks (see [RFC4960] and [RFC6528]); however, UDP offers no such protection. For this reason, IPFIX Message traffic transported via UDP and not secured via DTLS SHOULD be protected via segregation to a dedicated network.

11.6. Logging an IPFIX Attack

IPFIX Collecting Processes MUST detect potential IPFIX Message insertion or loss conditions by tracking the IPFIX Sequence Number and SHOULD provide a logging mechanism for reporting out-of-sequence messages. Note that an attacker may be able to exploit the handling of out-of-sequence messages at the Collecting Process, so care should be taken in handling these conditions. For example, a Collecting Process that simply resets the expected Sequence Number upon receipt of a later Sequence Number could be temporarily blinded by deliberate injection of later Sequence Numbers.
Top   ToC   RFC7011 - Page 59
   IPFIX Exporting and Collecting Processes SHOULD log any connection
   attempt that fails due to authentication failure, whether due to
   being presented an unauthorized or mismatched certificate during TLS
   or DTLS mutual authentication, or due to a connection attempt from an
   unauthorized IP address when TLS or DTLS is not in use.

   IPFIX Exporting and Collecting Processes SHOULD detect and log any
   SCTP association reset or TCP connection reset.

11.7. Securing the Collector

The security of the Collector and its implementation is important to achieve overall security; however, a complete set of security guidelines for Collector implementation is outside the scope of this document. As IPFIX uses length-prefix encodings, Collector implementors should take care to ensure the detection of inconsistent values that could impact IPFIX Message decoding, and proper operation in the presence of such inconsistent values. Specifically, IPFIX Message, Set, and variable-length Information Element lengths must be checked for consistency to avoid buffer- sizing vulnerabilities. Collector implementors should also pay special attention to UTF-8 encoding of string data types, as vulnerabilities may exist in the interpretation of ill-formed UTF-8 values; see Section 6.1.6.

11.8. Privacy Considerations for Collected Data

Flow data exported by Exporting Processes and collected by Collecting Processes typically contains information about traffic on the observed network. This information may be personally identifiable and privacy-sensitive. The storage of this data must be protected via technical as well as policy means to ensure that the privacy of the users of the measured network is protected. A complete specification of such means is out of scope for this document and is specific to the application and storage technology used.
Top   ToC   RFC7011 - Page 60

12. Management Considerations

[RFC6615] specifies a MIB module that defines managed objects for monitoring IPFIX Devices, including basic configuration. This MIB can be used to measure the impact of IPFIX export on the monitoring network; it contains tables covering: Transport Session, Cache definition, Observation Point definition, Template and Options Template definition, export features (failover, load-balancing, duplicate), and export statistics per Process, Session, and Template From an operational aspect, an important function of this MIB module is provided by the Transport Session Statistical table, which contains the rate (in bytes per second) at which the Collector receives or the Exporter sends out IPFIX Messages. Of particular interest to operations, the Transport Session Statistical table in Section 5.8.1 of this MIB module exposes the rate of collection or export of IPFIX Messages, which allows the measurement of the bandwidth used by IPFIX export. [RFC6727] describes extensions to the IPFIX-SELECTOR-MIB module specified in [RFC6615] and contains managed objects for providing information on applied packet selection functions and their parameters (filtering and sampling). Since the IPFIX-SELECTOR-MIB [RFC6615] and PSAMP-MIB [RFC6727] modules only contain read-only objects, they cannot be used for configuration of IPFIX Devices. [RFC6728] specifies a configuration data model for the IPFIX and PSAMP protocols, using the Network Configuration Protocol (NETCONF). This data model covers Selection Processes, Caches, Exporting Processes, and Collecting Processes on IPFIX and PSAMP Devices, and is defined using UML (Unified Modeling Language) class diagrams and formally specified using YANG. The configuration data is encoded in Extensible Markup Language (XML). A few mechanisms specified alongside the IPFIX protocol can help monitor and reduce bandwidth used for IPFIX Export: - a bandwidth-saving method for exporting redundant information in IPFIX [RFC5473] - an efficient method for exporting bidirectional flows [RFC5103] - a method for the definition and export of complex data structures [RFC6313]
Top   ToC   RFC7011 - Page 61
   Alternatively, PSAMP [RFC5474] can be used to export packets sampled
   by statistical and other methods, which may be applicable to many
   monitoring areas for which IPFIX is also suited.  PSAMP also provides
   control over the impact on the measured network through its sampling
   rate.  The set of packet selection techniques (Sampling, Filtering,
   and hashing) standardized by PSAMP is described in [RFC5475].  PSAMP
   also defines an explicitly configurable export rate limit in
   Section 8.4 of [RFC5474].

13. IANA Considerations

IANA has updated the "IPFIX Information Elements" registry [IANA-IPFIX] so that all references that previously pointed to RFC 5101 now point to this document instead. IPFIX Messages use two fields with assigned values. These are the IPFIX Version Number, indicating which version of the IPFIX protocol was used to export an IPFIX Message, and the IPFIX Set ID, indicating the type for each set of information within an IPFIX Message. The Information Elements used by IPFIX, and sub-registries of Information Element values, are managed by IANA [IANA-IPFIX], as are the Private Enterprise Numbers used by enterprise-specific Information Elements [IANA-PEN]. This document makes no changes to these registries. The IPFIX Version Number value of 0x000a (10) is reserved for the IPFIX protocol specified in this document. Set ID values of 0 and 1 are not used, for historical reasons [RFC3954]. The Set ID value of 2 is reserved for the Template Set. The Set ID value of 3 is reserved for the Options Template Set. All other Set ID values from 4 to 255 are reserved for future use. Set ID values above 255 are used for Data Sets. New assignments in either the "IPFIX Version Number" or "IPFIX Set IDs" sub-registries require a Standards Action [RFC5226], i.e., they are to be made via Standards Track RFCs approved by the IESG.


(next page on part 4)

Next Section