tech-invite   World Map     

IETF     RFCs     Groups     SIP     ABNFs    |    3GPP     Specs     Gloss.     Arch.     IMS     UICC    |    Misc.    |    search     info

RFC 7011

 Errata 
STD 77
Pages: 76
Top     in Index     Prev     Next
in Group Index     Prev in Group     Next in Group     Group: IPFIX

Specification of the IP Flow Information Export (IPFIX) Protocol for the Exchange of Flow Information

Part 1 of 4, p. 1 to 13
None       Next RFC Part

Obsoletes:    5101


Top       ToC       Page 1 
Internet Engineering Task Force (IETF)                    B. Claise, Ed.
Request for Comments: 7011                           Cisco Systems, Inc.
STD: 77                                                 B. Trammell, Ed.
Obsoletes: 5101                                               ETH Zurich
Category: Standards Track                                      P. Aitken
ISSN: 2070-1721                                      Cisco Systems, Inc.
                                                          September 2013


    Specification of the IP Flow Information Export (IPFIX) Protocol
                  for the Exchange of Flow Information

Abstract

   This document specifies the IP Flow Information Export (IPFIX)
   protocol, which serves as a means for transmitting Traffic Flow
   information over the network.  In order to transmit Traffic Flow
   information from an Exporting Process to a Collecting Process, a
   common representation of flow data and a standard means of
   communicating them are required.  This document describes how the
   IPFIX Data and Template Records are carried over a number of
   transport protocols from an IPFIX Exporting Process to an IPFIX
   Collecting Process.  This document obsoletes RFC 5101.

Status of This Memo

   This is an Internet Standards Track document.

   This document is a product of the Internet Engineering Task Force
   (IETF).  It represents the consensus of the IETF community.  It has
   received public review and has been approved for publication by the
   Internet Engineering Steering Group (IESG).  Further information on
   Internet Standards is available in Section 2 of RFC 5741.

   Information about the current status of this document, any errata,
   and how to provide feedback on it may be obtained at
   http://www.rfc-editor.org/info/rfc7011.

Page 2 
Copyright Notice

   Copyright (c) 2013 IETF Trust and the persons identified as the
   document authors.  All rights reserved.

   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents
   (http://trustee.ietf.org/license-info) in effect on the date of
   publication of this document.  Please review these documents
   carefully, as they describe your rights and restrictions with respect
   to this document.  Code Components extracted from this document must
   include Simplified BSD License text as described in Section 4.e of
   the Trust Legal Provisions and are provided without warranty as
   described in the Simplified BSD License.

Table of Contents

   1. Introduction ....................................................5
      1.1. Changes since RFC 5101 .....................................5
      1.2. IPFIX Documents Overview ...................................6
   2. Terminology .....................................................7
      2.1. Terminology Summary Table .................................13
   3. IPFIX Message Format ...........................................13
      3.1. Message Header Format .....................................15
      3.2. Field Specifier Format ....................................16
      3.3. Set and Set Header Format .................................18
           3.3.1. Set Format .........................................18
           3.3.2. Set Header Format ..................................19
      3.4. Record Format .............................................20
           3.4.1. Template Record Format .............................20
           3.4.2. Options Template Record Format .....................23
                  3.4.2.1. Scope .....................................23
                  3.4.2.2. Options Template Record Format ............24
           3.4.3. Data Record Format .................................27
   4. Specific Reporting Requirements ................................28
      4.1. The Metering Process Statistics Options Template ..........29
      4.2. The Metering Process Reliability Statistics
           Options Template ..........................................29
      4.3. The Exporting Process Reliability Statistics
           Options Template ..........................................31
      4.4. The Flow Keys Options Template ............................32
   5. Timing Considerations ..........................................32
      5.1. IPFIX Message Header Export Time and Flow Record Time .....32
      5.2. Supporting Timestamp Wraparound ...........................33

Top      ToC       Page 3 
   6. Linkage with the Information Model .............................34
      6.1. Encoding of IPFIX Data Types ..............................34
           6.1.1. Integral Data Types ................................34
           6.1.2. Address Types ......................................34
           6.1.3. float32 ............................................34
           6.1.4. float64 ............................................34
           6.1.5. boolean ............................................35
           6.1.6. string and octetArray ..............................35
           6.1.7. dateTimeSeconds ....................................35
           6.1.8. dateTimeMilliseconds ...............................35
           6.1.9. dateTimeMicroseconds ...............................35
           6.1.10. dateTimeNanoseconds ...............................36
      6.2. Reduced-Size Encoding .....................................36
   7. Variable-Length Information Element ............................37
   8. Template Management ............................................38
      8.1. Template Withdrawal and Redefinition ......................40
      8.2. Sequencing Template Management Actions ....................42
      8.3. Additional Considerations for Template Management
           over SCTP .................................................43
      8.4. Additional Considerations for Template Management
           over UDP ..................................................44
   9. The Collecting Process's Side ..................................45
      9.1. Collecting Process Handling of Malformed IPFIX Messages ...46
      9.2. Additional Considerations for SCTP Collecting Processes ...46
      9.3. Additional Considerations for UDP Collecting Processes ....46
   10. Transport Protocol ............................................47
      10.1. Transport Compliance and Transport Usage .................47
      10.2. SCTP .....................................................48
           10.2.1. Congestion Avoidance ..............................48
           10.2.2. Reliability .......................................49
           10.2.3. MTU ...............................................49
           10.2.4. Association Establishment and Shutdown ............49
           10.2.5. Failover ..........................................50
           10.2.6. Streams ...........................................50
      10.3. UDP ......................................................50
           10.3.1. Congestion Avoidance ..............................50
           10.3.2. Reliability .......................................51
           10.3.3. MTU ...............................................51
           10.3.4. Session Establishment and Shutdown ................51
           10.3.5. Failover and Session Duplication ..................51
      10.4. TCP ......................................................52
           10.4.1. Congestion Avoidance ..............................52
           10.4.2. Reliability .......................................52
           10.4.3. MTU ...............................................52
           10.4.4. Connection Establishment and Shutdown .............53
           10.4.5. Failover ..........................................53

Top      ToC       Page 4 
   11. Security Considerations .......................................54
      11.1. Applicability of TLS and DTLS ............................55
      11.2. Usage ....................................................56
      11.3. Mutual Authentication ....................................56
      11.4. Protection against DoS Attacks ...........................57
      11.5. When DTLS or TLS Is Not an Option ........................58
      11.6. Logging an IPFIX Attack ..................................58
      11.7. Securing the Collector ...................................59
      11.8. Privacy Considerations for Collected Data ................59
   12. Management Considerations .....................................60
   13. IANA Considerations ...........................................61
   Appendix A. IPFIX Encoding Examples ...............................62
      A.1. Message Header Example ....................................62
      A.2. Template Set Examples .....................................63
        A.2.1. Template Set Using IANA Information Elements ..........63
        A.2.2. Template Set Using Enterprise-Specific Information
               Elements ..............................................64
      A.3. Data Set Example ..........................................65
      A.4. Options Template Set Examples .............................66
        A.4.1. Options Template Set Using IANA Information Elements ..66
        A.4.2. Options Template Set Using Enterprise-Specific
               Information Elements ..................................66
        A.4.3. Options Template Set Using an Enterprise-Specific
               Scope .................................................67
        A.4.4. Data Set Using an Enterprise-Specific Scope ...........68
      A.5. Variable-Length Information Element Examples ..............69
        A.5.1. Example of Variable-Length Information Element with
               Length Less Than 255 Octets ...........................69
        A.5.2. Example of Variable-Length Information Element with
               3-Octet Length Encoding ...............................70
   Normative References ..............................................71
   Informative References ............................................71
   Acknowledgments ...................................................74
   Contributors ......................................................75

Top      ToC       Page 5 
1.  Introduction

   Traffic on a data network can be seen as consisting of flows passing
   through network elements.  For administrative or other purposes, it
   is often interesting, useful, or even necessary to have access to
   information about these flows that pass through the network elements.
   A Collecting Process should be able to receive the Flow information
   passing through multiple network elements within the data network.
   This requires uniformity in the method of representing the flow
   information and the means of communicating the flows from the network
   elements to the collection point.  This document specifies a protocol
   to achieve these requirements.  This document specifies in detail the
   representation of different flows, as well as the additional data
   required for flow interpretation, packet format, transport mechanisms
   used, security concerns, etc.

1.1.  Changes since RFC 5101

   This document obsoletes the Proposed Standard revision of the IPFIX
   Protocol Specification [RFC5101].  The protocol specified by this
   document is interoperable with the protocol as specified in
   [RFC5101].  The following changes have been made to this document
   with respect to the previous document:

   - All outstanding technical and editorial errata on [RFC5101] have
     been addressed.

   - As the [IANA-IPFIX] registry is now the normative reference for all
     Information Element definitions (see [RFC7012]), all definitions of
     Information Elements in Section 4 have been replaced with
     references to that registry.

   - The encoding of the dateTimeSeconds, dateTimeMilliseconds,
     dateTimeMicroseconds, and dateTimeNanoseconds data types, and the
     related encoding of the IPFIX Message Header Export Time field,
     have been clarified, especially with respect to the epoch upon
     which the timestamp data types are based.

   - A new Section 5.2 has been added to address wraparound of these
     timestamp data types after they overflow in the years 2032-2038.

   - Clarifications on encoding, especially in Section 6, have been
     made: all IPFIX values are encoded in network byte order.

Top      ToC       Page 6 
   - Template management, as described in Section 8, has been simplified
     and clarified: the specification has been relaxed with respect to
     [RFC5101], especially concerning potential failures in Template ID
     reuse.  Additional corner cases in template management have been
     addressed.  The new template management language is interoperable
     with that in [RFC5101] to the extent that the behavior was defined
     in the prior specification.

   - Section 11.3 (Mutual Authentication) has been improved to refer to
     current practices in Transport Layer Security (TLS) mutual
     authentication; references to Punycode were removed, as these are
     covered in [RFC6125].

   - Editorial improvements, including structural changes to Sections 8,
     9, and 10 to improve readability, have been applied.  Behavior
     common to all transport protocols has been separated out, with
     exceptions per transport specifically noted.  All template
     management language (on both Exporting and Collecting Processes)
     has been unified in Section 8.

   - A new Section 12 on management considerations has been added.

1.2.  IPFIX Documents Overview

   The IPFIX protocol provides network administrators with access to IP
   Flow information.  The architecture for the export of measured IP
   Flow information out of an IPFIX Exporting Process to a Collecting
   Process is defined in [RFC5470], per the requirements defined in
   [RFC3917].  This document specifies how IPFIX Data Records and
   Templates are carried via a number of transport protocols from IPFIX
   Exporting Processes to IPFIX Collecting Processes.

   Four IPFIX optimizations/extensions are currently specified: a
   bandwidth-saving method for the IPFIX protocol [RFC5473], an
   efficient method for exporting bidirectional flows [RFC5103], a
   method for the definition and export of complex data structures
   [RFC6313], and the specification of the Protocol on IPFIX Mediators
   [IPFIX-MED-PROTO] based on the IPFIX Mediation Framework [RFC6183].

   A "file-based transport" for IPFIX, which defines how IPFIX Messages
   can be stored in files for document-based workflows and for archival
   purposes, is discussed in [RFC5655].

   IPFIX has a formal description of IPFIX Information Elements -- their
   names, data types, and additional semantic information -- as
   specified in [RFC7012].  The registry is maintained by IANA
   [IANA-IPFIX].  The inline export of the Information Element type
   information is specified in [RFC5610].

Top      ToC       Page 7 
   The framework for packet selection and reporting [RFC5474] enables
   network elements to select subsets of packets by statistical and
   other methods, and to export a stream of reports on the selected
   packets to a Collector.  The set of packet selection techniques
   (Sampling, Filtering, and hashing) standardized by the Packet
   Sampling (PSAMP) protocol is described in [RFC5475].  The PSAMP
   protocol [RFC5476], which uses IPFIX as its export protocol,
   specifies the export of packet information from a PSAMP Exporting
   Process to a PSAMP Collector.  Instead of exporting PSAMP Packet
   Reports, the stream of selected packets may also serve as input to
   the generation of IPFIX Flow Records.  Like IPFIX, PSAMP has a formal
   description of its Information Elements: their names, types, and
   additional semantic information.  The PSAMP information model is
   defined in [RFC5477].

   [RFC6615] specifies a MIB module for monitoring, and [RFC6728]
   specifies a data model for configuring and monitoring IPFIX and
   PSAMP-compliant devices using the Network Configuration Protocol
   (NETCONF).  [RFC6727] specifies the PSAMP MIB module as an extension
   of the IPFIX SELECTOR MIB module defined in [RFC6615].

   In terms of development, [RFC5153] provides guidelines for the
   implementation and use of the IPFIX protocol, while [RFC5471]
   provides guidelines for testing.  Finally, [RFC5472] describes what
   types of applications can use the IPFIX protocol and how they can use
   the information provided.  It furthermore shows how the IPFIX
   framework relates to other architectures and frameworks.

2.  Terminology

   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
   "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and
   "OPTIONAL" in this document are to be interpreted as described in
   RFC 2119 [RFC2119].

   The definitions of the basic terms like Traffic Flow, Exporting
   Process, Collecting Process, Observation Points, etc. are
   semantically identical to those found in the IPFIX requirements
   document [RFC3917].  Some of the terms have been expanded for more
   clarity when defining the protocol.  Additional terms required for
   the protocol have also been defined.  Definitions in this document
   and in [RFC5470] are equivalent; definitions that are only relevant
   to the IPFIX protocol only appear here.

Top      ToC       Page 8 
   The terminology summary table in Section 2.1 gives a quick overview
   of the relationships among some of the different terms defined.

   Observation Point

      An Observation Point is a location in the network where packets
      can be observed.  Examples include a line to which a probe is
      attached; a shared medium, such as an Ethernet-based LAN; a single
      port of a router; or a set of interfaces (physical or logical) of
      a router.

      Note that every Observation Point is associated with an
      Observation Domain (defined below) and that one Observation Point
      may be a superset of several other Observation Points.  For
      example, one Observation Point can be an entire line card.  That
      would be the superset of the individual Observation Points at the
      line card's interfaces.

   Observation Domain

      An Observation Domain is the largest set of Observation Points for
      which Flow information can be aggregated by a Metering Process.
      For example, a router line card may be an Observation Domain if it
      is composed of several interfaces, each of which is an Observation
      Point.  In the IPFIX Message it generates, the Observation Domain
      includes its Observation Domain ID, which is unique per Exporting
      Process.  That way, the Collecting Process can identify the
      specific Observation Domain from the Exporter that sends the IPFIX
      Messages.  Every Observation Point is associated with an
      Observation Domain.  It is RECOMMENDED that Observation Domain IDs
      also be unique per IPFIX Device.

   Packet Treatment

      "Packet Treatment" refers to action(s) performed on a packet by a
      forwarding device or other middlebox, including forwarding,
      dropping, delaying for traffic-shaping purposes, etc.

Top      ToC       Page 9 
   Traffic Flow or Flow

      There are several definitions of the term 'flow' being used by the
      Internet community.  Within the context of IPFIX, we use the
      following definition:

      A Flow is defined as a set of packets or frames passing an
      Observation Point in the network during a certain time interval.
      All packets belonging to a particular Flow have a set of common
      properties.  Each property is defined as the result of applying a
      function to the values of:

      1. one or more packet header fields (e.g., destination IP
         address), transport header fields (e.g., destination port
         number), or application header fields (e.g., RTP header fields
         [RFC3550]).

      2. one or more characteristics of the packet itself (e.g., number
         of MPLS labels, etc.).

      3. one or more of the fields derived from Packet Treatment (e.g.,
         next-hop IP address, the output interface, etc.).

      A packet is defined as belonging to a Flow if it completely
      satisfies all the defined properties of the Flow.

      Note that the set of packets represented by a Flow may be empty;
      that is, a Flow may represent zero or more packets.  As sampling
      is a Packet Treatment, this definition includes packets selected
      by a sampling mechanism.

   Flow Key

      Each of the fields that:

      1. belong to the packet header (e.g., destination IP address), or

      2. are a property of the packet itself (e.g., packet length), or

      3. are derived from Packet Treatment (e.g., Autonomous System (AS)
         number),

      and that are used to define a Flow (i.e., are the properties
      common to all packets in the Flow) are termed Flow Keys.  As an
      example, the traditional '5-tuple' Flow Key of source and
      destination IP address, source and destination transport port, and
      transport protocol, groups together all packets belonging to a
      single direction of communication on a single socket.

Top      ToC       Page 10 
   Flow Record

      A Flow Record contains information about a specific Flow that was
      observed at an Observation Point.  A Flow Record contains measured
      properties of the Flow (e.g., the total number of bytes for all
      the Flow's packets) and usually contains characteristic properties
      of the Flow (e.g., source IP address).

   Metering Process

      The Metering Process generates Flow Records.  Inputs to the
      process are packet headers, characteristics, and Packet Treatment
      observed at one or more Observation Points.

      The Metering Process consists of a set of functions that includes
      packet header capturing, timestamping, sampling, classifying, and
      maintaining Flow Records.

      The maintenance of Flow Records may include creating new records,
      updating existing ones, computing Flow statistics, deriving
      further Flow properties, detecting Flow expiration, passing Flow
      Records to the Exporting Process, and deleting Flow Records.

   Exporting Process

      The Exporting Process sends IPFIX Messages to one or more
      Collecting Processes.  The Flow Records in the Messages are
      generated by one or more Metering Processes.

   Exporter

      A device that hosts one or more Exporting Processes is termed an
      Exporter.

   IPFIX Device

      An IPFIX Device hosts at least one Exporting Process.  It may host
      further Exporting Processes as well as arbitrary numbers of
      Observation Points and Metering Processes.

   Collecting Process

      A Collecting Process receives IPFIX Messages from one or more
      Exporting Processes.  The Collecting Process might process or
      store Flow Records received within these Messages, but such
      actions are out of scope for this document.

Top      ToC       Page 11 
   Collector

      A device that hosts one or more Collecting Processes is termed a
      Collector.

   Template

      A Template is an ordered sequence of <type, length> pairs used to
      completely specify the structure and semantics of a particular set
      of information that needs to be communicated from an IPFIX Device
      to a Collector.  Each Template is uniquely identifiable by means
      of a Template ID.

   IPFIX Message

      An IPFIX Message is a message that originates at the Exporting
      Process and carries the IPFIX records of this Exporting Process,
      and whose destination is a Collecting Process.  An IPFIX Message
      is encapsulated at the transport layer.

   Message Header

      The Message Header is the first part of an IPFIX Message; the
      Message Header provides basic information about the message, such
      as the IPFIX version, length of the message, message sequence
      number, etc.

   Template Record

      A Template Record defines the structure and interpretation of
      fields in a Data Record.

   Data Record

      A Data Record is a record that contains values of the parameters
      corresponding to a Template Record.

   Options Template Record

      An Options Template Record is a Template Record that defines the
      structure and interpretation of fields in a Data Record, including
      defining how to scope the applicability of the Data Record.

Top      ToC       Page 12 
   Set

      A Set is a collection of records that have a similar structure,
      prefixed by a header.  In an IPFIX Message, zero or more Sets
      follow the Message Header.  There are three different types of
      Sets: Template Sets, Options Template Sets, and Data Sets.

   Template Set

      A Template Set is a collection of one or more Template Records
      that have been grouped together in an IPFIX Message.

   Options Template Set

      An Options Template Set is a collection of one or more Options
      Template Records that have been grouped together in an IPFIX
      Message.

   Data Set

      A Data Set is one or more Data Records, of the same type, that are
      grouped together in an IPFIX Message.  Each Data Record is
      previously defined by a Template Record or an Options Template
      Record.

   Information Element

      An Information Element is a protocol- and encoding-independent
      description of an attribute that may appear in an IPFIX Record.
      Information Elements are defined in the IANA "IPFIX Information
      Elements" registry [IANA-IPFIX].  The type associated with an
      Information Element indicates constraints on what it may contain
      and also determines the valid encoding mechanisms for use in
      IPFIX.

   Transport Session

      In the Stream Control Transmission Protocol (SCTP), the Transport
      Session is known as the SCTP association, which is uniquely
      identified by the SCTP endpoints [RFC4960]; in TCP, the Transport
      Session is known as the TCP connection, which is uniquely
      identified by the combination of IP addresses and TCP ports used.
      In UDP, the Transport Session is known as the UDP session, which
      is uniquely identified by the combination of IP addresses and UDP
      ports used.

Top      ToC       Page 13 
2.1.  Terminology Summary Table

   Figure A shows a summary of IPFIX terminology.

    +------------------+---------------------------------------------+
    |                  |                 Contents                    |
    |                  +--------------------+------------------------+
    |       Set        |      Template      |         Record         |
    +------------------+--------------------+------------------------+
    |     Data Set     |          /         |     Data Record(s)     |
    +------------------+--------------------+------------------------+
    |   Template Set   | Template Record(s) |           /            |
    +------------------+--------------------+------------------------+
    | Options Template |  Options Template  |           /            |
    |       Set        |      Record(s)     |                        |
    +------------------+--------------------+------------------------+

                    Figure A: Terminology Summary Table

   A Data Set is composed of Data Record(s).  No Template Record is
   included.  A Template Record or an Options Template Record defines
   the Data Record.

   A Template Set contains only Template Record(s).

   An Options Template Set contains only Options Template Record(s).



(page 13 continued on part 2)

Next RFC Part