Tech-invite3GPPspaceIETF RFCsSIP
929190898887868584838281807978777675747372717069686766656463626160595857565554535251504948474645444342414039383736353433323130292827262524232221201918171615141312111009080706050403020100
in Index   Prev   Next

RFC 7011

Specification of the IP Flow Information Export (IPFIX) Protocol for the Exchange of Flow Information

Pages: 76
Internet Standard: 77
Errata
Obsoletes:  5101
Part 2 of 4 – Pages 13 to 38
First   Prev   Next

Top   ToC   RFC7011 - Page 13   prevText

3. IPFIX Message Format

An IPFIX Message consists of a Message Header, followed by zero or more Sets. The Sets can be any of these three possible types: Data Set, Template Set, or Options Template Set. The format of the IPFIX Message is shown in Figure B. +----------------------------------------------------+ | Message Header | +----------------------------------------------------+ | Set | +----------------------------------------------------+ | Set | +----------------------------------------------------+ ... +----------------------------------------------------+ | Set | +----------------------------------------------------+ Figure B: IPFIX Message Format
Top   ToC   RFC7011 - Page 14
   Following are some examples of IPFIX Messages:

   1. An IPFIX Message consisting of interleaved Template, Data, and
      Options Template Sets, as shown in Figure C.  Here, Template and
      Options Template Sets are transmitted "on demand", before the
      first Data Set whose structure they define.

     +--------+--------------------------------------------------------+
     |        | +----------+ +---------+     +-----------+ +---------+ |
     |Message | | Template | | Data    |     | Options   | | Data    | |
     | Header | | Set      | | Set     | ... | Template  | | Set     | |
     |        | |          | |         |     | Set       | |         | |
     |        | +----------+ +---------+     +-----------+ +---------+ |
     +--------+--------------------------------------------------------+

                     Figure C: IPFIX Message: Example 1

   2. An IPFIX Message consisting entirely of Data Sets, sent after the
      appropriate Template Records have been defined and transmitted to
      the Collecting Process, as shown in Figure D.

       +--------+----------------------------------------------+
       |        | +---------+     +---------+      +---------+ |
       |Message | | Data    |     | Data    |      | Data    | |
       | Header | | Set     | ... | Set     | ...  | Set     | |
       |        | +---------+     +---------+      +---------+ |
       +--------+----------------------------------------------+

                    Figure D: IPFIX Message: Example 2

   3. An IPFIX Message consisting entirely of Template and Options
      Template Sets, as shown in Figure E.  Such a message can be used
      to define or redefine Templates and Options Templates in bulk.

      +--------+-------------------------------------------------+
      |        | +----------+     +----------+      +----------+ |
      |Message | | Template |     | Template |      | Options  | |
      | Header | | Set      | ... | Set      | ...  | Template | |
      |        | |          |     |          |      | Set      | |
      |        | +----------+     +----------+      +----------+ |
      +--------+-------------------------------------------------+

                    Figure E: IPFIX Message: Example 3
Top   ToC   RFC7011 - Page 15

3.1. Message Header Format

The format of the IPFIX Message Header is shown in Figure F. 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Version Number | Length | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Export Time | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Sequence Number | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Observation Domain ID | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Figure F: IPFIX Message Header Format Each Message Header field is exported in network byte order. The fields are defined as follows: Version Version of IPFIX to which this Message conforms. The value of this field is 0x000a for the current version, incrementing by one the version used in the NetFlow services export version 9 [RFC3954]. Length Total length of the IPFIX Message, measured in octets, including Message Header and Set(s). Export Time Time at which the IPFIX Message Header leaves the Exporter, expressed in seconds since the UNIX epoch of 1 January 1970 at 00:00 UTC, encoded as an unsigned 32-bit integer.
Top   ToC   RFC7011 - Page 16
   Sequence Number

      Incremental sequence counter modulo 2^32 of all IPFIX Data Records
      sent in the current stream from the current Observation Domain by
      the Exporting Process.  Each SCTP Stream counts sequence numbers
      separately, while all messages in a TCP connection or UDP session
      are considered to be part of the same stream.  This value can be
      used by the Collecting Process to identify whether any IPFIX Data
      Records have been missed.  Template and Options Template Records
      do not increase the Sequence Number.

   Observation Domain ID

      A 32-bit identifier of the Observation Domain that is locally
      unique to the Exporting Process.  The Exporting Process uses the
      Observation Domain ID to uniquely identify to the Collecting
      Process the Observation Domain that metered the Flows.  It is
      RECOMMENDED that this identifier also be unique per IPFIX Device.
      Collecting Processes SHOULD use the Transport Session and the
      Observation Domain ID field to separate different export streams
      originating from the same Exporter.  The Observation Domain ID
      SHOULD be 0 when no specific Observation Domain ID is relevant for
      the entire IPFIX Message, for example, when exporting the
      Exporting Process Statistics, or in the case of a hierarchy of
      Collectors when aggregated Data Records are exported.

3.2. Field Specifier Format

Vendors need the ability to define proprietary Information Elements, because, for example, they are delivering a pre-standards product, or the Information Element is in some way commercially sensitive. This section describes the Field Specifier format for both IANA-registered Information Elements [IANA-IPFIX] and enterprise-specific Information Elements. The Information Elements are identified by the Information Element identifier. When the Enterprise bit is set to 0, the corresponding Information Element appears in [IANA-IPFIX], and the Enterprise Number MUST NOT be present. When the Enterprise bit is set to 1, the corresponding Information Element identifier identified an enterprise-specific Information Element; the Enterprise Number MUST be present. An example of this is shown in Appendix A.2.2.
Top   ToC   RFC7011 - Page 17
   The Field Specifier format is shown in Figure G.

     0                   1                   2                   3
     0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
     |E|  Information Element ident. |        Field Length           |
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
     |                      Enterprise Number                        |
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

                     Figure G: Field Specifier Format

   Where:

   E

      Enterprise bit.  This is the first bit of the Field Specifier.  If
      this bit is zero, the Information Element identifier identifies an
      Information Element in [IANA-IPFIX], and the four-octet Enterprise
      Number field MUST NOT be present.  If this bit is one, the
      Information Element identifier identifies an enterprise-specific
      Information Element, and the Enterprise Number field MUST be
      present.

   Information Element identifier

      A numeric value that represents the Information Element.  Refer to
      [IANA-IPFIX].

   Field Length

      The length of the corresponding encoded Information Element, in
      octets.  Refer to [IANA-IPFIX].  The Field Length may be smaller
      than that listed in [IANA-IPFIX] if the reduced-size encoding is
      used (see Section 6.2).  The value 65535 is reserved for variable-
      length Information Elements (see Section 7).

   Enterprise Number

      IANA enterprise number [IANA-PEN] of the authority defining the
      Information Element identifier in this Template Record.
Top   ToC   RFC7011 - Page 18

3.3. Set and Set Header Format

A Set is a generic term for a collection of records that have a similar structure. There are three different types of Sets: Template Sets, Options Template Sets, and Data Sets. Each of these Sets consists of a Set Header and one or more records. The Set Format and the Set Header Format are defined in the following sections.

3.3.1. Set Format

A Set has the format shown in Figure H. The record types can be either Template Records, Options Template Records, or Data Records. The record types MUST NOT be mixed within a Set. +--------------------------------------------------+ | Set Header | +--------------------------------------------------+ | record | +--------------------------------------------------+ | record | +--------------------------------------------------+ ... +--------------------------------------------------+ | record | +--------------------------------------------------+ | Padding (opt.) | +--------------------------------------------------+ Figure H: Set Format Set Header The Set Header Format is defined in Section 3.3.2. Record One of the record formats: Template Record, Options Template Record, or Data Record format. Padding The Exporting Process MAY insert some padding octets, so that the subsequent Set starts at an aligned boundary. For security reasons, the padding octet(s) MUST be composed of octets with value zero (0). The padding length MUST be shorter than any allowable record in this Set. If padding of the IPFIX Message is desired in combination with very short records, then the padding Information Element 'paddingOctets' can be used for padding
Top   ToC   RFC7011 - Page 19
      records such that their length is increased to a multiple of 4 or
      8 octets.  Because Template Sets are always 4-octet aligned by
      definition, padding is only needed in the case of other
      alignments, e.g., on 8-octet boundaries.

3.3.2. Set Header Format

Every Set contains a common header. This header is defined in Figure I. 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Set ID | Length | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Figure I: Set Header Format Each Set Header field is exported in network format. The fields are defined as follows: Set ID Identifies the Set. A value of 2 is reserved for Template Sets. A value of 3 is reserved for Options Template Sets. Values from 4 to 255 are reserved for future use. Values 256 and above are used for Data Sets. The Set ID values of 0 and 1 are not used, for historical reasons [RFC3954]. Length Total length of the Set, in octets, including the Set Header, all records, and the optional padding. Because an individual Set MAY contain multiple records, the Length value MUST be used to determine the position of the next Set.
Top   ToC   RFC7011 - Page 20

3.4. Record Format

IPFIX defines three record formats, as defined in the next sections: the Template Record format, the Options Template Record format, and the Data Record format.

3.4.1. Template Record Format

One of the essential elements in the IPFIX record format is the Template Record. Templates greatly enhance the flexibility of the record format because they allow the Collecting Process to process IPFIX Messages without necessarily knowing the interpretation of all Data Records. A Template Record contains any combination of IANA- assigned and/or enterprise-specific Information Element identifiers. The format of the Template Record is shown in Figure J. It consists of a Template Record Header and one or more Field Specifiers. Field Specifiers are defined in Figure G above. +--------------------------------------------------+ | Template Record Header | +--------------------------------------------------+ | Field Specifier | +--------------------------------------------------+ | Field Specifier | +--------------------------------------------------+ ... +--------------------------------------------------+ | Field Specifier | +--------------------------------------------------+ Figure J: Template Record Format The format of the Template Record Header is shown in Figure K. 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Template ID (> 255) | Field Count | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Figure K: Template Record Header Format
Top   ToC   RFC7011 - Page 21
   The Template Record Header Field definitions are as follows:

   Template ID

      Each Template Record is given a unique Template ID in the range
      256 to 65535.  This uniqueness is local to the Transport Session
      and Observation Domain that generated the Template ID.  Since
      Template IDs are used as Set IDs in the Sets they describe (see
      Section 3.4.3), values 0-255 are reserved for special Set types
      (e.g., Template Sets themselves), and Templates and Options
      Templates (see Section 3.4.2) cannot share Template IDs within a
      Transport Session and Observation Domain.  There are no
      constraints regarding the order of the Template ID allocation.  As
      Exporting Processes are free to allocate Template IDs as they see
      fit, Collecting Processes MUST NOT assume incremental Template
      IDs, or anything about the contents of a Template based on its
      Template ID alone.

   Field Count

      Number of fields in this Template Record.
Top   ToC   RFC7011 - Page 22
   The example in Figure L shows a Template Set with mixed IANA-assigned
   and enterprise-specific Information Elements.  It consists of a Set
   Header, a Template Header, and several Field Specifiers.

      0                   1                   2                   3
      0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
     |          Set ID = 2           |          Length               |
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
     |      Template ID = 256        |         Field Count = N       |
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
     |1| Information Element id. 1.1 |        Field Length 1.1       |
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
     |                    Enterprise Number  1.1                     |
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
     |0| Information Element id. 1.2 |        Field Length 1.2       |
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
     |             ...               |              ...              |
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
     |1| Information Element id. 1.N |        Field Length 1.N       |
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
     |                    Enterprise Number  1.N                     |
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
     |      Template ID = 257        |         Field Count = M       |
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
     |0| Information Element id. 2.1 |        Field Length 2.1       |
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
     |1| Information Element id. 2.2 |        Field Length 2.2       |
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
     |                    Enterprise Number  2.2                     |
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
     |             ...               |              ...              |
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
     |1| Information Element id. 2.M |        Field Length 2.M       |
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
     |                    Enterprise Number  2.M                     |
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
     |                          Padding (opt)                        |
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

                      Figure L: Template Set Example

   Information Element id.s 1.2 and 2.1 appear in [IANA-IPFIX]
   (Enterprise bit = 0) and therefore do not need an Enterprise Number
   to identify them.
Top   ToC   RFC7011 - Page 23

3.4.2. Options Template Record Format

Thanks to the notion of scope, The Options Template Record gives the Exporter the ability to provide additional information to the Collector that would not be possible with Flow Records alone. See Section 4 for specific Options Templates used for reporting metadata about IPFIX Exporting and Metering Processes.
3.4.2.1. Scope
The scope, which is only available in the Options Template Set, gives the context of the reported Information Elements in the Data Records. The scope is one or more Information Elements, specified in the Options Template Record. At a minimum, Collecting Processes SHOULD support as scope the observationDomainId, exportingProcessId, meteringProcessId, templateId, lineCardId, exporterIPv4Address, exporterIPv6Address, and ingressInterface Information Elements. The IPFIX protocol doesn't prevent the use of any Information Elements for scope. However, some Information Element types don't make sense if specified as scope (for example, the counter Information Elements). The IPFIX Message Header already contains the Observation Domain ID. If not zero, this Observation Domain ID can be considered as an implicit scope for the Data Records in the IPFIX Message. Multiple Scope Fields MAY be present in the Options Template Record, in which case the composite scope is the combination of the scopes. For example, if the two scopes are meteringProcessId and templateId, the combined scope is this Template for this Metering Process. If a different order of Scope Fields would result in a Record having a different semantic meaning, then the order of Scope Fields MUST be preserved by the Exporting Process. For example, in the context of PSAMP [RFC5476], if the first scope defines the filtering function, while the second scope defines the sampling function, the order of the scope is important. Applying the sampling function first, followed by the filtering function, would lead to potentially different Data Records than applying the filtering function first, followed by the sampling function.
Top   ToC   RFC7011 - Page 24
3.4.2.2. Options Template Record Format
An Options Template Record contains any combination of IANA-assigned and/or enterprise-specific Information Element identifiers. The format of the Options Template Record is shown in Figure M. It consists of an Options Template Record Header and one or more Field Specifiers. Field Specifiers are defined in Figure G above. +--------------------------------------------------+ | Options Template Record Header | +--------------------------------------------------+ | Field Specifier | +--------------------------------------------------+ | Field Specifier | +--------------------------------------------------+ ... +--------------------------------------------------+ | Field Specifier | +--------------------------------------------------+ Figure M: Options Template Record Format The format of the Options Template Record Header is shown in Figure N. 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Template ID (> 255) | Field Count | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Scope Field Count | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Figure N: Options Template Record Header Format
Top   ToC   RFC7011 - Page 25
   The Options Template Record Header Field definitions are as follows:

   Template ID

      Each Options Template Record is given a unique Template ID in the
      range 256 to 65535.  This uniqueness is local to the Transport
      Session and Observation Domain that generated the Template ID.
      Since Template IDs are used as Set IDs in the sets they describe
      (see Section 3.4.3), values 0-255 are reserved for special Set
      types (e.g., Template Sets themselves), and Templates and Options
      Templates cannot share Template IDs within a Transport Session and
      Observation Domain.  There are no constraints regarding the order
      of the Template ID allocation.  As Exporting Processes are free to
      allocate Template IDs as they see fit, Collecting Processes MUST
      NOT assume incremental Template IDs, or anything about the
      contents of an Options Template based on its Template ID alone.

   Field Count

      Number of all fields in this Options Template Record, including
      the Scope Fields.

   Scope Field Count

      Number of scope fields in this Options Template Record.  The Scope
      Fields are normal Fields, except that they are interpreted as
      scope at the Collector.  A scope field count of N specifies that
      the first N Field Specifiers in the Template Record are Scope
      Fields.  The Scope Field Count MUST NOT be zero.
Top   ToC   RFC7011 - Page 26
   The example in Figure O shows an Options Template Set with mixed
   IANA-assigned and enterprise-specific Information Elements.  It
   consists of a Set Header, an Options Template Header, and several
   Field Specifiers.

      0                   1                   2                   3
      0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
     |          Set ID = 3           |          Length               |
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
     |         Template ID = 258     |         Field Count = N + M   |
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
     |     Scope Field Count = N     |0|  Scope 1 Infor. Element id. |
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
     |     Scope 1 Field Length      |0|  Scope 2 Infor. Element id. |
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
     |     Scope 2 Field Length      |             ...               |
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
     |            ...                |1|  Scope N Infor. Element id. |
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
     |     Scope N Field Length      |   Scope N Enterprise Number  ...
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    ...  Scope N Enterprise Number   |1| Option 1 Infor. Element id. |
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
     |    Option 1 Field Length      |  Option 1 Enterprise Number  ...
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    ... Option 1 Enterprise Number   |              ...              |
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
     |             ...               |0| Option M Infor. Element id. |
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
     |     Option M Field Length     |      Padding (optional)       |
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

                  Figure O: Options Template Set Example
Top   ToC   RFC7011 - Page 27

3.4.3. Data Record Format

The Data Records are sent in Data Sets. The format of the Data Record is shown in Figure P. It consists only of one or more Field Values. The Template ID to which the Field Values belong is encoded in the Set Header field "Set ID", i.e., "Set ID" = "Template ID". +--------------------------------------------------+ | Field Value | +--------------------------------------------------+ | Field Value | +--------------------------------------------------+ ... +--------------------------------------------------+ | Field Value | +--------------------------------------------------+ Figure P: Data Record Format Note that Field Values do not necessarily have a length of 16 bits. Field Values are encoded according to their data type as specified in [RFC7012]. Interpretation of the Data Record format can be done only if the Template Record corresponding to the Template ID is available at the Collecting Process.
Top   ToC   RFC7011 - Page 28
   The example in Figure Q shows a Data Set.  It consists of a Set
   Header and several Field Values.

     0                   1                   2                   3
     0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |   Set ID = Template ID        |          Length               |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |   Record 1 - Field Value 1    |   Record 1 - Field Value 2    |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |   Record 1 - Field Value 3    |             ...               |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |   Record 2 - Field Value 1    |   Record 2 - Field Value 2    |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |   Record 2 - Field Value 3    |             ...               |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |   Record 3 - Field Value 1    |   Record 3 - Field Value 2    |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |   Record 3 - Field Value 3    |             ...               |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |              ...              |      Padding (optional)       |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

                Figure Q: Data Set, Containing Data Records

4. Specific Reporting Requirements

Some specific Options Templates and Options Template Records are necessary to provide extra information about the Flow Records and about the Metering Process. The Options Template and Options Template Records defined in these subsections, which impose some constraints on the Metering Process and Exporting Process implementations, MAY be implemented. If implemented, the specific Options Templates SHOULD be implemented as specified in these subsections. The minimum set of Information Elements is always specified in these Specific IPFIX Options Templates. Nevertheless, extra Information Elements may be used in these specific Options Templates. The Collecting Process MUST check the possible combinations of Information Elements within the Options Template Records to correctly interpret the following Options Templates.
Top   ToC   RFC7011 - Page 29

4.1. The Metering Process Statistics Options Template

The Metering Process Statistics Options Template specifies the structure of a Data Record for reporting Metering Process statistics. It SHOULD contain the following Information Elements, as defined in [IANA-IPFIX]: (scope) observationDomainId This Information Element MUST be defined as a Scope Field and MUST be present, unless the Observation Domain ID of the enclosing Message is non-zero. (scope) meteringProcessId If present, this Information Element MUST be defined as a Scope Field. exportedMessageTotalCount exportedFlowRecordTotalCount exportedOctetTotalCount The Exporting Process SHOULD export the Data Record specified by the Metering Process Statistics Options Template on a regular basis or based on some export policy. This periodicity or export policy SHOULD be configurable. Note that if several Metering Processes are available on the Exporter Observation Domain, the Information Element meteringProcessId MUST be specified as an additional Scope Field.

4.2. The Metering Process Reliability Statistics Options Template

The Metering Process Reliability Statistics Options Template specifies the structure of a Data Record for reporting lack of reliability in the Metering Process. It SHOULD contain the following Information Elements, as defined in [IANA-IPFIX]: (scope) observationDomainId This Information Element MUST be defined as a Scope Field and MUST be present, unless the Observation Domain ID of the enclosing Message is non-zero.
Top   ToC   RFC7011 - Page 30
      (scope) meteringProcessId

         If present, this Information Element MUST be defined as a Scope
         Field.

      ignoredPacketTotalCount

      ignoredOctetTotalCount

      time first packet ignored

         The timestamp of the first packet that was ignored by the
         Metering Process.  For this timestamp, any of the following
         timestamp Information Elements can be used:

            observationTimeSeconds,
            observationTimeMilliseconds,
            observationTimeMicroseconds, or
            observationTimeNanoseconds.

      time last packet ignored

         The timestamp of the last packet that was ignored by the
         Metering Process.  For this timestamp, any of the following
         timestamp Information Elements can be used:

            observationTimeSeconds,
            observationTimeMilliseconds,
            observationTimeMicroseconds, or
            observationTimeNanoseconds.

   The Exporting Process SHOULD export the Data Record specified by the
   Metering Process Reliability Statistics Options Template on a regular
   basis or based on some export policy.  This periodicity or export
   policy SHOULD be configurable.

   Note that if several Metering Processes are available on the Exporter
   Observation Domain, the Information Element meteringProcessId MUST be
   specified as an additional Scope Field.

   Since the Metering Process Reliability Statistics Options Template
   contains two identical timestamp Information Elements, and since the
   order of the Information Elements in the Template Records is not
   guaranteed, the Collecting Process interprets the time interval of
   ignored packets as the range between the two values; see Section 5.2
   for wraparound considerations.
Top   ToC   RFC7011 - Page 31

4.3. The Exporting Process Reliability Statistics Options Template

The Exporting Process Reliability Statistics Options Template specifies the structure of a Data Record for reporting lack of reliability in the Exporting Process. It SHOULD contain the following Information Elements, as defined in [IANA-IPFIX]: (scope) Exporting Process Identifier The identifier of the Exporting Process for which reliability is reported. Any of the exporterIPv4Address, exporterIPv6Address, or exportingProcessId Information Elements can be used for this field. This Information Element MUST be defined as a Scope Field. notSentFlowTotalCount notSentPacketTotalCount notSentOctetTotalCount time first flow dropped The time at which the first Flow Record was dropped by the Exporting Process. For this timestamp, any of the following timestamp Information Elements can be used: observationTimeSeconds, observationTimeMilliseconds, observationTimeMicroseconds, or observationTimeNanoseconds. time last flow dropped The time at which the last Flow Record was dropped by the Exporting Process. For this timestamp, any of the following timestamp Information Elements can be used: observationTimeSeconds, observationTimeMilliseconds, observationTimeMicroseconds, or observationTimeNanoseconds. The Exporting Process SHOULD export the Data Record specified by the Exporting Process Reliability Statistics Options Template on a regular basis or based on some export policy. This periodicity or export policy SHOULD be configurable.
Top   ToC   RFC7011 - Page 32
   Since the Exporting Process Reliability Statistics Options Template
   contains two identical timestamp Information Elements, and since the
   order of the Information Elements in the Template Records is not
   guaranteed, the Collecting Process interprets the time interval of
   dropped packets as the range between the two values; see Section 5.2
   for wraparound considerations.

4.4. The Flow Keys Options Template

The Flow Keys Options Template specifies the structure of a Data Record for reporting the Flow Keys of reported Flows. A Flow Keys Data Record extends a particular Template Record that is referenced by its templateId. The Template Record is extended by specifying which of the Information Elements contained in the corresponding Data Records describe Flow properties that serve as Flow Keys of the reported Flow. The Flow Keys Options Template SHOULD contain the following Information Elements, as defined in [IANA-IPFIX]: (scope) templateId This Information Element MUST be defined as a Scope Field. flowKeyIndicator

5. Timing Considerations

5.1. IPFIX Message Header Export Time and Flow Record Time

The IPFIX Message Header Export Time field is the time at which the IPFIX Message Header leaves the Exporter, using the same encoding as the dateTimeSeconds abstract data type [RFC7012], i.e., expressed in seconds since the UNIX epoch, 1 January 1970 at 00:00 UTC, encoded as an unsigned 32-bit integer. Certain time-related Information Elements may be expressed as an offset from this Export Time. For example, Data Records requiring a microsecond precision can export the flow start and end times with the flowStartMicroseconds and flowEndMicroseconds Information Elements, which encode the absolute time in microseconds in terms of the NTP epoch, 1 January 1900 at 00:00 UTC, in a 64-bit field. An alternate solution is to export the flowStartDeltaMicroseconds and flowEndDeltaMicroseconds Information Elements in the Data Record, which respectively report the flow start and end time as negative offsets from the Export Time, as an unsigned 32-bit integer. This latter solution lowers the export bandwidth requirement, saving four bytes per timestamp, while increasing the load on the Exporter,
Top   ToC   RFC7011 - Page 33
   as the Exporting Process must calculate the
   flowStartDeltaMicroseconds and flowEndDeltaMicroseconds of every
   single Data Record before exporting the IPFIX Message.

   It must be noted that timestamps based on the Export Time impose some
   time constraints on the Data Records contained within the IPFIX
   Message.  In the example of flowStartDeltaMicroseconds and
   flowEndDeltaMicroseconds Information Elements, the Data Record can
   only contain records with timestamps within 71 minutes of the Export
   Time.  Otherwise, the 32-bit counter would not be sufficient to
   contain the flow start time offset.

5.2. Supporting Timestamp Wraparound

The dateTimeSeconds abstract data type [RFC7012] and the Export Time Message Header field (Section 3.1) are encoded as 32-bit unsigned integers, expressed as seconds since the UNIX epoch, 1 January 1970 at 00:00 UTC, as defined in [POSIX.1]. These values will wrap around on 7 February 2106 at 06:28:16 UTC. In order to support continued use of the IPFIX protocol beyond this date, Exporting Processes SHOULD export dateTimeSeconds values and the Export Time Message Header field as the number of seconds since the UNIX epoch, 1 January 1970 at 00:00 UTC, modulo 2^32. Collecting Processes SHOULD use the current date, or other contextual information, to properly interpret dateTimeSeconds values and the Export Time Message Header field. There are similar considerations for the NTP-based dateTimeMicroseconds and dateTimeNanoseconds abstract data types [RFC7012]. Exporting Processes SHOULD export dateTimeMicroseconds and dateTimeNanoseconds values as if the NTP era [RFC5905] is implicit; Collecting Processes SHOULD use the current date, or other contextual information, to determine the NTP era in order to properly interpret dateTimeMicroseconds and dateTimeNanoseconds values in received Data Records. The dateTimeMilliseconds abstract data type will wrap around in approximately 500 billion years; the specification of the behavior of this abstract data type after that time is left as a subject of a future revision of this specification. The long-term storage of files [RFC5655] for archival purposes is affected by timestamp wraparound, as the use of the current date to interpret timestamp values in files stored on the order of multiple decades in the past may lead to incorrect values; therefore, it is RECOMMENDED that such files be stored with contextual information to assist in the interpretation of these timestamps.
Top   ToC   RFC7011 - Page 34

6. Linkage with the Information Model

As with values in the IPFIX Message Header and Set Header, values of all Information Elements [RFC7012], except for those of the string and octetArray data types, are encoded in canonical format in network byte order (also known as big-endian byte ordering).

6.1. Encoding of IPFIX Data Types

The following sections define the encoding of the data types specified in [RFC7012].

6.1.1. Integral Data Types

Integral data types -- unsigned8, unsigned16, unsigned32, unsigned64, signed8, signed16, signed32, and signed64 -- MUST be encoded using the default canonical format in network byte order. Signed integral data types are represented in two's complement notation.

6.1.2. Address Types

Address types -- macAddress, ipv4Address, and ipv6Address -- MUST be encoded the same way as the integral data types, as six, four, and sixteen octets in network byte order, respectively.

6.1.3. float32

The float32 data type MUST be encoded as an IEEE binary32 floating point type as specified in [IEEE.754.2008], in network byte order as specified in Section 3.6 of [RFC1014]. Note that on little-endian machines, byte swapping of the native representation is necessary before export. Note that the method for doing this may be implementation platform dependent.

6.1.4. float64

The float64 data type MUST be encoded as an IEEE binary64 floating point type as specified in [IEEE.754.2008], in network byte order as specified in Section 3.7 of [RFC1014]. Note that on little-endian machines, byte swapping of the native representation is necessary before export. Note that the method for doing this may be implementation platform dependent.
Top   ToC   RFC7011 - Page 35

6.1.5. boolean

The boolean data type is specified according to the TruthValue in [RFC2579]. It is encoded as a single-octet integer per Section 6.1.1, with the value 1 for true and value 2 for false. Every other value is undefined.

6.1.6. string and octetArray

The "string" data type represents a finite-length string of valid characters of the Unicode character encoding set. The string data type MUST be encoded in UTF-8 [RFC3629] format. The string is sent as an array of zero or more octets using an Information Element of fixed or variable length. IPFIX Exporting Processes MUST NOT send IPFIX Messages containing ill-formed UTF-8 string values for Information Elements of the string data type; Collecting Processes SHOULD detect and ignore such values. See [UTF8-EXPLOIT] for background on this issue. The octetArray data type has no encoding rules; it represents a raw array of zero or more octets, with the interpretation of the octets defined in the Information Element definition.

6.1.7. dateTimeSeconds

The dateTimeSeconds data type is an unsigned 32-bit integer in network byte order containing the number of seconds since the UNIX epoch, 1 January 1970 at 00:00 UTC, as defined in [POSIX.1]. dateTimeSeconds is encoded identically to the IPFIX Message Header Export Time field. It can represent dates between 1 January 1970 and 7 February 2106 without wraparound; see Section 5.2 for wraparound considerations.

6.1.8. dateTimeMilliseconds

The dateTimeMilliseconds data type is an unsigned 64-bit integer in network byte order containing the number of milliseconds since the UNIX epoch, 1 January 1970 at 00:00 UTC, as defined in [POSIX.1]. It can represent dates beginning on 1 January 1970 and for approximately the next 500 billion years without wraparound.

6.1.9. dateTimeMicroseconds

The dateTimeMicroseconds data type is a 64-bit field encoded according to the NTP Timestamp format as defined in Section 6 of [RFC5905]. This field is made up of two unsigned 32-bit integers in network byte order: Seconds and Fraction. The Seconds field is the number of seconds since the NTP epoch, 1 January 1900 at 00:00 UTC.
Top   ToC   RFC7011 - Page 36
   The Fraction field is the fractional number of seconds in units of
   1/(2^32) seconds (approximately 233 picoseconds).  It can represent
   dates between 1 January 1900 and 8 February 2036 in the current
   NTP era; see Section 5.2 for wraparound considerations.

   Note that dateTimeMicroseconds and dateTimeNanoseconds share an
   identical encoding.  The dateTimeMicroseconds data type is intended
   only to represent timestamps of microsecond precision.  Therefore,
   the bottom 11 bits of the Fraction field SHOULD be zero and MUST
   be ignored for all Information Elements of this data type
   (as 2^11 x 233 picoseconds = .477 microseconds).

6.1.10. dateTimeNanoseconds

The dateTimeNanoseconds data type is a 64-bit field encoded according to the NTP Timestamp format as defined in Section 6 of [RFC5905]. This field is made up of two unsigned 32-bit integers in network byte order: Seconds and Fraction. The Seconds field is the number of seconds since the NTP epoch, 1 January 1900 at 00:00 UTC. The Fraction field is the fractional number of seconds in units of 1/(2^32) seconds (approximately 233 picoseconds). It can represent dates between 1 January 1900 and 8 February 2036 in the current NTP era; see Section 5.2 for wraparound considerations. Note that dateTimeMicroseconds and dateTimeNanoseconds share an identical encoding. There is no restriction on the interpretation of the Fraction field for the dateTimeNanoseconds data type.

6.2. Reduced-Size Encoding

Information Elements encoded as signed, unsigned, or float data types MAY be encoded using fewer octets than those implied by their type in the information model definition, based on the assumption that the smaller size is sufficient to carry any value the Exporter may need to deliver. This reduces the network bandwidth requirement between the Exporter and the Collector. Note that the Information Element definitions [IANA-IPFIX] always define the maximum encoding size. For instance, the information model defines octetDeltaCount as an unsigned64 type, which would require 64 bits. However, if the Exporter will never locally encounter the need to send a value larger than 4294967295, it may choose to send the value as unsigned32 instead. This behavior is indicated by the Exporter by specifying a size in the Template with a smaller length than that associated with the assigned type of the Information Element. In the example above, the Exporter would place a length of 4 versus 8 in the Template.
Top   ToC   RFC7011 - Page 37
   Reduced-size encoding MAY be applied to the following integer types:
   unsigned64, signed64, unsigned32, signed32, unsigned16, and signed16.
   The signed versus unsigned property of the reported value MUST be
   preserved.  The reduction in size can be to any number of octets
   smaller than the original type if the data value still fits, i.e., so
   that only leading zeroes are dropped.  For example, an unsigned64 can
   be reduced in size to 7, 6, 5, 4, 3, 2, or 1 octet(s).

   Reduced-size encoding MAY be used to reduce float64 to float32.  The
   float32 not only has a reduced number range but, due to the smaller
   mantissa, is also less precise.  In this case, the float64 would be
   reduced in size to 4 octets.

   Reduced-size encoding MUST NOT be applied to any other data type
   defined in [RFC7012] that implies a fixed length, as these types
   either have internal structure (such as ipv4Address or
   dateTimeMicroseconds) or restricted ranges that are not suitable for
   reduced-size encoding (such as dateTimeMilliseconds).

   Information Elements of type octetArray and string may be exported
   using any length, subject to restrictions on length specific to each
   Information Element, as noted in that Information Element's
   description.

7. Variable-Length Information Element

The IPFIX Template mechanism is optimized for fixed-length Information Elements [RFC7012]. Where an Information Element has a variable length, the following mechanism MUST be used to carry the length information for both the IANA-assigned and enterprise-specific Information Elements. In the Template Set, the Information Element Field Length is recorded as 65535. This reserved length value notifies the Collecting Process that the length value of the Information Element will be carried in the Information Element content itself.
Top   ToC   RFC7011 - Page 38
   In most cases, the length of the Information Element will be less
   than 255 octets.  The following length-encoding mechanism optimizes
   the overhead of carrying the Information Element length in this more
   common case.  The length is carried in the octet before the
   Information Element, as shown in Figure R.

      0                   1                   2                   3
      0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
     | Length (< 255)|          Information Element                  |
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
     |                      ... continuing as needed                 |
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

            Figure R: Variable-Length Information Element (IE)
                           (Length < 255 Octets)

   The length may also be encoded into 3 octets before the Information
   Element, allowing the length of the Information Element to be greater
   than or equal to 255 octets.  In this case, the first octet of the
   Length field MUST be 255, and the length is carried in the second and
   third octets, as shown in Figure S.

     0                   1                   2                   3
     0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |      255      |      Length (0 to 65535)      |       IE      |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |                      ... continuing as needed                 |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

            Figure S: Variable-Length Information Element (IE)
                        (Length 0 to 65535 Octets)

   The octets carrying the length (either the first or the first
   three octets) MUST NOT be included in the length of the Information
   Element.



(page 38 continued on part 3)

Next Section