Tech-invite3GPPspaceIETFspace
96959493929190898887868584838281807978777675747372717069686766656463626160595857565554535251504948474645444342414039383736353433323130292827262524232221201918171615141312111009080706050403020100
in Index   Prev   Next

RFC 6313

Export of Structured Data in IP Flow Information Export (IPFIX)

Pages: 71
Proposed Standard
Errata
Updates:  5102
Part 2 of 4 – Pages 12 to 33
First   Prev   Next

Top   ToC   RFC6313 - Page 12   prevText

4. Linkage with the IPFIX Information Model

As in the IPFIX protocol specification [RFC5101], the new Information Elements specified in Section 4.3 MUST be sent in canonical format in network-byte order (also known as the big-endian byte ordering).

4.1. New Abstract Data Types

This document specifies three new abstract data types, as described below.

4.1.1. basicList

The type "basicList" represents a list of zero or more instances of any Information Element, primarily used for single-valued data types. Examples include a list of port numbers, a list of interface indexes, a list of AS in a BGP AS-PATH, etc.

4.1.2. subTemplateList

The type "subTemplateList" represents a list of zero or more instances of a structured data type, where the data type of each list element is the same and corresponds with a single Template Record. Examples include a structured data type composed of multiple pairs of ("MPLS label stack entry position", "MPLS label stack value"), a structured data type composed of performance metrics, and a structured data type composed of multiple pairs of IP address, etc.

4.1.3. subTemplateMultiList

The type "subTemplateMultiList" represents a list of zero or more instances of a structured data type, where the data type of each list element can be different and corresponds with different Template definitions. Examples include a structured data type composed of
Top   ToC   RFC6313 - Page 13
   multiple access-list entries, where entries can be composed of
   different criteria types.

4.2. New Data Type Semantic

This document specifies a new data type semantic, in addition to the ones specified in Section 3.2 of the IPFIX information model [RFC5102], as described below.

4.2.1. List

A list represents an arbitrary-length sequence of zero or more structured data Information Elements, either composed of regular Information Elements or composed of data conforming to a Template Record.

4.3. New Information Elements

This document specifies three new Information Elements, as described below.

4.3.1. basicList

A basicList specifies a generic Information Element with a basicList abstract data type as defined in Section 4.1.1 and list semantics as defined in Section 4.2.1. Examples include a list of port numbers, a list of interface indexes, etc.

4.3.2. subTemplateList

A subTemplateList specifies a generic Information Element with a subTemplateList abstract data type as defined in Section 4.1.2 and list semantics as defined in Section 4.2.1.

4.3.3. subTemplateMultiList

A subTemplateMultiList specifies a generic Information Element with a subTemplateMultiList abstract data type as defined in Section 4.1.3 and list semantics as defined in Section 4.2.1.

4.4. New Structured Data Type Semantics

Structured data type semantics are provided in order to express the relationship among multiple list elements in a Structured Data Information Element. These structured data type semantics require a new IPFIX subregistry, as specified in the "IANA Considerations" section. The semantics are specified in the following subsections.
Top   ToC   RFC6313 - Page 14

4.4.1. undefined

The "undefined" structured data type semantic specifies that the semantic of list elements is not specified and that, if a semantic exists, then it is up to the Collecting Process to draw its own conclusions. The "undefined" structured data type semantic, which is the default value, is used when no other structured data type semantic applies. For example, a mediator that wants to translate IPFIX [RFC5101] into the export of structured data according to the specifications in this document doesn't know what the semantic is; it can only guess, as the IPFIX specifications [RFC5101] does not contain any semantic. Therefore, the mediator should use the "undefined" semantic.

4.4.2. noneOf

The "noneOf" structured data type semantic specifies that none of the elements are actual properties of the Data Record. For example, a mediator might want to report to a Collector that a specific Flow is suspicious, but that it checked already that this Flow does not belong to the attack type 1, attack type 2, or attack type 3. So this Flow might need some further inspection. In such a case, the mediator would report the Flow Record with a basicList composed of (attack type 1, attack type 2, attack type 3) and the respective structured data type semantic of "noneOf". Another example is a router that monitors some specific BGP AS-PATHs and reports if a Flow belongs to any of them. If the router wants to export that a Flow does not belong to any of the monitored BGP AS- PATHs, the router reports a Data Record with a basicList composed of (BGP AS-PATH 1, BGP AS-PATH 2, BGP AS-PATH 3) and the respective structured data type semantic of "noneOf".

4.4.3. exactlyOneOf

The "exactlyOneOf" structured data type semantic specifies that only a single element from the structured data is an actual property of the Data Record. This is equivalent to a logical XOR operation. For example, if a Flow record contains a basicList of outgoing interfaces with the "exactlyOneOf" semantic, then it implies that the reported Flow only egressed from a single interface, although the Flow Record lists all of the possible outgoing interfaces. This is a typical example of a per destination load-balancing.
Top   ToC   RFC6313 - Page 15
   Another example is a mediator that must aggregate Data Records from
   different Observation Points and report an aggregated Observation
   Point.  However, the different Observation Points can be specified by
   different Information Element types depending on the Exporter.  For
   example:

      Exporter1 Observation Point is characterized by the
      exporterIPv4Address, so a specific Exporter can be represented.

      Exporter2 Observation Point is characterized by the
      exporterIPv4Address and a basicList of ingressInterface, so the
      Exporting Process can express that the observations were made on a
      series of input interfaces.

      Exporter3 Observation Point is characterized by the
      exporterIPv4Address and a specific lineCardId, so the Exporting
      Process can express that the observation was made on a specific
      linecard.

   If the mediator models the three different types of Observation
   Points with the three Template Records below:

      Template Record 1: exporterIPv4Address
      Template Record 2: exporterIPv4Address, basicList of
                         ingressInterface
      Template Record 3: exporterIPv4Address, lineCardId

   then it can represent the aggregated Observation Point with a
   subTemplateMultiList and the semantic "exactlyOneOf".  The aggregated
   Observation Point is modeled with the Data Records corresponding to
   either Template Record 1, Template Record 2, or Template Record 3 but
   not more than one of these.  This implies that the Flow was observed
   at exactly one of the Observation Points reported.

4.4.4. oneOrMoreOf

The "oneOrMoreOf" structured data type semantic specifies that one or more elements from the list in the structured data are actual properties of the Data Record. This is equivalent to a logical OR operation. Consider an example where a mediator must report an aggregated Flow (e.g., by aggregating IP addresses from IP prefixes), with an aggregated Observation Point. However, the different Observation Points can be specified by different Information Element types as described in Section 4.4.2.
Top   ToC   RFC6313 - Page 16
   If the mediator models the three different types of Observation
   Points with the three Template Records below:

          Template Record 1: exporterIPv4Address
          Template Record 2: exporterIPv4Address, basicList of
                             ingressInterface
          Template Record 3: exporterIPv4Address, lineCardId

   then it can represent the aggregated Observation Point with a
   subTemplateMultiList and the semantic "oneOrMoreOf".  The aggregated
   Observation Point is modeled with the Data Records corresponding to
   either Template Record 1, Template Record 2, or Template Record 3.
   This implies that the Flow was observed on at least one of the
   Observation Points reported, and potentially on multiple Observation
   Points.

4.4.5. allOf

The "allOf" structured data type semantic specifies that all of the list elements from the structured data are actual properties of the Data Record. For example, if a Record contains a basicList of outgoing interfaces with the "allOf" semantic, then the observed Flow is typically a multicast Flow where each packet in the Flow has been replicated to each outgoing interface in the basicList.

4.4.6. ordered

The "ordered" structured data type semantic specifies that elements from the list in the structured data are ordered. For example, an Exporter might want to export the AS10 AS20 AS30 AS40 BGP AS-PATH. In such a case, the Exporter would report a basicList composed of (AS10, AS20, AS30, AS40) and the respective structured data type semantic of "ordered".

4.5. Encoding of IPFIX Data Types

The following subsections define the encoding of the abstract data types defined in Section 4.1. These data types may be encoded using either fixed- or variable-length Information Elements, as discussed in Section 5.1. Like in the IPFIX specifications [RFC5101], all lengths are specified in octets.
Top   ToC   RFC6313 - Page 17

4.5.1. basicList

The basicList Information Element defined in Section 4.3.1 represents a list of zero or more instances of an Information Element and is encoded as follows: 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Semantic |0| Field ID | Element... | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | ...Length | basicList Content ... | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | ... | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | ... | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Figure 1: basicList Encoding Semantic The Semantic field indicates the relationship among the different Information Element values within this Structured Data Information Element. Refer to IANA's "IPFIX Structured Data Types Semantics" registry. Field ID Field ID is the Information Element identifier of the Information Element(s) contained in the list. Element Length Per Section 7 of [RFC5101], the Element Length field indicates the length, in octets, of each list element specified by Field ID, or contains the value 0xFFFF if the length is encoded as a variable- length Information Element at the start of the basicList Content. Effectively, the Element Length field is part of the header, so even in the case of a zero-element list, it MUST NOT be omitted. basicList Content A Collecting Process decodes list elements from the basicList Content until no further data remains. A field count is not included but can be derived when the Information Element is decoded.
Top   ToC   RFC6313 - Page 18
   Note that in the diagram above, Field ID is shown with the Enterprise
   bit (most significant bit) set to 0.  Instead, if the Enterprise bit
   is set to 1, a four-byte Enterprise Number MUST be encoded
   immediately after the Element Length as shown below.  See the "Field
   Specifier Format" section in the IPFIX protocol [RFC5101] for
   additional information.

    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |    Semantic   |1|         Field ID            |   Element...  |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   | ...Length     |               Enterprise Number ...           |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |      ...      |              basicList Content ...            |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                              ...                              |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

           Figure 2: basicList Encoding with Enterprise Number

   Also, note that if a basicList has zero elements, the encoded data
   contains the Semantic field, Field ID, the Element Length field, and
   the four-byte Enterprise Number (if present), while the basicList
   Content is empty.

   If the basicList is encoded as a variable-length Information Element
   in less than 255 octets, it MAY be encoded with the Length field per
   Section 7 of [RFC5101] as shown in Figure 3.  However, the three-byte
   length encoding, as shown in Figure 4, is RECOMMENDED (see Section
   5.1).

    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   | Length (< 255)|   Semantic    |0|          Field ID           |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |         Element Length        | basicList Content ...         |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                              ...                              |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                              ...                              |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

        Figure 3: Variable-Length basicList Encoding
                      (Length < 255 Octets)
Top   ToC   RFC6313 - Page 19
   If the basicList is encoded as a variable-length Information Element
   in 255 or more octets, it MUST be encoded with the Length field per
   Section 7 of [RFC5101] as follows:

    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |      255      |      Length (0 to 65535)      |   Semantic    |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |0|          Field ID           |        Element Length         |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                      basicList Content ...                    |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                              ...                              |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                              ...                              |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

     Figure 4: Variable-Length basicList Encoding
                  (Length 0 to 65535 Octets)

4.5.2. subTemplateList

The subTemplateList Information Element represents a list of zero or more Data Records corresponding to a specific Template. Because the Template Record referenced by a subTemplateList Information Element can itself contain other subTemplateList Information Elements, and because these Template Record references are part of the Information Elements content in the Data Record, it is possible to represent complex hierarchical data structures. The following diagram shows how a subTemplateList Information Element is encoded within a Data Record: 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Semantic | Template ID | ... | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | subTemplateList Content ... | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | ... | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Figure 5: subTemplateList Encoding
Top   ToC   RFC6313 - Page 20
   Semantic

      The Semantic field indicates the relationship among the different
      Data Records within this Structured Data Information Element.

   Template ID

      The Template ID field contains the ID of the Template used to
      encode and decode the subTemplateList Content.

   subTemplateList Content

      subTemplateList Content consists of zero or more instances of Data
      Records corresponding to the Template ID specified in the Template
      ID field.  A Collecting Process decodes the subTemplateList
      Content until no further data remains.  A record count is not
      included but can be derived when the subTemplateList is decoded.
      Encoding and decoding are performed recursively if the specified
      Template itself contains Structured Data Information Elements as
      described here.

   Note that, if a subTemplateList has zero elements, the encoded data
   contains only the Semantic field and the Template ID field, while the
   subTemplateList Content is empty.

   If the subTemplateList is encoded as a variable-length Information
   Element in less than 255 octets, it MAY be encoded with the Length
   field per Section 7 of [RFC5101] as shown in Figure 6.  However, the
   three-byte length encoding, as shown in Figure 7, is RECOMMENDED (see
   Section 5.1).

    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   | Length (< 255)|   Semantic    |         Template ID           |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                subTemplateList Content    ...                 |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                              ...                              |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

     Figure 6: Variable-Length subTemplateList Encoding
                       (Length < 255 Octets)
Top   ToC   RFC6313 - Page 21
   If the subTemplateList is encoded as a variable-length Information
   Element in 255 or more octets, it MUST be encoded with the Length
   field per Section 7 of [RFC5101] as follows:

    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |      255      |      Length (0 to 65535)      |   Semantic    |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |         Template ID           | subTemplateList Content ...   |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                              ...                              |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

     Figure 7: Variable-Length subTemplateList Encoding
                    (Length 0 to 65535 Octets)

4.5.3. subTemplateMultiList

Whereas each element in a subTemplateList Information Element corresponds to a single Template, it is sometimes useful for a list to contain elements corresponding to different Templates. To support this case, each top-level element in a subTemplateMultiList Information Element carries a Template ID, Length, and zero or more Data Records corresponding to the Template ID. The following diagram shows how a subTemplateMultiList Information Element is encoded within a Data Record. Note that the encoding following the Semantic field is consistent with the Set Header specified in [RFC5101]. 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Semantic | Template ID X |Data Records...| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | ... Length X | Data Record X.1 Content ... | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | ... | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | ... | Data Record X.2 Content ... | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | ... | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | ... | Data Record X.L Content ... | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | ... | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | ... | Template ID Y |Data Records...| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Top   ToC   RFC6313 - Page 22
   | ... Length Y  |        Data Record  Y.1 Content ...           |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                              ...                              |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |      ...      |        Data Record Y.2 Content ...            |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                              ...                              |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |      ...      |        Data Record Y.M Content ...            |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                              ...                              |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |      ...      |         Template ID Z         |Data Records...|
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   | ... Length Z  |        Data Record Z.1 Content ...            |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                              ...                              |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |      ...      |        Data Record Z.2 Content ...            |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                              ...                              |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |      ...      |        Data Record Z.N Content ...            |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                              ...                              |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |      ...      |
   +-+-+-+-+-+-+-+-+

                Figure 8: subTemplateMultiList Encoding

   Semantic

      The Semantic field indicates the top-level relationship among the
      series of Data Records corresponding to the different Template
      Records within this Structured Data Information Element.

   Template ID

      Unlike the subTemplateList Information Element, each element of
      the subTemplateMultiList contains a Template ID that specifies the
      encoding of the following Data Records.
Top   ToC   RFC6313 - Page 23
   Data Records Length

      This is the total length of the Data Records encoding for the
      Template ID previously specified, including the two bytes for the
      Template ID and the two bytes for the Data Records Length field
      itself.

   Data Record X.M

      The Data Record X.M consists of the Mth Data Record of the
      Template Record X.  A Collecting Process decodes the Data Records
      according to Template Record X until no further data remains,
      according to the Data Records Length X.  Further Template IDs and
      Data Records may then be decoded according to the overall
      subTemplateMultiList length.  A record count is not included but
      can be derived when the Element Content is decoded.  Encoding and
      decoding are performed recursively if the specified Template
      itself contains Structured Data Information Elements as described
      here.

   In the exceptional case of zero instances in the
   subTemplateMultiList, no data is encoded, only the Semantic field and
   Template ID field(s), and the Data Record Length field is set to
   zero.

   If the subTemplateMultiList is encoded as a variable-length
   Information Element in less than 255 octets, it MAY be encoded with
   the Length field per Section 7 of [RFC5101] as shown in Figure 9.
   However, the three-byte length encoding, as shown in Figure 10, is
   RECOMMENDED (see Section 5.1).

    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   | Length (< 255)|    Semantic   |         Template ID X         |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |      Data Records Length X    |  Data Record X.1 Content ...  |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                              ...                              |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |             ...               |   Data Record X.2 Content ... |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                              ...                              |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |             ...               |   Data Record X.L Content ... |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                              ...                              |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Top   ToC   RFC6313 - Page 24
   |             ...               |         Template ID Y         |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |      Data Records Length Y    |   Data Record Y.1 Content ... |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                              ...                              |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |             ...               |   Data Record Y.2 Content ... |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                              ...                              |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |             ...               |   Data Record Y.M Content ... |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                              ...                              |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |             ...               |         Template ID Z         |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |      Data Records Length Z    |   Data Record Z.1 Content ... |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                              ...                              |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |             ...               |   Data Record Z.2 Content ... |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                              ...                              |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |             ...               |   Data Record Z.N Content ... |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                              ...                              |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |             ...               |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

      Figure 9: Variable-Length subTemplateMultiList Encoding
                      (Length < 255 Octets)

   If the subTemplateMultiList is encoded as a variable-length
   Information Element in 255 or more octets, it MUST be encoded with
   the Length field per Section 7 of [RFC5101] as follows:

    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |      255      |      Length (0 to 65535)      |   Semantic    |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |         Template ID X         |    Data Records Length X      |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                   Data Record X.1 Content ...                 |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                              ...                              |
Top   ToC   RFC6313 - Page 25
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                   Data Record X.2 Content ...                 |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                              ...                              |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                   Data Record X.L Content ...                 |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                              ...                              |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |         Template ID Y         |    Data Records Length Y      |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                   Data Record  Y.1 Content ...                |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                              ...                              |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                   Data Record Y.2 Content ...                 |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                              ...                              |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                   Data Record Y.M Content ...                 |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                              ...                              |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |         Template ID Z         |    Data Records Length Z      |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                     Data Record Z.1 Content ...               |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                              ...                              |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                     Data Record Z.2 Content ...               |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                              ...                              |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                     Data Record Z.N Content ...               |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                              ...                              |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

     Figure 10: Variable-Length subTemplateMultiList Encoding
                        (Length 0 to 65535 Octets)

5. Structured Data Format

5.1. Length Encoding Considerations

The new Structured Data Information Elements represent a list that potentially carries complex hierarchical and repeated data.
Top   ToC   RFC6313 - Page 26
   When the encoding of a Structured Data Information Element has a
   fixed length (because, for example, it contains the same number of
   fixed-length elements, or if the permutations of elements in the list
   always produces the same total length), the element length can be
   encoded in the corresponding Template Record.

   However, when representing variable-length data, hierarchical data,
   and repeated data with variable element counts, where the number and
   length of elements can vary from record to record, we RECOMMEND that
   the Information Elements are encoded using the variable-length
   encoding described in Section 7 of [RFC5101], with the length carried
   before the Structured Data Information Element encoding.

   Because of the complex and repeated nature of the data, it is
   potentially difficult for the Exporting Process to efficiently know
   in advance the exact encoding size.  In this case, the Exporting
   Process may encode the available data starting at a fixed offset and
   fill in the final length afterwards.  Therefore, the three-byte
   length encoding is RECOMMENDED for variable-length Information
   Elements in all Template Records containing a Structured Data
   Information Element, even if the encoded length can be less than 255
   bytes, because the starting offset of the data is known in advance.

   When encoding such data, an Exporting Process MUST take care to not
   exceed the maximum allowed IPFIX message length of 65535 bytes as
   specified in [RFC5101].

5.2. Recursive Structured Data

It is possible to define recursive relationships between IPFIX structured data instances, for example, when representing a tree structure. The simplest case of this might be a basicList, where each element is itself a basicList, or a subTemplateList where one of the fields of the referenced Template is itself a subTemplateList referencing the same Template. Also, the Exporting Process MUST take care when encoding recursively-defined structured data not to exceed the maximum allowed length of an IPFIX Message (as noted in Length Encoding Considerations).

5.3. Structured Data Information Elements Applicability in Options Template Sets

Structured Data Information Elements MAY be used in Options Template Sets. As an example, consider a mediation function that must aggregate Data Records from multiple Observation Point types:
Top   ToC   RFC6313 - Page 27
      Router 1, (interface 1)
      Router 2, (linecard A)
      Router 3, (linecard B)
      Router 4, (linecard C, interface 2)

   In order to encode the PSAMP Selection Sequence Report Interpretation
   [RFC5476], the mediation function must express this combination of
   Observation Points as a single new Observation Point.  Recall from
   [RFC5476] that the PSAMP Selection Sequence Report Interpretation
   consists of the following fields:

     Scope:     selectionSequenceId
     Non-Scope: one Information Element mapping the Observation Point
                selectorId (one or more)

   Without structured data, there is clearly no way to express the
   complex aggregated Observation Point as "one Information Element
   mapping the Observation Point".  However, the desired result may be
   easily achieved using the structured data types.  Refer to Section
   9.5. for an encoding example related to this case study.

   Regarding the scope in the Options Template Record, the IPFIX
   specification [RFC5101] mentions that "the IPFIX protocol doesn't
   prevent the use of any Information Elements for scope".  Therefore, a
   Structured Data Information Element MAY be used as scope in an
   Options Template Set.

   Extending the previous example, the mediation function could export a
   given name for this complex aggregated Observation Point:

      Scope: Aggregated Observation Point (structured data)
      Non-Scope: a new Information Element containing the name

5.4. Usage Guidelines for Equivalent Data Representations

Because basicList, subTemplateList, and subTemplateMultiList are all lists, in several cases, there is more than one way to represent what is effectively the same data structure. However, in some cases, one approach has an advantage over the other, e.g., more compact, uses fewer resources, and is therefore preferred over an alternate representation. A subTemplateList can represent the same simple list of single-valued Information Elements as a basicList, if the Template referenced by the subTemplateList contains only one single-valued Information Element. Although the encoding is more compact than a basicList by two bytes, using a subTemplateList, in this case, requires a new
Top   ToC   RFC6313 - Page 28
   Template per Information Element.  The basicList requires no
   additional Template and is therefore RECOMMENDED in this case.

   Although a subTemplateMultiList with one Element can represent the
   contents of a subTemplateList, the subTemplateMultiList carries two
   additional bytes (Element Length).  It is also potentially useful to
   a Collecting Process to know in advance that a subTemplateList
   directly indicates that list element types are consistent.  The
   subTemplateList Information Element is therefore RECOMMENDED in this
   case.

   The Semantic field in a subTemplateMultiList indicates the top-level
   relationship among the series of Data Records corresponding to the
   different Template Records, within this Structured Data Information
   Element.  If a semantic is required to describe the relationship
   among the different Data Records corresponding to a single Template
   ID within the subTemplateMultiList, then an encoding based on a
   basicList of subTemplateLists should be used; refer to Section 5.6
   for more information.  Alternatively, if a semantic is required to
   describe the relationship among all Data Records within a
   subTemplateMultiList (regardless of the Template Record), an encoding
   based on a subTemplateMultiList with one Data Record corresponding to
   a single Template ID can be used.

   Note that the referenced Information Element(s) in the Structured
   Data Information Elements can be taken from the IPFIX information
   model [RFC5102], the PSAMP information model [RFC5477], any of the
   Information Elements defined in the IANA IPFIX registry [IANA-IPFIX],
   or enterprise-specific Information Elements.

   If a Template Record contains a subTemplateList as the only field, a
   Set encoding as specified in the IPFIX protocol specifications
   [RFC5101] should be considered, unless:

   - A relationship among multiple list elements must be exported, in
     which case, the semantic from the IPFIX Structured Data Information
     Element can convey this relationship.

   - The Exporting Process wants to convey the number of elements in the
     list, even in the special cases of zero or one element in the list.
     Indeed, the case of an empty list cannot be represented with the
     IPFIX protocol specifications [RFC5101].  In the case of a single
     element list, the Template Record specified in the IPFIX protocol
     specification [RFC5101] could be used.  However, on the top of the
     Template Record with the subTemplateList to export multiple list
     elements, this supplementary Template would impose some extra
Top   ToC   RFC6313 - Page 29
     management, both on the Exporting Process and on the Collecting
     Process, which might have to correlate the information from two
     Template Records.

   Similarly, if a Template Record contains a subTemplateMultiList as
   the only field, an IPFIX Message as described in the IPFIX protocol
   specification [RFC5101] should be considered, unless:

   - A relationship among top-level list elements must be exported, in
     which case, the semantic from the IPFIX Structured Data Information
     Element can convey this relationship.

   - The Exporting Process wants to convey the number of Data Records
     corresponding to every Template in the subTemplateMultiList.

5.5. Padding

The Exporting Process MAY insert some padding octets in structured data field values in a Data Record by including the 'paddingOctets' Information Element as described in [RFC5101], Section 3.3.1. The paddingOctets Information Element can be included in a Template Record referenced by a structured data Information Element for this purpose.

5.6. Semantic

Semantic interpretations of received Data Records at or beyond the Collecting Process remain explicitly undefined, unless that data is transmitted using this extension with explicit structured data type semantic information. It is not the Exporter's role to check the validity of the semantic representation of Data Records. More complex semantics can be expressed as a combination of the Semantic Data Information Elements specified in this document. For example, the export of the AS10 AS20 AS30 AS40 {AS50,AS60} BGP AS-PATH would be reported as a basicList of two elements, each element being a basicList of BGP AS, with the top-level structured data type semantic of "ordered". The first element would contain a basicList composed of (AS10,AS20,AS30,AS40) and the respective structured data type semantic of "ordered", while the second element would contain a basicList composed of (AS50, AS60) and the respective structured data type semantic of "exactlyOneOf". A high-level Data Record diagram would be represented as:
Top   ToC   RFC6313 - Page 30
        BGP AS-PATH = (basicList, ordered,

            (basicList, ordered, AS10,AS20,AS30,AS40),

            (basicList, exactlyOneOf, AS50, AS60)

        )

   If a semantic is required to describe the relationship among the
   different Data Records corresponding to a single Template ID within
   the subTemplateMultiList, then an encoding based on a basicList of
   subTemplateLists should be used, as shown in the next case study.

    Case study 1:

   In this example, an Exporter monitoring security attacks must export
   a list of security events consisting of attackers and targets.  For
   the sake of the example, assume that the Collector can differentiate
   the attacker (which is expressed using source fields) from the target
   (which is expressed using destination fields).  Imagine that
   attackers A1 or A2 may attack targets T1 and T2.

   The first case uses a subTemplateMultiList composed of two Template
   Records, one representing the attacker and one representing the
   target, each of them containing an IP address and a port.

        Attacker Template Record = (src IP address, src port)

        Target Template Record = (dst IP address, dst port)

   A high-level Data Record diagram would be represented as:

         Alert = (subTemplateMultiList, allOf,

            (Attacker Template Record, A1, A2),

            (Target Template Record, T1, T2)

         )

   The Collecting Process can only conclude that the list of attackers
   (A1, A2) and the list of targets (T1, T2) are present, without
   knowing the relationship amongst attackers and targets.  The
   Exporting Process would have to explicitly call out the relationship
   amongst attackers and targets as the top-level semantic offered by
   the subTemplateMultiList isn't sufficient.
Top   ToC   RFC6313 - Page 31
   The only proper encoding for the previous semantic (i.e., attacker A1
   or A2 may attack target T1 and T2) uses a basicList of
   subTemplateLists and is represented as follows:

        Attacker Template Record = (src IP address, src port)

        Target Template Record = (dst IP address, dst port)

        Alert = (basicList, allOf,

              (subTemplateList, exactlyOneOf, attacker A1, A2)

              (subTemplateList, allOf, target T1, T2)

        )

    Case study 2:

   In this example, an Exporter monitoring security attacks must export
   a list of attackers and targets.  For the sake of the example, assume
   that the Collector can differentiate the attacker (which is expressed
   using source fields) from the target (which is expressed using
   destination fields).  Imagine that attacker A1 or A2 is attacking
   target T1, while attacker A3 is attacking targets T2 and T3.  The
   first case uses a subTemplateMultiList that contains Data Records
   corresponding to two Template Records, one representing the attacker
   and one representing the target, each of them containing an IP
   address and a port.

        Attacker Template Record = (src IP address, src port)
        Target Template Record = (dst IP address, dst port)

   A high-level Data Record diagram would be represented as:

         Alert = (subTemplateMultiList, allOf,

            (Attacker Template Record, A1, A2, A3),

            (Target Template Record, T1, T2, T3)

         )

   The Collecting Process can only conclude that the list of attackers
   (A1, A2, A3), and the list of targets (T1, T2, T3) are present,
   without knowing the relationship amongst attackers and targets.
Top   ToC   RFC6313 - Page 32
   The second case could use a Data Record definition composed of the
   following:

         Alert = (subTemplateMultiList, allOf,

            (Attacker Template Record, A1, A2),

            (Target Template Record, T1),

            (Attacker Template Record, A3),

            (Target Template Record, T2, T3)

         )

   With the above representation, the Collecting Process can infer that
   the alert consists of the list of attackers (A1, A2), target (T1),
   attacker (A3), and list of targets (T2, T3).  From the sequence in
   which attackers and targets are encoded, the Collector can possibly
   deduce that some relationship exists among (A1, A2, T1) and (A2, T1,
   T2) but cannot understand what it is exactly.  So, there is a need
   for the Exporting Process to explicitly define the relationship
   between the attackers, and targets and the top-level semantic of the
   subTemplateMultiList is not sufficient.

   The only proper encoding for the previous semantic (i.e., attacker A1
   or A2 attacks target T1, attacker A3 attacks targets T2 and T3) uses
   a basicList of subTemplateLists and is represented as follows:

        Participant P1 =

        (basicList, allOf,

              (subTemplateList, exactlyOneOf, attacker A1, A2)

              (subTemplateList, undefined, target T1)

        )

        Participant P2 =

        (basicList, allOf,

              (subTemplateList, undefined, attacker A3,

              (subTemplateList, allOf, targets T2, T3)

        )
Top   ToC   RFC6313 - Page 33
   The security alert is represented as a subTemplateList of
   participants.

        Alert =

           (subTemplateList, allOf, Participant P1, Participant P2)

   Note that, in the particular case of a single element in a Structured
   Data Information Element, the Semantic field is actually not very
   useful since it specifies the relationship among multiple elements.
   Any choice of allOf, exactlyOneOf, or OneOrMoreOf would provide the
   same result semantically.  Therefore, in case of a single element in
   a Structured Data Information Element, the default "undefined"
   semantic SHOULD be used.



(page 33 continued on part 3)

Next Section