tech-invite   World Map     

IETF     RFCs     Groups     SIP     ABNFs    |    3GPP     Specs     Gloss.     Arch.     IMS     UICC    |    Misc.    |    search     info

RFC 5655

 
 
 

Specification of the IP Flow Information Export (IPFIX) File Format

Part 3 of 3, p. 36 to 64
Prev RFC Part

 


prevText      Top      Up      ToC       Page 36 
9.  Signing and Encryption of IPFIX Files

   In order to ensure the integrity of IPFIX Files and the identity of
   IPFIX File Writers, File Writers and File Readers SHOULD provide for
   an interoperable and easily implemented method for signing IPFIX
   Files, and verifying those signatures.  This section specifies method
   via CMS detached signatures.

   Note that while CMS specifies an encapsulation format that can be
   used for encryption as well as signing, no method is specified for
   encapsulation for confidentiality protection.  It is assumed that
   application-specific or process-specific requirements outweigh the
   need for interoperability for encrypted files.

9.1.  CMS Detached Signatures

   The Cryptographic Message Syntax (CMS) [RFC3852] defines an
   encapsulation syntax for data protection, used to digitally sign,
   authenticate, or encrypt arbitrary message content.  CMS can also be
   used to create detached signatures, in which the signature is stored
   in a separate file.  This arrangement maximizes interoperability, as
   File Readers that are not aware of CMS detached signatures and have
   no requirement for them can simply ignore them; the content of the
   IPFIX File itself is unchanged by the signature.

   The detached signature file for an IPFIX File SHOULD be stored,
   transported, or otherwise made available (e.g., by FTP or HTTP)
   alongside the signed IPFIX File, with the same filename as the IPFIX
   File, except that the file extension ".p7s" is added to the end,
   conforming to the naming convention in [RFC3851].

   Within the detached signature, the CMS ContentInfo type MUST always
   be present, and it MUST encapsulate the CMS SignedData content type,
   which in turn MUST NOT encapsulate the signed IPFIX File content.
   The CMS detached signature is summarized as follows:

Top      Up      ToC       Page 37 
   ContentInfo {
     contentType          id-signedData, -- (1.2.840.113549.1.7.2)
     content              SignedData
   }

   SignedData {
     version              CMSVersion, -- Always set to 3
     digestAlgorithms     DigestAlgorithmIdentifiers,
     encapContentInfo     EncapsulatedContentInfo,
     certificates         CertificateSet, -- File Writer certificate(s)
     crls                 CertificateRevocationLists, -- Optional
     signerInfos          SET OF SignerInfo -- Only one signer
   }

   SignerInfo {
     version              CMSVersion, -- Always set to 3
     sid                  SignerIdentifier,
     digestAlgorithm      DigestAlgorithmIdentifier,
     signedAttrs          SignedAttributes,
     signatureAlgorithm   SignatureAlgorithmIdentifier,
     signature            SignatureValue,
     unsignedAttrs        UnsignedAttributes
   }

   EncapsulatedContentInfo {
     eContentType         id-data, -- (1.2.840.113549.1.7.1)
     eContent             OCTET STRING  -- Always absent
   }

   The details of the contents of each CMS encapsulation are detailed in
   the subsections below.

9.1.1.  ContentInfo

   [RFC3852] requires the outer-most encapsulation to be ContentInfo;
   the fields of ContentInfo are as follows:

   contentType:  the type of the associated content.  For the detached
      signature file, the encapsulated type is always SignedData, so the
      id-signedData (1.2.840.113549.1.7.2) object identifier MUST be
      present in this field.

   content:  a SignedData content, detailed in Section 9.1.2.

Top      Up      ToC       Page 38 
9.1.2.  SignedData

   The SignedData content type contains the signature of the IPFIX File
   and information to aid in validation; the fields of SignedData are as
   follows:

   version:  MUST be 3.

   digestAlgorithms:  a collection of one-way hash function identifiers.
      It MUST contain the identifier used by the File Writer to generate
      the digital signature.

   encapContentInfo:  the signed content, including a content type
      identifier.  Since a detached signature is being created, it does
      not encapsulate the IPFIX File.  The EncapsulatedContentInfo is
      detailed in Section 9.1.4.

   certificates:  a collection of certificates.  It SHOULD include the
      X.509 certificate needed to validate the digital signature file.
      Certification Authority (CA) and File Writer certificates MUST
      conform to the certificate profile specified in [RFC5280].

   crls:  an optional collection of certificate revocation lists (CRLs).
      It SHOULD NOT contain any CRLs; any CRLs that are present MUST
      conform to the certificate profile specified in [RFC5280].

   signerInfos:  a collection of per-signer information; this identifies
      the File Writer.  More than one SignerInfo MAY appear to
      facilitate transitions between keys or algorithms.  The SignerInfo
      type is detailed in Section 9.1.3.

9.1.3.  SignerInfo

   The SignerInfo type identifies the File Writer; the fields of
   SignerInfo are as follows:

   version:  MUST be 3.

   sid:  identifies the File Writer's public key.  This identifier MUST
      match the value included in the subjectKeyIdentifier certificate
      extension on the File Writer's X.509 certificate.

   digestAlgorithm:  identifies the one-way hash function and associated
      parameters used to generate the signature.

Top      Up      ToC       Page 39 
   signedAttrs:  an optional set of attributes that are signed along
      with the content.

   digestAlgorithm:  identifies the digital signature algorithm and
      associated parameters used to generate the signature.

   signature:  the digital signature of the associated file.

   unsignedAttrs:  an optional set of attributes that are not signed.

9.1.4.  EncapsulatedContentInfo

   The EncapsulatedContentInfo structure contains a content type
   identifier.  Since a detached signature is being created, it does not
   encapsulate the IPFIX File.  The fields of EncapsulatedContentInfo
   are as follows:

   eContentType:  an object identifier that uniquely specifies the
      content type.  The content type associated with IPFIX File MUST be
      id-data (1.2.840.113549.1.7.1).

   eContent:  an optional field containing the signed content.  Since
      this is a detached signature, eContent MUST be absent.

9.2.  Encryption Error Resilience

   Note that single bit errors in the encrypted data stream can result
   in larger errors in the decrypted stream, depending on the encryption
   scheme used.

   In applications (e.g., archival storage) in which error resilience is
   very important, File Writers SHOULD use an encryption scheme that can
   resynchronize after bit errors.  A common example is a block cipher
   in CBC (Cipher Block Chaining) mode.  In this case, File Writers MAY
   also use the Message Checksum Options Template to attach a checksum
   to each IPFIX Message in the IPFIX File, in order to support the
   recognition of errors in the decrypted data.

10.  Compression of IPFIX Files

   Network traffic measurement data is generally highly compressible.
   IPFIX Templates tend to increase the information content per record
   by not requiring the export of irrelevant or non-present fields in
   records, and the technique described in [RFC5473] also reduces the
   export of redundant information.  However, even with these
   techniques, generalized compression can decrease storage requirements
   significantly; therefore, IPFIX File Writers and File Readers SHOULD
   support compression as described in this section.

Top      Up      ToC       Page 40 
10.1.  Supported Compression Formats

   IPFIX Files support two compression encapsulation formats: bzip2
   [bzip2] and gzip [RFC1952]. bzip2 provides better compression than
   gzip and, as a block compression algorithm, better error recovery
   characteristics, at the expense of slower compression. gzip is
   potentially a better choice when compression time is an issue.  These
   two algorithms and encapsulation formats were chosen for ubiquity and
   ease of implementation.

   IPFIX File Readers and Writers supporting compression MUST support
   bzip2, and SHOULD support gzip.

10.2.  Compression Recognition at the File Reader

   bzip2, gzip, and uncompressed IPFIX Files have distinct magic
   numbers.  IPFIX File Readers SHOULD use these magic numbers to
   determine what compression, if any, is in use for an IPFIX File, and
   invoke the proper decompression. bzip2 files are identified by the
   initial three-octet string 0x42, 0x5A, 0x68 ("BZh"). gzip files are
   identified by the initial two-octet string 0x1F, 0x8B.  IPFIX Files
   are identified by the initial two-octet string 0x00, 0x0A; these are
   the version bytes of the first IPFIX Message header in the File.

10.3.  Compression Error Resilience

   Compression at the file level, like encryption, is not particularly
   resilient to errors; in the worst case, a single bit error in a
   stream-compressed file could result in the loss of the entire file.

   Since block compression algorithms that support the identification
   and isolation of blocks containing errors limit the impact of errors
   on the recoverability of compressed data, the use of bzip2 in
   applications where error resilience is important is RECOMMENDED.

   Since the block boundary of a block-compressed IPFIX File may fall in
   the middle of an IPFIX Message, resynchronization of an IPFIX Message
   stream by a File Reader after a compression error requires some care.
   The beginning of an IPFIX Message may be identified by its header
   signature (the Version field of the Message Header, 0x00 0x0A,
   followed by a 16-bit Message Length), but simply searching for the
   first occurrence of the Version field is insufficient, since these
   two bytes may occur in valid IPFIX Template or Data Sets.

   Therefore, we specify the following algorithm for File Readers to
   resynchronize an IPFIX Message Stream after skipping a compressed
   block containing errors:

Top      Up      ToC       Page 41 
   1.  Search after the error for the first occurrence of the octet
       string 0x00, 0x0A (the IPFIX Message Header Version field).

   2.  Treat this field as the beginning of a candidate IPFIX Message.
       Read the two bytes following the Version field as a Message
       Length, and seek to that offset from the beginning of the
       candidate IPFIX Message.

   3.  If the first two octets after the candidate IPFIX Message are
       0x00, 0x0A (i.e., the IPFIX Message Header Version field of the
       next message in the stream), or if the end-of-file is reached
       precisely at the end of the candidate IPFIX Message, presume that
       the candidate IPFIX Message is valid, and begin reading the IPFIX
       File from the start of the candidate IPFIX Message.

   4.  If not, or if the seek reaches end-of-file or another block
       containing errors before finding the end of the candidate
       message, go back to step 1, starting the search two bytes from
       the start of the candidate IPFIX Message.

   The algorithm above will improperly identify a non-message as a
   message approximately 1 in 2^32 times, assuming random IPFIX data.
   It may be expanded to consider multiple candidate IPFIX Messages in
   order to increase reliability.

   In applications (e.g., archival storage) in which error resilience is
   very important, File Writers SHOULD use block compression algorithms,
   and MAY attempt to align IPFIX Messages within compression blocks to
   ease resynchronization after errors.  File Readers SHOULD use the
   resynchronization algorithm above to minimize data loss due to
   compression errors.

11.  Recommended File Integration Strategies

   This section describes methods for integrating IPFIX File data with
   other file formats.

11.1.  Encapsulation of Non-IPFIX Data in IPFIX Files

   At times, it may be useful to export or store non-IPFIX data inline
   in an IPFIX File or Message stream.  To do this cleanly, this data
   must be encapsulated into IPFIX Messages so that an IPFIX File Reader
   or Collecting Process can handle it without any need to interpret it.
   At the same time, this data must not be changed during transmission
   or storage.  The opaqueOctets Information Element, as defined in
   Section 8.2.17, is provided for this encapsulation.

Top      Up      ToC       Page 42 
   Processing the encapsulated non-IPFIX data is left to a separate
   processing mechanisms that can identify encapsulated non-IPFIX data
   in an IPFIX Message Stream, but need not have any other IPFIX
   handling capability, except the ability to skip over all IPFIX
   Messages that do not encapsulate non-IPFIX data.

   The Message Checksum Options Template, described in Section 8.1.1,
   may be used as a uniform mechanism to identify errors within
   encapsulated data.

   Note that this mechanism can only encapsulate data objects up to
   65,515 octets in length.  If the space available in one IPFIX Message
   is not enough for the amount of data to be encapsulated, then the
   data must be broken into smaller segments that are encapsulated into
   consecutive IPFIX Messages.  Any additional structuring or semantics
   of the raw data is outside the scope of IPFIX and must be implemented
   within the encapsulated binary data itself.  Furthermore, the raw
   encapsulated data cannot be assumed by an IPFIX File Reader to have
   any specific format.

11.2.  Encapsulation of IPFIX Files within Other File Formats

   Consequently, it may also be useful to reverse the encapsulation,
   that is, to export or store IPFIX data inline within a non-IPFIX File
   or data stream.  This makes sense when the other file format is not
   compatible with the encapsulation described above in Section 11.1.
   Generally speaking, the encapsulation here will be specific to the
   format of the containing file.  For example, IPFIX Files may be
   embedded in XML elements using hex or Base64 encoding, or in raw
   binary files using start and end delimiters or some form of run-
   length encoding.  As there are as many potential encapsulations here
   as there are potential file formats, the specifics of each are out of
   scope for this specification.

12.  Security Considerations

   The Security Considerations section of [RFC5101], on which the IPFIX
   File format is based, is largely concerned with the proper
   application of TLS and DTLS to ensure confidentiality and integrity
   when exporting IPFIX Messages.  By analogy, this document specifies
   the use of CMS [RFC3852] detached signatures to provide equivalent
   integrity protection to TLS and DTLS in Section 9.1.  However, aside
   from merely applying CMS for signatures, there are several security
   issues which much be considered in certain circumstances; these are
   covered in the subsections below.

Top      Up      ToC       Page 43 
12.1.  Relationship between IPFIX File and Transport Encryption

   The underlying protocol used to exchange the information that will be
   stored using the format proposed in this document must as well apply
   appropriate procedures to guarantee the integrity and confidentiality
   of the exported information.  Such issues are addressed in [RFC5101].
   Specifically, IPFIX Files that store data taken from an IPFIX
   Collecting Process using TLS or DTLS for transport security SHOULD be
   signed as in Section 9.1 and SHOULD be encrypted out of band; storage
   of such flow data without encryption may present a potential breach
   of confidentiality.  Conversely, flow data considered sensitive
   enough to require encryption in storage that is later transmitted
   using IPFIX SHOULD be transmitted using TLS or DTLS for transport
   security.

12.2.  End-to-End Assertions for IPFIX Files

   Note that while both TLS and CMS provide the ability to sign an IPFIX
   Transport Session or an IPFIX File, there exists no method for
   protecting data integrity end-to-end in the case in which a
   Collecting Process is collocated with a File Writer.  The channel
   between the Exporting Process to Collecting Process using IPFIX is
   signed by the Exporting Process key and protected via TLS and DTLS,
   while the File is signed by the File Writer key and protected via
   CMS.  The identity of the Exporting Process is not asserted in the
   file, and the records may be modified between the Collecting Process
   and the File Writer.

   There are two potential ways to address this issue.  The first is by
   fiat, and is appropriate only when the application allows the
   Collecting-Process-to-File-Writer channel to be trusted.  In this
   case, the File Writer's signature is an implicit assertion that the
   channel to the Exporting Process was protected, that the Exporting
   Process's signature was verified, and that the data was not changed
   after collection.  For this to work, a File Writer collocated with a
   Collecting Process SHOULD NOT sign a File as specified in Section 9.1
   unless the Transport Session over which the data was exported was
   protected via TLS or DTLS, and the Collecting Process positively
   identified the Exporting Process by its certificate.  The File Writer
   SHOULD include the Exporting Process and Collecting Process
   certificates within the File using the Export Session Detail Options
   Template in Section 8.1.3 or the Message Detail Options Template in
   Section 8.1.4 to allow for later verification.

   In situations in which the Collecting Process and/or File Writer
   cannot be trusted, end-to-end integrity can then be approximated by
   collocating the File Writer with the Metering Process, and removing
   the IPFIX protocol completely from the chain.  In this case, the File

Top      Up      ToC       Page 44 
   Writer's signature is an implicit assertion that the Metering Process
   is identified and is not tampering with the information as observed
   on the wire.

   Verification of these trust relationships is out of scope for this
   document, and should be considered on a per-implementation basis.

12.3.  Recommendations for Strength of Cryptography for IPFIX Files

   Note that when encrypting files for archival storage, the
   cryptographic strength is dependent on the length of time over which
   archival data is expected to be kept.  Long-term storage may require
   re-application of cryptographic protection, periodically resigning
   and reencrypting files with stronger keys.  In this case, it is
   recommended that the existing signed and/or encypted data be
   encapsulated within newer, stronger protection.  See [RFC4810] for a
   discussion of this issue.

13.  IANA Considerations

   This document specifies the creation of several new IPFIX Information
   Elements in the IPFIX Information Element registry located at
   http://www.iana.org, as defined in Section 8.2 above.  IANA has
   assigned the following Information Element numbers for their
   respective Information Elements as specified below:

   o  Information Element number 258 for the collectionTimeMilliseconds
      Information Element.

   o  Information Element number 274 for the collectorCertificate
      Information Element.

   o  Information Element number 275 for the exporterCertificate
      Information Element.

   o  Information Element number 259 for the exportSctpStreamId
      Information Element.

   o  Information Element number 260 for the maxExportSeconds
      Information Element.

   o  Information Element number 268 for the maxFlowEndMicroseconds
      Information Element.

   o  Information Element number 269 for the maxFlowEndMilliseconds
      Information Element.

Top      Up      ToC       Page 45 
   o  Information Element number 270 for the maxFlowEndNanoseconds
      Information Element.

   o  Information Element number 261 for the maxFlowEndSeconds
      Information Element.

   o  Information Element number 262 for the messageMD5Checksum
      Information Element.

   o  Information Element number 263 for the messageScope Information
      Element.

   o  Information Element number 264 for the minExportSeconds
      Information Element.

   o  Information Element number 271 for the minFlowStartMicroseconds
      Information Element.

   o  Information Element number 272 for the minFlowStartMilliseconds
      Information Element.

   o  Information Element number 273 for the minFlowStartNanoseconds
      Information Element.

   o  Information Element number 265 for the minFlowStartSeconds
      Information Element.

   o  Information Element number 266 for the opaqueOctets Information
      Element.

   o  Information Element number 267 for the sessionScope Information
      Element.

   IANA has created the media type application/ipfix for IPFIX data, as
   described by the following registration information:

   Type name:   application

   Subtype name:   ipfix

   Required parameters:   none

   Optional parameters:   none

   Encoding considerations:   IPFIX Files are binary, and therefore must
      be encoded in non-binary contexts.

Top      Up      ToC       Page 46 
   Security considerations:   See the Security Considerations
      (Section 12) of RFC 5655, and the Security Considerations of
      [RFC5101].

   Interoperability considerations:   See the "Detailed Specification"
      (Section 7) of RFC 5655.  The format is designed to be broadly
      interoperable, as any valid stream of IPFIX Messages over any
      transport specified in [RFC5101] MUST be recognizable as a valid
      IPFIX File.

   Published specification:   RFC 5655, especially Section 7, and
      [RFC5101].

   Applications that use this media type:   Various IPFIX
      implementations (see [RFC5153]) support the construction of IPFIX
      File Readers and Writers.

   Additional information:

      Magic number(s):   None, although the first two bytes of any IPFIX
         File are the first two bytes of a message header, the Version
         field, which as of [RFC5101] are always 10 in network byte
         order: 0x00, 0x0A.

      File extension(s):   .ipfix

      Macintosh file type code(s):   none

   Person & email address to contact for further information:   Brian
      Trammell <brian.trammell@hitachi-eu.com> for the authors of RFC
      5655; Nevil Brownlee <n.brownlee@auckland.ac.nz> for the IPFIX
      Working Group.

   Intended usage:   LIMITED USE

   Restrictions on usage:   none

   Change controller:   Brian Trammell <brian.trammell@hitachi-eu.com>
      for the authors of RFC 5655; Nevil Brownlee
      <n.brownlee@auckland.ac.nz> for the IPFIX Working Group.

14.  Acknowledgements

   Thanks to Maurizio Molina, Tom Kosnar, and Andreas Kind for technical
   assistance with the requirements for a standard flow storage format.
   Thanks to Benoit Claise, Paul Aitken, Andrew Johnson, Gerhard Muenz,
   and Nevil Brownlee for their reviews and feedback.  Thanks to Pasi
   Eronen for pointing out [RFC5485], and Russ Housley for writing it;

Top      Up      ToC       Page 47 
   it specifies a detached signature format, from which Section 9.1 is
   largely drawn.  Thanks to the PRISM project for its support of this
   work.

15.  References

15.1.  Normative References

   [RFC5101]    Claise, B., "Specification of the IP Flow Information
                Export (IPFIX) Protocol for the Exchange of IP Traffic
                Flow Information", RFC 5101, January 2008.

   [RFC5102]    Quittek, J., Bryant, S., Claise, B., Aitken, P., and J.
                Meyer, "Information Model for IP Flow Information
                Export", RFC 5102, January 2008.

   [RFC5610]    Boschi, E., Trammell, B., Mark, L., and T. Zseby,
                "Exporting Type Information for IP Flow Information
                Export (IPFIX) Information Elements", RFC 5610,
                July 2009.

   [RFC1321]    Rivest, R., "The MD5 Message-Digest Algorithm",
                RFC 1321, April 1992.

   [RFC1952]    Deutsch, P., Gailly, J-L., Adler, M., Deutsch, L., and
                G. Randers-Pehrson, "GZIP file format specification
                version 4.3", RFC 1952, May 1996.

   [RFC2119]    Bradner, S., "Key words for use in RFCs to Indicate
                Requirement Levels", BCP 14, RFC 2119, March 1997.

   [RFC3852]    Housley, R., "Cryptographic Message Syntax (CMS)",
                RFC 3852, July 2004.

   [RFC4810]    Wallace, C., Pordesch, U., and R. Brandner, "Long-Term
                Archive Service Requirements", RFC 4810, March 2007.

   [RFC5280]    Cooper, D., Santesson, S., Farrell, S., Boeyen, S.,
                Housley, R., and W. Polk, "Internet X.509 Public Key
                Infrastructure Certificate and Certificate Revocation
                List (CRL) Profile", RFC 5280, May 2008.

   [bzip2]      Seward, J., "bzip2 (http://www.bzip.org/)", March 2008.

Top      Up      ToC       Page 48 
15.2.  Informative References

   [RFC3917]    Quittek, J., Zseby, T., Claise, B., and S. Zander,
                "Requirements for IP Flow Information Export (IPFIX)",
                RFC 3917, October 2004.

   [RFC3954]    Claise, B., "Cisco Systems NetFlow Services Export
                Version 9", RFC 3954, October 2004.

   [RFC5153]    Boschi, E., Mark, L., Quittek, J., Stiemerling, M., and
                P. Aitken, "IP Flow Information Export (IPFIX)
                Implementation Guidelines", RFC 5153, April 2008.

   [RFC5470]    Sadasivan, G., Brownlee, N., Claise, B., and J. Quittek,
                "Architecture for IP Flow Information Export", RFC 5470,
                March 2009.

   [RFC5471]    Schmoll, C., Aitken, P., and B. Claise, "Guidelines for
                IP Flow Information Export (IPFIX) Testing", RFC 5471,
                March 2009.

   [RFC5472]    Zseby, T., Boschi, E., Brownlee, N., and B. Claise, "IP
                Flow Information Export (IPFIX) Applicability",
                RFC 5472, March 2009.

   [RFC5473]    Boschi, E., Mark, L., and B. Claise, "Reducing
                Redundancy in IP Flow Information Export (IPFIX) and
                Packet Sampling (PSAMP) Reports", RFC 5473, March 2009.

   [SAINT2007]  Trammell, B., Boschi, E., Mark, L., and T. Zseby,
                "Requirements for a standardized flow storage solution",
                 in Proceedings of the SAINT 2007 workshop on Internet
                Measurement Technology, Hiroshima, Japan, January 2007.

   [RFC3851]    Ramsdell, B., "Secure/Multipurpose Internet Mail
                Extensions (S/MIME) Version 3.1 Message Specification",
                RFC 3851, July 2004.

   [RFC4288]    Freed, N. and J. Klensin, "Media Type Specifications and
                Registration Procedures", BCP 13, RFC 4288,
                December 2005.

   [RFC5485]    Housley, R., "Digital Signatures on Internet-Draft
                Documents", RFC 5485, March 2009.

   [pcap]       "libpcap (http://www.tcpdump.org/)", October 2008.

Top      Up      ToC       Page 49 
Appendix A.  Example IPFIX File

   In this section we will explore an example IPFIX File that
   demonstrates the various features of the IPFIX File format.  This
   File contains flow records described by a single Template.  This File
   also contains a File Time Window record to note the start and end
   time of the data, and an Export Session Details record to record
   collection infrastructure information.  Each Message within this File
   also contains a Message Checksum record, as this File may be
   externally encrypted and/or stored as an archive.  The structure of
   this File is shown in Figure 2.

             +=================================================+
             | IPFIX Message                       seq. 0      |
             | +---------------------------------------------+ |
             | | Template Set (ID 2)                  1 rec  | |
             | |   Data Tmpl. ID 256                         | |
             | +---------------------------------------------+ |
             | | Options Template Set (ID 3)          3 recs | |
             | |   File Time Window Opt. Tmpl. ID 257        | |
             | |   Message Checksum Opt. Tmpl. ID 259        | |
             | |   Export Session Details Opt. Tmpl. ID 258  | |
             | +---------------------------------------------+ |
             | | Data Set (ID 259) [Message Checksum] 1 rec  | |
             | +---------------------------------------------+ |
             +=================================================+
             | IPFIX Message                       seq. 1      |
             | +---------------------------------------------+ |
             | | Data Set (ID 257) [File Time Window] 1 rec  | |
             | +---------------------------------------------+ |
             | | Data Set (ID 258) [Export Session]   1 rec  | |
             | +---------------------------------------------+ |
             | | Data Set (ID 259) [Message Checksum] 1 rec  | |
             | +---------------------------------------------+ |
             +=================================================+
             | IPFIX Message                       seq. 4      |
             | +---------------------------------------------+ |
             | | Data Set (ID 256)                   50 recs | |
             | |  contains flow data                         | |
             | +---------------------------------------------+ |
             | | Data Set (ID 259) [Message Checksum] 1 rec  | |
             | +---------------------------------------------+ |
             +=================================================+
             | IPFIX Message                       seq. 55     |
             |                    . . .                        |

                     Figure 2: File Example Structure

Top      Up      ToC       Page 50 
   The Template describing the data records contains a flow start
   timestamp, an IPv4 5-tuple, and packet and octet total counts.  The
   Template Set defining this is as shown in Figure 3 below:

                        1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |          Set ID = 2           |          Length =  40         |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |      Template ID = 256        |        Field Count = 8        |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |0| flowStartSeconds      = 150 |       Field Length =  4       |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |0| sourceIPv4Address     =   8 |       Field Length =  4       |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |0| dest.IPv4Address      =  12 |       Field Length =  4       |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |0| sourceTransportPort   =   7 |       Field Length =  2       |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |0| dest.TransportPort    =  11 |       Field Length =  2       |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |0| protocolIdentifier    =   4 |       Field Length =  1       |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |0| octetTotalCount       =  85 |       Field Length =  4       |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |0| packetTotalCount      =  86 |       Field Length =  4       |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

                   Figure 3: File Example Data Template

A.1.  Example Options Templates

   This is followed by an Options Template Set containing the Options
   Templates required to read the File: the File Time Window Options
   Template (defined in Section 8.1.2 above), the Export Session Details
   Options Template (defined in Section 8.1.3 above), and the Message
   Checksum Options Template (defined in Section 8.1.1 above).  This
   Options Template Set is shown in Figure 4 and Figure 5 below:

Top      Up      ToC       Page 51 
                        1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |          Set ID = 3           |          Length =  80         |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |      Template ID = 257        |        Field Count = 3        |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |    Scope Field Count = 1      |0| sessionScope          = 267 |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |       Field Length = 1        |0| minFlowStartSeconds   = 265 |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |       Field Length = 4        |0| maxFlowEndSeconds     = 261 |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |       Field Length = 4        |      Template ID = 259        |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |       Field Count = 2         |    Scope Field Count = 1      |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |0| messageScope          = 263 |       Field Length =  1       |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |0| messageMD5Checksum    = 262 |       Field Length = 16       |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

    Figure 4: File Example Options Templates (Time Window and Checksum)

Top      Up      ToC       Page 52 
                        1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |       Template ID = 258       |         Field Count = 9       |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |    Scope Field Count = 1      |0| sessionScope          = 267 |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |       Field Length =  1       |0| exporterIPv4Address   = 130 |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |       Field Length =  4       |0| collectorIPv4Address  = 211 |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |       Field Length =  4       |0| exporterTransportPort = 217 |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |       Field Length =  2       |0| col.TransportPort     = 216 |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |       Field Length =  2       |0| col.TransportProtocol = 215 |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |       Field Length =  1       |0| col.ProtocolVersion   = 214 |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |       Field Length =  1       |0| minExportSeconds      = 264 |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |       Field Length =  4       |0| maxExportSeconds      = 260 |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |       Field Length =  4       |     set padding (2 octets)    |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

   Figure 5: File Example Options Templates, Continued (Session Details)

A.2.  Example Supplemental Options Data

   Following the Templates required to decode the File is the
   supplemental IPFIX Options information used to describe the File's
   contents and type information.  First comes the File Time Window
   record; it notes that the File contains data from 9 October 2007
   between 00:01:13 and 23:56:27 UTC, and appears as in Figure 6:

Top      Up      ToC       Page 53 
                        1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |          Set ID = 257         |          Length =  13         |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   | sessionScope  |           minFlowStartSeconds
   |       0       |         2007-10-09 00:01:13 UTC           . . .
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
                   |            maxFlowEndSeconds
   . . .           |         2007-10-09 23:56:27 UTC           . . .
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
                   |
   . . .           |
   +-+-+-+-+-+-+-+-+

                    Figure 6: File Example Time Window

   This is followed by information about how the data in the File was
   collected, in the Export Session Details record.  This record notes
   that the session stored in this File was sent via SCTP from an
   Exporter at 192.0.2.30 port 32769 to a Collector at 192.0.2.40 port
   4739, and contains messages exported between 00:01:57 and 23:57:12
   UTC on 9 October 2007; it is represented in its Data Set as in
   Figure 7:

Top      Up      ToC       Page 54 
                       1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |          Set ID = 258         |          Length =  27         |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   | sessionScope  |           exporterIPv4Address
   |       0       |               192.0.2.30                  . . .
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
                   |           collectorIPv4Address
   . . .           |               192.0.2.31                  . . .
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
                   |     exporterTransportPort     |   cTPort
   . . .           |             32769             |    4739   . . .
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
                   |   cTProtocol  |  cPVersion    |
   . . .           |      132      |     10        |           . . .
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
                minExportSeconds                   |
   . . .     2007-10-09 00:01:57 UTC               |           . . .
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
                maxExportSeconds                   |
   . . .     2007-10-09 23:57:12 UTC               |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

               Figure 7: File Example Export Session Details

A.3.  Example Message Checksum

   Each IPFIX Message within the File is completed with a Message
   Checksum record; the structure of this record within its Data Set is
   as in Figure 8:

    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |          Set ID = 259         |          Length =  24         |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   | messageScope  |                                               |
   |       0       |                                               |
   +-+-+-+-+-+-+-+-+                                               |
   |                       messageMD5Checksum                      |
   |           (16-byte MD5 checksum of options message)           |
   |                                                               |
   |                                                               |
   |               +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |               |              set padding (3 octets)           |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

                  Figure 8: File Example Message Checksum

Top      Up      ToC       Page 55 
A.4.  File Example Data Set

   After the Templates and supplemental Options information comes the
   data itself.  The first record of an example Data Set is shown with
   its message and set headers in Figure 9:

                        1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |     Version = 10              |         Length = 1296         |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |          Export Time = 2007-10-09 00:01:57 UTC                |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                      Sequence Number = 4                      |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                   Observation Domain ID = 1                   |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |       Set ID = 256           |          Length = 1254         |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                      flowStartSeconds                         |
   |                    2007-10-09 00:01:13 UTC                    |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                      sourceIPv4Address                        |
   |                          192.0.2.2                            |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                    destinationIPv4Address                     |
   |                          192.0.2.3                            |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |      sourceTransportPort      |   destinationTransportPort    |
   |             32770             |               80              |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |  protocolId   |             totalOctetCount
   |       6       |                  18000                    . . .
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
                   |             totalPacketCount
   . . .           |                    65                     . . .
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
                   |             (49 more records)
   . . .           |
   +-+-+-+-+-+-+-+-+

                      Figure 9: File Example Data Set

A.5.  Complete File Example

   Bringing together the examples above and adding message headers as
   appropriate, a hex dump of the first 317 bytes of the example File
   constructed above would appear as in the annotated Figure 10 below.

Top      Up      ToC       Page 56 
     0:|00 0A 00 A0 47 0A B6 E5 00 00 00 00 00 00 00 01
      [^ first message header (length 160 bytes) -->
    16:|00 02 00 28 01 00 00 08 00 96 00 04 00 08 00 04
      [^ data template set -->
    32: 00 0C 00 04 00 07 00 02 00 0B 00 02 00 04 00 01

    48: 00 55 00 04 00 56 00 04|00 03 00 50 01 01 00 03
                              [^ opt template set -->
    64: 00 01 01 0B 00 01 01 09 00 04 01 05 00 04 01 03

    80: 00 02 00 01 01 07 00 01 01 06 00 10 01 02 00 09

    96: 00 01 01 0B 00 01 00 82 00 04 00 D3 00 04 00 D9

   112: 00 02 00 D8 00 02 00 D7 00 01 00 D0 00 01 01 08

   128: 00 04 01 04 00 04 00 00|01 03 00 18 00 73 F1 12
                              [^ checksum record -->
   144: D6 C7 58 BE 44 E6 60 06 4E 78 74 AE 7D 00 00 00

   176:|00 0A 00 50 47 0A B6 E5 00 00 00 01 00 00 00 01
      [^ second message header (length 80 bytes) -->
   192:|01 01 00 0E 00 47 0A B6 B9 47 0C 07 1B 00|01 02
      [^ time window rec -> [ session detail rec ^ -->
   208: 00 1C 00 C0 00 02 1E 0C 00 02 1F 80 01 12 83 84

   224: 0A 47 0A B6 E5 47 0C 07 48 00|01 03 00 18 00 3E
           [ message checksum record ^ -->
   240: 2B 37 08 CE B2 0E 30 11 32 12 4A 5F E3 AD DB 00

   256:|00 0A 05 10 47 0A B6 E5 00 00 00 06 00 00 00 01
      [^ third message header (length 1296 bytes) -->
   272:|01 00 04 E6|47 0A B6 B9 C0 00 02 02 C0 00 02 03
      [^ set hdr ][^ first data rec -->
   288: 80 02 00 50 06 00 00 46 50 00 00 00 41

                     Figure 10: File Example Hex Dump

Top      Up      ToC       Page 57 
Appendix B.  Applicability of IPFIX Files to NetFlow V9 Flow Storage

   As the IPFIX Message format is nearly a superset of the NetFlow V9
   packet format, IPFIX Files can be used for store NetFlow V9 data
   relatively easily.  This section describes a method for doing so.
   The differences between the two protocols are outlined in
   Appendix B.1 below.  A simple, lightweight, message-for-message
   translation method for transforming V9 Packets into IPFIX Messages
   for storage within IPFIX Files is described in Appendix B.2.  An
   example of this translation method is given in Appendix B.3.

B.1.  Comparing NetFlow V9 to IPFIX

   With a few caveats, the IPFIX protocol is a superset of the NetFlow
   V9 protocol, having evolved from it largely through a process of
   feature addition to bring it into compliance with the IPFIX
   Requirements and the needs of stakeholders within the IPFIX Working
   Group.  This appendix outlines the differences between the two
   protocols.  It is informative only, and presented as an exploration
   of the two protocols to motivate the usage of IPFIX Files to store
   V9-collected flow data.

B.1.1.  Message Header Format

   Both NetFlow V9 and IPFIX use streams of messages prefixed by a
   message header, though the message header differs significantly
   between the two.  Note that in NetFlow V9 terminology, these messages
   are called packets, and messages must be delimited by datagram
   boundaries.  IPFIX does not have this constraint.  The header formats
   are detailed below:

    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |       Version Number          |            Count              |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                           sysUpTime                           |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                           UNIX Secs                           |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                       Sequence Number                         |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                        Source ID                              |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

                Figure 11: NetFlow V9 Packet Header Format

Top      Up      ToC       Page 58 
    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |       Version Number          |            Length             |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                           Export Time                         |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                       Sequence Number                         |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                    Observation Domain ID                      |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

                  Figure 12: IPFIX Message Header Format

   Version Number:   The IPFIX Version Number MUST be 10, while the
      NetFlow V9 Version Number MUST be 9.

   Length vs. Count:   The Count field in the NetFlow V9 packet header
      counts records in the message (including Data and Template
      Records), while the Length field in the IPFIX Message Header
      counts octets in the message.  Note that this implies that NetFlow
      V9 collectors must rely on datagram boundaries or some other
      external delimiter; otherwise, they must completely consume a
      message before finding its end.

   System Uptime:   System uptime in milliseconds is exported in the
      NetFlow V9 packet header.  This field is not present in the IPFIX
      Message Header, and must be exported using an IPFIX Option if
      required.

   Export Time:   Aside from being called UNIX Secs in the NetFlow V9
      packet header specification, the export time in seconds since 1
      January 1970 at 0000 UTC appears in both NetFlow V9 and IPFIX
      message headers.

   Sequence Number:   The NetFlow V9 Sequence Number counts packets,
      while the IPFIX Sequence Number counts records in Data Sets.  Both
      are scoped to Observation Domain.

   Observation Domain ID:   Similarly, the NetFlow V9 sourceID has
      become the IPFIX Observation Domain ID.

B.1.2.  Set Header Format

   Set headers are identical between NetFlow V9 and IPFIX; that is, each
   Set (FlowSet in NetFlow V9 terminology) is prefixed by a 4-byte set
   header containing the Set ID and the length of the set in octets.

Top      Up      ToC       Page 59 
   Note that the special Set IDs are different between IPFIX and NetFlow
   V9.  IPFIX Template Sets are identified by Set ID 2, while NetFlow V9
   Template FlowSets are identified by Set ID 0.  Similarly, IPFIX
   Options Template Sets are identified by Set ID 3, while NetFlow V9
   Options Template FlowSets are identified by Set ID 1.

   Both protocols reserve Set IDs 0-255, and use Set IDs 256-65535 for
   Data Sets (or FlowSets, in NetFlow V9 terminology).

B.1.3.  Template Format

   Template FlowSets in NetFlow V9 support a subset of functionality of
   those in IPFIX.  Specifically, NetFlow V9 does not have any support
   for vendor-specific Information Elements as IPFIX does, so there is
   no enterprise bit or facility for associating a private enterprise
   number with an information element.  NetFlow V9 also does not support
   variable-length fields.

   Options Template FlowSets in NetFlow V9 are similar to Options
   Template Sets in IPFIX subject to the same caveats.

B.1.4.  Information Model

   The NetFlow V9 field type definitions are a compatible subset of, and
   have evolved in concert with, the IPFIX Information Model.  IPFIX
   Information Element identifiers in the range 1-127 are defined by the
   IPFIX Information Model [RFC5102] to be compatible with the
   corresponding NetFlow V9 field types.

B.1.5.  Template Management

   NetFlow V9 has no concept of a Transport Session as in IPFIX, as
   NetFlow V9 was designed with a connectionless transport in mind.
   Template IDs are therefore scoped to an Exporting Process lifetime
   (i.e., an Exporting Process instance between restarts).  There is no
   facility in NetFlow V9 as in IPFIX for Template withdrawal or
   Template ID reuse.  Template retransmission at the Exporter works as
   in UDP-based IPFIX Exporting Processes.

B.1.6.  Transport

   In practice, though NetFlow V9 is designed to be transport-
   independent, it is transported only over UDP.  There is no facility
   as in IPFIX for full connection-oriented transport without datagram
   boundaries, due to the use of a record count field as opposed to a
   message length field in the packet header.  There is no support in
   NetFlow V9 for transport layer security via TLS or DTLS.

Top      Up      ToC       Page 60 
B.2.  A Method for Transforming NetFlow V9 Messages to IPFIX

   This appendix describes a method for transforming NetFlow V9 Packets
   into IPFIX Messages, which can be used to store NetFlow V9 data in
   IPFIX Files.  A process transforming NetFlow V9 Packets into IPFIX
   Messages must handle the fact that NetFlow V9 Packets and IPFIX
   Messages are framed differently, that sequence numbering works
   differently, and that the NetFlow V9 field type definitions are only
   compatible with the IPFIX Information Model below Information Element
   identifier 128.

   For each incoming NetFlow V9 packet, the transformation process must:

   1.  Verify that the Version field of the packet header is 9.

   2.  Verify that the Sequence Number field of the packet header is
       valid.

   3.  Scan the packet to:

       1.  Verify that it contains no Templates with field types outside
           the range 1-127;

       2.  Verify that it contains no FlowSets with Set IDs between 2
           and 255 inclusive;

       3.  Verify that it contains the number of records in FlowSets,
           Template FlowSets, and Options Template FlowSets declared in
           the Count field of the packet header; and

       4.  Count the number of records in Data FlowSets for calculating
           the IPFIX Sequence Number.

   4.  Calculate a Sequence Number for each IPFIX Observation Domain by
       storing the last Sequence Number sent for each Observation Domain
       plus the count of records in Data FlowSets in the previous step
       to be sent as the Sequence Number for the IPFIX Message following
       this one within that Observation Domain.

   5.  Generate a new IPFIX Message Header with:

       1.  a Version field of 10;

       2.  a Length field with the number of octets in the IPFIX
           Message, generally available by subtracting 4 from the length
           of the NetFlow V9 packet as returned from the transport layer
           (accounting for the difference in message header lengths);

Top      Up      ToC       Page 61 
       3.  the Sequence Number calculated for this message by the
           Sequence Number calculation step; and

       4.  Export Time and Observation Domain ID taken from the UNIX
           secs and Source ID fields of the NetFlow V9 packet header,
           respectively.

   6.  Copy each FlowSet from the Netflow V9 packet to the IPFIX Message
       after the header.  Replace Set ID 0 with Set ID 2 for Template
       Sets, and Set ID 1 with Set ID 3 for Options Template Sets.

   Note that this process loses system uptime information; if such
   information is required, the transformation process will have to
   export that information using IPFIX Options.  This may require a more
   sophisticated transformation process structure.

B.3.  NetFlow V9 Transformation Example

   The following two figures show a single NetFlow V9 packet with
   templates and the corresponding IPFIX Message, exporting a single
   flow record representing 60,303 octets sent from 192.0.2.2 to
   192.0.2.3.  This would be the third packet exported in Observation
   Domain 33 from the NetFlow V9 exporter, containing records starting
   with the 12th record (packet and record sequence numbers count from
   0).

   The ** symbol in the IPFIX example shows those fields that required
   modification from the NetFlow V9 packet by the transformation
   process.

Top      Up      ToC       Page 62 
                        1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |           Version = 9          |         Count = 2             |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |               Uptime = 3750405 ms (1:02:30.405)               |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |   Export Time = 1171557627 epoch sec (2007-02-15 16:40:27)    |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                     Sequence Number = 2                       |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                 Observation Domain ID = 33                    |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |           Set ID = 0          |       Set Length = 20         |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |       Template ID = 256       |       Field Count = 3         |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   | IPV4_SRC_ADDR           =   8 |       Field Length = 4        |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   | IPV4_DST_ADDR           =  12 |       Field Length = 4        |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   | IN_BYTES                =   1 |       Field Length = 4        |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |          Set ID = 256         |       Set Length = 16         |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                         IPV4_SRC_ADDR                         |
   |                           192.0.2.2                           |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                         IPV4_DST_ADDR                         |
   |                           192.0.2.3                           |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                           IN_BYTES                            |
   |                             60303                             |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

                   Figure 13: Example NetFlow V9 Packet

Top      Up      ToC       Page 63 
                       1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   | **       Version = 10         | **      Length = 52           |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |   Export Time = 1171557627 epoch sec (2007-02-15 16:40:27)    |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   | **                   Sequence Number = 11                     |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                   Observation Domain ID = 33                  |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   | **         Set ID = 2         |       Set Length = 20         |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |       Template ID = 256       |       Field Count  = 3        |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |0| sourceIPv4Address      =  8 |       Field Length = 4        |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |0| destinationIPv4Address = 12 |       Field Length = 4        |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |0| octetDeltaCount        =  1 |       Field Length = 4        |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |          Set ID = 256         |       Set Length = 16         |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                       sourceIPv4Address                       |
   |                           192.0.2.2                           |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                     destinationIPv4Address                    |
   |                           192.0.2.3                           |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                        octetDeltaCount                        |
   |                             60303                             |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

              Figure 14: Corresponding Example IPFIX Message

Top      Up      ToC       Page 64 
Authors' Addresses

   Brian Trammell
   Hitachi Europe
   c/o ETH Zurich
   Gloriastrasse 35
   8092 Zurich
   Switzerland
   Phone: +41 44 632 70 13
   EMail: brian.trammell@hitachi-eu.com

   Elisa Boschi
   Hitachi Europe
   c/o ETH Zurich
   Gloriastrasse 35
   8092 Zurich
   Switzerland
   Phone: +41 44 632 70 57
   EMail: elisa.boschi@hitachi-eu.com

   Lutz Mark
   Fraunhofer IFAM
   Wiener Str. 12
   28359 Bremen
   Germany
   Phone: +49 421 2246206
   EMail: lutz.mark@ifam.fraunhofer.de

   Tanja Zseby
   Fraunhofer Institute for Open Communication Systems
   Kaiserin-Augusta-Allee 31
   10589 Berlin
   Germany
   Phone: +49 30 3463 7153
   EMail: tanja.zseby@fokus.fraunhofer.de

   Arno Wagner
   ETH Zurich
   Gloriastrasse 35
   8092 Zurich
   Switzerland
   Phone: +41 44 632 70 04
   EMail: arno@wagner.name