tech-invite   World Map     

3GPP     Specs     Glossaries     Architecture     IMS     UICC       IETF     RFCs     Groups     SIP     ABNFs       Search

RFC 3821

Proposed STD
Pages: 74
Top     in Index     Prev     Next
in Group Index     Prev in Group     Next in Group     Group: IPS

Fibre Channel Over TCP/IP (FCIP)

Part 1 of 3, p. 1 to 21
None       Next RFC Part

Updated by:    7146


Top       ToC       Page 1 
Network Working Group                                       M. Rajagopal
Request for Comments: 3821                                  E. Rodriguez
Category: Standards Track                                       R. Weber
                                                              July 2004


                    Fibre Channel Over TCP/IP (FCIP)

Status of this Memo

   This document specifies an Internet standards track protocol for the
   Internet community, and requests discussion and suggestions for
   improvements.  Please refer to the current edition of the "Internet
   Official Protocol Standards" (STD 1) for the standardization state
   and status of this protocol.  Distribution of this memo is unlimited.

Copyright Notice

   Copyright (C) The Internet Society (2004).

Abstract

   Fibre Channel Over TCP/IP (FCIP) describes mechanisms that allow the
   interconnection of islands of Fibre Channel storage area networks
   over IP-based networks to form a unified storage area network in a
   single Fibre Channel fabric.  FCIP relies on IP-based network
   services to provide the connectivity between the storage area network
   islands over local area networks, metropolitan area networks, or wide
   area networks.

Table Of Contents

   1.  Purpose, Motivation, and Objectives. . . . . . . . . . . . . .  3
   2.  Relationship to Fibre Channel Standards. . . . . . . . . . . .  4
       2.1.  Relevant Fibre Channel Standards . . . . . . . . . . . .  4
       2.2.  This Specification and Fibre Channel Standards . . . . .  5
   3.  Terminology. . . . . . . . . . . . . . . . . . . . . . . . . .  5
   4.  Protocol Summary . . . . . . . . . . . . . . . . . . . . . . .  7
   5.  The FCIP Model . . . . . . . . . . . . . . . . . . . . . . . .  9
       5.1.  FCIP Protocol Model. . . . . . . . . . . . . . . . . . .  9
       5.2.  FCIP Link. . . . . . . . . . . . . . . . . . . . . . . . 10
       5.3.  FC Entity. . . . . . . . . . . . . . . . . . . . . . . . 11
       5.4.  FCIP Entity. . . . . . . . . . . . . . . . . . . . . . . 12
       5.5.  FCIP Link Endpoint (FCIP_LEP). . . . . . . . . . . . . . 13
       5.6.  FCIP Data Engine (FCIP_DE) . . . . . . . . . . . . . . . 14
             5.6.1.  FCIP Encapsulation of FC Frames. . . . . . . . . 16
             5.6.2.  FCIP Data Engine Error Detection and Recovery. . 19
   6.  Checking FC Frame Transit Times in the IP Network. . . . . . . 22

Top      ToC       Page 2 
   7.  The FCIP Special Frame (FSF) . . . . . . . . . . . . . . . . . 23
       7.1.  FCIP Special Frame Format. . . . . . . . . . . . . . . . 23
       7.2.  Overview of FSF Usage in Connection Establishment. . . . 26
   8.  TCP Connection Management. . . . . . . . . . . . . . . . . . . 28
       8.1.  TCP Connection Establishment . . . . . . . . . . . . . . 28
             8.1.1.  Connection Establishment Model . . . . . . . . . 28
             8.1.2.  Creating New TCP Connections . . . . . . . . . . 29
             8.1.3.  Processing Incoming TCP Connect Requests . . . . 32
             8.1.4.  Simultaneous Connection Establishment. . . . . . 36
       8.2.  Closing TCP Connections. . . . . . . . . . . . . . . . . 36
       8.3.  TCP Connection Parameters. . . . . . . . . . . . . . . . 36
             8.3.1.  TCP Selective Acknowledgement Option . . . . . . 36
             8.3.2.  TCP Window Scale Option. . . . . . . . . . . . . 36
             8.3.3.  Protection Against Sequence Number Wrap. . . . . 37
             8.3.4.  TCP_NODELAY Option . . . . . . . . . . . . . . . 37
       8.4.  TCP Connection Considerations. . . . . . . . . . . . . . 37
       8.5.  Flow Control Mapping between TCP and FC. . . . . . . . . 37
   9.  Security . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
       9.1.  Threat Models. . . . . . . . . . . . . . . . . . . . . . 38
       9.2.  FC Fabric and IP Network Deployment Models . . . . . . . 40
       9.3.  FCIP Security Components . . . . . . . . . . . . . . . . 40
             9.3.1.  IPsec ESP Authentication and Confidentiality . . 40
             9.3.2.  Key Management . . . . . . . . . . . . . . . . . 41
             9.3.3.  ESP Replay Protection and Rekeying Issues. . . . 43
       9.4.  Secure FCIP Link Operation . . . . . . . . . . . . . . . 44
             9.4.1.  FCIP Link Initialization Steps . . . . . . . . . 44
             9.4.2.  TCP Connection Security Associations (SAs) . . . 44
             9.4.3.  Handling Data Integrity and Confidentiality
                     Violations . . . . . . . . . . . . . . . . . . . 45
   10. Performance. . . . . . . . . . . . . . . . . . . . . . . . . . 45
       10.1. Performance Considerations . . . . . . . . . . . . . . . 45
       10.2. IP Quality of Service (QoS) Support. . . . . . . . . . . 46
   11. References . . . . . . . . . . . . . . . . . . . . . . . . . . 47
       11.1. Normative References . . . . . . . . . . . . . . . . . . 47
       11.2. Informative References . . . . . . . . . . . . . . . . . 49
   12. Acknowledgments. . . . . . . . . . . . . . . . . . . . . . . . 50
   Appendix A. Fibre Channel Bit and Byte Numbering Guidance  . . . . 51
   Appendix B. IANA Considerations  . . . . . . . . . . . . . . . . . 51
   Appendix C. FCIP Usage of Addresses and Identifiers  . . . . . . . 52
   Appendix D. Example of synchronization Recovery Algorithm  . . . . 53
   Appendix E. Relationship between FCIP and IP over FC (IPFC)  . . . 58
   Appendix F. FC Frame Format  . . . . . . . . . . . . . . . . . . . 59
   Appendix G. FC Encapsulation Format  . . . . . . . . . . . . . . . 61
   Appendix H. FCIP Requirements on an FC Entity  . . . . . . . . . . 63
   Editors and Contributors Acknowledgements. . . . . . . . . . . . . 69
   Editors and Contributors Addresses . . . . . . . . . . . . . . . . 70
   Full Copyright Statement . . . . . . . . . . . . . . . . . . . . . 74

Top      ToC       Page 3 
1.  Purpose, Motivation, and Objectives

   Warning to Readers Familiar With Fibre Channel: Both Fibre Channel
   and IETF standards use the same byte transmission order.   However,
   the bit and byte numbering is different.  See appendix A for
   guidance.

   Fibre Channel (FC) is a gigabit or multi-gigabit speed networking
   technology primarily used to implement Storage Area Networks (SANs).
   See section 2 for information about how Fibre Channel is standardized
   and the relationship of this specification to Fibre Channel
   standards.  An overview of Fibre Channel can be found in [34].

   This specification describes mechanisms that allow the
   interconnection of islands of Fibre Channel SANs over IP Networks to
   form a unified SAN in a single Fibre Channel fabric.  The motivation
   behind defining these interconnection mechanisms is a desire to
   connect physically remote FC sites allowing remote disk access, tape
   backup, and live mirroring.

   Fibre Channel standards have chosen nominal distances between switch
   elements that are less than the distances available in an IP Network.
   Since Fibre Channel and IP Networking technologies are compatible, it
   is logical to turn to IP Networking for extending the allowable
   distances between Fibre Channel switch elements.

   The fundamental assumption made in this specification is that the
   Fibre Channel traffic is carried over the IP Network in such a manner
   that the Fibre Channel Fabric and all Fibre Channel devices on the
   Fabric are unaware of the presence of the IP Network.  This means
   that the FC datagrams must be delivered in such time as to comply
   with existing Fibre Channel specifications.  The FC traffic may span
   LANs, MANs, and WANs, so long as this fundamental assumption is
   adhered to.

   The objectives of this document are to:

   1) specify the encapsulation and mapping of Fibre Channel (FC) frames
      employing FC Frame Encapsulation [19].

   2) apply the mechanism described in 1) to an FC Fabric using an IP
      network as an interconnect between two or more islands in an FC
      Fabric.

   3) address any FC concerns arising from tunneling FC traffic over an
      IP-based network, including security, data integrity (loss),
      congestion, and performance.  This will be accomplished by
      utilizing the existing IETF-specified suite of protocols.

Top      ToC       Page 4 
   4) be compatible with the referenced FC standards.  While new work
      may be undertaken in T11 to optimize and enhance FC Fabrics, this
      specification REQUIRES conformance only to the referenced FC
      standards.

   5) be compatible with all applicable IETF standards so that the IP
      Network used to extend an FC Fabric can be used concurrently for
      other reasonable purposes.

   The objectives of this document do not include using an IP Network as
   a replacement for the Fibre Channel Arbitrated Loop interconnect.  No
   definition is provided for encapsulating loop primitive signals for
   transmission over an IP Network.

Conventions used in this document

   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
   "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
   document are to be interpreted as described in BCP 14, RFC 2119 [1].

2.  Relationship to Fibre Channel Standards

2.1.  Relevant Fibre Channel Standards

   FC is standardized as a family of American National Standards
   developed by the T11 technical committee of INCITS (InterNational
   Committee for Information Technology Standards).  T11 has specified a
   number of documents describing FC protocols, operations, and
   services.  T11 documents of interest to readers of this specification
   include (but are not limited to):

      -  FC-BB   - Fibre Channel Backbone [2]
      -  FC-BB-2 - Fibre Channel Backbone -2 [3]
      -  FC-SW-2 - Fibre Channel Switch Fabric -2 [4]
      -  FC-FS   - Fibre Channel Framing and Signaling [5]

   FC-BB and FC-BB-2 describe the relationship between an FC Fabric and
   interconnect technologies not defined by Fibre Channel standards
   (e.g., ATM and SONET).  FC-BB-2 is the Fibre Channel document
   describing the relationships between FC and TCP/IP, including the FC
   use of FCIP.

   FC-SW-2 describes the switch components of an FC Fabric and FC-FS
   describes the FC Frame format and basic control features of Fibre
   Channel.

   Additional information regarding T11 activities is available on the
   committee's web site www.t11.org.

Top      ToC       Page 5 
2.2.  This Specification and Fibre Channel Standards

   When considering the challenge of transporting FC Frames over an IP
   Network, it is logical to divide the standardization effort between
   TCP/IP requirements and Fibre Channel requirements.  This
   specification covers the TCP/IP requirements for transporting FC
   Frames; the Fibre Channel documents described in section 2.1 cover
   the Fibre Channel requirements.

   This specification addresses only the requirements necessary to
   properly utilize an IP Network as a conduit for FC Frames.  The
   result is a specification for an FCIP Entity (see section 5.4).

   A product that tunnels an FC Fabric through an IP Network MUST
   combine the FCIP Entity with an FC Entity (see section 5.3) using an
   implementation specific interface.  The requirements placed on an FC
   Entity by this specification to achieve proper delivery of FC Frames
   are summarized in appendix H.  More information about FC Entities can
   be found in the Fibre Channel standards and an example of an FC
   Entity can be found in FC-BB-2 [3].

   No attempt is being made to define a specific API between an FCIP
   Entity and an FC Entity.  The approach is to specify required
   functional interactions between an FCIP Entity and an FC Entity (both
   of which are required to forward FC frames across an IP Network), but
   allow implementers to choose how these interactions will be realized.

3.  Terminology

   Terms used to describe FCIP concepts are defined in this section.

   FC End Node - An FC device that uses the connection services provided
      by the FC Fabric.

   FC Entity - The Fibre Channel specific functional component that
      combines with an FCIP Entity to form an interface between an FC
      Fabric and an IP Network (see section 5.3).

   FC Fabric - An entity that interconnects various Nx_Ports (see [5])
      attached to it, and is capable of routing FC Frames using only the
      destination ID information in an FC Frame header (see appendix F).

   FC Fabric Entity - A Fibre Channel specific element containing one
      or more Interconnect_Ports (see FC-SW-2 [4]) and one or more
      FC/FCIP Entity pairs.  See FC-BB-2 [3] for details about FC Fabric
      Entities.

Top      ToC       Page 6 
   FC Frame - The basic unit of Fibre Channel data transfer (see
      appendix F).

   FC Frame Receiver Portal - The access point through which an FC
      Frame and time stamp enter an FCIP Data Engine from the FC Entity.

   FC Frame Transmitter Portal - The access point through which a
      reconstituted FC Frame and time stamp leave an FCIP Data Engine to
      the FC Entity.

   FC/FCIP Entity pair - The combination of one FC Entity and one FCIP
      entity.

   FCIP Data Engine (FCIP_DE) - The component of an FCIP Entity that
      handles FC Frame encapsulation, de-encapsulation, and transmission
      FCIP Frames through a single TCP Connection (see section 5.6).

   FCIP Entity - The entity responsible for the FCIP protocol exchanges
      on the IP Network and encompasses FCIP_LEP(s) and FCIP Control and
      Services module (see section 5.4).

   FCIP Frame - An FC Frame plus the FC Frame Encapsulation [19]
      header, encoded SOF and encoded EOF that contains the FC Frame
      (see section 5.6.1).

   FCIP Link - One or more TCP Connections that connect one FCIP_LEP to
      another (see section 5.2).

   FCIP Link Endpoint (FCIP_LEP) - The component of an FCIP Entity
      that handles a single FCIP Link and contains one or more FCIP_DEs
      (see section 5.5).

   Encapsulated Frame Receiver Portal - The TCP access point through
      which an FCIP Frame is received from the IP Network by an FCIP
      Data Engine.

   Encapsulated Frame Transmitter Portal - The TCP access point through
      which an FCIP Frame is transmitted to the IP Network by an FCIP
      Data Engine.

   FCIP Special Frame (FSF) - A specially formatted FC Frame containing
      information used by the FCIP protocol (see section 7).

Top      ToC       Page 7 
4.  Protocol Summary

   The FCIP protocol is summarized as follows:

   1) The primary function of an FCIP Entity is forwarding FC Frames,
      employing FC Frame Encapsulation described in [19].

   2) Viewed from the IP Network perspective, FCIP Entities are peers
      and communicate using TCP/IP.  Each FCIP Entity contains one or
      more TCP endpoints in the IP-based network.

   3) Viewed from the FC Fabric perspective, pairs of FCIP Entities, in
      combination with their associated FC Entities, forward FC Frames
      between FC Fabric elements.  The FC End Nodes are unaware of the
      existence of the FCIP Link.

   4) FC Primitive Signals, Primitive Sequences, and Class 1 FC Frames
      are not transmitted across an FCIP Link because they cannot be
      encoded using FC Frame Encapsulation [19].

   5) The path (route) taken by an encapsulated FC Frame follows the
      normal routing procedures of the IP Network.

   6) An FCIP Entity MAY contain multiple FCIP Link Endpoints, but each
      FCIP Link Endpoint (FCIP_LEP) communicates with exactly one other
      FCIP_LEP.

   7) When multiple FCIP_LEPs with multiple FCIP_DEs are in use,
      selection of which FCIP_DE to use for encapsulating and
      transmitting a given FC Frame is covered in FC-BB-2 [3].  FCIP
      Entities do not actively participate in FC Frame routing.

   8) The FCIP Control and Services module MAY use TCP/IP quality of
      service features (see section 10.2).

   9) It is necessary to statically or dynamically configure each FCIP
      entity with the IP addresses and TCP port numbers corresponding to
      FCIP Entities with which it is expected to initiate communication.
      If dynamic discovery of participating FCIP Entities is supported,
      the function SHALL be performed using the Service Location
      Protocol (SLPv2) [17].  It is outside the scope of this
      specification to describe any static configuration method for
      participating FCIP Entity discovery.  Refer to section 8.1.2.2 for
      a detailed description of dynamic discovery of participating FCIP
      Entities using SLPv2.

Top      ToC       Page 8 
  10) Before creating a TCP Connection to a peer FCIP Entity, the FCIP
      Entity attempting to create the TCP connection SHALL statically or
      dynamically determine the IP address, TCP port, expected FC Fabric
      Entity World Wide Name, TCP Connection Parameters, and Quality of
      Service Information.

  11) FCIP Entities do not actively participate in the discovery of FC
      source and destination identifiers.  Discovery of FC addresses
      (accessible via the FCIP Entity) is provided by techniques and
      protocols within the FC architecture as described in FC-FS [5] and
      FC-SW-2 [4].

  12) To support IP Network security (see section 9), FCIP Entities
      MUST:
      1) implement cryptographically protected authentication and
         cryptographic data integrity keyed to the authentication
         process, and
      2) implement data confidentiality security features.

  13) On an individual TCP Connection, this specification relies on
      TCP/IP to deliver a byte stream in the same order that it was
      sent.

  14) This specification assumes the presence of and requires the use of
      TCP and FC data loss and corruption mechanisms.  The error
      detection and recovery features described in this specification
      complement and support these existing mechanisms.

Top      ToC       Page 9 
5.  The FCIP Model

5.1.  FCIP Protocol Model

   The relationship between FCIP and other protocols is illustrated in
   figure 1.

   +------------------------+ FCIP Link +------------------------+
   |          FCIP          |===========|          FCIP          |
   +--------+------+--------+           +--------+------+--------+
   |  FC-2  |      |  TCP   |           |  TCP   |      |  FC-2  |
   +--------+      +--------+           +--------+      +--------+
   |  FC-1  |      |   IP   |           |   IP   |      |  FC-1  |
   +--------+      +--------+           +--------+      +--------+
   |  FC-0  |      |  LINK  |           |  LINK  |      |  FC-0  |
   +--------+      +--------+           +--------+      +--------+
        |          |   PHY  |           |   PHY  |           |
        |          +--------+           +--------+           |
        |               |                    |               |
        |               |     IP Network     |               |
        V               +--------------------+               V
     to Fibre                                             to Fibre
     Channel                                              Channel
     Fabric                                               Fabric

   Key: FC-0 - Fibre Channel Physical Media Layer
        FC-1 - Fibre Channel Encode and Decode Layer
        FC-2 - Fibre Channel Framing and Flow Control Layer
        TCP  - Transmission Control Protocol
        IP   - Internet Protocol
        LINK - IP Link Layer
        PHY  - IP Physical Layer

   Figure 1:  FCIP Protocol Stack Model

   Note that the objective of the FCIP Protocol is to create and
   maintain one or more FCIP Links to transport data.

Top      ToC       Page 10 
5.2.  FCIP Link

   The FCIP Link is the basic unit of service provided by the FCIP
   Protocol to an FC Fabric.  As shown in figure 2, an FCIP Link
   connects two portions of an FC Fabric using an IP Network as a
   transport to form a single FC Fabric.

   /\/\/\/\/\/\         /\/\/\/\/\/\         /\/\/\/\/\/\
   \    FC    /         \    IP    /         \    FC    /
   /  Fabric  \=========/  Network \=========/  Fabric  \
   \/\/\/\/\/\/         \/\/\/\/\/\/         \/\/\/\/\/\/
              |                              |
              |<--------- FCIP Link -------->|

   Figure:  2  FCIP Link Model

   At the points where the ends of the FCIP Link meet portions of the FC
   Fabric, an FCIP Entity (see section 5.4) combines with an FC Entity
   as described in section 5.3 to serve as the interface between FC and
   IP.

   An FCIP Link SHALL contain at least one TCP Connection and MAY
   contain more than one TCP Connection.  The endpoints of a single TCP
   Connection are FCIP Data Engines (see section 5.6).  The endpoints of
   a single FCIP Link are FCIP Link Endpoints (see section 5.5).

Top      ToC       Page 11 
5.3.  FC Entity

   An implementation that tunnels an FC Fabric through an IP Network
   MUST combine an FC Entity with an FCIP Entity (see section 5.4) to
   form a complete interface between the FC Fabric and IP Network as
   shown in figure 3.  An FC Fabric Entity may contain multiple
   instances of the FC/FCIP Entity pair shown on either the right-hand
   or left-hand side of figure 3.

              |<--------- FCIP Link -------->|
              |                              |
   +----------+         /\/\/\/\/\/\         +----------+
   |   FCIP   |         \    IP    /         |   FCIP   |
   |  Entity  |=========/  Network \=========|  Entity  |
   +----------+         \/\/\/\/\/\/         +----------+
   |    FC    |                              |    FC    |
   |  Entity  |                              |  Entity  |
   +----------+                              +----------+
        |                                         |
   /\/\/\/\/\/\                              /\/\/\/\/\/\
   \    FC    /                              \    FC    /
   /  Fabric  \                              /  Fabric  \
   \/\/\/\/\/\/                              \/\/\/\/\/\/

   Figure 3:  Model for Two Connected FC/FCIP Entity Pairs

   In general, the combination of an FCIP Link and two FC/FCIP Entity
   pairs is intended to provide a non-Fibre Channel backbone transport
   between Fibre Channel components.  For example, this combination can
   be used to function as the hard-wire connection between two Fibre
   Channel switches.

   The interface between the FC and FCIP Entities is implementation
   specific.  The functional requirements placed on an FC Entity by this
   specification are listed in appendix H.  More information about FC
   Entities can be found in the Fibre Channel standards and an example
   of an FC Entity can be found in FC-BB-2 [3].

Top      ToC       Page 12 
5.4.  FCIP Entity

   The model for an FCIP Entity is shown in figure 4.

    .......................................................
    : FCIP Entity                                         :
    :                                                     :
    :  +-----------+                                      :
    :  |   FCIP    |                                      :
    :  |Control and|------------------------------------+ :
    :  | Services  |                                    | :
    :  |  Module   |                                    | :
    :  +-----------+                                    | :
    :        |            +--------------------+        | :
    :        |   +-------+--------------------+|----+   | :
    :        |   |+-----+--------------------+|----+|   | :
    :        |   ||+----| FCIP Link Endpoint |----+||   | :
    :        |   |||    +--------------------+    |||   | :
    :.............................................|||.....:
             |   |||                              |||   |
             |   |||                              |||   o<--+
             |   |||                unique TCP    |||   |   |
             |   |||                connections-->|||   |   |
             |   |||                              |||   |   |
          +----------+                         /\/\/\/\/\/\ |
          |    FC    |                         \    IP    / |
          |  Entity  |                         /  Network \ |
          +----------+                         \/\/\/\/\/\/ |
               |                                            |
          /\/\/\/\/\/\                   +------------------+
          \    FC    /                   +->TCP port for
          /  Fabric  \                      incoming
          \/\/\/\/\/\/                      connections

    Figure 4:  FCIP Entity Model

   The FCIP Entity receives TCP connect requests on behalf of the
   FCIP_LEPs that it manages.  In support of this, the FCIP Entity is
   the sole owner of at least one TCP port/IP Address combination used
   to form TCP Connections.  The TCP port may be the FCIP well known
   port at a given IP Address.  An FC Fabric to IP Network interface
   product SHALL provide each FC/FCIP Entity pair contained in the
   product with a unique combination of FC Fabric Entity World Wide
   Identifier and FC/FCIP Entity Identifier values (see section 7).

   An FCIP Entity contains an FCIP Control and Services Module to
   control FCIP link initialization, FCIP link dissolution, and to
   provide the FC Entity with an interface to key IP Network features.

Top      ToC       Page 13 
   The interfaces to the IP Network features are implementation
   specific, however, REQUIRED TCP/IP functional support is specified in
   this document, including:

   -  TCP Connections - see section 8
   -  Security - see section 9
   -  Performance - see section 10
   -  Dynamic Discovery - see section 8.1.2.2

   The FCIP Link Endpoints in an FCIP Entity provide the FC Frame
   encapsulation and transmission features of FCIP.

5.5.  FCIP Link Endpoint (FCIP_LEP)

   As shown in figure 5, the FCIP Link Endpoint contains one FCIP Data
   Engine for each TCP Connection in the FCIP Link.

    ................................................
    : FCIP Link Endpoint                           :
    :                   +------------------+       :
    :          +-------+------------------+|----+  :
    :          |+-----+------------------+|----+|  :
    :          ||+----| FCIP Data Engine |----+||  :
    :          |||    +------------------+    |||  :
    :..............................................:
               |||                            |||
          +----------+                    /\/\/\/\/\/\
          |    FC    |                    \    IP    /
          |  Entity  |                    /  Network \
          +----------+                    \/\/\/\/\/\/
                |
          /\/\/\/\/\/\
          \    FC    /
          /  Fabric  \
          \/\/\/\/\/\/

   Figure 5:  FCIP Link Endpoint Model

   Each time a TCP Connection is formed with a new FC/FCIP Entity pair
   (including all the actions described in section  8.1), the FCIP
   Entity SHALL create a new FCIP Link Endpoint containing one FCIP Data
   Engine.

   An FCIP_LEP is a transparent data translation point between an FC
   Entity and an IP Network.  A pair of FCIP_LEPs communicating over one
   or more TCP Connections create an FCIP Link to join two islands of an
   FC Fabric, producing a single FC Fabric.

Top      ToC       Page 14 
   The IP Network over which the two FCIP_LEPs communicate is not aware
   of the FC payloads that it is carrying.  Likewise, the FC End Nodes
   connected to the FC Fabric are unaware of the TCP/IP based transport
   employed in the structure of the FC Fabric.

   An FCIP_LEP uses normal TCP based flow control mechanisms for
   managing its internal resources and matching them with the advertised
   TCP Receiver Window Size (see sections 8.3.2, 8.5).  An FCIP_LEP MAY
   communicate with its local FC Entity counterpart to coordinate flow
   control.

5.6.  FCIP Data Engine (FCIP_DE)

   The model for one of the multiple FCIP_DEs that MAY be present in an
   FCIP_LEP is shown in figure 6.

        +--------------------------------+
        |                                |
   F    |-+    +------------------+    +-|
   C    |p|    |  Encapsulation   |    |p|    N
     -->|1|--->|     Engine       |--->|2|--> e
   E    |-+    +------------------+    +-|    t
   n    |                                |  I w
   t    |-+    +------------------+    +-|  P o
   i    |p|    | De-Encapsulation |    |p|    r
   t <--|4|<---|     Engine       |<---|3|<-- k
   y    |-+    +------------------+    +-|
        |                                |
        +--------------------------------+

   Figure 6:  FCIP Data Engine Model

   Data enters and leaves the FCIP_DE through four portals (p1 - p4).
   The portals do not process or examine the data that passes through
   them.  They are only the named access points where the FCIP_DE
   interfaces with the external world.  The names of the portals are as
   follows:

   p1) FC Frame Receiver Portal - The interface through which an FC
       Frame and time stamp enters an FCIP_DE from the FC Entity.

   p2) Encapsulated Frame Transmitter Portal - The TCP interface through
       which an FCIP Frame is transmitted to the IP Network by an
       FCIP_DE.

   p3) Encapsulated Frame Receiver Portal - The TCP interface through
       which an FCIP Frame is received from the IP Network by an
       FCIP_DE.

Top      ToC       Page 15 
   p4) FC Frame Transmitter Portal - The interface through which a
       reconstituted FC Frame and time stamp exits an FCIP_DE to the FC
       Entity.

   The work of the FCIP_DE is done by the Encapsulation and De-
   Encapsulation Engines.  The Engines have two functions:

   1) Encapsulating and de-encapsulating FC Frames using the
      encapsulation format described in FC Frame Encapsulation [19] and
      in section 5.6.1 of this document, and

   2) Detecting some data transmission errors and performing minimal
      error recovery as described in section 5.6.2.

   Data flows through a pair of IP Network connected FCIP_DEs in the
   following seven steps:

   1) An FC Frame and time stamp arrives at the FC Frame Receiver Portal
      and is passed to the Encapsulation Engine.  The FC Frame is
      assumed to have been processed by the FC Entity according to the
      applicable FC rules and is not validated by the FCIP_DE.  If the
      FC Entity is in the Unsynchronized state with respect to a time
      base as described in the FC Frame Encapsulation [19]
      specification, the time stamp delivered with the FC Frame SHALL be
      zero.

   2) In the Encapsulation Engine, the encapsulation format described in
      FC Frame Encapsulation [19] and in section 5.6.1 of this document
      SHALL be applied to prepare the FC Frame and associated time stamp
      for transmission over the IP Network.

   3) The entire encapsulated FC Frame (a.k.a. the FCIP Frame) SHALL be
      passed to the Encapsulated Frame Transmitter Portal where it SHALL
      be inserted in the TCP byte stream.

   4) Transmission of the FCIP Frame over the IP Network follows all the
      TCP rules of operation.  This includes, but is not limited to, the
      in-order delivery of bytes in the stream, as specified by TCP [6].

   5) The FCIP Frame arrives at the partner FCIP Entity where it enters
      the FCIP_DE through the Encapsulated Frame Receiver Portal and is
      passed to the De-Encapsulation Engine for processing.

   6) The De-Encapsulation Engine SHALL validate the incoming TCP byte
      stream as described in section 5.6.2.2 and SHALL de-encapsulate
      the FC Frame and associated time stamp according to the
      encapsulation format described in FC Frame Encapsulation [19] and
      in section 5.6.1 of this document.

Top      ToC       Page 16 
   7) In the absence of errors, the de-encapsulated FC Frame and time
      stamp SHALL be passed to the FC Frame Transmitter Portal for
      delivery to the FC Entity.  Error handling is discussed in section
      5.6.2.2.

   Every FC Frame that arrives at the FC Frame Receiver Portal SHALL be
   transmitted on the IP Network as described in steps 1 through 4
   above.  In the absence of errors, data bytes arriving at the
   Encapsulated Frame Receiver Portal SHALL be de-encapsulated and
   forwarded to the FC Frame Transmitter Portal as described in steps 5
   through 7.

5.6.1.  FCIP Encapsulation of FC Frames

   The FCIP encapsulation of FC Frames employs FC Frame Encapsulation
   [19].

   The features from FC Frame Encapsulation that are unique to
   individual protocols SHALL be applied as follows for the FCIP
   encapsulation of FC Frames.

   The Protocol# field SHALL contain 1 in accordance with the IANA
   Considerations annex of FC Frame Encapsulation [19].

   The Protocol Specific field SHALL have the format shown in figure 7.
   Note: the word numbers in figure 7 are relative to the complete FC
   Frame Encapsulation header, not to the Protocol Specific field.

   W|------------------------------Bit------------------------------|
   o|                                                               |
   r|                    1 1 1 1 1 1 1 1 1 1 2 2 2 2 2 2 2 2 2 2 3 3|
   d|0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1|
    +---------------------------------------------------------------+
   1|               replication of encapsulation word 0             |
    +---------------+---------------+---------------+---------------+
   2|    pFlags     |    Reserved   |    -pFlags    |  -Reserved    |
    +---------------+---------------+---------------+---------------+

   Figure 7:  FCIP Usage of FC Frame Encapsulation Protocol Specific
   field

   Word 1 of the Protocol Specific field SHALL contain an exact copy of
   word 0 in FC Frame Encapsulation [19].

   The pFlags (protocol specific flags) field provides information about
   the protocol specific usage of the FC Encapsulation Header.  Figure 8
   shows the defined pFlags bits.

Top      ToC       Page 17 
   |----------------Bit--------------------|
   |                                       |
   |  0    1    2    3    4    5    6    7 |
   +----+-----------------------------+----+
   | Ch |          Reserved           | SF |
   +----+-----------------------------+----+

   Figure 8:  pFlags Field Bits

   The SF (Special Frame) bit indicates whether the FCIP Frame is an
   encapsulated FC Frame or an FSF (FCIP Special Frame, see section 7).
   When the FCIP Frame contains an encapsulated FC Frame, the SF bit
   SHALL be 0.  When the FCIP Frame is an FSF, the SF bit SHALL be 1.

   The FSF SHALL only be sent as the first bytes transmitted in each
   direction on a newly formed TCP Connection and only one FSF SHALL be
   transmitted in each direction at that time (see section 8.1).  After
   that all FCIP Frames SHALL have the SF bit set to 0.

   The Ch (Changed) bit indicates whether an echoed FSF has been
   intentionally altered (see section 8.1.3).  The Ch bit SHALL be 0
   unless the FSF bit is 1.  When the initial TCP Connection FSF is
   sent, the Ch bit SHALL be 0.  If the recipient of a TCP connect
   request echoes the FSF without any changes, then the Ch bit SHALL
   continue to be 0.  If the recipient of a TCP connect request alters
   the FSF before echoing it, then the Ch bit SHALL be changed to 1.

   The -pFlags field SHALL contain the ones complement of the contents
   of the pFlags field.

Top      ToC       Page 18 
   Table 1 summarizes the usage of the pFlags SF and Ch bits.

   +----+----+------------+--------------------------------------+
   |    |    | Originated |                                      |
   | SF | Ch | or Echoed  | Validity/Description                 |
   +----+----+------------+--------------------------------------+
   |  0 |  0 |    n/a     | Encapsulated FC Frame                |
   +----+----+------------+--------------------------------------+
   |  0 |  1 |    n/a     | Always Illegal                       |
   +----+----+------------+--------------------------------------+
   |  1 |  0 | Originated | Originated FSF                       |
   +----+----+------------+--------------------------------------+
   |  1 |  1 | Originated | Always Illegal                       |
   +----+----+------------+--------------------------------------+
   |  1 |  0 |   Echoed   | Echoed FSF without changes           |
   +----+----+------------+--------------------------------------+
   |  1 |  1 |   Echoed   | Echoed FSF with changes              |
   +----+----+------------+--------------------------------------+
   | Note 1: Echoed FSFs may contain changes resulting from      |
   | transmission errors, necessitating the comparison between   |
   | sent and received FSF bytes by the FSF originator described |
   | in section 8.1.2.3.                                         |
   |                                                             |
   | Note 2: Column positions in this table do not reflect the   |
   | bit positions of the SF and Ch bits in the pFlags field.    |
   +-------------------------------------------------------------+

   Table 1:  pFlags SF and Ch bit usage summary

   The Reserved pFlags bits SHALL be 0.

   The Reserved field (bits 23-16 in word 2): SHALL contain 0.

   The -Reserved field (bits 7-0 in word 2): SHALL contain 255 (or
   0xFF).

   The CRCV (CRC Valid) Flag SHALL be set to 0.

   The CRC field SHALL be set to 0.

   In FCIP, the SOF and EOF codes listed as Class 2, Class 3, and Class
   4 in the FC Frame Encapsulation [19] are legal.

Top      ToC       Page 19 
5.6.2.  FCIP Data Engine Error Detection and Recovery

5.6.2.1.  TCP Assistance With Error Detection and Recovery

   TCP [6] requires in order delivery, generation of TCP checksums, and
   checking of TCP checksums.  Thus, the byte stream passed from TCP to
   the FCIP_LEP will be in order and free of errors detectable by the
   TCP checksum.  The FCIP_LEP relies on TCP to perform these functions.

5.6.2.2.  Errors in FCIP Headers and Discarding FCIP Frames

   Bytes delivered through the Encapsulated Frame Receiver Portal that
   are not correctly delimited as defined by the FC Frame Encapsulation
   [19] are considered to be in error.

   The failure of the Protocol# and Version fields in the FCIP Frame
   header to contain the values defined for an FCIP Frame SHALL be
   considered an error.

   Further, some errors in the encapsulation will result in the FCIP_DE
   losing synchronization with the FC Frames in the byte stream entering
   through the Encapsulated Frame Receiver Portal.

   The Frame Length field in the FC Frame Encapsulation header is used
   to determine where in the data stream the next FC Encapsulated Header
   is located.  The following tests SHALL be performed to verify
   synchronization with the byte stream entering the Encapsulated Frame
   Receiver Portal, and synchronization SHALL be considered lost if any
   of the tests fail:

   1) Frame Length field validation -- 15 < Frame Length < 545;

   2) Comparison of Frame Length field to its ones complement; and

   3) A valid EOF is found in the word preceding the start of the next
      FCIP header as indicated by the Frame Length field, to be tested
      as follows:

      1) Bits 24-31 and 16-23 contain identical legal EOF values (the
         list of legal EOF values is in the FC Frame Encapsulation
         [19]); and

      2) Bits 8-15 and 0-7 contain the ones complement of the EOF value
         found in bits 24-31.

   Note: The range of valid Frame Length values is derived as follows.
   The FCIP Frame header is seven words, one word each is required for
   the encoded SOF and EOF values, the FC Frame header is six words, and

Top      ToC       Page 20 
   the FC CRC requires one word, yielding a base Frame Length of 16
   (7+1+1+6+1) words, if no FC Payload is present.  Since the FC Payload
   is optional, any Frame Length value greater than 15 is valid.  The
   maximum FC Payload size is 528 words, meaning that any Frame Length
   value up to and including 544 (528+16) is valid.

   If synchronization is lost, the FC Frame SHALL NOT be forwarded on to
   the FC Entity and further recovery SHALL be handled as defined by
   section 5.6.2.3.

   In addition to the tests above, the validity and positioning of the
   following FCIP Frame information SHOULD be used to detect
   encapsulation errors that may or may not affect synchronization:

      a)  Protocol# ones complement field (1 test);
      b)  Version ones complement field (1 test);
      c)  Replication of encapsulation word 0 in word 1 (1 test);
      d)  Reserved field and its ones complement (2 tests);
      e)  Flags field and its ones complement (2 tests);
      f)  CRC field is equal to zero (1 test);
      g)  SOF fields and ones complement fields (4 tests);
      h)  Format and values of FC header (1 test);
      i)  CRC of FC Frame (2 tests);
      j)  FC Frame Encapsulation header information in the next FCIP
          Frame (1 test).

   At least 3 of the 16 tests listed above SHALL be performed.  Failure
   of any of the above tests actually performed SHALL indicate an
   encapsulation error and the FC Frame SHALL NOT be forwarded on to the
   FC Entity.  Further, such errors SHOULD be considered carefully,
   since some may be synchronization errors.

   Whenever an FCIP_DE discards bytes delivered through the Encapsulated
   Frame Receiver Portal, it SHALL cause the FCIP Entity to notify the
   FC Entity of the condition and provide a suitable description of the
   reason bytes were discarded.

   The burden for recovering from discarded data falls on the FC Entity
   and other components of the FC Fabric, and is outside the scope of
   this specification.

Top      ToC       Page 21 
5.6.2.3.  Synchronization Failures

   If an FCIP_DE determines that it cannot find the next FCIP Frame
   header in the byte stream entering through the Encapsulated Frame
   Receiver Portal, the FCIP_DE SHALL do one of the following:

   a) close the TCP Connection [6] [7] and notify the FC Entity with the
      reason for the closure;

   b) recover synchronization by searching the bytes delivered by the
      Encapsulated Frame Receiver Portal for a valid FCIP Frame header
      having the correct properties (see section 5.6.2.2), and
      discarding bytes delivered by the Encapsulated Frame Receiver
      Portal until a valid FCIP Frame header is found; or

   c) attempt to recover synchronization as described in b) and if
      synchronization cannot be recovered, close the TCP Connection as
      described in a), including notification of the FC Entity with the
      reason for the closure.

   If the FCIP_DE attempts to recover synchronization, the
   resynchronization algorithm used SHALL meet the following
   requirements:

   a) discard or identify with an EOFa (see appendix section F.1) those
      FC Frames and fragments of FC Frames identified before
      synchronization has again been completely verified.  The number of
      FC Frames not forwarded may vary based on the algorithm used;

   b) return to forwarding FC Frames through the FC Frame Transmitter
      Portal only after synchronization on the transmitted FCIP Frame
      stream has been verified; and

   c) close the TCP/IP connection if the algorithm ends without
      verifying successful synchronization.  The probability of failing
      to synchronize successfully and the time necessary to determine
      whether or not synchronization was successful may vary with the
      algorithm used.

   An example algorithm meeting these requirements can be found in
   appendix D.

   The burden for recovering from the discarding of FCIP Frames during
   the optional resynchronization process described in this section
   falls on the FC Entity and other components of the FC Fabric, and is
   outside the scope of this specification.


Next RFC Part