tech-invite   World Map     

IETF     RFCs     Groups     SIP     ABNFs    |    3GPP     Specs     Glossaries     Architecture     IMS     UICC    |    search

RFC 3720

 
 
 

Internet Small Computer Systems Interface (iSCSI)

Part 4 of 9, p. 67 to 99
Prev RFC Part       Next RFC Part

 


prevText      Top      Up      ToC       Page 67 
6.  iSCSI Error Handling and Recovery

6.1.  Overview

6.1.1.  Background

   The following two considerations prompted the design of much of the
   error recovery functionality in iSCSI:

      i)  An iSCSI PDU may fail the digest check and be dropped, despite
          being received by the TCP layer.  The iSCSI layer must
          optionally be allowed to recover such dropped PDUs.
      ii) A TCP connection may fail at any time during the data
          transfer.  All the active tasks must optionally be allowed to
          continue on a different TCP connection within the same
          session.

   Implementations have considerable flexibility in deciding what degree
   of error recovery to support, when to use it and by which mechanisms
   to achieve the required behavior.  Only the externally visible
   actions of the error recovery mechanisms must be standardized to
   ensure interoperability.

   This chapter describes a general model for recovery in support of
   interoperability.  See Appendix E.  - Algorithmic Presentation of
   Error Recovery Classes - for further detail on how the described
   model may be implemented.  Compliant implementations do not have to
   match the implementation details of this model as presented, but the
   external behavior of such implementations must correspond to the
   externally observable characteristics of the presented model.

6.1.2.  Goals

   The major design goals of the iSCSI error recovery scheme are as
   follows:

      a)  Allow iSCSI implementations to meet different requirements by
          defining a collection of error recovery mechanisms that
          implementations may choose from.
      b)  Ensure interoperability between any two implementations
          supporting different sets of error recovery capabilities.
      c)  Define the error recovery mechanisms to ensure command
          ordering even in the face of errors, for initiators that
          demand ordering.

Top      Up      ToC       Page 68 
      d)  Do not make additions in the fast path, but allow moderate
          complexity in the error recovery path.
      e)  Prevent both the initiator and target from attempting to
          recover the same set of PDUs at the same time.  For example,
          there must be a clear "error recovery functionality
          distribution" between the initiator and target.

6.1.3.  Protocol Features and State Expectations

   The initiator mechanisms defined in connection with error recovery
   are:

      a)  NOP-OUT to probe sequence numbers of the target (section
          10.18)
      b)  Command retry (section 6.2.1)
      c)  Recovery R2T support (section 6.7)
      d)  Requesting retransmission of status/data/R2T using the SNACK
          facility (section 10.16)
      e)  Acknowledging the receipt of the data (section 10.16)
      f)  Reassigning the connection allegiance of a task to a different
          TCP connection (section 6.2.2)
      g)  Terminating the entire iSCSI session to start afresh (section
          6.1.4.4)

   The target mechanisms defined in connection with error recovery are:

      a)  NOP-IN to probe sequence numbers of the initiator (section
          10.19)
      b)  Requesting retransmission of data using the recovery R2T
          feature (section 6.7)
      c)  SNACK support (section 10.16) d)  Requesting that parts of
          read data be acknowledged (section 10.7.2)
      e)  Allegiance reassignment support (section 6.2.2)
      f)  Terminating the entire iSCSI session to force the initiator to
          start over (section 6.1.4.4)

   For any outstanding SCSI command, it is assumed that iSCSI, in
   conjunction with SCSI at the initiator, is able to keep enough
   information to be able to rebuild the command PDU, and that outgoing
   data is available (in host memory) for retransmission while the
   command is outstanding.  It is also assumed that at the target,
   incoming data (read data) MAY be kept for recovery or it can be
   reread from a device server.

   It is further assumed that a target will keep the "status & sense"
   for a command it has executed if it supports status retransmission.
   A target that agrees to support data retransmission is expected to be
   prepared to retransmit the outgoing data (i.e., Data-In) on request

Top      Up      ToC       Page 69 
   until either the status for the completed command is acknowledged, or
   the data in question has been separately acknowledged.

6.1.4.  Recovery Classes

   iSCSI enables the following classes of recovery (in the order of
   increasing scope of affected iSCSI tasks):

      - Within a command (i.e., without requiring command restart).
      - Within a connection (i.e., without requiring the connection to
        be rebuilt, but perhaps requiring command restart).
      - Connection recovery (i.e., perhaps requiring connections to be
        rebuilt and commands to be reissued).
      - Session recovery.

   The recovery scenarios detailed in the rest of this section are
   representative rather than exclusive.  In every case, they detail the
   lowest class recovery that MAY be attempted.  The implementer is left
   to decide under which circumstances to escalate to the next recovery
   class and/or what recovery classes to implement.  Both the iSCSI
   target and initiator MAY escalate the error handling to an error
   recovery class, which impacts a larger number of iSCSI tasks in any
   of the cases identified in the following discussion.

   In all classes, the implementer has the choice of deferring errors to
   the SCSI initiator (with an appropriate response code), in which case
   the task, if any, has to be removed from the target and all the side
   effects, such as ACA, must be considered.

   Use of within-connection and within-command recovery classes MUST NOT
   be attempted before the connection is in Full Feature Phase.

   In the detailed description of the recovery classes, the mandating
   terms (MUST, SHOULD, MAY, etc.) indicate normative actions to be
   executed if the recovery class is supported and used.

6.1.4.1.  Recovery Within-command

   At the target, the following cases lend themselves to
   within-command recovery:

    -  Lost data PDU - realized through one of the following:

       a)  Data digest error - dealt with as specified in Section 6.7
           Digest Errors, using the option of a recovery R2T.

Top      Up      ToC       Page 70 
       b)  Sequence reception timeout (no data or
           partial-data-and-no-F-bit) - considered an implicit sequence
           error and dealt with as specified in Section 6.8 Sequence
           Errors, using the option of a recovery R2T.
       c)  Header digest error, which manifests as a sequence reception
           timeout or a sequence error - dealt with as specified in
           Section 6.8 Sequence Errors, using the option of a recovery
           R2T.

   At the initiator, the following cases lend themselves to
   within-command recovery:

       Lost data PDU or lost R2T - realized through one of the
       following:

       a)  Data digest error - dealt with as specified in Section 6.7
           Digest Errors, using the option of a SNACK.
       b)  Sequence reception timeout (no status) or response reception
           timeout - dealt with as specified in Section 6.8 Sequence
           Errors, using the option of a SNACK.
       c)  Header digest error, which manifests as a sequence reception
           timeout or a sequence error - dealt with as specified in
           Section 6.8 Sequence Errors, using the option of a SNACK.

   To avoid a race with the target, which may already have a recovery
   R2T or a termination response on its way, an initiator SHOULD NOT
   originate a SNACK for an R2T based on its internal timeouts (if any).
   Recovery in this case is better left to the target.

   The timeout values used by the initiator and target are outside the
   scope of this document.  Sequence reception timeout is generally a
   large enough value to allow the data sequence transfer to be
   complete.

6.1.4.2.  Recovery Within-connection

   At the initiator, the following cases lend themselves to
   within-connection recovery:

    -  Requests not acknowledged for a long time.  Requests are
       acknowledged explicitly through ExpCmdSN or implicitly by
       receiving data and/or status.  The initiator MAY retry
       non-acknowledged commands as specified in Section 6.2 Retry and
       Reassign in Recovery.

Top      Up      ToC       Page 71 
    -  Lost iSCSI numbered Response.  It is recognized by either
       identifying a data digest error on a Response PDU or a Data-In
       PDU carrying the status, or by receiving a Response PDU with a
       higher StatSN than expected.  In the first case, digest error
       handling is done as specified in Section 6.7 Digest Errors using
       the option of a SNACK.  In the second case, sequence error
       handling is done as specified in Section 6.8 Sequence Errors,
       using the option of a SNACK.

   At the target, the following cases lend themselves to
   within-connection recovery:

    -  Status/Response not acknowledged for a long time.  The target MAY
       issue a NOP-IN (with a valid Target Transfer Tag or otherwise)
       that carries the next status sequence number it is going to use
       in the StatSN field.  This helps the initiator detect any missing
       StatSN(s) and issue a SNACK for the status.

   The timeout values used by the initiator and the target are outside
   the scope of this document.

6.1.4.3.  Connection Recovery

   At an iSCSI initiator, the following cases lend themselves to
   connection recovery:

    - TCP connection failure: The initiator MUST close the connection.
      It then MUST either implicitly or explicitly logout the failed
      connection with the reason code "remove the connection for
      recovery" and reassign connection allegiance for all commands
      still in progress associated with the failed connection on one or
      more connections (some or all of which MAY be newly established
      connections) using the "Task reassign" task management function
      (see Section 10.5.1 Function). For an initiator, a command is in
      progress as long as it has not received a response or a Data-In
      PDU including status.

      Note: The logout function is mandatory. However, a new connection
      establishment is only mandatory if the failed connection was the
      last or only connection in the session.

    - Receiving an Asynchronous Message that indicates one or all
      connections in a session has been dropped.  The initiator MUST
      handle it as a TCP connection failure for the connection(s)
      referred to in the Message.

Top      Up      ToC       Page 72 
   At an iSCSI target, the following cases lend themselves to connection
   recovery:

    - TCP connection failure. The target MUST close the connection and,
      if more than one connection is available, the target SHOULD send
      an Asynchronous Message that indicates it has dropped the
      connection. Then, the target will wait for the initiator to
      continue recovery.

6.1.4.4.  Session Recovery

   Session recovery should be performed when all other recovery attempts
   have failed.  Very simple initiators and targets MAY perform session
   recovery on all iSCSI errors and rely on recovery on the SCSI layer
   and above.

   Session recovery implies the closing of all TCP connections,
   internally aborting all executing and queued tasks for the given
   initiator at the target, terminating all outstanding SCSI commands
   with an appropriate SCSI service response at the initiator, and
   restarting a session on a new set of connection(s) (TCP connection
   establishment and login on all new connections).

   For possible clearing effects of session recovery on SCSI and iSCSI
   objects, refer to Appendix F. - Clearing Effects of Various Events on
   Targets -.

6.1.5.  Error Recovery Hierarchy

   The error recovery classes described so far are organized into a
   hierarchy for ease in understanding and to limit the implementation
   complexity. With few and well defined recovery levels
   interoperability is easier to achieve.  The attributes of this
   hierarchy are as follows:

      a)  Each level is a superset of the capabilities of the previous
          level. For example, Level 1 support implies supporting all
          capabilities of Level 0 and more.
      b)  As a corollary, supporting a higher error recovery level means
          increased sophistication and possibly an increase in resource
          requirements.
      c)  Supporting error recovery level "n" is advertised and
          negotiated by each iSCSI entity by exchanging the text key
          "ErrorRecoveryLevel=n".  The lower of the two exchanged values
          is the operational ErrorRecoveryLevel for the session.

Top      Up      ToC       Page 73 
   The following diagram represents the error recovery hierarchy.

                         +
                        /
                       / 2 \       <-- Connection recovery
                      +-----+
                     /   1   \     <-- Digest failure recovery
                    +---------+
                   /     0     \   <-- Session failure recovery
                  +-------------+

   The following table lists the error recovery capabilities expected
   from the implementations that support each error recovery level.

   +-------------------+--------------------------------------------+
   |ErrorRecoveryLevel |  Associated Error recovery capabilities    |
   +-------------------+--------------------------------------------+
   |        0          |  Session recovery class                    |
   |                   |  (Section 6.1.4.4 Session Recovery)        |
   +-------------------+--------------------------------------------+
   |        1          |  Digest failure recovery (See Note below.) |
   |                   |  plus the capabilities of ER Level 0       |
   +-------------------+--------------------------------------------+
   |        2          |  Connection recovery class                 |
   |                   |  (Section 6.1.4.3 Connection Recovery)     |
   |                   |  plus the capabilities of ER Level 1       |
   +-------------------+--------------------------------------------+

   Note: Digest failure recovery is comprised of two recovery classes:
   Within-Connection recovery class (Section 6.1.4.2 Recovery Within-
   connection) and Within-Command recovery class (Section 6.1.4.1
   Recovery Within-command).

   When a defined value of ErrorRecoveryLevel is proposed by an
   originator in a text negotiation, the originator MUST support the
   functionality defined for the proposed value and additionally, the
   functionality corresponding to any defined value numerically less
   than the proposed.  When a defined value of ErrorRecoveryLevel is
   returned by a responder in a text negotiation, the responder MUST
   support the functionality corresponding to the ErrorRecoveryLevel it
   is accepting.

   When either party attempts to use error recovery functionality beyond
   what is negotiated, the recovery attempts MAY fail unless an a priori
   agreement outside the scope of this document exists between the two
   parties to provide such support.

Top      Up      ToC       Page 74 
   Implementations MUST support error recovery level "0", while the rest
   are OPTIONAL to implement.  In implementation terms, the above
   striation means that the following incremental sophistication with
   each level is required.

   +-------------------+---------------------------------------------+
   |Level transition   |  Incremental requirement                    |
   +-------------------+---------------------------------------------+
   |        0->1       |  PDU retransmissions on the same connection |
   +-------------------+---------------------------------------------+
   |        1->2       |  Retransmission across connections and      |
   |                   |  allegiance reassignment                    |
   +-------------------+---------------------------------------------+

6.2.  Retry and Reassign in Recovery

   This section summarizes two important and somewhat related iSCSI
   protocol features used in error recovery.

6.2.1.  Usage of Retry

   By resending the same iSCSI command PDU ("retry") in the absence of a
   command acknowledgement (by way of an ExpCmdSN update) or a response,
   an initiator attempts to "plug" (what it thinks are) the
   discontinuities in CmdSN ordering on the target end.  Discarded
   command PDUs, due to digest errors, may have created these
   discontinuities.

   Retry MUST NOT be used for reasons other than plugging command
   sequence gaps, and in particular, cannot be used for requesting PDU
   retransmissions from a target.  Any such PDU retransmission requests
   for a currently allegiant command in progress may be made using the
   SNACK mechanism described in section 10.16, although the usage of
   SNACK is OPTIONAL.

   If initiators, as part of plugging command sequence gaps as described
   above, inadvertently issue retries for allegiant commands already in
   progress (i.e., targets did not see the discontinuities in CmdSN
   ordering), the duplicate commands are silently ignored by targets as
   specified in section 3.2.2.1.

   When an iSCSI command is retried, the command PDU MUST carry the
   original Initiator Task Tag and the original operational attributes
   (e.g., flags, function names, LUN, CDB etc.) as well as the original
   CmdSN.  The command being retried MUST be sent on the same connection
   as the original command unless the original connection was already
   successfully logged out.

Top      Up      ToC       Page 75 
6.2.2.  Allegiance Reassignment

   By issuing a "task reassign" task management request (Section 10.5.1
   Function), the initiator signals its intent to continue an already
   active command (but with no current connection allegiance) as part of
   connection recovery.  This means that a new connection allegiance is
   requested for the command, which seeks to associate it to the
   connection on which the task management request is being issued.
   Before the allegiance reassignment is attempted for a task, an
   implicit or explicit Logout with the reason code "remove the
   connection for recovery" ( see section 10.14) MUST be successfully
   completed for the previous connection to which the task was
   allegiant.

   In reassigning connection allegiance for a command, the targets
   SHOULD continue the command from its current state.  For example,
   when reassigning read commands, the target SHOULD take advantage of
   the ExpDataSN field provided by the Task Management function request
   (which must be set to zero if there was no data transfer) and bring
   the read command to completion by sending the remaining data and
   sending (or resending) the status.  ExpDataSN acknowledges all data
   sent up to, but not including, the Data-In PDU and or R2T with DataSN
   (or R2TSN) equal to ExpDataSN.  However, targets may choose to
   send/receive all unacknowledged data or all of the data on a
   reassignment of connection allegiance if unable to recover or
   maintain an accurate state.  Initiators MUST not subsequently request
   data retransmission through Data SNACK for PDUs numbered less than
   ExpDataSN (i.e., prior to the acknowledged sequence number).  For all
   types of commands, a reassignment request implies that the task is
   still considered in progress by the initiator and the target must
   conclude the task appropriately if the target returns the "Function
   Complete" response to the reassignment request.  This might possibly
   involve retransmission of data/R2T/status PDUs as necessary, but MUST
   involve the (re)transmission of the status PDU.

   It is OPTIONAL for targets to support the allegiance reassignment.
   This capability is negotiated via the ErrorRecoveryLevel text key
   during the login time.  When a target does not support allegiance
   reassignment, it MUST respond with a Task Management response code of
   "Allegiance reassignment not supported".  If allegiance reassignment
   is supported by the target, but the task is still allegiant to a
   different connection, or a successful recovery Logout of the
   previously allegiant connection was not performed, the target MUST
   respond with a Task Management response code of "Task still
   allegiant".

Top      Up      ToC       Page 76 
   If allegiance reassignment is supported by the target, the Task
   Management response to the reassignment request MUST be issued before
   the reassignment becomes effective.

   If a SCSI Command that involves data input is reassigned, any SNACK
   Tag it holds for a final response from the original connection is
   deleted and the default value of 0 MUST be used instead.

6.3.  Usage Of Reject PDU in Recovery

   Targets MUST NOT implicitly terminate an active task by sending a
   Reject PDU for any PDU exchanged during the life of the task.  If the
   target decides to terminate the task, a Response PDU (SCSI, Text,
   Task, etc.) must be returned by the target to conclude the task.  If
   the task had never been active before the Reject (i.e., the Reject is
   on the command PDU), targets should not send any further responses
   because the command itself is being discarded.

   The above rule means that the initiator can eventually expect a
   response on receiving Rejects, if the received Reject is for a PDU
   other than the command PDU itself.  The non-command Rejects only have
   diagnostic value in logging the errors, and they can be used for
   retransmission decisions by the initiators.

   The CmdSN of the rejected command PDU (if it is a non-immediate
   command) MUST NOT be considered received by the target (i.e., a
   command sequence gap must be assumed for the CmdSN), even though the
   CmdSN of the rejected command PDU may be reliably ascertained.  Upon
   receiving the Reject, the initiator MUST plug the CmdSN gap in order
   to continue to use the session.  The gap may be plugged either by
   transmitting a command PDU with the same CmdSN, or by aborting the
   task (see section 6.9 on how an abort may plug a CmdSN gap).

   When a data PDU is rejected and its DataSN can be ascertained, a
   target MUST advance ExpDataSN for the current data burst if a
   recovery R2T is being generated.  The target MAY advance its
   ExpDataSN if it does not attempt to recover the lost data PDU.

6.4.  Connection Timeout Management

   iSCSI defines two session-global timeout values (in seconds)
   - Time2Wait and Time2Retain - that are applicable when an iSCSI Full
   Feature Phase connection is taken out of service either intentionally
   or by an exception.  Time2Wait is the initial "respite time" before
   attempting an explicit/implicit Logout for the CID in question or
   task reassignment for the affected tasks (if any).  Time2Retain is
   the maximum time after the initial respite interval that the task
   and/or connection state(s) is/are guaranteed to be maintained on the

Top      Up      ToC       Page 77 
   target to cater to a possible recovery attempt.  Recovery attempts
   for the connection and/or task(s) SHOULD NOT be made before Time2Wait
   seconds, but MUST be completed within Time2Retain seconds after that
   initial Time2Wait waiting period.

6.4.1.  Timeouts on Transport Exception Events

   A transport connection shutdown or a transport reset without any
   preceding iSCSI protocol interactions informing the end-points of the
   fact causes a Full Feature Phase iSCSI connection to be abruptly
   terminated.  The timeout values to be used in this case are the
   negotiated values of defaultTime2Wait (Section 12.15
   DefaultTime2Wait) and DefaultTime2Retain (Section 12.16
   DefaultTime2Retain) text keys for the session.

6.4.2.  Timeouts on Planned Decommissioning

   Any planned decommissioning of a Full Feature Phase iSCSI connection
   is preceded by either a Logout Response PDU, or an Async Message PDU.
   The Time2Wait and Time2Retain field values (section 10.15) in a
   Logout Response PDU, and the Parameter2 and Parameter3 fields of an
   Async Message (AsyncEvent types "drop the connection" or "drop all
   the connections"; section 10.9.1) specify the timeout values to be
   used in each of these cases.

   These timeout values are only applicable for the affected connection,
   and the tasks active on that connection.  These timeout values have
   no bearing on initiator timers (if any) that are already running on
   connections or tasks associated with that session.

6.5.  Implicit Termination of Tasks

   A target implicitly terminates the active tasks due to iSCSI protocol
   dynamics in the following cases:

      a)  When a connection is implicitly or explicitly logged out with
          the reason code of "Close the connection" and there are active
          tasks allegiant to that connection.

      b)  When a connection fails and the connection state eventually
          times out (state transition M1 in Section 7.2.2 State
          Transition Descriptions for Initiators and Targets) and there
          are active tasks allegiant to that connection.

      c)  When a successful Logout with the reason code of "remove the
          connection for recovery" is performed while there are active
          tasks allegiant to that connection, and those tasks eventually

Top      Up      ToC       Page 78 
          time out after the Time2Wait and Time2Retain periods without
          allegiance reassignment.

      d)  When a connection is implicitly or explicitly logged out with
          the reason code of "Close the session" and there are active
          tasks in that session.

   If the tasks terminated in the above cases a), b, c) and d)are SCSI
   tasks, they must be internally terminated as if with CHECK CONDITION
   status.  This status is only meaningful for appropriately handling
   the internal SCSI state and SCSI side effects with respect to
   ordering because this status is never communicated back as a
   terminating status to the initiator.  However additional actions may
   have to be taken at SCSI level depending on the SCSI context as
   defined by the SCSI standards (e.g., queued commands and ACA, in
   cases a), b), and c), after the tasks are terminated, the target MUST
   report a Unit Attention condition on the next command processed on
   any connection for each affected I_T_L nexus with the status of CHECK
   CONDITION, and the ASC/ASCQ value of 47h/7Fh - "SOME COMMANDS CLEARED
   BY ISCSI PROTOCOL EVENT" , etc. - see [SAM2] and [SPC3]).

6.6.  Format Errors

   The following two explicit violations of PDU layout rules are format
   errors:

      a)  Illegal contents of any PDU header field except the Opcode
          (legal values are specified in Section 10 iSCSI PDU Formats).
      b)  Inconsistent field contents (consistent field contents are
          specified in Section 10 iSCSI PDU Formats).

   Format errors indicate a major implementation flaw in one of the
   parties.

   When a target or an initiator receives an iSCSI PDU with a format
   error, it MUST immediately terminate all transport connections in the
   session either with a connection close or with a connection reset and
   escalate the format error to session recovery (see Section 6.1.4.4
   Session Recovery).

6.7.  Digest Errors

   The discussion of the legal choices in handling digest errors below
   excludes session recovery as an explicit option, but either party
   detecting a digest error may choose to escalate the error to session
   recovery.

Top      Up      ToC       Page 79 
   When a target or an initiator receives any iSCSI PDU, with a header
   digest error, it MUST either discard the header and all data up to
   the beginning of a later PDU or close the connection.  Because the
   digest error indicates that the length field of the header may have
   been corrupted, the location of the beginning of a later PDU needs to
   be reliably ascertained by other means such as the operation of a
   sync and steering layer.

   When a target receives any iSCSI PDU with a payload digest error, it
   MUST answer with a Reject PDU with a reason code of
   Data-Digest-Error and discard the PDU.

      -  If the discarded PDU is a solicited or unsolicited iSCSI data
         PDU (for immediate data in a command PDU, non-data PDU rule
         below applies), the target MUST do one of the following:
         a) Request retransmission with a recovery R2T.
         b) Terminate the task with a response PDU with a CHECK
            CONDITION Status and an iSCSI Condition of "protocol service
            CRC error" (Section 10.4.7.2 Sense Data).  If the target
            chooses to implement this option, it MUST wait to receive
            all the data (signaled by a Data PDU with the final bit set
            for all outstanding R2Ts) before sending the response PDU.
            A task management command (such as an abort task) from the
            initiator during this wait may also conclude the task.
      -  No further action is necessary for targets if the discarded PDU
         is a non-data PDU.  In case of immediate data being present on
         a discarded command, the immediate data is implicitly recovered
         when the task is retried (see section 6.2.1), followed by the
         entire data transfer for the task.

   When an initiator receives any iSCSI PDU with a payload digest error,
   it MUST discard the PDU.

   -  If the discarded PDU is an iSCSI data PDU, the initiator MUST do
      one of the following:

      a) Request the desired data PDU through SNACK.  In response to the
         SNACK, the target MUST either resend the data PDU or reject the
         SNACK with a Reject PDU with a reason code of "SNACK reject" in
         which case:
         i)  If the status has not already been sent for the command,
             the target MUST terminate the command with a CHECK
             CONDITION Status and an iSCSI Condition of "SNACK rejected"
             (Section 10.4.7.2 Sense Data).
         ii) If the status was already sent, no further action is
             necessary for the target.  The initiator in this case MUST
             wait for the status to be received and then discard it, so
             as to internally signal the completion with CHECK CONDITION

Top      Up      ToC       Page 80 
             Status and an iSCSI Condition of "protocol service CRC
             error" (Section 10.4.7.2 Sense Data).
      b) Abort the task and terminate the command with an error.

   -  If the discarded PDU is a response PDU, the initiator MUST do one
      of the following:

      a) Request PDU retransmission with a status SNACK.
      b) Logout the connection for recovery and continue the tasks on a
         different connection instance as described in Section 6.2 Retry
         and Reassign in Recovery.
      c) Logout to close the connection (abort all the commands
         associated with the connection).

   -  No further action is necessary for initiators if the discarded PDU
      is an unsolicited PDU (e.g., Async, Reject).  Task timeouts as in
      the initiator waiting for a command completion, or process
      timeouts, as in the target waiting for a Logout, will ensure that
      the correct operational behavior will result in these cases
      despite the discarded PDU.

6.8.  Sequence Errors

   When an initiator receives an iSCSI R2T/data PDU with an out of order
   R2TSN/DataSN or a SCSI response PDU with an ExpDataSN that implies
   missing data PDU(s), it means that the initiator must have detected a
   header or payload digest error on one or more earlier R2T/data PDUs.
   The initiator MUST address these implied digest errors as described
   in Section 6.7 Digest Errors.  When a target receives a data PDU with
   an out of order DataSN, it means that the target must have hit a
   header or payload digest error on at least one of the earlier data
   PDUs.  The target MUST address these implied digest errors as
   described in Section 6.7 Digest Errors.

   When an initiator receives an iSCSI status PDU with an out of order
   StatSN that implies missing responses, it MUST address the one or
   more missing status PDUs as described in Section 6.7 Digest Errors.
   As a side effect of receiving the missing responses, the initiator
   may discover missing data PDUs.  If the initiator wants to recover
   the missing data for a command, it MUST NOT acknowledge the received
   responses that start from the StatSN of the relevant command, until
   it has completed receiving all the data PDUs of the command.

   When an initiator receives duplicate R2TSNs (due to proactive
   retransmission of R2Ts by the target) or duplicate DataSNs (due to
   proactive SNACKs by the initiator), it MUST discard the duplicates.

Top      Up      ToC       Page 81 
6.9.  SCSI Timeouts

   An iSCSI initiator MAY attempt to plug a command sequence gap on the
   target end (in the absence of an acknowledgement of the command by
   way of ExpCmdSN) before the ULP timeout by retrying the
   unacknowledged command, as described in Section 6.2 Retry and
   Reassign in Recovery.

   On a ULP timeout for a command (that carried a CmdSN of n), if the
   iSCSI initiator intends to continue the session, it MUST abort the
   command by either using an appropriate Task Management function
   request for the specific command, or a "close the connection" Logout.
   When using an ABORT TASK, if the ExpCmdSN is still less than (n+1),
   the target may see the abort request while missing the original
   command itself due to one of the following reasons:

      -  Original command was dropped due to digest error.
      -  Connection on which the original command was sent was
         successfully logged out.  Upon logout, the unacknowledged
         commands issued on the connection being logged out are
         discarded.

   If the abort request is received and the original command is missing,
   targets MUST consider the original command with that RefCmdSN to be
   received and issue a Task Management response with the response code:
   "Function Complete".  This response concludes the task on both ends.
   If the abort request is received and the target can determine (based
   on the Referenced Task Tag) that the command was received and
   executed and also that the response was sent prior to the abort, then
   the target MUST respond with the response code of "Task Does Not
   Exist".

6.10.  Negotiation Failures

   Text request and response sequences, when used to set/negotiate
   operational parameters, constitute the negotiation/parameter setting.
   A negotiation failure is considered to be one or more of the
   following:

      -  None of the choices, or the stated value, is acceptable to one
         of the sides in the negotiation.
      -  The text request timed out and possibly terminated.
      -  The text request was answered with a Reject PDU.

Top      Up      ToC       Page 82 
   The following two rules should be used to address negotiation
   failures:

      -  During Login, any failure in negotiation MUST be considered a
         login process failure and the Login Phase must be terminated,
         and with it, the connection.  If the target detects the
         failure, it must terminate the login with the appropriate Login
         Response code.

      -  A failure in negotiation, while in the Full Feature Phase, will
         terminate the entire negotiation sequence that may consist of a
         series of text requests that use the same Initiator Task Tag.
         The operational parameters of the session or the connection
         MUST continue to be the values agreed upon during an earlier
         successful negotiation (i.e., any partial results of this
         unsuccessful negotiation MUST NOT take effect and MUST be
         discarded).

6.11.  Protocol Errors

   Mapping framed messages over a "stream" connection, such as TCP,
   makes the proposed mechanisms vulnerable to simple software framing
   errors.  On the other hand, the introduction of framing mechanisms to
   limit the effects of these errors may be onerous on performance for
   simple implementations.  Command Sequence Numbers and the above
   mechanisms for connection drop and reestablishment help handle this
   type of mapping errors.

   All violations of iSCSI PDU exchange sequences specified in this
   document are also protocol errors.  This category of errors can only
   be addressed by fixing the implementations; iSCSI defines Reject and
   response codes to enable this.

6.12.  Connection Failures

   iSCSI can keep a session in operation if it is able to
   keep/establish at least one TCP connection between the initiator and
   the target in a timely fashion.  Targets and/or initiators may
   recognize a failing connection by either transport level means (TCP),
   a gap in the command sequence number, a response stream that is not
   filled for a long time, or by a failing iSCSI NOP (acting as a ping).
   The latter MAY be used periodically to increase the speed and
   likelihood of detecting connection failures.  Initiators and targets
   MAY also use the keep-alive option on the TCP connection to enable
   early link failure detection on otherwise idle links.

Top      Up      ToC       Page 83 
   On connection failure, the initiator and target MUST do one of the
   following:

      -  Attempt connection recovery within the session (Section 6.1.4.3
         Connection Recovery).

      -  Logout the connection with the reason code "closes the
         connection" (Section 10.14.5 Implicit termination of tasks),
         re-issue missing commands, and implicitly terminate all active
         commands.  This option requires support for the
         within-connection recovery class (Section 6.1.4.2 Recovery
         Within-connection).

      -  Perform session recovery (Section 6.1.4.4 Session Recovery).

   Either side may choose to escalate to session recovery (via the
   initiator dropping all the connections, or via an Async Message that
   announces the similar intent from a target), and the other side MUST
   give it precedence.  On a connection failure, a target MUST terminate
   and/or discard all of the active immediate commands regardless of
   which of the above options is used (i.e., immediate commands are not
   recoverable across connection failures).

6.13.  Session Errors

   If all of the connections of a session fail and cannot be
   reestablished in a short time, or if initiators detect protocol
   errors repeatedly, an initiator may choose to terminate a session and
   establish a new session.

   In this case, the initiator takes the following actions:

      -  Resets or closes all the transport connections.
      -  Terminates all outstanding requests with an appropriate
         response before initiating a new session.  If the same I_T
         nexus is intended to be reestablished, the initiator MUST
         employ session reinstatement (see section 5.3.5).

   When the session timeout (the connection state timeout for the last
   failed connection) happens on the target, it takes the following
   actions:

      -  Resets or closes the TCP connections (closes the session).
      -  Terminates all active tasks that were allegiant to the
         connection(s) that constituted the session.

   A target MUST also be prepared to handle a session reinstatement
   request from the initiator, that may be addressing session errors.

Top      Up      ToC       Page 84 
7.  State Transitions

   iSCSI connections and iSCSI sessions go through several well-defined
   states from the time they are created to the time they are cleared.

   The connection state transitions are described in two separate but
   dependent state diagrams for ease in understanding.  The first
   diagram, "standard connection state diagram", describes the
   connection state transitions when the iSCSI connection is not waiting
   for, or undergoing, a cleanup by way of an explicit or implicit
   Logout.  The second diagram, "connection cleanup state diagram",
   describes the connection state transitions while performing the iSCSI
   connection cleanup.

   The "session state diagram" describes the state transitions an iSCSI
   session would go through during its lifetime, and it depends on the
   states of possibly multiple iSCSI connections that participate in the
   session.

   States and state transitions are described in the text, tables and
   diagrams.  The diagrams are used for illustration.  The text and the
   tables are the governing specification.

7.1.  Standard Connection State Diagrams

7.1.1.  State Descriptions for Initiators and Targets

   State descriptions for the standard connection state diagram are as
   follows:

   -S1: FREE
        -initiator: State on instantiation, or after successful
         connection closure.
        -target: State on instantiation, or after successful connection
         closure.
   -S2: XPT_WAIT
        -initiator: Waiting for a response to its transport connection
         establishment request.
        -target: Illegal
   -S3: XPT_UP
        -initiator: Illegal
        -target: Waiting for the Login process to commence.
   -S4: IN_LOGIN
        -initiator: Waiting for the Login process to conclude, possibly
         involving several PDU exchanges.
        -target: Waiting for the Login process to conclude, possibly
         involving several PDU exchanges.

Top      Up      ToC       Page 85 
   -S5: LOGGED_IN
        -initiator: In Full Feature Phase, waiting for all internal,
         iSCSI, and transport events.
        -target: In Full Feature Phase, waiting for all internal, iSCSI,
         and transport events.
   -S6: IN_LOGOUT
        -initiator: Waiting for a Logout response.
        -target: Waiting for an internal event signaling completion of
         logout processing.
   -S7: LOGOUT_REQUESTED
        -initiator: Waiting for an internal event signaling readiness to
         proceed with Logout.
        -target: Waiting for the Logout process to start after having
         requested a Logout via an Async Message.
   -S8: CLEANUP_WAIT
        -initiator: Waiting for the context and/or resources to initiate
         the cleanup processing for this CSM.
        -target: Waiting for the cleanup process to start for this CSM.

7.1.2.  State Transition Descriptions for Initiators and Targets

   -T1:
        -initiator: Transport connect request was made (e.g., TCP SYN
            sent).
        -target: Illegal
   -T2:
        -initiator: Transport connection request timed out, a transport
            reset was received, or an internal event of receiving a
            Logout response (success) on another connection for a
            "close the session"  Logout request was received.
        -target:Illegal
   -T3:
        -initiator: Illegal
        -target: Received a valid transport connection request that
            establishes the transport connection.
   -T4:
        -initiator: Transport connection established, thus prompting the
            initiator to start the iSCSI Login.
        -target: Initial iSCSI Login Request was received.
   -T5:
        -initiator: The final iSCSI Login Response with a Status-Class
            of zero was received.
        -target: The final iSCSI Login Request to conclude the Login
            Phase was received, thus prompting the target to send the
            final iSCSI Login Response with a Status-Class of zero.

Top      Up      ToC       Page 86 
   -T6:
        -initiator: Illegal
        -target: Timed out waiting for an iSCSI Login, transport
            disconnect indication was received, transport reset was
            received, or an internal event indicating a transport
            timeout was received.  In all these cases, the connection is
            to be closed.
   -T7:
        -initiator - one of the following events caused the transition:
            - The final iSCSI Login Response was received with a
              non-zero Status-Class.
            - Login timed out.
            - A transport disconnect indication was received.
            - A transport reset was received.
            - An internal event was received indicating a transport
              timeout.
            - An internal event of receiving a Logout response (success)
              on another connection for a "close the session" Logout
              request was received.

        In all these cases, the transport connection is closed.

        -target - one of the following events caused the transition:
            - The final iSCSI Login Request to conclude the Login Phase
              was received, prompting the target to send the final iSCSI
              Login Response with a non-zero Status-Class.
            - Login timed out.
            - Transport disconnect indication was received.
            - Transport reset was received.
            - An internal event indicating a transport timeout was
              received.
            - On another connection a "close the session" Logout request
              was received.
        In all these cases, the connection is to be closed.
   -T8:
        -initiator: An internal event of receiving a Logout response
            (success) on another connection for a "close the session"
            Logout request was received, thus closing this connection
            requiring no further cleanup.
        -target: An internal event of sending a Logout response
            (success) on another connection for a "close the session"
            Logout request was received, or an internal event of a
            successful connection/session reinstatement is received,
            thus prompting the target to close this connection cleanly.

Top      Up      ToC       Page 87 
   -T9, T10:
        -initiator: An internal event that indicates the readiness to
            start the Logout process was received, thus prompting an
            iSCSI Logout to be sent by the initiator.
        -target: An iSCSI Logout request was received.
   -T11, T12:
        -initiator: Async PDU with AsyncEvent "Request Logout" was
            received.
        -target: An internal event that requires the decommissioning of
            the connection is received, thus causing an Async PDU with
            an AsyncEvent "Request Logout" to be sent.
   -T13:
        -initiator: An iSCSI Logout response (success) was received, or
            an internal event of receiving a Logout response (success)
            on another connection for a "close the session" Logout
            request was received.
        -target: An internal event was received that indicates
            successful processing of the Logout, which prompts an iSCSI
            Logout response (success) to be sent; an internal event of
            sending a Logout response (success) on another connection
            for a "close the session" Logout request was received; or an
            internal event of a successful connection/session
            reinstatement is received.  In all these cases, the
            transport connection is closed.

   -T14:
        -initiator: Async PDU with AsyncEvent "Request Logout" was
            received again.
        -target: Illegal
   -T15, T16:
        -initiator: One or more of the following events caused this
            transition:
            -Internal event that indicates a transport connection
               timeout was received thus prompting transport RESET or
               transport connection closure.
            -A transport RESET.
            -A transport disconnect indication.
            -Async PDU with AsyncEvent "Drop connection" (for this CID).
            -Async PDU with AsyncEvent "Drop all connections".
        -target: One or more of the following events caused this
            transition:
            -Internal event that indicates a transport connection
               timeout was received, thus prompting transport RESET or
               transport connection closure.
            -An internal event of a failed connection/session
               reinstatement is received.
            -A transport RESET.
            -A transport disconnect indication.

Top      Up      ToC       Page 88 
            -Internal emergency cleanup event was received which prompts
               an Async PDU with AsyncEvent "Drop connection" (for this
               CID), or event "Drop all connections".
   -T17:
        -initiator: One or more of the following events caused this
            transition:
            -Logout response, (failure i.e., a non-zero status) was
               received, or Logout timed out.
            -Any of the events specified for T15 and T16.
        -target:  One or more of the following events caused this
            transition:
            -Internal event that indicates a failure of the Logout
               processing was received, which prompts a Logout response
               (failure, i.e., a non-zero status) to be sent.
            -Any of the events specified for T15 and T16.
   -T18:
        -initiator: An internal event of receiving a Logout response
            (success) on another connection for a "close the session"
            Logout request was received.
        -target: An internal event of sending a Logout response
            (success) on another connection for a "close the session"
            Logout request was received, or an internal event of a
            successful connection/session reinstatement is received.  In
            both these cases, the connection is closed.

   The CLEANUP_WAIT state (S8) implies that there are possible iSCSI
   tasks that have not reached conclusion and are still considered busy.

7.1.3.  Standard Connection State Diagram for an Initiator

   Symbolic names for States:

      S1: FREE
      S2: XPT_WAIT
      S4: IN_LOGIN
      S5: LOGGED_IN
      S6: IN_LOGOUT
      S7: LOGOUT_REQUESTED
      S8: CLEANUP_WAIT

Top      Up      ToC       Page 89 
   States S5, S6, and S7 constitute the Full Feature Phase operation of
   the connection.

   The state diagram is as follows:

                     -------<-------------+
         +--------->/ S1    \<----+       |
      T13|       +->\       /<-+   \      |
         |      /    ---+---    \   \     |
         |     /        |     T2 \   |    |
         |  T8 |        |T1       |  |    |
         |     |        |        /   |T7  |
         |     |        |       /    |    |
         |     |        |      /     |    |
         |     |        V     /     /     |
         |     |     ------- /     /      |
         |     |    / S2    \     /       |
         |     |    \       /    /        |
         |     |     ---+---    /         |
         |     |        |T4    /          |
         |     |        V     /           | T18
         |     |     ------- /            |
         |     |    / S4    \             |
         |     |    \       /             |
         |     |     ---+---              |         T15
         |     |        |T5      +--------+---------+
         |     |        |       /T16+-----+------+  |
         |     |        |      /   -+-----+--+   |  |
         |     |        |     /   /  S7   \  |T12|  |
         |     |        |    / +->\       /<-+   V  V
         |     |        |   / /    -+-----       -------
         |     |        |  / /T11   |T10        /  S8   \
         |     |        V / /       V  +----+   \       /
         |     |      ---+-+-      ----+--  |    -------
         |     |     / S5    \T9  / S6    \<+    ^
         |     +-----\       /--->\       / T14  |
         |            -------      --+----+------+T17
         +---------------------------+

Top      Up      ToC       Page 90 
   The following state transition table represents the above diagram.
   Each row represents the starting state for a given transition, which
   after taking a transition marked in a table cell would end in the
   state represented by the column of the cell.  For example, from state
   S1, the connection takes the T1 transition to arrive at state S2.
   The fields marked "-" correspond to undefined transitions.

         +----+---+---+---+---+----+---+
         |S1  |S2 |S4 |S5 |S6 |S7  |S8 |
      ---+----+---+---+---+---+----+---+
       S1| -  |T1 | - | - | - | -  | - |
      ---+----+---+---+---+---+----+---+
       S2|T2  |-  |T4 | - | - | -  | - |
      ---+----+---+---+---+---+----+---+
       S4|T7  |-  |-  |T5 | - | -  | - |
      ---+----+---+---+---+---+----+---+
       S5|T8  |-  |-  | - |T9 |T11 |T15|
      ---+----+---+---+---+---+----+---+
       S6|T13 |-  |-  | - |T14|-   |T17|
      ---+----+---+---+---+---+----+---+
       S7|T18 |-  |-  | - |T10|T12 |T16|
      ---+----+---+---+---+---+----+---+
       S8| -  |-  |-  | - | - | -  | - |
      ---+----+---+---+---+---+----+---+

7.1.4.  Standard Connection State Diagram for a Target

   Symbolic names for States:

      S1: FREE
      S3: XPT_UP
      S4: IN_LOGIN
      S5: LOGGED_IN
      S6: IN_LOGOUT
      S7: LOGOUT_REQUESTED
      S8: CLEANUP_WAIT

   States S5, S6, and S7 constitute the Full Feature Phase operation of
   the connection.

Top      Up      ToC       Page 91 
   The state diagram is as follows:

                        -------<-------------+
            +--------->/ S1    \<----+       |
         T13|       +->\       /<-+   \      |
            |      /    ---+---    \   \     |
            |     /        |     T6 \   |    |
            |  T8 |        |T3       |  |    |
            |     |        |        /   |T7  |
            |     |        |       /    |    |
            |     |        |      /     |    |
            |     |        V     /     /     |
            |     |     ------- /     /      |
            |     |    / S3    \     /       |
            |     |    \       /    /        | T18
            |     |     ---+---    /         |
            |     |        |T4    /          |
            |     |        V     /           |
            |     |     ------- /            |
            |     |    / S4    \             |
            |     |    \       /             |
            |     |     ---+---         T15  |
            |     |        |T5      +--------+---------+
            |     |        |       /T16+-----+------+  |
            |     |        |      /  -+-----+---+   |  |
            |     |        |     /   /  S7   \  |T12|  |
            |     |        |    / +->\       /<-+   V  V
            |     |        |   / /    -+-----       -------
            |     |        |  / /T11   |T10        /  S8   \
            |     |        V / /       V           \       /
            |     |      ---+-+-      -------       -------
            |     |     / S5    \T9  / S6    \        ^
            |     +-----\       /--->\       /        |
            |            -------      --+----+--------+T17
            +---------------------------+

   The following state transition table represents the above diagram,
   and follows the conventions described for the initiator diagram.

Top      Up      ToC       Page 92 
      +----+---+---+---+---+----+---+
      |S1  |S3 |S4 |S5 |S6 |S7  |S8 |
   ---+----+---+---+---+---+----+---+
    S1| -  |T3 | - | - | - | -  | - |
   ---+----+---+---+---+---+----+---+
    S3|T6  |-  |T4 | - | - | -  | - |
   ---+----+---+---+---+---+----+---+
    S4|T7  |-  |-  |T5 | - | -  | - |
   ---+----+---+---+---+---+----+---+
    S5|T8  |-  |-  | - |T9 |T11 |T15|
   ---+----+---+---+---+---+----+---+
    S6|T13 |-  |-  | - |-  |-   |T17|
   ---+----+---+---+---+---+----+---+
    S7|T18 |-  |-  | - |T10|T12 |T16|
   ---+----+---+---+---+---+----+---+
    S8| -  |-  |-  | - | - | -  | - |
   ---+----+---+---+---+---+----+---+

7.2.  Connection Cleanup State Diagram for Initiators and Targets

   Symbolic names for states:

      R1: CLEANUP_WAIT (same as S8)
      R2: IN_CLEANUP
      R3: FREE (same as S1)

   Whenever a connection state machine (e.g., CSM-C) enters the
   CLEANUP_WAIT state (S8), it must go through the state transitions
   described in the connection cleanup state diagram either a) using a
   separate full-feature phase connection (let's call it CSM-E) in the
   LOGGED_IN state in the same session, or b) using a new transport
   connection (let's call it CSM-I) in the FREE state that is to be
   added to the same session.  In the CSM-E case, an explicit logout for
   the CID that corresponds to CSM-C (either as a connection or session
   logout) needs to be performed to complete the cleanup.  In the CSM-I
   case, an implicit logout for the CID that corresponds to CSM-C needs
   to be performed by way of connection reinstatement (section 5.3.4)
   for that CID.  In either case, the protocol exchanges on CSM-E or
   CSM-I determine the state transitions for CSM-C.  Therefore, this
   cleanup state diagram is only applicable to the instance of the
   connection in cleanup (i.e., CSM-C).  In the case of an implicit
   logout for example, CSM-C reaches FREE (R3) at the time CSM-I reaches
   LOGGED_IN.  In the case of an explicit logout, CSM-C reaches FREE
   (R3) when CSM-E receives a successful logout response while
   continuing to be in the LOGGED_IN state.

Top      Up      ToC       Page 93 
   An initiator must initiate an explicit or implicit connection logout
   for a connection in the CLEANUP_WAIT state, if the initiator intends
   to continue using the associated iSCSI session.

   The following state diagram applies to both initiators and targets.

                        -------
                       / R1    \
                    +--\       /<-+
                   /    ---+---
                  /        |        \ M3
               M1 |        |M2       |
                  |        |        /
                  |        |       /
                  |        |      /
                  |        V     /
                  |     ------- /
                  |    / R2    \
                  |    \       /
                  |     -------
                  |        |
                  |        |M4
                  |        |
                  |        |
                  |        |
                  |        V
                  |      -------
                  |     / R3    \
                  +---->\       /
                         -------

   The following state transition table represents the above diagram,
   and follows the same conventions as in earlier sections.

        +----+----+----+
        |R1  |R2  |R3  |
   -----+----+----+----+
    R1  | -  |M2  |M1  |
   -----+----+----+----+
    R2  |M3  | -  |M4  |
   -----+----+----+----+
    R3  | -  | -  | -  |
   -----+----+----+----+

Top      Up      ToC       Page 94 
7.2.1.  State Descriptions for Initiators and Targets

   -R1: CLEANUP_WAIT (Same as S8)
        -initiator: Waiting for the internal event to initiate the
            cleanup processing for CSM-C.
        -target: Waiting for the cleanup process to start for CSM-C.
   -R2: IN_CLEANUP
        -initiator: Waiting for the connection cleanup process to
            conclude for CSM-C.
        -target: Waiting for the connection cleanup process to conclude
            for CSM-C.
   -R3: FREE (Same as S1)
        -initiator: End state for CSM-C.
        -target: End state for CSM-C.

7.2.2.  State Transition Descriptions for Initiators and Targets

   -M1: One or more of the following events was received:
        -initiator:
            -An internal event that indicates connection state timeout.
            -An internal event of receiving a successful Logout response
               on a different connection for a "close the session"
               Logout.
        -target:
            -An internal event that indicates connection state timeout.
            -An internal event of sending a Logout response (success) on
               a different connection for a "close the session" Logout
               request.

   -M2: An implicit/explicit logout process was initiated by the
        initiator.
        -In CSM-I usage:
            -initiator: An internal event requesting the connection (or
               session) reinstatement was received, thus prompting a
               connection (or session) reinstatement Login to be sent
               transitioning CSM-I to state IN_LOGIN.
            -target: A connection/session reinstatement Login was
               received while in state XPT_UP.
        -In CSM-E usage:
            -initiator: An internal event that indicates that an
               explicit logout was sent for this CID in state LOGGED_IN.
            -target: An explicit logout was received for this CID in
               state LOGGED_IN.

Top      Up      ToC       Page 95 
   -M3: Logout failure detected
        -In CSM-I usage:
            -initiator: CSM-I failed to reach LOGGED_IN and arrived into
               FREE instead.
            -target: CSM-I failed to reach LOGGED_IN and arrived into
               FREE instead.
        -In CSM-E usage:
            -initiator: CSM-E either moved out of LOGGED_IN, or Logout
               timed out and/or aborted, or Logout response (failure)
               was received.
            -target: CSM-E either moved out of LOGGED_IN,  Logout timed
               out and/or aborted, or an internal event that indicates a
               failed Logout processing was received.  A Logout response
               (failure) was sent in the last case.

   -M4: Successful implicit/explicit logout was performed.

        - In CSM-I usage:
            -initiator: CSM-I reached state LOGGED_IN, or an internal
               event of receiving a Logout response (success) on another
               connection for a "close the session" Logout request was
               received.
            -target: CSM-I reached state LOGGED_IN, or an internal event
               of sending a Logout response (success) on a different
               connection for a "close the session" Logout request was
               received.
        - In CSM-E usage:
            -initiator: CSM-E stayed in LOGGED_IN and received a Logout
               response (success), or an internal event of receiving a
               Logout response (success) on another connection for a
               "close the session" Logout request was received.
            -target: CSM-E stayed in LOGGED_IN and an internal event
               indicating a successful Logout processing was received,
               or an internal event of sending a Logout response
               (success) on a different connection for a "close the
               session" Logout request was received.

7.3.  Session State Diagrams

7.3.1.  Session State Diagram for an Initiator

   Symbolic Names for States:

        Q1: FREE
        Q3: LOGGED_IN
        Q4: FAILED

   State Q3 represents the Full Feature Phase operation of the session.

Top      Up      ToC       Page 96 
   The state diagram is as follows:

                          -------
                         / Q1    \
                 +------>\       /<-+
                /         ---+---   |
               /             |      |N3
           N6 |              |N1    |
              |              |      |
              |    N4        |      |
              |  +--------+  |     /
              |  |        |  |    /
              |  |        |  |   /
              |  |        V  V  /
             -+--+--      -----+-
            / Q4    \ N5 / Q3    \
            \       /<---\       /
             -------      -------

   The state transition table is as follows:

        +----+----+----+
        |Q1  |Q3  |Q4  |
   -----+----+----+----+
    Q1  | -  |N1  | -  |
   -----+----+----+----+
    Q3  |N3  | -  |N5  |
   -----+----+----+----+
    Q4  |N6  |N4  | -  |
   -----+----+----+----+

7.3.2.  Session State Diagram for a Target

   Symbolic Names for States:

     Q1: FREE
     Q2: ACTIVE
     Q3: LOGGED_IN
     Q4: FAILED
     Q5: IN_CONTINUE

   State Q3 represents the Full Feature Phase operation of the session.

Top      Up      ToC       Page 97 
   The state diagram is as follows:

                                    -------
               +------------------>/ Q1    \
              /    +-------------->\       /<-+
              |    |                ---+---   |
              |    |                ^  |      |N3
           N6 |    |N11           N9|  V N1   |
              |    |                +------   |
              |    |               / Q2    \  |
              |    |               \       /  |
              |  --+----            +--+---   |
              | / Q5    \              |      |
              | \       / N10          |      |
              |  +-+---+------------+  |N2   /
              |  ^ |                |  |    /
              |N7| |N8              |  |   /
              |  | |                |  V  /
             -+--+-V                V----+-
            / Q4    \ N5           / Q3    \
            \       /<-------------\       /
             -------                -------

   The state transition table is as follows:

        +----+----+----+----+----+
        |Q1  |Q2  |Q3  |Q4  |Q5  |
   -----+----+----+----+----+----+
    Q1  | -  |N1  | -  | -  | -  |
   -----+----+----+----+----+----+
    Q2  |N9  | -  |N2  | -  | -  |
   -----+----+----+----+----+----+
    Q3  |N3  | -  | -  |N5  | -  |
   -----+----+----+----+----+----+
    Q4  |N6  | -  | -  | -  |N7  |
   -----+----+----+----+----+----+
    Q5  |N11 | -  |N10 |N8  | -  |
   -----+----+----+----+----+----+

7.3.3.  State Descriptions for Initiators and Targets

   -Q1: FREE
        -initiator: State on instantiation or after cleanup.
        -target: State on instantiation or after cleanup.

Top      Up      ToC       Page 98 
   -Q2: ACTIVE
        -initiator: Illegal.
        -target: The first iSCSI connection in the session transitioned
            to IN_LOGIN, waiting for it to complete the login process.

   -Q3: LOGGED_IN
        -initiator: Waiting for all session events.
        -target: Waiting for all session events.

   -Q4: FAILED
        -initiator: Waiting for session recovery or session
            continuation.
        -target: Waiting for session recovery or session continuation.

   -Q5: IN_CONTINUE
        -initiator: Illegal.
        -target: Waiting for session continuation attempt to reach a
            conclusion.

7.3.4.  State Transition Descriptions for Initiators and Targets

   -N1:
        -initiator: At least one transport connection reached the
            LOGGED_IN state.
        -target: The first iSCSI connection in the session had reached
            the IN_LOGIN state.

   -N2:
        -initiator: Illegal.
        -target: At least one iSCSI connection reached the LOGGED_IN
            state.

   -N3:
        -initiator: Graceful closing of the session via session closure
            (Section 5.3.6 Session Continuation and Failure).
        -target: Graceful closing of the session via session closure
            (Section 5.3.6 Session Continuation and Failure) or a
            successful session reinstatement cleanly closed the session.

   -N4:
        -initiator: A session continuation attempt succeeded.
        -target: Illegal.

   -N5:
        -initiator: Session failure (Section 5.3.6 Session Continuation
            and Failure) occurred.
        -target: Session failure (Section 5.3.6 Session Continuation and
            Failure) occurred.

Top      Up      ToC       Page 99 
   -N6:
        -initiator: Session state timeout occurred, or a session
            reinstatement cleared this session instance.  This results
            in the freeing of all associated resources and the session
            state is discarded.
        -target: Session state timeout occurred, or a session
            reinstatement cleared this session instance.  This results
            in the freeing of all associated resources and the session
            state is discarded.

   -N7:
        -initiator: Illegal.
        -target: A session continuation attempt is initiated.

   -N8:
        -initiator: Illegal.
        -target: The last session continuation attempt failed.

   -N9:
        -initiator: Illegal.
        -target: Login attempt on the leading connection failed.

   -N10:
        -initiator: Illegal.
        -target: A session continuation attempt succeeded.

   -N11:
        -initiator: Illegal.
        -target: A successful session reinstatement cleanly closed the
            session.



(page 99 continued on part 5)

Next RFC Part