Internet Engineering Task Force (IETF)                    M. Chadalapaka
Request for Comments: 7143                                     Microsoft
Obsoletes: 3720, 3980, 4850, 5048                              J. Satran
Updates: 3721                                             Infinidat Ltd.
Category: Standards Track                                        K. Meth
ISSN: 2070-1721                                                      IBM
                                                                D. Black
                                                                     EMC
                                                              April 2014


       Internet Small Computer System Interface (iSCSI) Protocol
                             (Consolidated)

Abstract

   This document describes a transport protocol for SCSI that works on
   top of TCP.  The iSCSI protocol aims to be fully compliant with the
   standardized SCSI Architecture Model (SAM-2).  RFC 3720 defined the
   original iSCSI protocol.  RFC 3721 discusses iSCSI naming examples
   and discovery techniques.  Subsequently, RFC 3980 added an additional
   naming format to the iSCSI protocol.  RFC 4850 followed up by adding
   a new public extension key to iSCSI.  RFC 5048 offered a number of
   clarifications as well as a few improvements and corrections to the
   original iSCSI protocol.

   This document obsoletes RFCs 3720, 3980, 4850, and 5048 by
   consolidating them into a single document and making additional
   updates to the consolidated specification.  This document also
   updates RFC 3721.  The text in this document thus supersedes the text
   in all the noted RFCs wherever there is a difference in semantics.

Status of This Memo

   This is an Internet Standards Track document.

   This document is a product of the Internet Engineering Task Force
   (IETF).  It represents the consensus of the IETF community.  It has
   received public review and has been approved for publication by the
   Internet Engineering Steering Group (IESG).  Further information on
   Internet Standards is available in Section 2 of RFC 5741.

   Information about the current status of this document, any errata,
   and how to provide feedback on it may be obtained at
   http://www.rfc-editor.org/info/rfc7143.

Copyright Notice

   Copyright (c) 2014 IETF Trust and the persons identified as the
   document authors.  All rights reserved.

   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents
   (http://trustee.ietf.org/license-info) in effect on the date of
   publication of this document.  Please review these documents
   carefully, as they describe your rights and restrictions with respect
   to this document.  Code Components extracted from this document must
   include Simplified BSD License text as described in Section 4.e of
   the Trust Legal Provisions and are provided without warranty as
   described in the Simplified BSD License.

Table of Contents

   1. Introduction ...................................................11
   2. Acronyms, Definitions, and Document Summary ....................11
      2.1. Acronyms ..................................................11
      2.2. Definitions ...............................................13
      2.3. Summary of Changes ........................................19
      2.4. Conventions ...............................................20
   3. UML Conventions ................................................20
      3.1. UML Conventions Overview ..................................20
      3.2. Multiplicity Notion .......................................21
      3.3. Class Diagram Conventions .................................22
      3.4. Class Diagram Notation for Associations ...................23
      3.5. Class Diagram Notation for Aggregations ...................24
      3.6. Class Diagram Notation for Generalizations ................25
   4. Overview .......................................................25
      4.1. SCSI Concepts .............................................25
      4.2. iSCSI Concepts and Functional Overview ....................26
           4.2.1. Layers and Sessions ................................27
           4.2.2. Ordering and iSCSI Numbering .......................28
                  4.2.2.1. Command Numbering and Acknowledging .......28
                  4.2.2.2. Response/Status Numbering and
                           Acknowledging .............................32
                  4.2.2.3. Response Ordering .........................32
                           4.2.2.3.1. Need for Response Ordering .....32
                           4.2.2.3.2. Response Ordering Model
                                      Description ....................33
                           4.2.2.3.3. iSCSI Semantics with
                                      the Interface Model ............33
                           4.2.2.3.4. Current List of Fenced
                                      Response Use Cases .............34
                  4.2.2.4. Data Sequencing ...........................35

           4.2.3. iSCSI Task Management ..............................36
                  4.2.3.1. Task Management Overview ..................36
                  4.2.3.2. Notion of Affected Tasks ..................36
                  4.2.3.3. Standard Multi-Task Abort Semantics .......37
                  4.2.3.4. FastAbort Multi-Task Abort Semantics ......38
                  4.2.3.5. Affected Tasks Shared across
                           Standard and FastAbort Sessions ...........40
                  4.2.3.6. Rationale behind the FastAbort Semantics ..41
           4.2.4. iSCSI Login ........................................42
           4.2.5. iSCSI Full Feature Phase ...........................44
                  4.2.5.1. Command Connection Allegiance .............44
                  4.2.5.2. Data Transfer Overview ....................45
                  4.2.5.3. Tags and Integrity Checks .................46
                  4.2.5.4. SCSI Task Management during iSCSI
                           Full Feature Phase ........................47
           4.2.6. iSCSI Connection Termination .......................47
           4.2.7. iSCSI Names ........................................47
                  4.2.7.1. iSCSI Name Properties .....................48
                  4.2.7.2. iSCSI Name Encoding .......................50
                  4.2.7.3. iSCSI Name Structure ......................51
                  4.2.7.4. Type "iqn." (iSCSI Qualified Name) ........52
                  4.2.7.5. Type "eui." (IEEE EUI-64 Format) ..........53
                  4.2.7.6. Type "naa." (Network Address Authority) ...54
           4.2.8. Persistent State ...................................55
           4.2.9. Message Synchronization and Steering ...............55
                  4.2.9.1. Sync/Steering and iSCSI PDU Length ........56
      4.3. iSCSI Session Types .......................................56
      4.4. SCSI-to-iSCSI Concepts Mapping Model ......................57
           4.4.1. iSCSI Architecture Model ...........................58
           4.4.2. SCSI Architecture Model ............................59
           4.4.3. Consequences of the Model ..........................61
                  4.4.3.1. I_T Nexus State ...........................62
                  4.4.3.2. Reservations ..............................63
      4.5. iSCSI UML Model ...........................................64
      4.6. Request/Response Summary ..................................66
           4.6.1. Request/Response Types Carrying SCSI Payload .......66
                  4.6.1.1. SCSI Command ..............................66
                  4.6.1.2. SCSI Response .............................66
                  4.6.1.3. Task Management Function Request ..........67
                  4.6.1.4. Task Management Function Response .........68
                  4.6.1.5. SCSI Data-Out and SCSI Data-In ............68
                  4.6.1.6. Ready To Transfer (R2T) ...................69
           4.6.2. Requests/Responses Carrying SCSI and iSCSI
                  Payload ............................................69
                  4.6.2.1. Asynchronous Message ......................69

           4.6.3. Requests/Responses Carrying iSCSI-Only Payload .....69
                  4.6.3.1. Text Requests and Text Responses ..........69
                  4.6.3.2. Login Requests and Login Responses ........70
                  4.6.3.3. Logout Requests and Logout Responses ......71
                  4.6.3.4. SNACK Request .............................71
                  4.6.3.5. Reject ....................................71
                  4.6.3.6. NOP-Out Request and NOP-In Response .......71
   5. SCSI Mode Parameters for iSCSI .................................72
   6. Login and Full Feature Phase Negotiation .......................72
      6.1. Text Format ...............................................73
      6.2. Text Mode Negotiation .....................................76
           6.2.1. List Negotiations ..................................80
           6.2.2. Simple-Value Negotiations ..........................80
      6.3. Login Phase ...............................................81
           6.3.1. Login Phase Start ..................................84
           6.3.2. iSCSI Security Negotiation .........................87
           6.3.3. Operational Parameter Negotiation during
                  the Login Phase ....................................87
           6.3.4. Connection Reinstatement ...........................88
           6.3.5. Session Reinstatement, Closure, and Timeout ........89
                  6.3.5.1. Loss of Nexus Notification ................90
           6.3.6. Session Continuation and Failure ...................90
      6.4. Operational Parameter Negotiation outside the
           Login Phase ...............................................90
   7. iSCSI Error Handling and Recovery ..............................92
      7.1. Overview ..................................................92
           7.1.1. Background .........................................92
           7.1.2. Goals ..............................................92
           7.1.3. Protocol Features and State Expectations ...........93
           7.1.4. Recovery Classes ...................................94
                  7.1.4.1. Recovery Within-command ...................95
                  7.1.4.2. Recovery Within-connection ................96
                  7.1.4.3. Connection Recovery .......................96
                  7.1.4.4. Session Recovery ..........................97
           7.1.5. Error Recovery Hierarchy ...........................97
      7.2. Retry and Reassign in Recovery ............................99
           7.2.1. Usage of Retry .....................................99
           7.2.2. Allegiance Reassignment ...........................100
      7.3. Usage of Reject PDU in Recovery ..........................101
      7.4. Error Recovery Considerations for Discovery Sessions .....102
           7.4.1. ErrorRecoveryLevel for Discovery Sessions .........102
           7.4.2. Reinstatement Semantics for Discovery Sessions ....102
                  7.4.2.1. Unnamed Discovery Sessions ...............103
                  7.4.2.2. Named Discovery Sessions .................103
           7.4.3. Target PDUs during Discovery ......................103

      7.5. Connection Timeout Management ............................104
           7.5.1. Timeouts on Transport Exception Events ............104
           7.5.2. Timeouts on Planned Decommissioning ...............104
      7.6. Implicit Termination of Tasks ............................104
      7.7. Format Errors ............................................105
      7.8. Digest Errors ............................................106
      7.9. Sequence Errors ..........................................107
      7.10. Message Error Checking ..................................108
      7.11. SCSI Timeouts ...........................................108
      7.12. Negotiation Failures ....................................109
      7.13. Protocol Errors .........................................110
      7.14. Connection Failures .....................................110
      7.15. Session Errors ..........................................111
   8. State Transitions .............................................112
      8.1. Standard Connection State Diagrams .......................112
           8.1.1. State Descriptions for Initiators and Targets .....112
           8.1.2. State Transition Descriptions for
                  Initiators and Targets ............................114
           8.1.3. Standard Connection State Diagram for an
                  Initiator .........................................118
           8.1.4. Standard Connection State Diagram for a Target ....120
      8.2. Connection Cleanup State Diagram for Initiators
           and Targets ..............................................122
           8.2.1. State Descriptions for Initiators and Targets .....124
           8.2.2. State Transition Descriptions for
                  Initiators and Targets ............................124
      8.3. Session State Diagrams ...................................126
           8.3.1. Session State Diagram for an Initiator ............126
           8.3.2. Session State Diagram for a Target ................127
           8.3.3. State Descriptions for Initiators and Targets .....129
           8.3.4. State Transition Descriptions for
                  Initiators and Targets ............................129
   9. Security Considerations .......................................131
      9.1. iSCSI Security Mechanisms ................................132
      9.2. In-Band Initiator-Target Authentication ..................132
           9.2.1. CHAP Considerations ...............................134
           9.2.2. SRP Considerations ................................136
           9.2.3. Kerberos Considerations ...........................136
      9.3. IPsec ....................................................137
           9.3.1. Data Authentication and Integrity .................137
           9.3.2. Confidentiality ...................................138
           9.3.3. Policy, Security Associations, and
                  Cryptographic Key Management ......................139
      9.4. Security Considerations for the X#NodeArchitecture Key ...141
      9.5. SCSI Access Control Considerations .......................143

   10. Notes to Implementers ........................................143
      10.1. Multiple Network Adapters ...............................143
           10.1.1. Conservative Reuse of ISIDs ......................143
           10.1.2. iSCSI Name, ISID, and TPGT Use ...................144
      10.2. Autosense and Auto Contingent Allegiance (ACA) ..........146
      10.3. iSCSI Timeouts ..........................................146
      10.4. Command Retry and Cleaning Old Command Instances ........147
      10.5. Sync and Steering Layer, and Performance ................147
      10.6. Considerations for State-Dependent Devices and
            Long-Lasting SCSI Operations ............................147
           10.6.1. Determining the Proper ErrorRecoveryLevel ........148
      10.7. Multi-Task Abort Implementation Considerations ..........149
   11. iSCSI PDU Formats ............................................150
      11.1. iSCSI PDU Length and Padding ............................150
      11.2. PDU Template, Header, and Opcodes .......................150
           11.2.1. Basic Header Segment (BHS) .......................152
                  11.2.1.1. I (Immediate) Bit .......................152
                  11.2.1.2. Opcode ..................................152
                  11.2.1.3. F (Final) Bit ...........................154
                  11.2.1.4. Opcode-Specific Fields ..................154
                  11.2.1.5. TotalAHSLength ..........................154
                  11.2.1.6. DataSegmentLength .......................154
                  11.2.1.7. LUN .....................................154
                  11.2.1.8. Initiator Task Tag ......................154
           11.2.2. Additional Header Segment (AHS) ..................155
                  11.2.2.1. AHSType .................................155
                  11.2.2.2. AHSLength ...............................155
                  11.2.2.3. Extended CDB AHS ........................156
                  11.2.2.4. Bidirectional Read Expected Data
                            Transfer Length AHS .....................156
           11.2.3. Header Digest and Data Digest ....................156
           11.2.4. Data Segment .....................................157
      11.3. SCSI Command ............................................158
           11.3.1. Flags and Task Attributes (Byte 1) ...............159
           11.3.2. CmdSN - Command Sequence Number ..................159
           11.3.3. ExpStatSN ........................................160
           11.3.4. Expected Data Transfer Length ....................160
           11.3.5. CDB - SCSI Command Descriptor Block ..............160
           11.3.6. Data Segment - Command Data ......................161
      11.4. SCSI Response ...........................................161
           11.4.1. Flags (Byte 1) ...................................162
           11.4.2. Status ...........................................163
           11.4.3. Response .........................................163
           11.4.4. SNACK Tag ........................................164

           11.4.5. Residual Count ...................................164
                  11.4.5.1. Field Semantics .........................164
                  11.4.5.2. Residuals Concepts Overview .............164
                  11.4.5.3. SCSI REPORT LUNS Command and
                            Residual Overflow .......................165
           11.4.6. Bidirectional Read Residual Count ................166
           11.4.7. Data Segment - Sense and Response Data Segment ...167
                  11.4.7.1. SenseLength .............................167
                  11.4.7.2. Sense Data ..............................168
           11.4.8. ExpDataSN ........................................168
           11.4.9. StatSN - Status Sequence Number ..................168
           11.4.10. ExpCmdSN - Next Expected CmdSN from This
                    Initiator .......................................169
           11.4.11. MaxCmdSN - Maximum CmdSN from This Initiator ....169
      11.5. Task Management Function Request ........................170
           11.5.1. Function .........................................170
           11.5.2. TotalAHSLength and DataSegmentLength .............173
           11.5.3. LUN ..............................................173
           11.5.4. Referenced Task Tag ..............................173
           11.5.5. RefCmdSN .........................................174
           11.5.6. ExpDataSN ........................................174
      11.6. Task Management Function Response .......................175
           11.6.1. Response .........................................176
           11.6.2. TotalAHSLength and DataSegmentLength .............177
      11.7. SCSI Data-Out and SCSI Data-In ..........................178
           11.7.1. F (Final) Bit ....................................180
           11.7.2. A (Acknowledge) Bit ..............................180
           11.7.3. Flags (Byte 1) ...................................181
           11.7.4. Target Transfer Tag and LUN ......................181
           11.7.5. DataSN ...........................................182
           11.7.6. Buffer Offset ....................................182
           11.7.7. DataSegmentLength ................................182
      11.8. Ready To Transfer (R2T) .................................183
           11.8.1. TotalAHSLength and DataSegmentLength .............184
           11.8.2. R2TSN ............................................184
           11.8.3. StatSN ...........................................185
           11.8.4. Desired Data Transfer Length and Buffer Offset ...185
           11.8.5. Target Transfer Tag ..............................185
      11.9. Asynchronous Message ....................................186
           11.9.1. AsyncEvent .......................................187
           11.9.2. AsyncVCode .......................................189
           11.9.3. LUN ..............................................189
           11.9.4. Sense Data and iSCSI Event Data ..................190
                  11.9.4.1. SenseLength .............................190

      11.10. Text Request ...........................................191
           11.10.1. F (Final) Bit ...................................192
           11.10.2. C (Continue) Bit ................................192
           11.10.3. Initiator Task Tag ..............................192
           11.10.4. Target Transfer Tag .............................192
           11.10.5. Text ............................................193
      11.11. Text Response ..........................................194
           11.11.1. F (Final) Bit ...................................194
           11.11.2. C (Continue) Bit ................................195
           11.11.3. Initiator Task Tag ..............................195
           11.11.4. Target Transfer Tag .............................195
           11.11.5. StatSN ..........................................196
           11.11.6. Text Response Data ..............................196
      11.12. Login Request ..........................................196
           11.12.1. T (Transit) Bit .................................197
           11.12.2. C (Continue) Bit ................................197
           11.12.3. CSG and NSG .....................................198
           11.12.4. Version .........................................198
                  11.12.4.1. Version-max ............................198
                  11.12.4.2. Version-min ............................198
           11.12.5. ISID ............................................199
           11.12.6. TSIH ............................................200
           11.12.7. Connection ID (CID) .............................200
           11.12.8. CmdSN ...........................................201
           11.12.9. ExpStatSN .......................................201
           11.12.10. Login Parameters ...............................201
      11.13. Login Response .........................................202
           11.13.1. Version-max .....................................202
           11.13.2. Version-active ..................................203
           11.13.3. TSIH ............................................203
           11.13.4. StatSN ..........................................203
           11.13.5. Status-Class and Status-Detail ..................203
           11.13.6. T (Transit) Bit .................................206
           11.13.7. C (Continue) Bit ................................206
           11.13.8. Login Parameters ................................207
      11.14. Logout Request .........................................207
           11.14.1. Reason Code .....................................209
           11.14.2. TotalAHSLength and DataSegmentLength ............209
           11.14.3. CID .............................................210
           11.14.4. ExpStatSN .......................................210
           11.14.5. Implicit Termination of Tasks ...................210
      11.15. Logout Response ........................................211
           11.15.1. Response ........................................212
           11.15.2. TotalAHSLength and DataSegmentLength ............212
           11.15.3. Time2Wait .......................................212
           11.15.4. Time2Retain .....................................212

      11.16. SNACK Request ..........................................213
           11.16.1. Type ............................................214
           11.16.2. Data Acknowledgment .............................215
           11.16.3. Resegmentation ..................................215
           11.16.4. Initiator Task Tag ..............................216
           11.16.5. Target Transfer Tag or SNACK Tag ................216
           11.16.6. BegRun ..........................................216
           11.16.7. RunLength .......................................216
      11.17. Reject .................................................217
           11.17.1. Reason ..........................................218
           11.17.2. DataSN/R2TSN ....................................219
           11.17.3. StatSN, ExpCmdSN, and MaxCmdSN ..................219
           11.17.4. Complete Header of Bad PDU ......................219
      11.18. NOP-Out ................................................220
           11.18.1. Initiator Task Tag ..............................221
           11.18.2. Target Transfer Tag .............................221
           11.18.3. Ping Data .......................................221
      11.19. NOP-In .................................................222
           11.19.1. Target Transfer Tag .............................223
           11.19.2. StatSN ..........................................223
           11.19.3. LUN .............................................223
   12. iSCSI Security Text Keys and Authentication Methods ..........223
      12.1. AuthMethod ..............................................224
           12.1.1. Kerberos .........................................226
           12.1.2. Secure Remote Password (SRP) .....................226
           12.1.3. Challenge Handshake Authentication
                   Protocol (CHAP) ..................................228
   13. Login/Text Operational Text Keys .............................229
      13.1. HeaderDigest and DataDigest .............................230
      13.2. MaxConnections ..........................................232
      13.3. SendTargets .............................................232
      13.4. TargetName ..............................................232
      13.5. InitiatorName ...........................................233
      13.6. TargetAlias .............................................233
      13.7. InitiatorAlias ..........................................234
      13.8. TargetAddress ...........................................234
      13.9. TargetPortalGroupTag ....................................235
      13.10. InitialR2T .............................................236
      13.11. ImmediateData ..........................................236
      13.12. MaxRecvDataSegmentLength ...............................237
      13.13. MaxBurstLength .........................................238
      13.14. FirstBurstLength .......................................238
      13.15. DefaultTime2Wait .......................................239
      13.16. DefaultTime2Retain .....................................239
      13.17. MaxOutstandingR2T ......................................239
      13.18. DataPDUInOrder .........................................240
      13.19. DataSequenceInOrder ....................................240
      13.20. ErrorRecoveryLevel .....................................241

      13.21. SessionType ............................................241
      13.22. The Private Extension Key Format .......................242
      13.23. TaskReporting ..........................................242
      13.24. iSCSIProtocolLevel Negotiation .........................243
      13.25. Obsoleted Keys .........................................243
      13.26. X#NodeArchitecture .....................................244
           13.26.1. Definition ......................................244
           13.26.2. Implementation Requirements .....................244
   14. Rationale for Revised IANA Considerations ....................245
   15. IANA Considerations ..........................................246
   16. References ...................................................248
      16.1. Normative References ....................................248
      16.2. Informative References ..................................251
   Appendix A. Examples .............................................254
     A.1. Read Operation Example ....................................254
     A.2. Write Operation Example ...................................255
     A.3. R2TSN/DataSN Use Examples .................................256
          A.3.1. Output (Write) Data DataSN/R2TSN Example ...........256
          A.3.2. Input (Read) Data DataSN Example ...................257
          A.3.3. Bidirectional DataSN Example .......................258
          A.3.4. Unsolicited and Immediate Output (Write) Data
                 with DataSN Example ................................259
     A.4. CRC Examples ..............................................259
   Appendix B. Login Phase Examples .................................261
   Appendix C. SendTargets Operation ................................268
   Appendix D. Algorithmic Presentation of Error Recovery
               Classes ..............................................272
     D.1. General Data Structure and Procedure Description ..........273
     D.2. Within-command Error Recovery Algorithms ..................274
          D.2.1. Procedure Descriptions .............................274
          D.2.2. Initiator Algorithms ...............................275
          D.2.3. Target Algorithms ..................................277
     D.3. Within-connection Recovery Algorithms .....................279
          D.3.1. Procedure Descriptions .............................279
          D.3.2. Initiator Algorithms ...............................280
          D.3.3. Target Algorithms ..................................283
     D.4. Connection Recovery Algorithms ............................283
          D.4.1. Procedure Descriptions .............................283
          D.4.2. Initiator Algorithms ...............................284
          D.4.3. Target Algorithms ..................................286
   Appendix E. Clearing Effects of Various Events on Targets ........288
     E.1. Clearing Effects on iSCSI Objects .........................288
     E.2. Clearing Effects on SCSI Objects ..........................293
   Acknowledgments ..................................................294