tech-invite   World Map     

IETF     RFCs     Groups     SIP     ABNFs    |    3GPP     Specs     Gloss.     Arch.     IMS     UICC    |    Misc.    |    search     info

RFC 5042

Proposed STD
Pages: 52
Top     in Index     Prev     Next
in Group Index     Prev in Group     Next in Group     Group: RDDP

Direct Data Placement Protocol (DDP) / Remote Direct Memory Access Protocol (RDMAP) Security

Part 1 of 3, p. 1 to 18
None       Next RFC Part

Updated by:    7146


Top       ToC       Page 1 
Network Working Group                                       J. Pinkerton
Request for Comments: 5042                         Microsoft Corporation
Category: Standards Track                                   E. Deleganes
                                                                    Self
                                                            October 2007


                Direct Data Placement Protocol (DDP) /
         Remote Direct Memory Access Protocol (RDMAP) Security

Status of This Memo

   This document specifies an Internet standards track protocol for the
   Internet community, and requests discussion and suggestions for
   improvements.  Please refer to the current edition of the "Internet
   Official Protocol Standards" (STD 1) for the standardization state
   and status of this protocol.  Distribution of this memo is unlimited.

Abstract

   This document analyzes security issues around implementation and use
   of the Direct Data Placement Protocol (DDP) and Remote Direct Memory
   Access Protocol (RDMAP).  It first defines an architectural model for
   an RDMA Network Interface Card (RNIC), which can implement DDP or
   RDMAP and DDP.  The document reviews various attacks against the
   resources defined in the architectural model and the countermeasures
   that can be used to protect the system.  Attacks are grouped into
   those that can be mitigated by using secure communication channels
   across the network, attacks from Remote Peers, and attacks from Local
   Peers.  Attack categories include spoofing, tampering, information
   disclosure, denial of service, and elevation of privilege.

Top       Page 2 
Table of Contents

   1. Introduction ....................................................4
   2. Architectural Model .............................................6
      2.1. Components .................................................7
      2.2. Resources ..................................................9
           2.2.1. Stream Context Memory ...............................9
           2.2.2. Data Buffers .......................................10
           2.2.3. Page Translation Tables ............................10
           2.2.4. Protection Domain (PD) .............................11
           2.2.5. STag Namespace and Scope ...........................11
           2.2.6. Completion Queues ..................................12
           2.2.7. Asynchronous Event Queue ...........................12
           2.2.8. RDMA Read Request Queue ............................13
      2.3. RNIC Interactions .........................................13
           2.3.1. Privileged Control Interface Semantics .............13
           2.3.2. Non-Privileged Data Interface Semantics ............13
           2.3.3. Privileged Data Interface Semantics ................14
           2.3.4. Initialization of RNIC Data Structures for
                  Data Transfer ......................................14
           2.3.5. RNIC Data Transfer Interactions ....................16
   3. Trust and Resource Sharing .....................................17
   4. Attacker Capabilities ..........................................18
   5. Attacks That Can Be Mitigated with End-to-End Security .........18
      5.1. Spoofing ..................................................19
           5.1.1. Impersonation ......................................19
           5.1.2. Stream Hijacking ...................................20
           5.1.3. Man-in-the-Middle Attack ...........................20
      5.2. Tampering - Network-Based Modification of Buffer Content ..21
      5.3. Information Disclosure - Network-Based Eavesdropping ......21
      5.4. Specific Requirements for Security Services ...............21
           5.4.1. Introduction to Security Options ...................21
           5.4.2. TLS Is Inappropriate for DDP/RDMAP Security ........22
           5.4.3. DTLS and RDDP ......................................23
           5.4.4. ULPs That Provide Security .........................23
           5.4.5. Requirements for IPsec Encapsulation of DDP ........23
   6. Attacks from Remote Peers ......................................24
      6.1. Spoofing ..................................................25
           6.1.1. Using an STag on a Different Stream ................25
      6.2. Tampering .................................................26
           6.2.1. Buffer Overrun - RDMA Write or Read Response .......26
           6.2.2. Modifying a Buffer after Indication ................27
           6.2.3. Multiple STags to Access the Same Buffer ...........27
      6.3. Information Disclosure ....................................28
           6.3.1. Probing Memory Outside of the Buffer Bounds ........28
           6.3.2. Using RDMA Read to Access Stale Data ...............28
           6.3.3. Accessing a Buffer after the Transfer ..............28
           6.3.4. Accessing Unintended Data with a Valid STag ........29

Top      ToC       Page 3 
           6.3.5. RDMA Read into an RDMA Write Buffer ................29
           6.3.6. Using Multiple STags That Alias to the Same
                  Buffer .............................................29
      6.4. Denial of Service (DOS) ...................................30
           6.4.1. RNIC Resource Consumption ..........................30
           6.4.2. Resource Consumption by Idle ULPs ..................31
           6.4.3. Resource Consumption by Active ULPs ................32
                  6.4.3.1. Multiple Streams Sharing Receive Buffers ..32
                  6.4.3.2. Remote or Local Peer Attacking a
                           Shared CQ .................................34
                  6.4.3.3. Attacking the RDMA Read Request Queue .....36
           6.4.4. Exercise of Non-Optimal Code Paths .................37
           6.4.5. Remote Invalidate an STag Shared on
                  Multiple Streams ...................................37
           6.4.6. Remote Peer Attacking an Unshared CQ ...............38
      6.5. Elevation of Privilege ....................................38
   7. Attacks from Local Peers .......................................38
      7.1. Local ULP Attacking a Shared CQ ...........................39
      7.2. Local Peer Attacking the RDMA Read Request Queue ..........39
      7.3. Local ULP Attacking the PTT and STag Mapping ..............39
   8. Security considerations ........................................40
   9. IANA Considerations ............................................40
   10. References ....................................................40
      10.1. Normative References .....................................40
      10.2. Informative References ...................................41
   Appendix A. ULP Issues for RDDP Client/Server Protocols ...........43
   Appendix B. Summary of RNIC and ULP Implementation Requirements ...46
   Appendix C. Partial Trust Taxonomy ................................47
   Acknowledgments ...................................................49

Top      ToC       Page 4 
1.  Introduction

   RDMA enables new levels of flexibility when communicating between two
   parties compared to current conventional networking practice (e.g., a
   stream-based model or datagram model).  This flexibility brings new
   security issues that must be carefully understood when designing
   Upper Layer Protocols (ULPs) utilizing RDMA and when implementing
   RDMA-aware NICs (RNICs).  Note that for the purposes of this security
   analysis, an RNIC may implement RDMAP [RDMAP] and DDP [DDP], or just
   DDP.  Also, a ULP may be an application or it may be a middleware
   library.

   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
   "SHOULD", "SHOULD NOT", "RECOMMENDED",  "MAY", and "OPTIONAL" in this
   document are to be interpreted as described in RFC 2119.
   Additionally, the security terminology defined in [RFC4949] is used
   in this specification.

   The document first develops an architectural model that is relevant
   for the security analysis.  Section 2 details components, resources,
   and system properties that may be attacked.  The document uses Local
   Peer to represent the RDMA/DDP protocol implementation on the local
   end of a Stream (implemented with a transport protocol, such as
   [RFC793] or [RFC4960]).  The local Upper-Layer-Protocol (ULP) is used
   to represent the application or middle-ware layer above the Local
   Peer.  The document does not attempt to differentiate between a
   Remote Peer and a Remote ULP (an RDMA/DDP protocol implementation on
   the remote end of a Stream versus the application on the remote end)
   for several reasons: often, the source of the attack is difficult to
   know for sure and, regardless of the source, the mitigations required
   of the Local Peer or local ULP are the same.  Thus, the document
   generically refers to a Remote Peer rather than trying to further
   delineate the attacker.

   The document then defines what resources a local ULP may share across
   Streams and what resources the local ULP may share with the Remote
   Peer across Streams in Section 3.

   Intentional sharing of resources between multiple Streams may imply
   some level of trust between the Streams.  However, some types of
   resource sharing have unmitigated security attacks, which would
   mandate not sharing a specific type of resource unless there is some
   level of trust between the Streams sharing resources.

Top      ToC       Page 5 
   This document defines a new term, "Partial Mutual Trust", to address
   this concept:

      Partial Mutual Trust - a collection of RDMAP/DDP Streams, which
      represent the local and remote end points of the Stream that are
      willing to assume that the Streams from the collection will not
      perform malicious attacks against any of the other Streams in the
      collection.

   ULPs have explicit control of which collection of endpoints is in a
   Partial Mutual Trust collection through tools discussed in Appendix
   C, Partial Trust Taxonomy.

   An untrusted peer relationship is appropriate when a ULP wishes to
   ensure that it will be robust and uncompromised even in the face of a
   deliberate attack by its peer.  For example, a single ULP that
   concurrently supports multiple unrelated Streams (e.g., a server)
   would presumably treat each of its peers as an untrusted peer.  For a
   collection of Streams that share Partial Mutual Trust, the assumption
   is that any Stream not in the collection is untrusted.  For the
   untrusted peer, a brief list of capabilities is enumerated in Section
   4.

   The rest of the document is focused on analyzing attacks and
   recommending specific mitigations to the attacks.  Attacks are
   categorized into attacks mitigated by end-to-end security, attacks
   initiated by Remote Peers, and attacks initiated by Local Peers.  For
   each attack, possible countermeasures are reviewed.

   ULPs within a host are divided into two categories - Privileged and
   Non-Privileged.  Both ULP types can send and receive data and request
   resources.  The key differences between the two are:

      The Privileged ULP is trusted by the local system not to
      maliciously attack the operating environment, but it is not
      trusted to optimize resource allocation globally.  For example,
      the Privileged ULP could be a kernel ULP; thus, the kernel
      presumably has in some way vetted the ULP before allowing it to
      execute.

      A Non-Privileged ULP's capabilities are a logical sub-set of the
      Privileged ULP's.  It is assumed by the local system that a Non-
      Privileged ULP is untrusted.  All Non-Privileged ULP interactions
      with the RNIC Engine that could affect other ULPs need to be done
      through a trusted intermediary that can verify the Non-Privileged
      ULP requests.

Top      ToC       Page 6 
   The appendices provide focused summaries of this specification.
   Appendix A, ULP Issues for RDDP Client/Server Protocols, focuses on
   implementers of traditional client/server protocols.  Appendix B,
   Summary of RNIC and ULP Implementation Requirements, summarizes all
   normative requirements in this specification.  Appendix C, Partial
   Trust Taxonomy, provides an abstract model for categorizing trust
   boundaries.

   If an RDMAP/DDP protocol implementation uses the mitigations
   recommended in this document, that implementation should not exhibit
   additional security vulnerabilities above and beyond those of an
   implementation of the transport protocol (i.e., TCP or SCTP) and
   protocols beneath it (e.g., IP) without RDMAP/DDP.

2.  Architectural Model

   This section describes an RDMA architectural reference model that is
   used as security issues are examined.  It introduces the components
   of the model, the resources that can be attacked, the types of
   interactions possible between components and resources, and the
   system properties that must be preserved.

   Figure 1 shows the components comprising the architecture and the
   interfaces where potential security attacks could be launched.
   External attacks can be injected into the system from a ULP that sits
   above the RNIC Interface or from the network.

   The intent here is to describe high level components and capabilities
   that affect threat analysis, and not focus on specific implementation
   options.  Also note that the architectural model is an abstraction,
   and an actual implementation may choose to subdivide its components
   along different boundary lines from those defined here.  For example,
   the Privileged Resource Manager may be partially or completely
   encapsulated in the Privileged ULP.  Regardless, it is expected that
   the security analysis of the potential threats and countermeasures
   still apply.

   Note that the model below is derived from several specific RDMA
   implementations.  A few of note are [VERBS-RDMAC], [VERBS-RDMAC-
   Overview], and [INFINIBAND].

Top      ToC       Page 7 
             +-------------+
             |  Privileged |
             |  Resource   |
    Admin<-+>|  Manager    |     ULP Control Interface
           | |             |<------+-------------------+
           | +-------------+       |                   |
           |       ^               v                   v
           |       |         +-------------+   +-----------------+
           +---------------->| Privileged  |   |  Non-Privileged |
                   |         | ULP         |   |  ULP            |
                   |         +-------------+   +-----------------+
                   |               ^                   ^
                   |Privileged     |Privileged         |Non-Privileged
                   |Control        |Data               |Data
                   |Interface      |Interface          |Interface
   RNIC            |               |                   |
   Interface       v               v                   v
   =================================================================

                 +--------------------------------------+
                 |                                      |
                 |               RNIC Engine            |
                 |                                      |
                 +--------------------------------------+
                                   ^
                                   |
                                   v
                                Internet

                      Figure 1 - RDMA Security Model

2.1.  Components

   The components shown in Figure 1 - RDMA Security Model are:

   *   RDMA Network Interface Controller Engine (RNIC) - The component
       that implements the RDMA protocol and/or DDP protocol.

   *   Privileged Resource Manager - The component responsible for
       managing and allocating resources associated with the RNIC
       Engine.  The Resource Manager does not send or receive data.
       Note that whether the Resource Manager is an independent
       component, part of the RNIC, or part of the ULP is implementation
       dependent.

Top      ToC       Page 8 
   *   Privileged ULP - See Section 1, Introduction, for a definition of
       Privileged ULP.  The local host infrastructure can enable the
       Privileged ULP to map a Data Buffer directly from the RNIC Engine
       to the host through the RNIC Interface, but it does not allow the
       Privileged ULP to directly consume RNIC Engine resources.

   *   Non-Privileged ULP - See Section 1, Introduction, for a
       definition of Non-Privileged ULP.

   A design goal of the DDP and RDMAP protocols is to allow, under
   constrained conditions, Non-Privileged ULP to send and receive data
   directly to/from the RDMA Engine without Privileged Resource Manager
   intervention, while ensuring that the host remains secure.  Thus, one
   of the primary goals of this document is to analyze this usage model
   for the enforcement that is required in the RNIC Engine to ensure
   that the system remains secure.

   DDP provides two mechanisms for transferring data:

   *   Untagged Data Transfer - The incoming payload simply consumes the
       first buffer in a queue of buffers that are in the order
       specified by the receiving Peer (commonly referred to as the
       Receive Queue), and

   *   Tagged Data Transfer - The Peer transmitting the payload
       explicitly states which destination buffer is targeted, through
       use of an STag.  STag-based transfers allow the receiving ULP to
       be indifferent to what order (or in what messages) the opposite
       Peer sent the data, or in what order packets are received.

   Both data transfer mechanisms are also enabled through RDMAP, with
   additional control semantics.  Typically, Tagged Data Transfer can be
   used for payload transfer, while Untagged Data Transfer is best used
   for control messages.  However, each Upper Layer Protocol can
   determine the optimal use of Tagged and Untagged messages for itself.
   See [APPLICABILITY] for more information on application applicability
   for the two transfer mechanisms.

   For DDP, the two forms correspond to Untagged and Tagged DDP
   Messages, respectively.  For RDMAP, the two forms correspond to Send
   Type Messages and RDMA Messages (either RDMA Read or RDMA Write
   Messages), respectively.

Top      ToC       Page 9 
   The host interfaces that could be exercised include:

   *   Privileged Control Interface - A Privileged Resource Manager uses
       the RNIC Interface to allocate and manage RNIC Engine resources,
       control the state within the RNIC Engine, and monitor various
       events from the RNIC Engine.  It also uses this interface to act
       as a proxy for some operations that a Non-Privileged ULP may
       require (after performing appropriate countermeasures).

   *   ULP Control Interface - A ULP uses this interface to the
       Privileged Resource Manager to allocate RNIC Engine resources.
       The Privileged Resource Manager implements countermeasures to
       ensure that, if the Non-Privileged ULP launches an attack, it can
       prevent the attack from affecting other ULPs.

   *   Non-Privileged Data Transfer Interface - A Non-Privileged ULP
       uses this interface to initiate and check the status of data
       transfer operations.

   *   Privileged Data Transfer Interface - A superset of the
       functionality provided by the Non-Privileged Data Transfer
       Interface.  The ULP is allowed to directly manipulate RNIC Engine
       mapping resources to map an STag to a ULP Data Buffer.

   If Internet control messages, such as ICMP, ARP, RIPv4, etc. are
   processed by the RNIC Engine, the threat analyses for those protocols
   is also applicable, but outside the scope of this document.

2.2.  Resources

   This section describes the primary resources in the RNIC Engine that
   could be affected if under attack.  For RDMAP, all the defined
   resources apply.  For DDP, all the resources except the RDMA Read
   Queue apply.

2.2.1.  Stream Context Memory

   The state information for each Stream is maintained in memory, which
   could be located in a number of places - on the NIC, inside RAM
   attached to the NIC, in host memory, or in any combination of the
   three, depending on the implementation.

   Stream Context Memory includes state associated with Data Buffers.
   For Tagged Buffers, this includes how STag names, Data Buffers, and
   Page Translation Tables (see Section 2.2.3) interrelate.  It also
   includes the list of Untagged Data Buffers posted for reception of
   Untagged Messages (commonly called the Receive Queue), and a list of
   operations to perform to send data (commonly called the Send Queue).

Top      ToC       Page 10 
2.2.2.  Data Buffers

   As mentioned previously, there are two different ways to expose a
   local ULP's Data Buffers for data transfer: Untagged Data Transfer,
   where a buffer can be exposed for receiving RDMAP Send Type Messages
   (a.k.a. DDP Untagged Messages) on DDP Queue zero, or Tagged Data
   Transfer, where the buffer can be exposed for remote access through
   STags (a.k.a. DDP Tagged Messages).  This distinction is important
   because the attacks and the countermeasures used to protect against
   the attack are different depending on the method for exposing the
   buffer to the network.

   For the purposes of the security discussion, for Tagged Data
   Transfer, a single logical Data Buffer is exposed with a single STag
   on a given Stream.  Actual implementations may support scatter/gather
   capabilities to enable multiple physical data buffers to be accessed
   with a single STag, but from a threat analysis perspective, it is
   assumed that a single STag enables access to a single logical Data
   Buffer.

   In any event, it is the responsibility of the Privileged Resource
   Manager to ensure that no STag can be created that exposes memory
   that the consumer had no authority to expose.

   A Data Buffer has specific access rights.  The local ULP can control
   whether a Data Buffer is exposed for local only, or local and remote
   access, and assign specific access privileges (read, write, read and
   write) on a per Stream basis.

   For DDP, when an STag is Advertised, the Remote Peer is presumably
   given write access rights to the data (otherwise, there would not be
   much point to the Advertisement).  For RDMAP, when a ULP Advertises
   an STag, it can enable write-only, read-only, or both write and read
   access rights.

   Similarly, some ULPs may wish to provide a single buffer with
   different access rights on a per Stream basis.  For example, some
   Streams may have read-only access, some may have remote read and
   write access, while on other Streams, only the local ULP/Local Peer
   is allowed access.

2.2.3.  Page Translation Tables

   Page Translation Tables are the structures used by the RNIC to be
   able to access ULP memory for data transfer operations.  Even though
   these structures are called "Page" Translation Tables, they may not
   reference a page at all - conceptually, they are used to map a ULP
   address space representation (e.g., a virtual address) of a buffer to

Top      ToC       Page 11 
   the physical addresses that are used by the RNIC Engine to move data.
   If, on a specific system, a mapping is not used, then a subset of the
   attacks examined may be appropriate.  Note that the Page Translation
   Table may or may not be a shared resource.

2.2.4.  Protection Domain (PD)

   A Protection Domain (PD) is a local construct to the RDMA
   implementation, and never visible over the wire.  Protection Domains
   are assigned to three of the resources of concern - Stream Context
   Memory, STags associated with Page Translation Table entries, and
   Data Buffers.  A correct implementation of a Protection Domain
   requires that resources that belong to a given Protection Domain
   cannot be used on a resource belonging to another Protection Domain,
   because Protection Domain membership is checked by the RNIC prior to
   taking any action involving such a resource.  Protection Domains are
   therefore used to ensure that an STag can only be used to access an
   associated Data Buffer on one or more Streams that are associated
   with the same Protection Domain as the specific STag.

   If an implementation chooses not to share resources between Streams,
   it is recommended that each Stream be associated with its own, unique
   Protection Domain.  If an implementation chooses to allow resource
   sharing, it is recommended that Protection Domain be limited to the
   collection of Streams that have Partial Mutual Trust with each other.

   Note that a ULP (either Privileged or Non-Privileged) can potentially
   have multiple Protection Domains.  This could be used, for example,
   to ensure that multiple clients of a server do not have the ability
   to corrupt each other.  The server would allocate a Protection Domain
   per client to ensure that resources covered by the Protection Domain
   could not be used by another (untrusted) client.

2.2.5.  STag Namespace and Scope

   The DDP specification defines a 32-bit namespace for the STag.
   Implementations may vary in terms of the actual number of STags that
   are supported.  In any case, this is a bounded resource that can come
   under attack.  Depending upon STag namespace allocation algorithms,
   the actual name space to attack may be significantly less than 2^32.

   The scope of an STag is the set of DDP/RDMAP Streams on which the
   STag is valid.  If an STag is valid on a particular DDP/RDMAP Stream,
   then that stream can modify the buffer, subject to the access rights
   that the stream has for the STag (see Section 2.2.2, Data Buffers,
   for additional information).

Top      ToC       Page 12 
   The analysis presented in this document assumes two mechanisms for
   limiting the scope of Streams for which the STag is valid:

   *   Protection Domain scope.  The STag is valid if used on any Stream
       within a specific Protection Domain, and is invalid if used on
       any Stream that is not a member of the Protection Domain.

   *   Single Stream scope.  The STag is valid on a single Stream,
       regardless of what the Stream association is to a Protection
       Domain.  If used on any other Stream, it is invalid.

2.2.6.  Completion Queues

   Completion Queues (CQ) are used in this document to conceptually
   represent how the RNIC Engine notifies the ULP about the completion
   of the transmission of data, or the completion of the reception of
   data through the Data Transfer Interface (specifically for Untagged
   Data Transfer; Tagged Data Transfer cannot cause a completion to
   occur).  Because there could be many transmissions or receptions in
   flight at any one time, completions are modeled as a queue rather
   than as a single event.  An implementation may also use the
   Completion Queue to notify the ULP of other activities; for example,
   the completion of a mapping of an STag to a specific ULP buffer.
   Completion Queues may be shared by a group of Streams, or may be
   designated to handle a specific Stream's traffic.  Limiting
   Completion Queue association to one, or a small number, of RDMAP/DDP
   Streams can prevent several forms of attacks by sharply limiting the
   scope of the attack's effect.

   Some implementations may allow this queue to be manipulated directly
   by both Non-Privileged and Privileged ULPs.

2.2.7.  Asynchronous Event Queue

   The Asynchronous Event Queue is a queue from the RNIC to the
   Privileged Resource Manager of bounded size.  It is used by the RNIC
   to notify the host of various events that might require management
   action, including protocol violations, Stream state changes, local
   operation errors, low water marks on receive queues, and possibly
   other events.

   The Asynchronous Event Queue is a resource that can be attacked
   because Remote or Local Peers and/or ULPs can cause events to occur
   that have the potential of overflowing the queue.

   Note that an implementation is at liberty to implement the functions
   of the Asynchronous Event Queue in a variety of ways, including
   multiple queues or even simple callbacks.  All vulnerabilities

Top      ToC       Page 13 
   identified are intended to apply, regardless of the implementation of
   the Asynchronous Event Queue.  For example, a callback function may
   be viewed simply as a very short queue.

2.2.8.  RDMA Read Request Queue

   The RDMA Read Request Queue is the memory that holds state
   information for one or more RDMA Read Request Messages that have
   arrived, but for which the RDMA Read Response Messages have not yet
   been completely sent.  Because potentially more than one RDMA Read
   Request can be outstanding at one time, the memory is modeled as a
   queue of bounded size.  Some implementations may enable sharing of a
   single RDMA Read Request Queue across multiple Streams.

2.3.  RNIC Interactions

   With RNIC resources and interfaces defined, it is now possible to
   examine the interactions supported by the generic RNIC functional
   interfaces through each of the 3 interfaces: Privileged Control
   Interface, Privileged Data Interface, and Non-Privileged Data
   Interface.  As mentioned previously in Section 2.1, Components, there
   are two data transfer mechanisms to be examined, Untagged Data
   Transfer and Tagged Data Transfer.

2.3.1.  Privileged Control Interface Semantics

   Generically, the Privileged Control Interface controls the RNIC's
   allocation, de-allocation, and initialization of RNIC global
   resources.  This includes allocation and de-allocation of Stream
   Context Memory, Page Translation Tables, STag names, Completion
   Queues, RDMA Read Request Queues, and Asynchronous Event Queues.

   The Privileged Control Interface is also typically used for managing
   Non-Privileged ULP resources for the Non-Privileged ULP (and possibly
   for the Privileged ULP as well).  This includes initialization and
   removal of Page Translation Table resources, and managing RNIC events
   (possibly managing all events for the Asynchronous Event Queue).

2.3.2.  Non-Privileged Data Interface Semantics

   The Non-Privileged Data Interface enables data transfer (transmit and
   receive) but does not allow initialization of the Page Translation
   Table resources.  However, once the Page Translation Table resources
   have been initialized, the interface may enable a specific STag
   mapping to be enabled and disabled by directly communicating with the
   RNIC, or create an STag mapping for a buffer that has been previously
   initialized in the RNIC.

Top      ToC       Page 14 
   For RDMAP, ULP data can be sent by one of the previously described
   data transfer mechanisms: Untagged Data Transfer or Tagged Data
   Transfer.  Two RDMAP data transfer mechanisms are defined, one using
   Untagged Data Transfer (Send Type Messages), and one using Tagged
   Data Transfer (RDMA Read Responses and RDMA Writes).  ULP data
   reception through RDMAP can be done by receiving Send Type Messages
   into buffers that have been posted on the Receive Queue or Shared
   Receive Queue.  Thus, a Receive Queue or Shared Receive Queue can
   only be affected by Untagged Data Transfer.  Data reception can also
   be done by receiving RDMA Write and RDMA Read Response Messages into
   buffers that have previously been exposed for external write access
   through Advertisement of an STag (i.e., Tagged Data Transfer).
   Additionally, to cause ULP data to be pulled (read) across the
   network, RDMAP uses an RDMA Read Request Message (which only contains
   RDMAP control information necessary to access the ULP buffer to be
   read), to cause an RDMA Read Response Message to be generated that
   contains the ULP data.

   For DDP, transmitting data means sending DDP Tagged or Untagged
   Messages.  For data reception, DDP can receive Untagged Messages into
   buffers that have been posted on the Receive Queue or Shared Receive
   Queue.  It can also receive Tagged DDP Messages into buffers that
   have previously been exposed for external write access through
   Advertisement of an STag.

   Completion of data transmission or reception generally entails
   informing the ULP of the completed work by placing completion
   information on the Completion Queue.  For data reception, only an
   Untagged Data Transfer can cause completion information to be put in
   the Completion Queue.

2.3.3.  Privileged Data Interface Semantics

   The Privileged Data Interface semantics are a superset of the Non-
   Privileged Data Transfer semantics.  The interface can do everything
   defined in the prior section, as well as create/destroy buffer to
   STag mappings directly.  This generally entails initialization or
   clearing of Page Translation Table state in the RNIC.

2.3.4.  Initialization of RNIC Data Structures for Data Transfer

   Initialization of the mapping between an STag and a Data Buffer can
   be viewed in the abstract as two separate operations:

   a.  Initialization of the allocated Page Translation Table entries
       with the location of the Data Buffer, and

Top      ToC       Page 15 
   b.  Initialization of a mapping from an allocated STag name to a set
       of Page Translation Table entry(s) or partial entries.

   Note that an implementation may not have a Page Translation Table
   (i.e., it may support a direct mapping between an STag and a Data
   Buffer).  If there is no Page Translation Table, then attacks based
   on changing its contents or exhausting its resources are not
   possible.

   Initialization of the contents of the Page Translation Table can be
   done by either the Privileged ULP or by the Privileged Resource
   Manager as a proxy for the Non-Privileged ULP.  By definition, the
   Non-Privileged ULP is not trusted to directly manipulate the Page
   Translation Table.  In general, the concern is that the Non-
   Privileged ULP may try to maliciously initialize the Page Translation
   Table to access a buffer for which it does not have permission.

   The exact resource allocation algorithm for the Page Translation
   Table is outside the scope of this document.  It may be allocated for
   a specific Data Buffer, or as a pooled resource to be consumed by
   potentially multiple Data Buffers, or be managed in some other way.
   This document attempts to abstract implementation dependent issues,
   and group them into higher level security issues, such as resource
   starvation and sharing of resources between Streams.

   The next issue is how an STag name is associated with a Data Buffer.
   For the case of an Untagged Data Buffer (i.e., Untagged Data
   Transfer), there is no wire visible mapping between an STag and the
   Data Buffer.  Note that there may, in fact, be an STag that
   represents the buffer, if an implementation chooses to internally
   represent Untagged Data Buffer using STags.  However, because the
   STag, by definition, is not visible on the wire, this is a local
   host, implementation-specific issue that should be analyzed in the
   context of a local host implementation-specific security analysis,
   and thus, is outside the scope of this document.

   For a Tagged Data Buffer (i.e., Tagged Data Transfer), either the
   Privileged ULP or the Privileged Resource Manager acting on behalf of
   the Non-Privileged ULP may initialize a mapping from an STag to a
   Page Translation Table, or may have the ability to simply
   enable/disable an existing STag to Page Translation Table mapping.
   There may also be multiple STag names that map to a specific group of
   Page Translation Table entries (or sub-entries).  Specific security
   issues with this level of flexibility are examined in Section 6.2.3,
   Multiple STags to Access the Same Buffer.

Top      ToC       Page 16 
   There are a variety of implementation options for initialization of
   Page Translation Table entries and mapping an STag to a group of Page
   Translation Table entries that have security repercussions.  This
   includes support for separation of mapping an STag versus mapping a
   set of Page Translation Table entries, and support for ULPs directly
   manipulating STag to Page Translation Table entry mappings (versus
   requiring access through the Privileged Resource Manager).

2.3.5.  RNIC Data Transfer Interactions

   RNIC Data Transfer operations can be subdivided into send and receive
   operations.

   For send operations, there is typically a queue that enables the ULP
   to post multiple operation requests to send data (referred to as the
   Send Queue).  Depending upon the implementation, Data Buffers used in
   the operations may or may not have Page Translation Table entries
   associated with them, and may or may not have STags associated with
   them.  Because this is a local host specific implementation issue
   rather than a protocol issue, the security analysis of threats and
   mitigations is left to the host implementation.

   Receive operations are different for Tagged Data Buffers versus
   Untagged Data Buffers (i.e., Tagged Data Transfer vs. Untagged Data
   Transfer).  For Untagged Data Transfer, if more than one Untagged
   Data Buffer can be posted by the ULP, the DDP specification requires
   that they be consumed in sequential order (the RDMAP specification
   also requires this).  Thus, the most general implementation is that
   there is a sequential queue of receive Untagged Data Buffers (Receive
   Queue).  Some implementations may also support sharing of the
   sequential queue between multiple Streams.  In this case, defining
   "sequential" becomes non-trivial - in general, the buffers for a
   single Stream are consumed from the queue in the order that they were
   placed on the queue, but there is no consumption order guarantee
   between Streams.

   For receive Tagged Data Transfer (i.e., Tagged Data Buffers, RDMA
   Write Buffers, or RDMA Read Buffers), at some time prior to data
   transfer, the mapping of the STag to specific Page Translation Table
   entries (if present) and the mapping from the Page Translation Table
   entries to the Data Buffer must have been initialized (see Section
   2.3.4 for interaction details).

Top      ToC       Page 17 
3.  Trust and Resource Sharing

   It is assumed that, in general, the Local and Remote Peer are
   untrusted, and thus attacks by either should have mitigations in
   place.

   A separate, but related issue is resource sharing between multiple
   Streams.  If local resources are not shared, the resources are
   dedicated on a per Stream basis.  Resources are defined in Section
   2.2, Resources.  The advantage of not sharing resources between
   Streams is that it reduces the types of attacks that are possible.
   The disadvantage of not sharing resources is that ULPs might run out
   of resources.  Thus, there can be a strong incentive for sharing
   resources, if the security issues associated with the sharing of
   resources can be mitigated.

   It is assumed in this document that the component that implements the
   mechanism to control sharing of the RNIC Engine resources is the
   Privileged Resource Manager.  The RNIC Engine exposes its resources
   through the RNIC Interface to the Privileged Resource Manager.  All
   Privileged and Non-Privileged ULPs request resources from the
   Resource Manager (note that by definition both the Non-Privileged and
   the Privileged application might try to greedily consume resources,
   thus creating a potential Denial of Service (DOS) attack).  The
   Resource Manager implements resource management policies to ensure
   fair access to resources.  The Resource Manager should be designed to
   take into account security attacks detailed in this document.  Note
   that for some systems the Privileged Resource Manager may be
   implemented within the Privileged ULP.

   All Non-Privileged ULP interactions with the RNIC Engine that could
   affect other ULPs MUST be done using the Privileged Resource Manager
   as a proxy.  All ULP resource allocation requests for scarce
   resources MUST also be done using a Privileged Resource Manager.

   The sharing of resources across Streams should be under the control
   of the ULP, both in terms of the trust model the ULP wishes to
   operate under, as well as the level of resource sharing the ULP
   wishes to give local processes.  For more discussion on types of
   trust models that combine partial trust and sharing of resources, see
   Appendix C, Partial Trust Taxonomy.

   The Privileged Resource Manager MUST NOT assume that different
   Streams share Partial Mutual Trust unless there is a mechanism to
   ensure that the Streams do indeed share Partial Mutual Trust.  This
   can be done in several ways, including explicit notification from the
   ULP that owns the Streams.

Top      ToC       Page 18 
4.  Attacker Capabilities

   An attacker's capabilities delimit the types of attacks that the
   attacker is able to launch.  RDMAP and DDP require that the initial
   LLP Stream (and connection) be set up prior to transferring RDMAP/DDP
   Messages.  This requires at least one round-trip handshake to occur.

   If the attacker is not the Remote Peer that created the initial
   connection, then the attacker's capabilities can be segmented into
   send only capabilities or send and receive capabilities.  Attacking
   with send only capabilities requires the attacker to first guess the
   current LLP Stream parameters before they can attack RNIC resources
   (e.g., TCP sequence number).  If this class of attacker also has
   receive capabilities and the ability to pose as the receiver to the
   sender and the sender to the receiver, they are typically referred to
   as a "man-in-the-middle" attacker [RFC3552].  A man-in-the-middle
   attacker has a much wider ability to attack RNIC resources.  The
   breadth of attack is essentially the same as that of an attacking
   Remote Peer (i.e., the Remote Peer that set up the initial LLP
   Stream).



(page 18 continued on part 2)

Next RFC Part