Tech-invite3GPPspaceIETFspace
959493929190898887868584838281807978777675747372717069686766656463626160595857565554535251504948474645444342414039383736353433323130292827262524232221201918171615141312111009080706050403020100
in Index   Prev   Next

RFC 5042

Direct Data Placement Protocol (DDP) / Remote Direct Memory Access Protocol (RDMAP) Security

Pages: 52
Proposed Standard
Updated by:  7146
Part 1 of 3 – Pages 1 to 18
None   None   Next

Top   ToC   RFC5042 - Page 1
Network Working Group                                       J. Pinkerton
Request for Comments: 5042                         Microsoft Corporation
Category: Standards Track                                   E. Deleganes
                                                                    Self
                                                            October 2007


                Direct Data Placement Protocol (DDP) /
         Remote Direct Memory Access Protocol (RDMAP) Security

Status of This Memo

   This document specifies an Internet standards track protocol for the
   Internet community, and requests discussion and suggestions for
   improvements.  Please refer to the current edition of the "Internet
   Official Protocol Standards" (STD 1) for the standardization state
   and status of this protocol.  Distribution of this memo is unlimited.

Abstract

This document analyzes security issues around implementation and use of the Direct Data Placement Protocol (DDP) and Remote Direct Memory Access Protocol (RDMAP). It first defines an architectural model for an RDMA Network Interface Card (RNIC), which can implement DDP or RDMAP and DDP. The document reviews various attacks against the resources defined in the architectural model and the countermeasures that can be used to protect the system. Attacks are grouped into those that can be mitigated by using secure communication channels across the network, attacks from Remote Peers, and attacks from Local Peers. Attack categories include spoofing, tampering, information disclosure, denial of service, and elevation of privilege.
Top   ToC   RFC5042 - Page 2

Table of Contents

1. Introduction ....................................................4 2. Architectural Model .............................................6 2.1. Components .................................................7 2.2. Resources ..................................................9 2.2.1. Stream Context Memory ...............................9 2.2.2. Data Buffers .......................................10 2.2.3. Page Translation Tables ............................10 2.2.4. Protection Domain (PD) .............................11 2.2.5. STag Namespace and Scope ...........................11 2.2.6. Completion Queues ..................................12 2.2.7. Asynchronous Event Queue ...........................12 2.2.8. RDMA Read Request Queue ............................13 2.3. RNIC Interactions .........................................13 2.3.1. Privileged Control Interface Semantics .............13 2.3.2. Non-Privileged Data Interface Semantics ............13 2.3.3. Privileged Data Interface Semantics ................14 2.3.4. Initialization of RNIC Data Structures for Data Transfer ......................................14 2.3.5. RNIC Data Transfer Interactions ....................16 3. Trust and Resource Sharing .....................................17 4. Attacker Capabilities ..........................................18 5. Attacks That Can Be Mitigated with End-to-End Security .........18 5.1. Spoofing ..................................................19 5.1.1. Impersonation ......................................19 5.1.2. Stream Hijacking ...................................20 5.1.3. Man-in-the-Middle Attack ...........................20 5.2. Tampering - Network-Based Modification of Buffer Content ..21 5.3. Information Disclosure - Network-Based Eavesdropping ......21 5.4. Specific Requirements for Security Services ...............21 5.4.1. Introduction to Security Options ...................21 5.4.2. TLS Is Inappropriate for DDP/RDMAP Security ........22 5.4.3. DTLS and RDDP ......................................23 5.4.4. ULPs That Provide Security .........................23 5.4.5. Requirements for IPsec Encapsulation of DDP ........23 6. Attacks from Remote Peers ......................................24 6.1. Spoofing ..................................................25 6.1.1. Using an STag on a Different Stream ................25 6.2. Tampering .................................................26 6.2.1. Buffer Overrun - RDMA Write or Read Response .......26 6.2.2. Modifying a Buffer after Indication ................27 6.2.3. Multiple STags to Access the Same Buffer ...........27 6.3. Information Disclosure ....................................28 6.3.1. Probing Memory Outside of the Buffer Bounds ........28 6.3.2. Using RDMA Read to Access Stale Data ...............28 6.3.3. Accessing a Buffer after the Transfer ..............28 6.3.4. Accessing Unintended Data with a Valid STag ........29
Top   ToC   RFC5042 - Page 3
           6.3.5. RDMA Read into an RDMA Write Buffer ................29
           6.3.6. Using Multiple STags That Alias to the Same
                  Buffer .............................................29
      6.4. Denial of Service (DOS) ...................................30
           6.4.1. RNIC Resource Consumption ..........................30
           6.4.2. Resource Consumption by Idle ULPs ..................31
           6.4.3. Resource Consumption by Active ULPs ................32
                  6.4.3.1. Multiple Streams Sharing Receive Buffers ..32
                  6.4.3.2. Remote or Local Peer Attacking a
                           Shared CQ .................................34
                  6.4.3.3. Attacking the RDMA Read Request Queue .....36
           6.4.4. Exercise of Non-Optimal Code Paths .................37
           6.4.5. Remote Invalidate an STag Shared on
                  Multiple Streams ...................................37
           6.4.6. Remote Peer Attacking an Unshared CQ ...............38
      6.5. Elevation of Privilege ....................................38
   7. Attacks from Local Peers .......................................38
      7.1. Local ULP Attacking a Shared CQ ...........................39
      7.2. Local Peer Attacking the RDMA Read Request Queue ..........39
      7.3. Local ULP Attacking the PTT and STag Mapping ..............39
   8. Security considerations ........................................40
   9. IANA Considerations ............................................40
   10. References ....................................................40
      10.1. Normative References .....................................40
      10.2. Informative References ...................................41
   Appendix A. ULP Issues for RDDP Client/Server Protocols ...........43
   Appendix B. Summary of RNIC and ULP Implementation Requirements ...46
   Appendix C. Partial Trust Taxonomy ................................47
   Acknowledgments ...................................................49
Top   ToC   RFC5042 - Page 4

1. Introduction

RDMA enables new levels of flexibility when communicating between two parties compared to current conventional networking practice (e.g., a stream-based model or datagram model). This flexibility brings new security issues that must be carefully understood when designing Upper Layer Protocols (ULPs) utilizing RDMA and when implementing RDMA-aware NICs (RNICs). Note that for the purposes of this security analysis, an RNIC may implement RDMAP [RDMAP] and DDP [DDP], or just DDP. Also, a ULP may be an application or it may be a middleware library. The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119. Additionally, the security terminology defined in [RFC4949] is used in this specification. The document first develops an architectural model that is relevant for the security analysis. Section 2 details components, resources, and system properties that may be attacked. The document uses Local Peer to represent the RDMA/DDP protocol implementation on the local end of a Stream (implemented with a transport protocol, such as [RFC793] or [RFC4960]). The local Upper-Layer-Protocol (ULP) is used to represent the application or middle-ware layer above the Local Peer. The document does not attempt to differentiate between a Remote Peer and a Remote ULP (an RDMA/DDP protocol implementation on the remote end of a Stream versus the application on the remote end) for several reasons: often, the source of the attack is difficult to know for sure and, regardless of the source, the mitigations required of the Local Peer or local ULP are the same. Thus, the document generically refers to a Remote Peer rather than trying to further delineate the attacker. The document then defines what resources a local ULP may share across Streams and what resources the local ULP may share with the Remote Peer across Streams in Section 3. Intentional sharing of resources between multiple Streams may imply some level of trust between the Streams. However, some types of resource sharing have unmitigated security attacks, which would mandate not sharing a specific type of resource unless there is some level of trust between the Streams sharing resources.
Top   ToC   RFC5042 - Page 5
   This document defines a new term, "Partial Mutual Trust", to address
   this concept:

      Partial Mutual Trust - a collection of RDMAP/DDP Streams, which
      represent the local and remote end points of the Stream that are
      willing to assume that the Streams from the collection will not
      perform malicious attacks against any of the other Streams in the
      collection.

   ULPs have explicit control of which collection of endpoints is in a
   Partial Mutual Trust collection through tools discussed in Appendix
   C, Partial Trust Taxonomy.

   An untrusted peer relationship is appropriate when a ULP wishes to
   ensure that it will be robust and uncompromised even in the face of a
   deliberate attack by its peer.  For example, a single ULP that
   concurrently supports multiple unrelated Streams (e.g., a server)
   would presumably treat each of its peers as an untrusted peer.  For a
   collection of Streams that share Partial Mutual Trust, the assumption
   is that any Stream not in the collection is untrusted.  For the
   untrusted peer, a brief list of capabilities is enumerated in Section
   4.

   The rest of the document is focused on analyzing attacks and
   recommending specific mitigations to the attacks.  Attacks are
   categorized into attacks mitigated by end-to-end security, attacks
   initiated by Remote Peers, and attacks initiated by Local Peers.  For
   each attack, possible countermeasures are reviewed.

   ULPs within a host are divided into two categories - Privileged and
   Non-Privileged.  Both ULP types can send and receive data and request
   resources.  The key differences between the two are:

      The Privileged ULP is trusted by the local system not to
      maliciously attack the operating environment, but it is not
      trusted to optimize resource allocation globally.  For example,
      the Privileged ULP could be a kernel ULP; thus, the kernel
      presumably has in some way vetted the ULP before allowing it to
      execute.

      A Non-Privileged ULP's capabilities are a logical sub-set of the
      Privileged ULP's.  It is assumed by the local system that a Non-
      Privileged ULP is untrusted.  All Non-Privileged ULP interactions
      with the RNIC Engine that could affect other ULPs need to be done
      through a trusted intermediary that can verify the Non-Privileged
      ULP requests.
Top   ToC   RFC5042 - Page 6
   The appendices provide focused summaries of this specification.
   Appendix A, ULP Issues for RDDP Client/Server Protocols, focuses on
   implementers of traditional client/server protocols.  Appendix B,
   Summary of RNIC and ULP Implementation Requirements, summarizes all
   normative requirements in this specification.  Appendix C, Partial
   Trust Taxonomy, provides an abstract model for categorizing trust
   boundaries.

   If an RDMAP/DDP protocol implementation uses the mitigations
   recommended in this document, that implementation should not exhibit
   additional security vulnerabilities above and beyond those of an
   implementation of the transport protocol (i.e., TCP or SCTP) and
   protocols beneath it (e.g., IP) without RDMAP/DDP.

2. Architectural Model

This section describes an RDMA architectural reference model that is used as security issues are examined. It introduces the components of the model, the resources that can be attacked, the types of interactions possible between components and resources, and the system properties that must be preserved. Figure 1 shows the components comprising the architecture and the interfaces where potential security attacks could be launched. External attacks can be injected into the system from a ULP that sits above the RNIC Interface or from the network. The intent here is to describe high level components and capabilities that affect threat analysis, and not focus on specific implementation options. Also note that the architectural model is an abstraction, and an actual implementation may choose to subdivide its components along different boundary lines from those defined here. For example, the Privileged Resource Manager may be partially or completely encapsulated in the Privileged ULP. Regardless, it is expected that the security analysis of the potential threats and countermeasures still apply. Note that the model below is derived from several specific RDMA implementations. A few of note are [VERBS-RDMAC], [VERBS-RDMAC- Overview], and [INFINIBAND].
Top   ToC   RFC5042 - Page 7
             +-------------+
             |  Privileged |
             |  Resource   |
    Admin<-+>|  Manager    |     ULP Control Interface
           | |             |<------+-------------------+
           | +-------------+       |                   |
           |       ^               v                   v
           |       |         +-------------+   +-----------------+
           +---------------->| Privileged  |   |  Non-Privileged |
                   |         | ULP         |   |  ULP            |
                   |         +-------------+   +-----------------+
                   |               ^                   ^
                   |Privileged     |Privileged         |Non-Privileged
                   |Control        |Data               |Data
                   |Interface      |Interface          |Interface
   RNIC            |               |                   |
   Interface       v               v                   v
   =================================================================

                 +--------------------------------------+
                 |                                      |
                 |               RNIC Engine            |
                 |                                      |
                 +--------------------------------------+
                                   ^
                                   |
                                   v
                                Internet

                      Figure 1 - RDMA Security Model

2.1. Components

The components shown in Figure 1 - RDMA Security Model are: * RDMA Network Interface Controller Engine (RNIC) - The component that implements the RDMA protocol and/or DDP protocol. * Privileged Resource Manager - The component responsible for managing and allocating resources associated with the RNIC Engine. The Resource Manager does not send or receive data. Note that whether the Resource Manager is an independent component, part of the RNIC, or part of the ULP is implementation dependent.
Top   ToC   RFC5042 - Page 8
   *   Privileged ULP - See Section 1, Introduction, for a definition of
       Privileged ULP.  The local host infrastructure can enable the
       Privileged ULP to map a Data Buffer directly from the RNIC Engine
       to the host through the RNIC Interface, but it does not allow the
       Privileged ULP to directly consume RNIC Engine resources.

   *   Non-Privileged ULP - See Section 1, Introduction, for a
       definition of Non-Privileged ULP.

   A design goal of the DDP and RDMAP protocols is to allow, under
   constrained conditions, Non-Privileged ULP to send and receive data
   directly to/from the RDMA Engine without Privileged Resource Manager
   intervention, while ensuring that the host remains secure.  Thus, one
   of the primary goals of this document is to analyze this usage model
   for the enforcement that is required in the RNIC Engine to ensure
   that the system remains secure.

   DDP provides two mechanisms for transferring data:

   *   Untagged Data Transfer - The incoming payload simply consumes the
       first buffer in a queue of buffers that are in the order
       specified by the receiving Peer (commonly referred to as the
       Receive Queue), and

   *   Tagged Data Transfer - The Peer transmitting the payload
       explicitly states which destination buffer is targeted, through
       use of an STag.  STag-based transfers allow the receiving ULP to
       be indifferent to what order (or in what messages) the opposite
       Peer sent the data, or in what order packets are received.

   Both data transfer mechanisms are also enabled through RDMAP, with
   additional control semantics.  Typically, Tagged Data Transfer can be
   used for payload transfer, while Untagged Data Transfer is best used
   for control messages.  However, each Upper Layer Protocol can
   determine the optimal use of Tagged and Untagged messages for itself.
   See [APPLICABILITY] for more information on application applicability
   for the two transfer mechanisms.

   For DDP, the two forms correspond to Untagged and Tagged DDP
   Messages, respectively.  For RDMAP, the two forms correspond to Send
   Type Messages and RDMA Messages (either RDMA Read or RDMA Write
   Messages), respectively.
Top   ToC   RFC5042 - Page 9
   The host interfaces that could be exercised include:

   *   Privileged Control Interface - A Privileged Resource Manager uses
       the RNIC Interface to allocate and manage RNIC Engine resources,
       control the state within the RNIC Engine, and monitor various
       events from the RNIC Engine.  It also uses this interface to act
       as a proxy for some operations that a Non-Privileged ULP may
       require (after performing appropriate countermeasures).

   *   ULP Control Interface - A ULP uses this interface to the
       Privileged Resource Manager to allocate RNIC Engine resources.
       The Privileged Resource Manager implements countermeasures to
       ensure that, if the Non-Privileged ULP launches an attack, it can
       prevent the attack from affecting other ULPs.

   *   Non-Privileged Data Transfer Interface - A Non-Privileged ULP
       uses this interface to initiate and check the status of data
       transfer operations.

   *   Privileged Data Transfer Interface - A superset of the
       functionality provided by the Non-Privileged Data Transfer
       Interface.  The ULP is allowed to directly manipulate RNIC Engine
       mapping resources to map an STag to a ULP Data Buffer.

   If Internet control messages, such as ICMP, ARP, RIPv4, etc. are
   processed by the RNIC Engine, the threat analyses for those protocols
   is also applicable, but outside the scope of this document.

2.2. Resources

This section describes the primary resources in the RNIC Engine that could be affected if under attack. For RDMAP, all the defined resources apply. For DDP, all the resources except the RDMA Read Queue apply.

2.2.1. Stream Context Memory

The state information for each Stream is maintained in memory, which could be located in a number of places - on the NIC, inside RAM attached to the NIC, in host memory, or in any combination of the three, depending on the implementation. Stream Context Memory includes state associated with Data Buffers. For Tagged Buffers, this includes how STag names, Data Buffers, and Page Translation Tables (see Section 2.2.3) interrelate. It also includes the list of Untagged Data Buffers posted for reception of Untagged Messages (commonly called the Receive Queue), and a list of operations to perform to send data (commonly called the Send Queue).
Top   ToC   RFC5042 - Page 10

2.2.2. Data Buffers

As mentioned previously, there are two different ways to expose a local ULP's Data Buffers for data transfer: Untagged Data Transfer, where a buffer can be exposed for receiving RDMAP Send Type Messages (a.k.a. DDP Untagged Messages) on DDP Queue zero, or Tagged Data Transfer, where the buffer can be exposed for remote access through STags (a.k.a. DDP Tagged Messages). This distinction is important because the attacks and the countermeasures used to protect against the attack are different depending on the method for exposing the buffer to the network. For the purposes of the security discussion, for Tagged Data Transfer, a single logical Data Buffer is exposed with a single STag on a given Stream. Actual implementations may support scatter/gather capabilities to enable multiple physical data buffers to be accessed with a single STag, but from a threat analysis perspective, it is assumed that a single STag enables access to a single logical Data Buffer. In any event, it is the responsibility of the Privileged Resource Manager to ensure that no STag can be created that exposes memory that the consumer had no authority to expose. A Data Buffer has specific access rights. The local ULP can control whether a Data Buffer is exposed for local only, or local and remote access, and assign specific access privileges (read, write, read and write) on a per Stream basis. For DDP, when an STag is Advertised, the Remote Peer is presumably given write access rights to the data (otherwise, there would not be much point to the Advertisement). For RDMAP, when a ULP Advertises an STag, it can enable write-only, read-only, or both write and read access rights. Similarly, some ULPs may wish to provide a single buffer with different access rights on a per Stream basis. For example, some Streams may have read-only access, some may have remote read and write access, while on other Streams, only the local ULP/Local Peer is allowed access.

2.2.3. Page Translation Tables

Page Translation Tables are the structures used by the RNIC to be able to access ULP memory for data transfer operations. Even though these structures are called "Page" Translation Tables, they may not reference a page at all - conceptually, they are used to map a ULP address space representation (e.g., a virtual address) of a buffer to
Top   ToC   RFC5042 - Page 11
   the physical addresses that are used by the RNIC Engine to move data.
   If, on a specific system, a mapping is not used, then a subset of the
   attacks examined may be appropriate.  Note that the Page Translation
   Table may or may not be a shared resource.

2.2.4. Protection Domain (PD)

A Protection Domain (PD) is a local construct to the RDMA implementation, and never visible over the wire. Protection Domains are assigned to three of the resources of concern - Stream Context Memory, STags associated with Page Translation Table entries, and Data Buffers. A correct implementation of a Protection Domain requires that resources that belong to a given Protection Domain cannot be used on a resource belonging to another Protection Domain, because Protection Domain membership is checked by the RNIC prior to taking any action involving such a resource. Protection Domains are therefore used to ensure that an STag can only be used to access an associated Data Buffer on one or more Streams that are associated with the same Protection Domain as the specific STag. If an implementation chooses not to share resources between Streams, it is recommended that each Stream be associated with its own, unique Protection Domain. If an implementation chooses to allow resource sharing, it is recommended that Protection Domain be limited to the collection of Streams that have Partial Mutual Trust with each other. Note that a ULP (either Privileged or Non-Privileged) can potentially have multiple Protection Domains. This could be used, for example, to ensure that multiple clients of a server do not have the ability to corrupt each other. The server would allocate a Protection Domain per client to ensure that resources covered by the Protection Domain could not be used by another (untrusted) client.

2.2.5. STag Namespace and Scope

The DDP specification defines a 32-bit namespace for the STag. Implementations may vary in terms of the actual number of STags that are supported. In any case, this is a bounded resource that can come under attack. Depending upon STag namespace allocation algorithms, the actual name space to attack may be significantly less than 2^32. The scope of an STag is the set of DDP/RDMAP Streams on which the STag is valid. If an STag is valid on a particular DDP/RDMAP Stream, then that stream can modify the buffer, subject to the access rights that the stream has for the STag (see Section 2.2.2, Data Buffers, for additional information).
Top   ToC   RFC5042 - Page 12
   The analysis presented in this document assumes two mechanisms for
   limiting the scope of Streams for which the STag is valid:

   *   Protection Domain scope.  The STag is valid if used on any Stream
       within a specific Protection Domain, and is invalid if used on
       any Stream that is not a member of the Protection Domain.

   *   Single Stream scope.  The STag is valid on a single Stream,
       regardless of what the Stream association is to a Protection
       Domain.  If used on any other Stream, it is invalid.

2.2.6. Completion Queues

Completion Queues (CQ) are used in this document to conceptually represent how the RNIC Engine notifies the ULP about the completion of the transmission of data, or the completion of the reception of data through the Data Transfer Interface (specifically for Untagged Data Transfer; Tagged Data Transfer cannot cause a completion to occur). Because there could be many transmissions or receptions in flight at any one time, completions are modeled as a queue rather than as a single event. An implementation may also use the Completion Queue to notify the ULP of other activities; for example, the completion of a mapping of an STag to a specific ULP buffer. Completion Queues may be shared by a group of Streams, or may be designated to handle a specific Stream's traffic. Limiting Completion Queue association to one, or a small number, of RDMAP/DDP Streams can prevent several forms of attacks by sharply limiting the scope of the attack's effect. Some implementations may allow this queue to be manipulated directly by both Non-Privileged and Privileged ULPs.

2.2.7. Asynchronous Event Queue

The Asynchronous Event Queue is a queue from the RNIC to the Privileged Resource Manager of bounded size. It is used by the RNIC to notify the host of various events that might require management action, including protocol violations, Stream state changes, local operation errors, low water marks on receive queues, and possibly other events. The Asynchronous Event Queue is a resource that can be attacked because Remote or Local Peers and/or ULPs can cause events to occur that have the potential of overflowing the queue. Note that an implementation is at liberty to implement the functions of the Asynchronous Event Queue in a variety of ways, including multiple queues or even simple callbacks. All vulnerabilities
Top   ToC   RFC5042 - Page 13
   identified are intended to apply, regardless of the implementation of
   the Asynchronous Event Queue.  For example, a callback function may
   be viewed simply as a very short queue.

2.2.8. RDMA Read Request Queue

The RDMA Read Request Queue is the memory that holds state information for one or more RDMA Read Request Messages that have arrived, but for which the RDMA Read Response Messages have not yet been completely sent. Because potentially more than one RDMA Read Request can be outstanding at one time, the memory is modeled as a queue of bounded size. Some implementations may enable sharing of a single RDMA Read Request Queue across multiple Streams.

2.3. RNIC Interactions

With RNIC resources and interfaces defined, it is now possible to examine the interactions supported by the generic RNIC functional interfaces through each of the 3 interfaces: Privileged Control Interface, Privileged Data Interface, and Non-Privileged Data Interface. As mentioned previously in Section 2.1, Components, there are two data transfer mechanisms to be examined, Untagged Data Transfer and Tagged Data Transfer.

2.3.1. Privileged Control Interface Semantics

Generically, the Privileged Control Interface controls the RNIC's allocation, de-allocation, and initialization of RNIC global resources. This includes allocation and de-allocation of Stream Context Memory, Page Translation Tables, STag names, Completion Queues, RDMA Read Request Queues, and Asynchronous Event Queues. The Privileged Control Interface is also typically used for managing Non-Privileged ULP resources for the Non-Privileged ULP (and possibly for the Privileged ULP as well). This includes initialization and removal of Page Translation Table resources, and managing RNIC events (possibly managing all events for the Asynchronous Event Queue).

2.3.2. Non-Privileged Data Interface Semantics

The Non-Privileged Data Interface enables data transfer (transmit and receive) but does not allow initialization of the Page Translation Table resources. However, once the Page Translation Table resources have been initialized, the interface may enable a specific STag mapping to be enabled and disabled by directly communicating with the RNIC, or create an STag mapping for a buffer that has been previously initialized in the RNIC.
Top   ToC   RFC5042 - Page 14
   For RDMAP, ULP data can be sent by one of the previously described
   data transfer mechanisms: Untagged Data Transfer or Tagged Data
   Transfer.  Two RDMAP data transfer mechanisms are defined, one using
   Untagged Data Transfer (Send Type Messages), and one using Tagged
   Data Transfer (RDMA Read Responses and RDMA Writes).  ULP data
   reception through RDMAP can be done by receiving Send Type Messages
   into buffers that have been posted on the Receive Queue or Shared
   Receive Queue.  Thus, a Receive Queue or Shared Receive Queue can
   only be affected by Untagged Data Transfer.  Data reception can also
   be done by receiving RDMA Write and RDMA Read Response Messages into
   buffers that have previously been exposed for external write access
   through Advertisement of an STag (i.e., Tagged Data Transfer).
   Additionally, to cause ULP data to be pulled (read) across the
   network, RDMAP uses an RDMA Read Request Message (which only contains
   RDMAP control information necessary to access the ULP buffer to be
   read), to cause an RDMA Read Response Message to be generated that
   contains the ULP data.

   For DDP, transmitting data means sending DDP Tagged or Untagged
   Messages.  For data reception, DDP can receive Untagged Messages into
   buffers that have been posted on the Receive Queue or Shared Receive
   Queue.  It can also receive Tagged DDP Messages into buffers that
   have previously been exposed for external write access through
   Advertisement of an STag.

   Completion of data transmission or reception generally entails
   informing the ULP of the completed work by placing completion
   information on the Completion Queue.  For data reception, only an
   Untagged Data Transfer can cause completion information to be put in
   the Completion Queue.

2.3.3. Privileged Data Interface Semantics

The Privileged Data Interface semantics are a superset of the Non- Privileged Data Transfer semantics. The interface can do everything defined in the prior section, as well as create/destroy buffer to STag mappings directly. This generally entails initialization or clearing of Page Translation Table state in the RNIC.

2.3.4. Initialization of RNIC Data Structures for Data Transfer

Initialization of the mapping between an STag and a Data Buffer can be viewed in the abstract as two separate operations: a. Initialization of the allocated Page Translation Table entries with the location of the Data Buffer, and
Top   ToC   RFC5042 - Page 15
   b.  Initialization of a mapping from an allocated STag name to a set
       of Page Translation Table entry(s) or partial entries.

   Note that an implementation may not have a Page Translation Table
   (i.e., it may support a direct mapping between an STag and a Data
   Buffer).  If there is no Page Translation Table, then attacks based
   on changing its contents or exhausting its resources are not
   possible.

   Initialization of the contents of the Page Translation Table can be
   done by either the Privileged ULP or by the Privileged Resource
   Manager as a proxy for the Non-Privileged ULP.  By definition, the
   Non-Privileged ULP is not trusted to directly manipulate the Page
   Translation Table.  In general, the concern is that the Non-
   Privileged ULP may try to maliciously initialize the Page Translation
   Table to access a buffer for which it does not have permission.

   The exact resource allocation algorithm for the Page Translation
   Table is outside the scope of this document.  It may be allocated for
   a specific Data Buffer, or as a pooled resource to be consumed by
   potentially multiple Data Buffers, or be managed in some other way.
   This document attempts to abstract implementation dependent issues,
   and group them into higher level security issues, such as resource
   starvation and sharing of resources between Streams.

   The next issue is how an STag name is associated with a Data Buffer.
   For the case of an Untagged Data Buffer (i.e., Untagged Data
   Transfer), there is no wire visible mapping between an STag and the
   Data Buffer.  Note that there may, in fact, be an STag that
   represents the buffer, if an implementation chooses to internally
   represent Untagged Data Buffer using STags.  However, because the
   STag, by definition, is not visible on the wire, this is a local
   host, implementation-specific issue that should be analyzed in the
   context of a local host implementation-specific security analysis,
   and thus, is outside the scope of this document.

   For a Tagged Data Buffer (i.e., Tagged Data Transfer), either the
   Privileged ULP or the Privileged Resource Manager acting on behalf of
   the Non-Privileged ULP may initialize a mapping from an STag to a
   Page Translation Table, or may have the ability to simply
   enable/disable an existing STag to Page Translation Table mapping.
   There may also be multiple STag names that map to a specific group of
   Page Translation Table entries (or sub-entries).  Specific security
   issues with this level of flexibility are examined in Section 6.2.3,
   Multiple STags to Access the Same Buffer.
Top   ToC   RFC5042 - Page 16
   There are a variety of implementation options for initialization of
   Page Translation Table entries and mapping an STag to a group of Page
   Translation Table entries that have security repercussions.  This
   includes support for separation of mapping an STag versus mapping a
   set of Page Translation Table entries, and support for ULPs directly
   manipulating STag to Page Translation Table entry mappings (versus
   requiring access through the Privileged Resource Manager).

2.3.5. RNIC Data Transfer Interactions

RNIC Data Transfer operations can be subdivided into send and receive operations. For send operations, there is typically a queue that enables the ULP to post multiple operation requests to send data (referred to as the Send Queue). Depending upon the implementation, Data Buffers used in the operations may or may not have Page Translation Table entries associated with them, and may or may not have STags associated with them. Because this is a local host specific implementation issue rather than a protocol issue, the security analysis of threats and mitigations is left to the host implementation. Receive operations are different for Tagged Data Buffers versus Untagged Data Buffers (i.e., Tagged Data Transfer vs. Untagged Data Transfer). For Untagged Data Transfer, if more than one Untagged Data Buffer can be posted by the ULP, the DDP specification requires that they be consumed in sequential order (the RDMAP specification also requires this). Thus, the most general implementation is that there is a sequential queue of receive Untagged Data Buffers (Receive Queue). Some implementations may also support sharing of the sequential queue between multiple Streams. In this case, defining "sequential" becomes non-trivial - in general, the buffers for a single Stream are consumed from the queue in the order that they were placed on the queue, but there is no consumption order guarantee between Streams. For receive Tagged Data Transfer (i.e., Tagged Data Buffers, RDMA Write Buffers, or RDMA Read Buffers), at some time prior to data transfer, the mapping of the STag to specific Page Translation Table entries (if present) and the mapping from the Page Translation Table entries to the Data Buffer must have been initialized (see Section 2.3.4 for interaction details).
Top   ToC   RFC5042 - Page 17

3. Trust and Resource Sharing

It is assumed that, in general, the Local and Remote Peer are untrusted, and thus attacks by either should have mitigations in place. A separate, but related issue is resource sharing between multiple Streams. If local resources are not shared, the resources are dedicated on a per Stream basis. Resources are defined in Section 2.2, Resources. The advantage of not sharing resources between Streams is that it reduces the types of attacks that are possible. The disadvantage of not sharing resources is that ULPs might run out of resources. Thus, there can be a strong incentive for sharing resources, if the security issues associated with the sharing of resources can be mitigated. It is assumed in this document that the component that implements the mechanism to control sharing of the RNIC Engine resources is the Privileged Resource Manager. The RNIC Engine exposes its resources through the RNIC Interface to the Privileged Resource Manager. All Privileged and Non-Privileged ULPs request resources from the Resource Manager (note that by definition both the Non-Privileged and the Privileged application might try to greedily consume resources, thus creating a potential Denial of Service (DOS) attack). The Resource Manager implements resource management policies to ensure fair access to resources. The Resource Manager should be designed to take into account security attacks detailed in this document. Note that for some systems the Privileged Resource Manager may be implemented within the Privileged ULP. All Non-Privileged ULP interactions with the RNIC Engine that could affect other ULPs MUST be done using the Privileged Resource Manager as a proxy. All ULP resource allocation requests for scarce resources MUST also be done using a Privileged Resource Manager. The sharing of resources across Streams should be under the control of the ULP, both in terms of the trust model the ULP wishes to operate under, as well as the level of resource sharing the ULP wishes to give local processes. For more discussion on types of trust models that combine partial trust and sharing of resources, see Appendix C, Partial Trust Taxonomy. The Privileged Resource Manager MUST NOT assume that different Streams share Partial Mutual Trust unless there is a mechanism to ensure that the Streams do indeed share Partial Mutual Trust. This can be done in several ways, including explicit notification from the ULP that owns the Streams.
Top   ToC   RFC5042 - Page 18

4. Attacker Capabilities

An attacker's capabilities delimit the types of attacks that the attacker is able to launch. RDMAP and DDP require that the initial LLP Stream (and connection) be set up prior to transferring RDMAP/DDP Messages. This requires at least one round-trip handshake to occur. If the attacker is not the Remote Peer that created the initial connection, then the attacker's capabilities can be segmented into send only capabilities or send and receive capabilities. Attacking with send only capabilities requires the attacker to first guess the current LLP Stream parameters before they can attack RNIC resources (e.g., TCP sequence number). If this class of attacker also has receive capabilities and the ability to pose as the receiver to the sender and the sender to the receiver, they are typically referred to as a "man-in-the-middle" attacker [RFC3552]. A man-in-the-middle attacker has a much wider ability to attack RNIC resources. The breadth of attack is essentially the same as that of an attacking Remote Peer (i.e., the Remote Peer that set up the initial LLP Stream).


(page 18 continued on part 2)

Next Section