Tech-invite3GPPspaceIETFspace
959493929190898887868584838281807978777675747372717069686766656463626160595857565554535251504948474645444342414039383736353433323130292827262524232221201918171615141312111009080706050403020100
in Index   Prev   Next

RFC 5042

Direct Data Placement Protocol (DDP) / Remote Direct Memory Access Protocol (RDMAP) Security

Pages: 52
Proposed Standard
Updated by:  7146
Part 2 of 3 – Pages 18 to 38
First   Prev   Next

Top   ToC   RFC5042 - Page 18   prevText

5. Attacks That Can Be Mitigated with End-to-End Security

This section describes the RDMAP/DDP attacks where the only solution is to implement some form of end-to-end security. The analysis includes a detailed description of each attack, what is being attacked, and a description of the countermeasures that can be taken to thwart the attack. Some forms of attack involve modifying the RDMAP or DDP payload by a network-based attacker or involve monitoring the traffic to discover private information. An effective tool to ensure confidentiality is to encrypt the data stream through mechanisms, such as IPsec encryption. Additionally, authentication protocols, such as IPsec authentication, are an effective tool to ensure the remote entity is who they claim to be, as well as ensuring that the payload is unmodified as it traverses the network. Note that connection setup and tear down is presumed to be done in stream mode (i.e., no RDMA encapsulation of the payload), so there are no new attacks related to connection setup/tear down beyond what is already present in the LLP (e.g., TCP or SCTP). Note, however, that RDMAP/DDP parameters may be exchanged in stream mode, and if they are corrupted by an attacker unintended consequences will result. Therefore, any existing mitigations for LLP Spoofing, Tampering, Repudiation, Information Disclosure, Denial of Service, or
Top   ToC   RFC5042 - Page 19
   Elevation of Privilege continue to apply (and are out of scope of
   this document).  Thus, the analysis in this section focuses on
   attacks that are present, regardless of the LLP Stream type.

   Tampering is any modification of the legitimate traffic (machine
   internal or network).  Spoofing attack is a special case of tampering
   where the attacker falsifies an identity of the Remote Peer (identity
   can be an IP address, machine name, ULP level identity, etc.).

5.1. Spoofing

Spoofing attacks can be launched by the Remote Peer, or by a network-based attacker. A network-based spoofing attack applies to all Remote Peers. This section analyzes the various types of spoofing attacks applicable to RDMAP and DDP.

5.1.1. Impersonation

A network-based attacker can impersonate a legal RDMAP/DDP Peer (by spoofing a legal IP address). This can either be done as a blind attack (see [RFC3552]) or by establishing an RDMAP/DDP Stream with the victim. Because an RDMAP/DDP Stream requires an LLP Stream to be fully initialized (e.g., for [RFC793], it is in the ESTABLISHED state), existing transport layer protection mechanisms against blind attacks remain in place. For a blind attack to succeed, it requires the attacker to inject a valid transport layer segment (e.g., for TCP, it must match at least the 4-tuple as well as guess a sequence number within the window) while also guessing valid RDMAP or DDP parameters. There are many ways to attack the RDMAP/DDP protocol if the transport protocol is assumed to be vulnerable. For example, for Tagged Messages, this entails guessing the STag and TO values. If the attacker wishes to simply terminate the connection, it can do so by correctly guessing the transport and network layer values, and providing an invalid STag. Per the DDP specification, if an invalid STag is received, the Stream is torn down and the Remote Peer is notified with an error. If an attacker wishes to overwrite an Advertised Buffer, it must successfully guess the correct STag and TO. Given that the TO will often start at zero, this is straightforward. The value of the STag should be chosen at random, as discussed in Section 6.1.1, Using an STag on a Different Stream. For Untagged Messages, if the MSN is invalid then the connection may be torn down. If it is valid, then the receive buffers can be corrupted. End-to-end authentication (e.g., IPsec or ULP authentication) provides protection against either the blind attack or the connected attack.
Top   ToC   RFC5042 - Page 20

5.1.2. Stream Hijacking

Stream hijacking happens when a network-based attacker eavesdrops on the LLP connection through the Stream establishment phase, and waits until the authentication phase (if such a phase exists) is completed successfully. The attacker then spoofs the IP address and re-directs the Stream from the victim to its own machine. For example, an attacker can wait until an iSCSI authentication is completed successfully, and then hijack the iSCSI Stream. The best protection against this form of attack is end-to-end integrity protection and authentication, such as IPsec, to prevent spoofing. Another option is to provide a physically segregated network for security. Discussion of physical security is out of scope for this document. Because the connection and/or Stream itself is established by the LLP, some LLPs are more difficult to hijack than others. Please see the relevant LLP documentation on security issues around connection and/or Stream hijacking.

5.1.3. Man-in-the-Middle Attack

If a network-based attacker has the ability to delete or modify packets that will still be accepted by the LLP (e.g., TCP sequence number is correct), then the Stream can be exposed to a man-in-the- middle attack. One style of attack is for the man-in-the-middle to send Tagged Messages (either RDMAP or DDP). If it can discover a buffer that has been exposed for STag enabled access, then the man- in-the-middle can use an RDMA Read operation to read the contents of the associated Data Buffer, perform an RDMA Write Operation to modify the contents of the associated Data Buffer, or invalidate the STag to disable further access to the buffer. The best protection against this form of attack is end-to-end integrity protection and authentication, such as IPsec, to prevent spoofing or tampering. If authentication and integrity protections are not used, then physical protection must be employed to prevent man-in-the-middle attacks. Because the connection/Stream itself is established by the LLP, some LLPs are more exposed to man-in-the-middle attack than others. Please see the relevant LLP documentation on security issues around connection and/or Stream hijacking.
Top   ToC   RFC5042 - Page 21
   Another approach is to restrict access to only the local subnet/link,
   and provide some mechanism to limit access, such as physical security
   or 802.1.x.  This model is an extremely limited deployment scenario,
   and will not be further examined here.

5.2. Tampering - Network-Based Modification of Buffer Content

This is actually a man-in-the-middle attack, but only on the content of the buffer, as opposed to the man-in-the-middle attack presented above, where both the signaling and content can be modified. See Section 5.1.3, Man-in-the-Middle Attack.

5.3. Information Disclosure - Network-Based Eavesdropping

An attacker that is able to eavesdrop on the network can read the content of all read and write accesses to a Peer's buffers. To prevent information disclosure, the read/written data must be encrypted. See also Section 5.1.3, Man-in-the-Middle Attack. The encryption can be done either by the ULP, or by a protocol that can provide security services to RDMAP and DDP (e.g., IPsec).

5.4. Specific Requirements for Security Services

Generally speaking, Stream confidentiality protects against eavesdropping. Stream and/or session authentication and integrity protection is a counter measurement against various spoofing and tampering attacks. The effectiveness of authentication and integrity against a specific attack depends on whether the authentication is machine level authentication (such as IPsec), or ULP authentication.

5.4.1. Introduction to Security Options

The following security services can be applied to an RDMAP/DDP Stream: 1. Session confidentiality - Protects against eavesdropping (Section 5.3). 2. Per-packet data source authentication - Protects against the following spoofing attacks: network-based impersonation (Section 5.1.1) and Stream hijacking (Section 5.1.2). 3. Per-packet integrity - Protects against tampering done by network-based modification of buffer content (Section 5.2) and when combined with authentication, also protects against man-in- the-middle attacks (Section 5.1.3).
Top   ToC   RFC5042 - Page 22
   4.  Packet sequencing - protects against replay attacks, which is a
       special case of the above tampering attack.

   If an RDMAP/DDP Stream may be subject to impersonation attacks, or
   Stream hijacking attacks, it is recommended that the Stream be
   authenticated, integrity protected, and protected from replay
   attacks; it may use confidentiality protection to protect from
   eavesdropping (in case the RDMAP/DDP Stream traverses a public
   network).

   IPsec is a protocol suite that is used to secure communication at the
   network layer between two peers.  The IPsec protocol suite is
   specified within the IP Security Architecture [RFC2401], IKE
   [RFC2409], IPsec Authentication Header (AH) [RFC2402], and IPsec
   Encapsulating Security Payload (ESP) [RFC2406] documents.  IKE is the
   key management protocol, while AH and ESP are used to protect IP
   traffic.  Please see those RFCs for a complete description of the
   respective protocols.

   IPsec is capable of providing the above security services for IP and
   TCP traffic, respectively.  ULP protocols are able to provide only
   part of the above security services.

5.4.2. TLS Is Inappropriate for DDP/RDMAP Security

TLS [RFC4346] provides Stream authentication, integrity and confidentiality for TCP based ULPs. TLS supports one-way (server only) or mutual certificates based authentication. If TLS is layered underneath RDMAP, TLS's connection orientation makes TLS inappropriate for DDP/RDMA security. If a stream cipher or block cipher in CBC mode is used for bulk encryption, then a packet can be decrypted only after all the packets preceding it have already arrived. If TLS is used to protect DDP/RDMAP traffic, then TCP must gather all out-of-order packets before TLS can decrypt them. Only after this is done can RDMAP/DDP place them into the ULP buffer. Thus, one of the primary features of DDP/RDMAP - enabling implementations to have a flow-through architecture with little to no buffering - cannot be achieved if TLS is used to protect the data stream. If TLS is layered on top of RDMAP or DDP, TLS does not protect the RDMAP and/or DDP headers. Thus, a man-in-the-middle attack can still occur by modifying the RDMAP/DDP header to place the data into the wrong buffer, thus effectively corrupting the data stream. For these reasons, it is not RECOMMENDED that TLS be layered on top of RDMAP or DDP.
Top   ToC   RFC5042 - Page 23

5.4.3. DTLS and RDDP

DTLS [DTLS] provides security services for datagram protocols, including unreliable datagram protocols. These services include anti-replay based on a mechanism adapted from IPsec that is intended to operate on packets as they are received from the network. For these and other reasons, DTLS is best applied to RDDP by employing DTLS beneath TCP, yielding a layering of RDDP over TCP over DTLS over UDP/IP. Such a layering inserts DTLS at roughly the same level in the protocol stack as IPsec, making DTLS's security services an alternative to IPsec's services from an RDDP standpoint. For RDDP, IPsec is the better choice for a security framework, and hence is mandatory-to-implement (as specified elsewhere in this document). An important contributing factor to the specification of IPsec rather than DTLS is that the non-RDDP versions of two initial adopters of RDDP (iSCSI [iSCSI][iSER] and NFSv4 [NFSv4][NFSv4.1]) are compatible with IPsec but neither of these protocols currently uses either TLS or DTLS. For the specific case of iSCSI, IPsec is the basis for mandatory-to-implement security services [RFC3723]. Therefore, this document and the RDDP protocol specifications contain mandatory implementation requirements for IPsec rather than for DTLS.

5.4.4. ULPs That Provide Security

ULPs that provide integrated security but wish to leverage lower- layer protocol security, should be aware of security concerns around correlating a specific channel's security mechanisms to the authentication performed by the ULP. See [NFSv4CHANNEL] for additional information on a promising approach called "channel binding". From [NFSv4CHANNEL]: "The concept of channel bindings allows applications to prove that the end-points of two secure channels at different network layers are the same by binding authentication at one channel to the session protection at the other channel. The use of channel bindings allows applications to delegate session protection to lower layers, which may significantly improve performance for some applications."

5.4.5. Requirements for IPsec Encapsulation of DDP

The IP Storage working group has spent significant time and effort to define the normative IPsec requirements for IP Storage [RFC3723]. Portions of that specification are applicable to a wide variety of protocols, including the RDDP protocol suite. In order not to replicate this effort, an RNIC implementation MUST follow the requirements defined in RFC 3723, Section 2.3 and Section 5,
Top   ToC   RFC5042 - Page 24
   including the associated normative references for those sections.
   Note that this means that support for IPSEC ESP mode is normative.

   Additionally, since IPsec acceleration hardware may only be able to
   handle a limited number of active IKE Phase 2 SAs, Phase 2 delete
   messages may be sent for idle SAs as a means of keeping the number of
   active Phase 2 SAs to a minimum.  The receipt of an IKE Phase 2
   delete message MUST NOT be interpreted as a reason for tearing down a
   DDP/RDMA Stream.  Rather, it is preferable to leave the Stream up,
   and if additional traffic is sent on it, to bring up another IKE
   Phase 2 SA to protect it.  This avoids the potential for continually
   bringing Streams up and down.

   Note that there are serious security issues if IPsec is not
   implemented end-to-end.  For example, if IPsec is implemented as a
   tunnel in the middle of the network, any hosts between the Peer and
   the IPsec tunneling device can freely attack the unprotected Stream.

   The IPsec requirements for RDDP are based on the version of IPsec
   specified in RFC 2401 [RFC2401] and related RFCs, as profiled by RFC
   3723 [RFC3723], despite the existence of a newer version of IPsec
   specified in RFC 4301 [RFC4301] and related RFCs.  One of the
   important early applications of the RDDP protocols is their use with
   iSCSI [iSER]; RDDP's IPsec requirements follow those of IPsec in
   order to facilitate that usage by allowing a common profile of IPsec
   to be used with iSCSI and the RDDP protocols.  In the future, RFC
   3723 may be updated to the newer version of IPsec; the IPsec security
   requirements of any such update should apply uniformly to iSCSI and
   the RDDP protocols.

6. Attacks from Remote Peers

This section describes remote attacks that are possible against the RDMA system defined in Figure 1 - RDMA Security Model and the RNIC Engine resources defined in Section 2.2. The analysis includes a detailed description of each attack, what is being attacked, and a description of the countermeasures that can be taken to thwart the attack. The attacks are classified into five categories: Spoofing, Tampering, Information Disclosure, Denial of Service (DoS) attacks, and Elevation of Privileges. As mentioned previously, tampering is any modification of the legitimate traffic (machine internal or network). A spoofing attack is a special case of tampering where the attacker falsifies an identity of the Remote Peer (identity can be an IP address, machine name, ULP level identity, etc.).
Top   ToC   RFC5042 - Page 25

6.1. Spoofing

This section analyzes the various types of spoofing attacks applicable to RDMAP and DDP. Spoofing attacks can be launched by the Remote Peer or by a network-based attacker. For countermeasures against a network-based attacker, see Section 5, Attacks That Can Be Mitigated with End-to-End Security.

6.1.1. Using an STag on a Different Stream

One style of attack from the Remote Peer is for it to attempt to use STag values that it is not authorized to use. Note that if the Remote Peer sends an invalid STag to the Local Peer, per the DDP and RDMAP specifications, the Stream must be torn down. Thus, the threat exists if an STag has been enabled for Remote Access on one Stream and a Remote Peer is able to use it on an unrelated Stream. If the attack is successful, the attacker could potentially be able to either perform RDMA Read operations to read the contents of the associated Data Buffer, perform RDMA Write operations to modify the contents of the associated data buffer, or invalidate the STag to disable further access to the buffer. An attempt by a Remote Peer to access a buffer with an STag on a different Stream in the same Protection Domain may or may not be an attack, depending on whether resource sharing is intended (i.e., whether the Streams shared Partial Mutual Trust). For some ULPs, using an STag on multiple Streams within the same Protection Domain could be desired behavior. For other ULPs, attempting to use an STag on a different Stream could be considered an attack. Since this varies by ULP, a ULP typically would need to be able to control the scope of the STag. In the case where an implementation does not share resources between Streams (including STags), this attack can be defeated by assigning each Stream to a different Protection Domain. Before allowing remote access to the buffer, the Protection Domain of the Stream where the access attempt was made is matched against the Protection Domain of the STag. If the Protection Domains do not match, access to the buffer is denied, an error is generated, and the RDMAP Stream associated with the attacking Stream is terminated. For implementations that share resources between multiple Streams, it may not be practical to separate each Stream into its own Protection Domain. In this case, the ULP can still limit the scope of any of the STags to a single Stream (if it is enabling it for remote access). If the STag scope has been limited to a single Stream, any attempt to use that STag on a different Stream will result in an error, and the RDMAP Stream is terminated.
Top   ToC   RFC5042 - Page 26
   Thus, for implementations that do not share STags between Streams,
   each Stream MUST either be in a separate Protection Domain or the
   scope of an STag MUST be limited to a single Stream.

   An RNIC MUST ensure that a specific Stream in a specific Protection
   Domain cannot access an STag in a different Protection Domain.

   An RNIC MUST ensure that, if an STag is limited in scope to a single
   Stream, no other Stream can use the STag.

   An additional issue may be unintended sharing of STags (i.e., a bug
   in the ULP) or a bug in the Remote Peer that causes an off-by-one
   STag to be used.  For additional protection, an implementation should
   allocate STags in such a fashion that it is difficult to predict the
   next allocated STag number, and also ensure that STags are reused at
   as slow a rate as possible.  Any allocation method that would lead to
   intentional or unintentional reuse of an STag by the peer should be
   avoided (e.g., a method that always starts with a given STag and
   monotonically increases it for each new allocation, or a method that
   always uses the same STag for each operation).

6.2. Tampering

A Remote Peer or a network-based attacker can attempt to tamper with the contents of Data Buffers on a Local Peer that have been enabled for remote write access. The types of tampering attacks from a Remote Peer are outlined in the sections that follow. For countermeasures against a network-based attacker, see Section 5, Attacks That Can Be Mitigated with End-to-End Security.

6.2.1. Buffer Overrun - RDMA Write or Read Response

This attack is an attempt by the Remote Peer to perform an RDMA Write or RDMA Read Response to memory outside of the valid length range of the Data Buffer enabled for remote write access. This attack can occur even when no resources are shared across Streams. This issue can also arise if the ULP has a bug. The countermeasure for this type of attack must be in the RNIC implementation, leveraging the STag. When the local ULP specifies to the RNIC the base address and the umber of bytes in the buffer that it wishes to make accessible, the RNIC must ensure that the base and bounds check are applied to any access to the buffer referenced by the STag before the STag is enabled for access. When an RDMA data transfer operation (which includes an STag) arrives on a Stream, a base and bounds byte granularity access check must be performed to ensure that the operation accesses only memory locations within the buffer described by that STag.
Top   ToC   RFC5042 - Page 27
   Thus an RNIC implementation MUST ensure that a Remote Peer is not
   able to access memory outside of the buffer specified when the STag
   was enabled for remote access.

6.2.2. Modifying a Buffer after Indication

This attack can occur if a Remote Peer attempts to modify the contents of an STag referenced buffer by performing an RDMA Write or an RDMA Read Response after the Remote Peer has indicated to the Local Peer or local ULP (by a variety of means) that the STag Data Buffer contents are ready for use. This attack can occur even when no resources are shared across Streams. Note that a bug in a Remote Peer, or network-based tampering, could also result in this problem. For example, assume that the STag referenced buffer contains ULP control information as well as ULP payload, and the ULP sequence of operation is to first validate the control information and then perform operations on the control information. If the Remote Peer can perform an additional RDMA Write or RDMA Read Response (thus, changing the buffer) after the validity checks have been completed but before the control data is operated on, the Remote Peer could force the ULP down operational paths that were never intended. The local ULP can protect itself from this type of attack by revoking remote access when the original data transfer has completed and before it validates the contents of the buffer. The local ULP can do this either by explicitly revoking remote access rights for the STag when the Remote Peer indicates the operation has completed, or by checking to make sure the Remote Peer invalidated the STag through the RDMAP Remote Invalidate capability. If the Remote Peer did not invalidate the STag, the local ULP then explicitly revokes the STag remote access rights. (See Section 6.4.5, Remote Invalidate an STag Shared on Multiple Streams for a definition of Remote Invalidate.) The local ULP SHOULD follow the above procedure to protect the buffer before it validates the contents of the buffer (or uses the buffer in any way). An RNIC MUST ensure that network packets using the STag for a previously Advertised Buffer can no longer modify the buffer after the ULP revokes remote access rights for the specific STag.

6.2.3. Multiple STags to Access the Same Buffer

See Section 6.3.6 Using Multiple STags That Alias to the Same Buffer, for this analysis.
Top   ToC   RFC5042 - Page 28

6.3. Information Disclosure

The main potential source for information disclosure is through a local buffer that has been enabled for remote access. If the buffer can be probed by a Remote Peer on another Stream, then there is potential for information disclosure. The potential attacks that could result in unintended information disclosure and countermeasures are detailed in the following sections.

6.3.1. Probing Memory Outside of the Buffer Bounds

This is essentially the same attack as described in Section 6.2.1, Buffer Overrun - RDMA Write or Read Response, except that an RDMA Read Request is used to mount the attack. The same countermeasure applies.

6.3.2. Using RDMA Read to Access Stale Data

If a buffer is being used for some combination of reads and writes (either remote or local), and is exposed to a Remote Peer with at least remote read access rights before it is initialized with the correct data, there is a potential race condition where the Remote Peer can view the prior contents of the buffer. This becomes a security issue if the prior contents of the buffer were not intended to be shared with the Remote Peer. To eliminate this race condition, the local ULP SHOULD ensure that no stale data is contained in the buffer before remote read access rights are granted (this can be done by zeroing the contents of the memory, for example). This ensures that the Remote Peer cannot access the buffer until the stale data has been removed.

6.3.3. Accessing a Buffer after the Transfer

If the Remote Peer has remote read access to a buffer and, by some mechanism, tells the local ULP that the transfer has been completed, but the local ULP does not disable remote access to the buffer before modifying the data, it is possible for the Remote Peer to retrieve the new data. This is similar to the attack defined in Section 6.2.2, Modifying a Buffer after Indication. The same countermeasures apply. In addition, the local ULP SHOULD grant remote read access rights only for the amount of time needed to retrieve the data.
Top   ToC   RFC5042 - Page 29

6.3.4. Accessing Unintended Data with a Valid STag

If the ULP enables remote access to a buffer using an STag that references the entire buffer, but intends only a portion of the buffer to be accessed, it is possible for the Remote Peer to access the other parts of the buffer anyway. To prevent this attack, the ULP SHOULD set the base and bounds of the buffer when the STag is initialized to expose only the data to be retrieved.

6.3.5. RDMA Read into an RDMA Write Buffer

One form of disclosure can occur if the access rights on the buffer enabled remote read, when only remote write access was intended. If the buffer contained ULP data, or data from a transfer on an unrelated Stream, the Remote Peer could retrieve the data through an RDMA Read operation. Note that an RNIC implementation is not required to support STags that have both read and write access. The most obvious countermeasure for this attack is to not grant remote read access if the buffer is intended to be write-only. Then the Remote Peer would not be able to retrieve data associated with the buffer. An attempt to do so would result in an error and the RDMAP Stream associated with the Stream would be terminated. Thus, if a ULP only intends a buffer to be exposed for remote write access, it MUST set the access rights to the buffer to only enable remote write access. Note that this requirement is not meant to restrict the use of zero-length RDMA Reads. Zero-length RDMA Reads do not expose ULP data. Because they are intended to be used as a mechanism to ensure that all RDMA Writes have been received, and do not even require a valid STag, their use is permitted even if a buffer has only been enabled for write access.

6.3.6. Using Multiple STags That Alias to the Same Buffer

Multiple STags that alias to the same buffer at the same time can result in unintentional information disclosure if the STags are used by different, mutually untrusted Remote Peers. This model applies specifically to client/server communication, where the server is communicating with multiple clients, each of which do not mutually trust each other. If only read access is enabled, then the local ULP has complete control over information disclosure. Thus, a server that intended to expose the same data (i.e., buffer) to multiple clients by using multiple STags to the same buffer creates no new security issues
Top   ToC   RFC5042 - Page 30
   beyond what has already been described in this document.  Note that
   if the server did not intend to expose the same data to the clients,
   it should use separate buffers for each client (and separate STags).

   When one STag has remote read access enabled and a different STag has
   remote write access enabled to the same buffer, it is possible for
   one Remote Peer to view the contents that have been written by
   another Remote Peer.

   If both STags have remote write access enabled and the two Remote
   Peers do not mutually trust each other, it is possible for one Remote
   Peer to overwrite the contents that have been written by the other
   Remote Peer.

   Thus, a ULP with multiple Remote Peers that do not share Partial
   Mutual Trust MUST NOT grant write access to the same buffer through
   different STags.  A buffer should be exposed to only one untrusted
   Remote Peer at a time to ensure that no information disclosure or
   information tampering occurs between peers.

6.4. Denial of Service (DOS)

A DOS attack is one of the primary security risks of RDMAP. This is because RNIC resources are valuable and scarce, and many ULP environments require communication with untrusted Remote Peers. If the Remote Peer can be authenticated or the ULP payload encrypted, clearly, the DOS profile can be reduced. For the purposes of this analysis, it is assumed that the RNIC must be able to operate in untrusted environments, which are open to DOS-style attacks. Denial of service attacks against RNIC resources are not the typical unknown party spraying packets at a random host (such as a TCP SYN attack). Because the connection/Stream must be fully established (e.g., a 3-message transport layer handshake has occurred), the attacker must be able to both send and receive messages over that connection/Stream, or be able to guess a valid packet on an existing RDMAP Stream. This section outlines the potential attacks and the countermeasures available for dealing with each attack.

6.4.1. RNIC Resource Consumption

This section covers attacks that fall into the general category of a local ULP attempting to unfairly allocate scarce (i.e., bounded) RNIC resources. The local ULP may be attempting to allocate resources on its own behalf, or on behalf of a Remote Peer. Resources that fall into this category include Protection Domains, Stream Context Memory,
Top   ToC   RFC5042 - Page 31
   Translation and Protection Tables, and STag namespace.  These can be
   due to attacks by currently active local ULPs or ones that allocated
   resources earlier but are now idle.

   This type of attack can occur regardless of whether resources are
   shared across Streams.

   The allocation of all scarce resources MUST be placed under the
   control of a Privileged Resource Manager.  This allows the Privileged
   Resource Manager to:

   *   prevent a local ULP from allocating more than its fair share of
       resources.

   *   detect if a Remote Peer is attempting to launch a DOS attack by
       attempting to create an excessive number of Streams (with
       associated resources) and take corrective action (such as
       refusing the request or applying network layer filters against
       the Remote Peer).

   This analysis assumes that the Resource Manager is responsible for
   handing out Protection Domains, and that RNIC implementations will
   provide enough Protection Domains to allow the Resource Manager to be
   able to assign a unique Protection Domain for each unrelated,
   untrusted local ULP (for a bounded, reasonable number of local ULPs).
   This analysis further assumes that the Resource Manager implements
   policies to ensure that untrusted local ULPs are not able to consume
   all the Protection Domains through a DOS attack.  Note that
   Protection Domain consumption cannot result from a DOS attack
   launched by a Remote Peer, unless a local ULP is acting on the Remote
   Peer's behalf.

6.4.2. Resource Consumption by Idle ULPs

The simplest form of a DOS attack, given a fixed amount of resources, is for the Remote Peer to create an RDMAP Stream to a Local Peer, request dedicated resources, and then do no actual work. This allows the Remote Peer to be very light weight (i.e., only negotiate resources, but do no data transfer) and consumes a disproportionate amount of resources at the Local Peer. A general countermeasure for this style of attack is to monitor active RDMAP Streams and, if resources are getting low, to reap the resources from RDMAP Streams that are not transferring data and possibly terminate the Stream. This would presumably be under administrative control.
Top   ToC   RFC5042 - Page 32
   Refer to Section 6.4.1 for the analysis and countermeasures for this
   style of attack on the following RNIC resources: Stream Context
   Memory, Page Translation Tables, and STag namespace.

   Note that some RNIC resources are not at risk of this type of attack
   from a Remote Peer because an attack requires the Remote Peer to send
   messages in order to consume the resource.  Receive Data Buffers,
   Completion Queue, and RDMA Read Request Queue resources are examples.
   These resources are, however, at risk from a local ULP that attempts
   to allocate resources, then goes idle.  This could also be created if
   the ULP negotiates the resource levels with the Remote Peer, which
   causes the Local Peer to consume resources; however, the Remote Peer
   never sends data to consume them.  The general countermeasure
   described in this section can be used to free resources allocated by
   an idle Local Peer.

6.4.3. Resource Consumption by Active ULPs

This section describes DOS attacks from Local and Remote Peers that are actively exchanging messages. Attacks on each RDMA NIC resource are examined and specific countermeasures are identified. Note that attacks on Stream Context Memory, Page Translation Tables, and STag namespace are covered in Section 6.4.1, RNIC Resource Consumption, so they are not included here.
6.4.3.1. Multiple Streams Sharing Receive Buffers
The Remote Peer can attempt to consume more than its fair share of receive Data Buffers (i.e., Untagged Buffers for DDP or Send Type Messages for RDMAP) if receive buffers are shared across multiple Streams. If resources are not shared across multiple Streams, then this attack is not possible because the Remote Peer will not be able to consume more buffers than were allocated to the Stream. The worst case scenario is that the Remote Peer can consume more receive buffers than the local ULP allowed, resulting in no buffers being available, which could cause the Remote Peer's Stream to the Local Peer to be torn down, and all allocated resources to be released. If local receive Data Buffers are shared among multiple Streams, then the Remote Peer can attempt to consume more than its fair share of the receive buffers, causing a different Stream to be short of receive buffers, and thus, possibly causing the other Stream to be torn down. For example, if the Remote Peer sent enough one-byte Untagged Messages, they might be able to consume all locally shared, receive queue resources with little effort on their part.
Top   ToC   RFC5042 - Page 33
   One method the Local Peer could use is to recognize that a Remote
   Peer is attempting to use more than its fair share of resources and
   terminate the Stream (causing the allocated resources to be
   released).  However, if the Local Peer is sufficiently slow, it may
   be possible for the Remote Peer to still mount a denial of service
   attack.  One countermeasure that can protect against this attack is
   implementing a low-water notification.  The low-water notification
   alerts the ULP if the number of buffers in the receive queue is less
   than a threshold.

   If all the following conditions are true, then the Local Peer or
   local ULP can size the amount of local receive buffers posted on the
   receive queue to ensure a DOS attack can be stopped.

   *   A low-water notification is enabled, and

   *   The Local Peer is able to bound the amount of time that it takes
       to replenish receive buffers, and

   *   The Local Peer maintains statistics to determine which Remote
       Peer is consuming buffers.

   The above conditions enable the low-water notification to arrive
   before resources are depleted, and thus, the Local Peer or local ULP
   can take corrective action (e.g., terminate the Stream of the
   attacking Remote Peer).

   A different, but similar, attack is if the Remote Peer sends a
   significant number of out-of-order packets and the RNIC has the
   ability to use the ULP buffer (i.e., the Untagged Buffer for DDP or
   the buffer consumed by a Send Type Message for RDMAP) as a reassembly
   buffer.  In this case, the Remote Peer can consume a significant
   number of ULP buffers, but never send enough data to enable the ULP
   buffer to be completed to the ULP.

   An effective countermeasure is to create a high-water notification
   that alerts the ULP if there is more than a specified number of
   receive buffers "in process" (partially consumed, but not completed).
   The notification is generated when more than the specified number of
   buffers are in process simultaneously on a specific Stream (i.e.,
   packets have started to arrive for the buffer, but the buffer has not
   yet been delivered to the ULP).

   A different countermeasure is for the RNIC Engine to provide the
   capability to limit the Remote Peer's ability to consume receive
   buffers on a per Stream basis.  Unfortunately, this requires a large
   amount of state to be tracked in each RNIC on a per Stream basis.
Top   ToC   RFC5042 - Page 34
   Thus, if an RNIC Engine provides the ability to share receive buffers
   across multiple Streams, the combination of the RNIC Engine and the
   Privileged Resource Manager MUST be able to detect if the Remote Peer
   is attempting to consume more than its fair share of resources so
   that the Local Peer or local ULP can apply countermeasures to detect
   and prevent the attack.

6.4.3.2. Remote or Local Peer Attacking a Shared CQ
For an overview of the shared CQ attack model, see Section 7.1. The Remote Peer can attack a shared CQ by consuming more than its fair share of CQ entries by using one of the following methods: * The ULP protocol allows the Remote Peer to cause the local ULP to reserve a specified number of CQ entries, possibly leaving insufficient entries for other Streams that are sharing the CQ. * If the Remote Peer, Local Peer, or local ULP (or any combination) can attack the CQ by overwhelming the CQ with completions, then completion processing on other Streams sharing that Completion Queue can be affected (e.g., the Completion Queue overflows and stops functioning). The first method of attack can be avoided if the ULP does not allow a Remote Peer to reserve CQ entries, or if there is a trusted intermediary, such as a Privileged Resource Manager. Unfortunately, it is often unrealistic not to allow a Remote Peer to reserve CQ entries, particularly if the number of completion entries is dependent on other ULP negotiated parameters, such as the amount of buffering required by the ULP. Thus, an implementation MUST implement a Privileged Resource Manager to control the allocation of CQ entries. See Section 2.1, Components, for a definition of a Privileged Resource Manager. One way that a Local or Remote Peer can attempt to overwhelm a CQ with completions is by sending minimum length RDMAP/DDP Messages to cause as many completions (receive completions for the Remote Peer, send completions for the Local Peer) per second as possible. If it is the Remote Peer attacking, and we assume that the Local Peer's receive queue(s) do not run out of receive buffers (if they do, then this is a different attack, documented in Section 6.4.3.1 Multiple Streams Sharing Receive Buffers), then it might be possible for the Remote Peer to consume more than its fair share of Completion Queue entries. Depending upon the CQ implementation, this could either cause the CQ to overflow (if it is not large enough to handle all the completions generated) or for another Stream not to be able to generate CQ entries (if the RNIC had flow control on generation of CQ
Top   ToC   RFC5042 - Page 35
   entries into the CQ).  In either case, the CQ will stop functioning
   correctly, and any Streams expecting completions on the CQ will stop
   functioning.

   This attack can occur regardless of whether all the Streams
   associated with the CQ are in the same or different Protection
   Domains - the key issue is that the number of Completion Queue
   entries is less than the number of all outstanding operations that
   can cause a completion.

   The Local Peer can protect itself from this type of attack using
   either of the following methods:

   *   Size the CQ to the appropriate level, as specified below (note
       that if the CQ currently exists and needs to be resized, resizing
       the CQ is not required to succeed in all cases, so the CQ resize
       should be done before sizing the Send Queue and Receive Queue on
       the Stream), OR

   *   Grant fewer resources than the Remote Peer requested (not
       supplying the number of Receive Data Buffers requested).

   The proper sizing of the CQ is dependent on whether the local ULP(s)
   will post as many resources to the various queues as the size of the
   queue enables.  If the local ULP(s) can be trusted to post a number
   of resources that is smaller than the size of the specific resource's
   queue, then a correctly sized CQ means that the CQ is large enough to
   hold completion status for all the outstanding Data Buffers (both
   send and receive buffers), or:

            CQ_MIN_SIZE = SUM(MaxPostedOnEachRQ)
                          + SUM(MaxPostedOnEachSRQ)
                          + SUM(MaxPostedOnEachSQ)

   Where:

           MaxPostedOnEachRQ = the maximum number of requests that
                  can cause a completion that will be posted on a
                  specific Receive Queue.

           MaxPostedOnEachSRQ = the maximum number of requests that
                  can cause a completion that will be posted on a
                  specific Shared Receive Queue.

           MaxPostedOnEachSQ = the maximum number of requests that
                  can cause a completion that will be posted on a
                  specific Send Queue.
Top   ToC   RFC5042 - Page 36
   If the local ULP must be able to completely fill the queues, or
   cannot be trusted to observe a limit smaller than the queues, then
   the CQ must be sized to accommodate the maximum number of operations
   that it is possible to post at any one time.  Thus, the equation
   becomes:

            CQ_MIN_SIZE = SUM(SizeOfEachRQ)
                          + SUM(SizeOfEachSRQ)
                          + SUM(SizeOfEachSQ)

   Where:

          SizeOfEachRQ = the maximum number of requests that
                  can cause a completion that can ever be posted
                  on a specific Receive Queue.

          SizeOfEachSRQ = the maximum number of requests that
                  can cause a completion that can ever be posted
                  on a specific Shared Receive Queue.

          SizeOfEachSQ = the maximum number of requests that
                  can cause a completion that can ever be posted
                  on a specific Send Queue.

   MaxPosted*OnEach*Q and SizeOfEach*Q vary on a per Stream or per
   Shared Receive Queue basis.

   If the ULP is sharing a CQ across multiple Streams that do not share
   Partial Mutual Trust, then the ULP MUST implement a mechanism to
   ensure that the Completion Queue does not overflow.  Note that it is
   possible to share CQs even if the Remote Peers accessing the CQs are
   untrusted if either of the above two formulas are implemented.  If
   the ULP can be trusted not to post more than MaxPostedOnEachRQ,
   MaxPostedOnEachSRQ, and MaxPostedOnEachSQ, then the first formula
   applies.  If the ULP cannot be trusted to obey the limit, then the
   second formula applies.

6.4.3.3. Attacking the RDMA Read Request Queue
The RDMA Read Request Queue can be attacked if the Remote Peer sends more RDMA Read Requests than the depth of the RDMA Read Request Queue at the Local Peer. If the RDMA Read Request Queue is a shared resource, this could corrupt the queue. If the queue is not shared, then the worst case is that the current Stream is no longer functional (e.g., torn down). One approach to solving the shared RDMA Read Request Queue would be to create thresholds, similar to those described in Section 6.4.3.1, Multiple Streams Sharing Receive Buffers. A simpler approach is to not share RDMA Read Request Queue
Top   ToC   RFC5042 - Page 37
   resources among Streams or to enforce hard limits of consumption per
   Stream.  Thus, RDMA Read Request Queue resource consumption MUST be
   controlled by the Privileged Resource Manager such that RDMAP/DDP
   Streams that do not share Partial Mutual Trust do not share RDMA Read
   Request Queue resources.

   If the issue is a bug in the Remote Peer's implementation, but not a
   malicious attack, the issue can be solved by requiring the Remote
   Peer's RNIC to throttle RDMA Read Requests.  By properly configuring
   the Stream at the Remote Peer through a trusted agent, the RNIC can
   be made not to transmit RDMA Read Requests that exceed the depth of
   the RDMA Read Request Queue at the Local Peer.  If the Stream is
   correctly configured, and if the Remote Peer submits more requests
   than the Local Peer's RDMA Read Request Queue can handle, the
   requests would be queued at the Remote Peer's RNIC until previous
   requests complete.  If the Remote Peer's Stream is not configured
   correctly, the RDMAP Stream is terminated when more RDMA Read
   Requests arrive at the Local Peer than the Local Peer can handle
   (assuming that the prior paragraph's recommendation is implemented).
   Thus, an RNIC implementation SHOULD provide a mechanism to cap the
   number of outstanding RDMA Read Requests.  The configuration of this
   limit is outside the scope of this document.

6.4.4. Exercise of Non-Optimal Code Paths

Another form of a DOS attack is to attempt to exercise data paths that can consume a disproportionate amount of resources. An example might be if error cases are handled on a "slow path" (consuming either host or RNIC computational resources), and an attacker generates excessive numbers of errors in an attempt to consume these resources. Note that for most RDMAP or DDP errors, the attacking Stream will simply be torn down. Thus, for this form of attack to be effective, the Remote Peer needs to exercise data paths that do not cause the Stream to be torn down. If an RNIC implementation contains "slow paths" that do not result in the tear down of the Stream, it is recommended that an implementation provide the ability to detect the above condition and allow an administrator to act, including potentially administratively tearing down the RDMAP Stream associated with the Stream that is exercising data paths, which consume a disproportionate amount of resources.

6.4.5. Remote Invalidate an STag Shared on Multiple Streams

If a Local Peer has enabled an STag for remote access, the Remote Peer could attempt to remotely invalidate the STag by using the RDMAP Send with Invalidate or Send with SE and Invalidate Message. If the STag is only valid on the current Stream, then the only side effect
Top   ToC   RFC5042 - Page 38
   is that the Remote Peer can no longer use the STag; thus, there are
   no security issues.

   If the STag is valid across multiple Streams, then the Remote Peer
   can prevent other Streams from using that STag by using the Remote
   Invalidate functionality.

   Thus, if RDDP Streams do not share Partial Mutual Trust (i.e., the
   Remote Peer may attempt to remotely invalidate the STag prematurely),
   the ULP MUST NOT enable an STag that would be valid across multiple
   Streams.

6.4.6. Remote Peer Attacking an Unshared CQ

The Remote Peer can attack an unshared CQ if the Local Peer does not size the CQ correctly. For example, if the Local Peer enables the CQ to handle completions of received buffers, and the receive buffer queue is longer than the Completion Queue, then an overflow can potentially occur. The effect on the attacker's Stream is catastrophic. However, if an RNIC does not have the proper protections in place, then an attack to overflow the CQ can also cause corruption and/or termination of an unrelated Stream. Thus, an RNIC MUST ensure that if a CQ overflows, any Streams that do not use the CQ MUST remain unaffected.

6.5. Elevation of Privilege

The RDMAP/DDP Security Architecture explicitly differentiates between three levels of privilege: Non-Privileged, Privileged, and the Privileged Resource Manager. If a Non-Privileged ULP is able to elevate its privilege level to a Privileged ULP, then mapping a physical address list to an STag can provide local and remote access to any physical address location on the node. If a Privileged Mode ULP is able to promote itself to be a Resource Manager, then it is possible for it to perform denial of service type attacks where substantial amounts of local resources could be consumed. In general, elevation of privilege is a local implementation specific issue and is thus outside the scope of this document.


(page 38 continued on part 3)

Next Section