tech-invite   World Map     

IETF     RFCs     Groups     SIP     ABNFs    |    3GPP     Specs     Glossaries     Architecture     IMS     UICC    |    search     info

RFC 7325

 
 
 

MPLS Forwarding Compliance and Performance Requirements

Part 3 of 4, p. 30 to 48
Prev RFC Part       Next RFC Part

 


prevText      Top      Up      ToC       Page 30 
2.5.  MPLS-TP and UHP

   MPLS-TP introduces forwarding demands that will be extremely
   difficult to meet in a core network.  Most troublesome is the
   requirement for Ultimate Hop Popping (UHP), the opposite of
   Penultimate Hop Popping (PHP).  Using UHP opens the possibility of
   one or more MPLS pop operations plus an MPLS swap operation for each
   packet.  The potential for multiple lookups and multiple counter
   instances per packet exists.

   As networks grow and tunneling of LDP LSPs into RSVP-TE LSPs is used,
   and/or RSVP-TE hierarchy is used, the requirement to perform one or
   more MPLS pop operations plus an MPLS swap operation (and possibly a
   push or two) increases.  If MPLS-TP LM (link monitoring) OAM is
   enabled at each layer, then a packet and byte count MUST be
   maintained for each pop and swap operation so as to offer OAM for
   each layer.

2.6.  Local Delivery of Packets

   There are a number of situations in which packets are destined to a
   local address or where a return packet must be generated.  There is a
   need to mitigate the potential for outage as a result of either
   attacks on network infrastructure, or in some cases unintentional
   misconfiguration resulting in processor overload.  Some hardware
   assistance is needed for all traffic destined to the general-purpose
   CPU that is used in processing of the MPLS control protocol or the
   network management protocol and in most cases to other general-
   purpose CPUs residing on an LSR.  This is due to the ease of
   overwhelming such a processor with traffic arriving on LSR high-speed
   interfaces, whether the traffic is malicious or not.

Top      Up      ToC       Page 31 
   Denial of service (DoS) protection is an area requiring hardware
   support that is often overlooked or inadequately considered.
   Hardware assists are also needed for OAM, particularly the more
   demanding MPLS-TP OAM.

2.6.1.  DoS Protection

   Modern equipment supports a number of control-plane and management-
   plane protocols.  Generally, no single means of protecting network
   equipment from DoS attacks is sufficient, particularly for high-speed
   interfaces.  This problem is not specific to MPLS but is a topic that
   cannot be ignored when implementing or evaluating MPLS
   implementations.

   Two types of protections are often cited as the primary means of
   protecting against attacks of all kinds.

   Isolated Control/Management Traffic
       Control and management traffic can be carried out-of-band (OOB),
       meaning not intermixed with payload.  For MPLS, use of G-ACh and
       GAL to carry control and management traffic provides a means of
       isolation from potentially malicious payloads.  Used alone, the
       compromise of a single node, including a small computer at a
       network operations center, could compromise an entire network.
       Implementations that send all G-ACh/GAL traffic directly to a
       routing engine CPU are subject to DoS attack as a result of such
       a compromise.

   Cryptographic Authentication
       Cryptographic authentication can very effectively prevent
       malicious injection of control or management traffic.
       Cryptographic authentication can in some circumstances be subject
       to DoS attack by overwhelming the capacity of the decryption with
       a high volume of malicious traffic.  For very-low-speed
       interfaces, cryptographic authentication can be performed by the
       general-purpose CPU used as a routing engine.  For all other
       cases, cryptographic hardware may be needed.  For very-high-speed
       interfaces, even cryptographic hardware can be overwhelmed.

   Some control and management protocols are often carried with payload
   traffic.  This is commonly the case with BGP, T-LDP, and SNMP.  It is
   often the case with RSVP-TE.  Even when carried over G-ACh/GAL,
   additional measures can reduce the potential for a minor breach to be
   leveraged to a full network attack.

   Some of the additional protections are supported by hardware packet
   filtering.

Top      Up      ToC       Page 32 
   GTSM
       [RFC5082] defines a mechanism that uses the IPv4 TTL or IPv6 Hop
       Limit fields to ensure control traffic that can only originate
       from an immediate neighbor is not forged and is not originating
       from a distant source.  GTSM can be applied to many control
       protocols that are routable, for example, LDP [RFC6720].

   IP Filtering
       At the very minimum, packet filtering plus classification and use
       of multiple queues supporting rate limiting is needed for traffic
       that could potentially be sent to a general-purpose CPU used as a
       routing engine.  The first level of filtering only allows
       connections to be initiated from specific IP prefixes to specific
       destination ports and then preferably passes traffic directly to
       a cryptographic engine and/or rate limits.  The second level of
       filtering passes connected traffic, such as TCP connections
       having received at least one authenticated SYN or having been
       locally initiated.  The second level of filtering only passes
       traffic to specific address and port pairs to be checked for
       cryptographic authentication.

   The cryptographic authentication is generally the last resort in DoS
   attack mitigation.  If a packet must be first sent to a general-
   purpose CPU, then sent to a cryptographic engine, a DoS attack is
   possible on high-speed interfaces.  Only where hardware can fully
   process a cryptographic authentication without intervention from a
   general-purpose CPU (to find the authentication field and to identify
   the portion of packet to run the cryptographic algorithm over) is
   cryptographic authentication beneficial in protecting against DoS
   attacks.

   For chips supporting multiple 100 Gb/s interfaces, only a very large
   number of parallel cryptographic engines can provide the processing
   capacity to handle a large-scale DoS or distributed DoS (DDoS)
   attack.  For many forwarding chips, this much processing power
   requires significant chip real estate and power, and therefore
   reduces system space and power density.  For this reason,
   cryptographic authentication is not considered a viable first line of
   defense.

   For some networks, the first line of defense is some means of
   supporting OOB control and management traffic.  In the past, this OOB
   channel might make use of overhead bits in SONET or OTN or a
   dedicated DWDM wavelength.  G-ACh and GAL provide an alternative OOB
   mechanism that is independent of underlying layers.  In other
   networks, including most IP/MPLS networks, perimeter filtering serves
   a similar purpose, though it is less effective without extreme
   vigilance.

Top      Up      ToC       Page 33 
   A second line of defense is filtering, including GTSM.  For protocols
   such as EBGP, GTSM and other filtering are often the first line of
   defense.  Cryptographic authentication is usually the last line of
   defense and insufficient by itself to mitigate DoS or DDoS attacks.

2.6.2.  MPLS OAM

   [RFC4377] defines requirements for MPLS OAM that predate MPLS-TP.
   [RFC4379] defines what is commonly referred to as LSP Ping and LSP
   Traceroute.  [RFC4379] is updated by [RFC6424], which supports MPLS
   tunnels and stitched LSP and P2MP LSP.  [RFC4379] is updated by
   [RFC6425], which supports P2MP LSP.  [RFC4379] is updated by
   [RFC6426] to support MPLS-TP connectivity verification (CV) and route
   tracing.

   [RFC4950] extends the ICMP format to support TTL expiration that may
   occur when using IP Traceroute within an MPLS tunnel.  The ICMP
   message generation can be implemented in forwarding hardware, but if
   the ICMP packets are sent to a general-purpose CPU, this packet flow
   must be rate limited to avoid a potential DoS attack.

   [RFC5880] defines Bidirectional Forwarding Detection (BFD), a
   protocol intended to detect faults in the bidirectional path between
   two forwarding engines.  [RFC5884] and [RFC5885] define BFD for MPLS.
   BFD can provide failure detection on any kind of path between
   systems, including direct physical links, virtual circuits, tunnels,
   MPLS Label Switched Paths (LSPs), multihop routed paths, and
   unidirectional links as long as there is some return path.

   The processing requirements for BFD are less than for LSP Ping,
   making BFD somewhat better suited for relatively high-rate proactive
   monitoring.  BFD does not verify that the data plane matches the
   control plane, where LSP Ping does.  LSP Ping is somewhat better
   suited for on-demand monitoring including relatively low-rate
   periodic verification of the data plane and as a diagnostic tool.

   Hardware assistance is often provided for BFD response where BFD
   setup or parameter change is not involved and may be necessary for
   relatively high-rate proactive monitoring.  If both BFD and LSP Ping
   are recognized in filtering prior to passing traffic to a general-
   purpose CPU, appropriate DoS protection can be applied (see
   Section 2.6.1).  Failure to recognize BFD and LSP Ping and at least
   to rate limit creates the potential for misconfiguration to cause
   outages rather than cause errors in the misconfigured OAM.

Top      Up      ToC       Page 34 
2.6.3.  Pseudowire OAM

   Pseudowire OAM makes use of the control channel provided by Virtual
   Circuit Connectivity Verification (VCCV) [RFC5085].  VCCV makes use
   of the pseudowire Control Word.  BFD support over VCCV is defined by
   [RFC5885].  [RFC5885] is updated by [RFC6478] in support of static
   pseudowires.  [RFC4379] is updated by [RFC6829] to support LSP Ping
   for Pseudowire FEC advertised over IPv6.

   G-ACh/GAL (defined in [RFC5586]) is the preferred MPLS-TP OAM control
   channel and applies to any MPLS-TP endpoints, including pseudowire.
   See Section 2.6.4 for an overview of MPLS-TP OAM.

2.6.4.  MPLS-TP OAM

   [RFC6669] summarizes the MPLS-TP OAM toolset, the set of protocols
   supporting the MPLS-TP OAM requirements specified in [RFC5860] and
   supported by the MPLS-TP OAM framework defined in [RFC6371].

   The MPLS-TP OAM toolset includes:

   CC-CV
       [RFC6428] defines BFD extensions to support proactive Continuity
       Check and Connectivity Verification (CC-CV) applications.
       [RFC6426] provides LSP Ping extensions that are used to implement
       on-demand connectivity verification.

   RDI
       Remote Defect Indication (RDI) is triggered by failure of
       proactive CC-CV, which is BFD based.  For fast RDI, RDI SHOULD be
       initiated and handled by hardware if BFD is handled in forwarding
       hardware.  [RFC6428] provides an extension for BFD that includes
       the RDI in the BFD format and a specification of how this
       indication is to be used.

   Route Tracing
       [RFC6426] specifies that the LSP Ping enhancements for MPLS-TP
       on-demand connectivity verification include information on the
       use of LSP Ping for route tracing of an MPLS-TP path.

   Alarm Reporting
       [RFC6427] describes the details of a new protocol supporting
       Alarm Indication Signal (AIS), Link Down Indication (LDI), and
       fault management.  Failure to support this functionality in
       forwarding hardware can potentially result in failure to meet
       protection recovery time requirements; therefore, support of this
       functionality is strongly recommended.

Top      Up      ToC       Page 35 
   Lock Instruct
       Lock instruct is initiated on demand and therefore need not be
       implemented in forwarding hardware.  [RFC6435] defines a lock
       instruct protocol.

   Lock Reporting
       [RFC6427] covers lock reporting.  Lock reporting need not be
       implemented in forwarding hardware.

   Diagnostic
       [RFC6435] defines protocol support for loopback.  Loopback
       initiation is on demand and therefore need not be implemented in
       forwarding hardware.  Loopback of packet traffic SHOULD be
       implemented in forwarding hardware on high-speed interfaces.

   Packet Loss and Delay Measurement
       [RFC6374] and [RFC6375] define a protocol and profile for Packet
       Loss Measurement (LM) and Delay Measurement (DM).  LM requires a
       very accurate capture and insertion of packet and byte counters
       when a packet is transmitted and capture of packet and byte
       counters when a packet is received.  This capture and insertion
       MUST be implemented in forwarding hardware for LM OAM if high
       accuracy is needed.  DM requires very accurate capture and
       insertion of a timestamp on transmission and capture of timestamp
       when a packet is received.  This timestamp capture and insertion
       MUST be implemented in forwarding hardware for DM OAM if high
       accuracy is needed.

   See Section 2.6.2 for discussion of hardware support necessary for
   BFD and LSP Ping.

   CC-CV and alarm reporting is tied to protection and therefore SHOULD
   be supported in forwarding hardware in order to provide protection
   for a large number of affected LSPs within target response intervals.
   When using MPLS-TP, since CC-CV is supported by BFD, providing
   hardware assistance for BFD processing helps ensure that protection
   recovery time requirements can be met even for faults affecting a
   large number of LSPs.

   MPLS-TP Protection State Coordination (PSC) is defined by [RFC6378]
   and updated by [RFC7324], which corrects some errors in [RFC6378].

2.6.5.  MPLS OAM and Layer 2 OAM Interworking

   [RFC6670] provides the reasons for selecting a single MPLS-TP OAM
   solution and examines the consequences were ITU-T to develop a second
   OAM solution that is based on Ethernet encodings and mechanisms.

Top      Up      ToC       Page 36 
   [RFC6310] and [RFC7023] specify the mapping of defect states between
   many types of hardware Attachment Circuits (ACs) and associated
   pseudowires (PWs).  This functionality SHOULD be supported in
   forwarding hardware.

   It is beneficial if an MPLS OAM implementation can interwork with the
   underlying server layer and provide a means to interwork with a
   client layer.  For example, [RFC6427] specifies an inter-layer
   propagation of AIS and LDI from MPLS server layer to client MPLS
   layers.  Where the server layer uses a Layer 2 mechanism, such as
   Ethernet, PPP over SONET/SDH, or GFP over OTN, interwork among layers
   is also beneficial.  For high-speed interfaces, supporting this
   interworking in forwarding hardware helps ensure that protection
   based on this interworking can meet recovery time requirements even
   for faults affecting a large number of LSPs.

2.6.6.  Extent of OAM Support by Hardware

   Where certain requirements must be met, such as relatively high CC-CV
   rates and a large number of interfaces, or strict protection recovery
   time requirements and a moderate number of affected LSPs, some OAM
   functionality must be supported by forwarding hardware.  In other
   cases, such as highly accurate LM and DM OAM or strict protection
   recovery time requirements with a large number of affected LSPs, OAM
   functionality must be entirely implemented in forwarding hardware.

   Where possible, implementation in forwarding hardware should be in
   programmable hardware such that if standards are later changed or
   extended these changes are likely to be accommodated with hardware
   reprogramming rather than replacement.

   For some functionality, there is a strong case for an implementation
   in dedicated forwarding hardware.  Examples include packet and byte
   counters needed for LM OAM as well as needed for management
   protocols.  Similarly, the capture and insertion of packet and byte
   counts or timestamps needed for transmitted LM or DM or time
   synchronization packets MUST be implemented in forwarding hardware if
   high accuracy is required.

   For some functions, there is a strong case to provide limited support
   in forwarding hardware, but an external general-purpose processor may
   be used if performance criteria can be met.  For example, origination
   of RDI triggered by CC-CV, response to RDI, and Protection State
   Coordination (PSC) functionality may be supported by hardware, but
   expansion to a large number of client LSPs and transmission of AIS or
   RDI to the client LSPs may occur in a general-purpose processor.
   Some forwarding hardware supports one or more on-chip general-purpose
   processors that may be well suited for such a role.  [RFC7324], being

Top      Up      ToC       Page 37 
   a very recent document that affects a protection state machine that
   requires hardware support, underscores the importance of having a
   degree of programmability in forwarding hardware.

   The customer (system supplier or provider) should not dictate design,
   but should independently validate target functionality and
   performance.  However, it is not uncommon for service providers and
   system implementers to insist on reviewing design details (under a
   non-disclosure agreement) due to past experiences with suppliers and
   to reject suppliers who are unwilling to provide details.

2.6.7.  Support for IPFIX in Hardware

   The IPFIX architecture is defined by [RFC5470].  IPFIX supports per-
   flow statistics.  IPFIX information elements (IEs) are defined in
   [RFC7012] and include IEs for MPLS.

   The forwarding chips used in core routers are not optimized for high-
   touch applications like IPFIX.  Often, support for IPFIX in core
   routers is limited to optional IPFIX metering, which involves a
   1-in-N packet sampling, limited filtering support, and redirection to
   either an internal CPU or an external interface.  The CPU or device
   at the other end of the external interface then implements the full
   IPFIX filtering and IPFIX collector functionality.

   LSRs that are intended to be deployed further from the core may
   support lower-capacity interfaces but support higher-touch
   applications on the forwarding hardware and may provide dedicated
   hardware to support a greater subset of IPFIX functionality before
   handing off to a general-purpose CPU.  In some cases, far from the
   core the entire IPFIX functionality up to and including the collector
   may be implemented in hardware and firmware in the forwarding
   silicon.  It is also worth noting that at lower speeds a general-
   purpose CPU may become adequate to implement IPFIX, particularly if
   metering is used.

2.7.  Number and Size of Flows

   Service provider networks may carry up to hundreds of millions of
   flows on 10 Gb/s links.  Most flows are very short lived, many under
   a second.  A subset of the flows are low capacity and somewhat long
   lived.  When Internet traffic dominates capacity, a very small subset
   of flows are high capacity and/or very long lived.

Top      Up      ToC       Page 38 
   Two types of limitations with regard to number and size of flows have
   been observed.

   1.  Some hardware cannot handle some high-capacity flows because of
       internal paths that are limited, such as per-packet backplane
       paths or paths internal or external to chips such as buffer
       memory paths.  Such designs can handle aggregates of smaller
       flows.  Some hardware with acknowledged limitations has been
       successfully deployed but may be increasingly problematic if the
       capacity of large microflows in deployed networks continues to
       grow.

   2.  Some hardware approaches cannot handle a large number of flows,
       or a large number of large flows, due to attempting to count per
       flow, rather than deal with aggregates of flows.  Hash techniques
       scale with regard to number of flows due to a fixed hash size
       with many flows falling into the same hash bucket.  Techniques
       that identify individual flows have been implemented but have
       never successfully deployed for Internet traffic.

3.  Questions for Suppliers

   The following questions should be asked of a supplier.  These
   questions are grouped into broad categories and are intended to be
   open-ended questions to the supplier.  The tests in Section 4 are
   intended to verify whether the supplier disclosed any compliance or
   performance limitations completely and accurately.

3.1.  Basic Compliance

   Q#1   Can the implementation forward packets with an arbitrarily
         large stack depth?  What limitations exist, and under what
         circumstances do further limitations come into play (such as
         high packet rate or specific features enabled or specific types
         of packet processing)?  See Section 2.1.

   Q#2   Is the entire set of basic MPLS functionality described in
         Section 2.1 supported?

   Q#3   Is the set of MPLS special-purpose labels handled correctly and
         with adequate performance?  Are extended special-purpose labels
         handled correctly and with adequate performance?  See
         Section 2.1.1.

   Q#4   Are mappings of label value and TC to PHB handled correctly,
         including L-LSP mappings (RFC 3270) and CT mappings (RFC 4124)
         to PHB?  See Section 2.1.2.

Top      Up      ToC       Page 39 
   Q#5   Is time synchronization adequately supported in forwarding
         hardware?

         A.  Are both PTP and NTP formats supported?

         B.  Is the accuracy of timestamp insertion and incoming
             stamping sufficient?

         See Section 2.1.3.

   Q#6   Is link bundling supported?

         A.  Can an LSP be pinned to specific components?

         B.  Is the "all-ones" component link supported?

         See Section 2.1.5.

   Q#7   Is MPLS hierarchy supported?

         A.  Are both PHP and UHP supported?  What limitations exist on
             the number of pop operations with UHP?

         B.  Are the pipe, short-pipe, and uniform models supported?
             Are TTL and TC values updated correctly at egress where
             applicable?

         See Section 2.1.6 regarding MPLS hierarchy.  See [RFC3443]
         regarding PHP, UHP, and pipe, short-pipe, and uniform models.

   Q#8   Is FRR supported?

         A.  Are both "One-to-One Backup" and "Facility Backup"
             supported?

         B.  What forms of IP/LDP FRR are supported?

         C.  How quickly does protection recovery occur?

         D.  Does protection recovery speed increase when a fault
             affects a large number of protected LSPs?  And if so, by
             how much?

         See Section 2.1.7.

   Q#9   Are pseudowire Sequence Numbers handled correctly?  See
         Section 2.1.8.1.

Top      Up      ToC       Page 40 
   Q#10  Is VPN LER functionality handled correctly and without
         performance issues?  See Section 2.1.9.

   Q#11  Is MPLS multicast (P2MP and MP2MP) handled correctly?

         A.  Are packets dropped on uncongested outputs if some outputs
             are congested?

         B.  Is performance limited in high-fanout situations?

         See Section 2.2.

3.2.  Basic Performance

   Q#12 Can very small packets be forwarded at full line rate on all
        interfaces indefinitely?  What limitations exist?  And under
        what circumstances do further limitations come into play (such
        as specific features enabled or specific types of packet
        processing)?

   Q#13 Customers must decide whether to relax the prior requirement and
        to what extent.  If the answer to the prior question indicates
        that limitations exist, then:

        A.  What is the smallest packet size where full line rate
            forwarding can be supported?

        B.  What is the longest burst of full-rate small packets that
            can be supported?

        Specify circumstances (such as specific features enabled or
        specific types of packet processing) that often impact these
        rates and burst sizes.

   Q#14 How many pop operations can be supported along with a swap
        operation at full line rate while maintaining per-LSP packet and
        byte counts for each pop and swap?  This requirement is
        particularly relevant for MPLS-TP.

   Q#15 How many label push operations can be supported.  While this
        limitation is rarely an issue, it applies to both PHP and UHP,
        unlike the pop limit that applies to UHP.

   Q#16 For a worst case where all packets arrive on one LSP, what is
        the counter overflow time?  Are any means provided to avoid
        polling all counters at short intervals?  This applies to both
        MPLS and MPLS-TP.

Top      Up      ToC       Page 41 
3.3.  Multipath Capabilities and Performance

   Multipath capabilities and performance do not apply to MPLS-TP, but
   they apply to MPLS and apply if MPLS-TP is carried in MPLS.

   Q#17 How are large microflows accommodated?  Is there active
        management of the hash space mapping to output ports?  See
        Section 2.4.2.

   Q#18 How many MPLS labels can be included in a hash based on the MPLS
        label stack?

   Q#19 Is packet rate performance decreased beyond some number of
        labels?

   Q#20 Can the IP header and payload information below the MPLS stack
        be used in the hash?  If so, which IP fields, payload types, and
        payload fields are supported?

   Q#21 At what maximum MPLS label stack depth can Bottom of Stack and
        an IP header appear without impacting packet rate performance?

   Q#22 Are special-purpose labels excluded from the label stack hash?
        Are extended special-purpose labels excluded from the label
        stack hash?  See Section 2.4.5.1.

   Q#23 How is multipath performance affected by high-capacity flows, an
        extremely large number of flows, or very short-lived flows?  See
        Section 2.7.

3.4.  Pseudowire Capabilities and Performance

   Q#24 Is the pseudowire Control Word supported?

   Q#25 What is the maximum rate of pseudowire encapsulation and
        decapsulation?  Apply the same questions as in Section 3.2
        ("Basic Performance") for any packet-based pseudowire, such as
        IP VPN or Ethernet.

   Q#26 Does inclusion of a pseudowire Control Word impact performance?

   Q#27 Are Flow Labels supported?

   Q#28 If so, what fields are hashed on for the Flow Label for
        different types of pseudowires?

   Q#29 Does inclusion of a Flow Label impact performance?

Top      Up      ToC       Page 42 
3.5.  Entropy Label Support and Performance

   Q#30 Can an Entropy Label be added when acting as an ingress LER, and
        can it be removed when acting as an egress LER?

   Q#31 If an Entropy Label can be added, what fields are hashed on for
        the Entropy Label?

   Q#32 Does adding or removing an Entropy Label impact packet rate
        performance?

   Q#33 Can an Entropy Label be detected in the label stack, used in the
        hash, and properly terminate the search for further information
        to hash on?

   Q#34 Does using an Entropy Label have any negative impact on
        performance?  It should have no impact or a positive impact.

3.6.  DoS Protection

   Q#35 For each control- and management-plane protocol in use, what
        measures are taken to provide DoS attack hardening?

   Q#36 Have DoS attack tests been performed?

   Q#37 Can compromise of an internal computer on a management subnet be
        leveraged for any form of attack including DoS attack?

3.7.  OAM Capabilities and Performance

   Q#38 What OAM proactive and on-demand mechanisms are supported?

   Q#39 What performance limits exist under high proactive monitoring
        rates?

   Q#40 Can excessively high proactive monitoring rates impact control-
        plane performance or cause control-plane instability?

   Q#41 Ask the prior questions for each of the following.

        A.  MPLS OAM

        B.  Pseudowire OAM

        C.  MPLS-TP OAM

Top      Up      ToC       Page 43 
        D.  Layer 2 OAM Interworking

        See Section 2.6.

4.  Forwarding Compliance and Performance Testing

   Packet rate performance of equipment supporting a large number of 10
   Gb/s or 100 Gb/s links is not possible using desktop computers or
   workstations.  The use of high-end workstations as a source of test
   traffic was barely viable 20 years ago but is no longer at all
   viable.  Though custom microcode has been used on specialized router
   forwarding cards to serve the purpose of generating test traffic and
   measuring it, for the most part, performance testing will require
   specialized test equipment.  There are multiple sources of suitable
   equipment.

   The set of tests listed here do not correspond one-to-one to the set
   of questions in Section 3.  The same categorization is used, and
   these tests largely serve to validate answers provided to the prior
   questions.  They can also provide answers where a supplier is
   unwilling to disclose compliance or performance.

   Performance testing is the domain of the IETF Benchmark Methodology
   Working Group (BMWG).  Below are brief descriptions of conformance
   and performance tests.  Some very basic tests, specified in
   [RFC5695], partially cover only the basic performance test T#3.

   The following tests should be performed by the systems designer or
   deployer; or, if it is not practical for the potential customer to
   perform the tests directly, they may be performed by the supplier on
   their behalf.  These tests are grouped into broad categories.

   The tests in Section 4.1 should be repeated under various conditions
   to retest basic performance when critical capabilities are enabled.
   Complete repetition of the performance tests enabling each capability
   and combinations of capabilities would be very time intensive;
   therefore, a reduced set of performance tests can be used to gauge
   the impact of enabling specific capabilities.

4.1.  Basic Compliance

   T#1  Test forwarding at a high rate for packets with varying number
        of label entries.  While packets with more than a dozen label
        entries are unlikely to be used in any practical scenario today,
        it is useful to know if limitations exists.

Top      Up      ToC       Page 44 
   T#2  For each of the questions listed under "Basic Compliance" in
        Section 3, verify the claimed compliance.  For any functionality
        considered critical to a deployment, the applicable performance
        using each capability under load should be verified in addition
        to basic compliance.

4.2.  Basic Performance

   T#3  Test packet forwarding at full line rate with small packets.
        See [RFC5695].  The most likely case to fail is the smallest
        packet size.  Also, test with packet sizes in 4-byte increments
        ranging from payload sizes of 40 to 128 bytes.

   T#4  If the prior tests did not succeed for all packet sizes, then
        perform the following tests.

        A.  Increase the packet size by 4 bytes until a size is found
            that can be forwarded at full rate.

        B.  Inject bursts of consecutive small packets into a stream of
            larger packets.  Allow some time for recovery between
            bursts.  Increase the number of packets in the burst until
            packets are dropped.

   T#5  Send test traffic where a swap operation is required.  Also set
        up multiple LSPs carried over other LSPs where the device under
        test (DUT) is the egress of these LSPs.  Create test packets
        such that the swap operation is performed after pop operations,
        increasing the number of pop operations until forwarding of
        small packets at full line rate can no longer be supported.
        Also, check to see how many pop operations can be supported
        before the full set of counters can no longer be maintained.
        This requirement is particularly relevant for MPLS-TP.

   T#6  Send all traffic on one LSP and see if the counters become
        inaccurate.  Often, counters on silicon are much smaller than
        the 64-bit packet and byte counters in various IETF MIBs.
        System developers should consider what counter polling rate is
        necessary to maintain accurate counters and whether those
        polling rates are practical.  Relevant MIBs for MPLS are
        discussed in [RFC4221] and [RFC6639].

   T#7  [RFC6894] provides a good basis for MPLS FRR testing.  Similar
        testing should be performed to determine restoration times;
        however, this testing is far more difficult to perform due to
        the need for a simulated test topology that is capable of
        simulating the signaling used in restoration.  The simulated
        topology should be comparable with the target deployment in the

Top      Up      ToC       Page 45 
        number of nodes and links and in resource usage flooding and
        setup delays.  Some commercial test equipment can support this
        type of testing.

4.3.  Multipath Capabilities and Performance

   Multipath capabilities do not apply to MPLS-TP but apply to MPLS and
   apply if MPLS-TP is carried in MPLS.

   T#8  Send traffic at a rate well exceeding the capacity of a single
        multipath component link, and where entropy exists only below
        the top of stack.  If only the top label is used, this test will
        fail immediately.

   T#9  Move the labels with entropy down in the stack until either the
        full forwarding rate can no longer be supported or most or all
        packets try to use the same component link.

   T#10 Repeat the two tests above with the entropy contained in IP
        headers or IP payload fields below the label stack rather than
        in the label stack.  Test with the set of IP headers or IP
        payload fields considered relevant to the deployment or to the
        target market.

   T#11 Determine whether traffic that contains a pseudowire Control
        Word is interpreted as IP traffic.  Information in the payload
        MUST NOT be used in the load balancing if the first nibble of
        the packet is not 4 or 6 (IPv4 or IPv6).

   T#12 Determine whether special-purpose labels and extended special-
        purpose labels are excluded from the label stack hash.  They
        MUST be excluded.

   T#13 Perform testing in the presence of combinations of:

        A.  Very large microflows.

        B.  Relatively short-lived high-capacity flows.

        C.  Extremely large numbers of flows.

        D.  Very short-lived small flows.

Top      Up      ToC       Page 46 
4.4.  Pseudowire Capabilities and Performance

   T#14 Ensure that pseudowire can be set up with a pseudowire label and
        pseudowire Control Word added at ingress and the pseudowire
        label and pseudowire Control Word removed at egress.

   T#15 For pseudowire that contains variable-length payload packets,
        repeat performance tests listed under "Basic Performance" for
        pseudowire ingress and egress functions.

   T#16 Repeat pseudowire performance tests with and without a
        pseudowire Control Word.

   T#17 Determine whether pseudowire can be set up with a pseudowire
        label, Flow Label, and pseudowire Control Word added at ingress
        and the pseudowire label, Flow Label, and pseudowire Control
        Word removed at egress.

   T#18 Determine which payload fields are used to create the Flow Label
        and whether the set of fields and algorithm provide sufficient
        entropy for load balancing.

   T#19 Repeat pseudowire performance tests with Flow Labels included.

4.5.  Entropy Label Support and Performance

   T#20 Determine whether Entropy Labels can be added at ingress and
        removed at egress.

   T#21 Determine which fields are used to create an Entropy Label.
        Labels further down in the stack, including Entropy Labels
        further down and IP headers or IP payload fields where
        applicable, should be used.  Determine whether the set of fields
        and algorithm provide sufficient entropy for load balancing.

   T#22 Repeat performance tests under "Basic Performance" when Entropy
        Labels are used, where ingress or egress is the device under
        test (DUT).

   T#23 Determine whether an ELI is detected when acting as a midpoint
        LSR and whether the search for further information on which to
        base the load balancing is used.  Information below the Entropy
        Label SHOULD NOT be used.

   T#24 Ensure that the Entropy Label indicator and Entropy Label (ELI
        and EL) are removed from the label stack during UHP and PHP
        operations.

Top      Up      ToC       Page 47 
   T#25 Ensure that operations on the TC field when adding and removing
        Entropy Label are correctly carried out.  If TC is changed
        during a swap operation, the ability to transfer that change
        MUST be provided.  The ability to suppress the transfer of TC
        MUST also be provided.  See the pipe, short-pipe, and uniform
        models in [RFC3443].

   T#26 Repeat performance tests for a midpoint LSR with Entropy Labels
        found at various label stack depths.

4.6.  DoS Protection

   T#27 Actively attack LSRs under high protocol churn load and
        determine control-plane performance impact or successful DoS
        under test conditions.  Specifically, test for the following.

        A.  TCP SYN attack against control-plane and management-plane
            protocols using TCP, including CLI access (typically SSH-
            protected login), NETCONF, etc.

        B.  High traffic volume attack against control-plane and
            management-plane protocols not using TCP.

        C.  Attacks that can be performed from a compromised management
            subnet computer, but not one with authentication keys.

        D.  Attacks that can be performed from a compromised peer within
            the control plane (internal domain and external domain).
            Assume that keys that are per peering and keys that are per
            router ID, rather than network-wide keys, are in use.

        See Section 2.6.1.

4.7.  OAM Capabilities and Performance

   T#28 Determine maximum sustainable rates of BFD traffic.  If BFD
        requires CPU intervention, determine both maximum rates and CPU
        loading when multiple interfaces are active.

   T#29 Verify LSP Ping and LSP Traceroute capability.

   T#30 Determine maximum rates of MPLS-TP CC-CV traffic.  If CC-CV
        requires CPU intervention, determine both maximum rates and CPU
        loading when multiple interfaces are active.

   T#31 Determine MPLS-TP DM precision.

   T#32 Determine MPLS-TP LM accuracy.

Top      Up      ToC       Page 48 
   T#33 Verify MPLS-TP AIS/RDI and Protection State Coordination (PSC)
        functionality, protection speed, and AIS/RDI notification speed
        when a large number of Maintenance Entities (MEs) must be
        notified with AIS/RDI.



(page 48 continued on part 4)

Next RFC Part