tech-invite   World Map     

IETF     RFCs     Groups     SIP     ABNFs    |    3GPP     Specs     Glossaries     Architecture     IMS     UICC    |    search     info

RFC 7640

 
 
 

Traffic Management Benchmarking

Part 3 of 3, p. 32 to 51
Prev RFC Part

 


prevText      Top      Up      ToC       Page 32 
6.3.  Shaper Tests

   Like a queue, a traffic shaper is memory based, but with the added
   intelligence of an active traffic scheduler.  The same concepts as
   those described in Section 6.2 (queue testing) can be applied to
   testing a network device shaper.

   Again, the tests are divided into two sections: individual shaper
   benchmark tests and then full-capacity shaper benchmark tests.

6.3.1.  Shaper Individual Tests

   A traffic shaper generally has three (3) components that can be
   configured:

   -  Ingress Queue bytes

   -  Shaper Rate (SR), bps

   -  Burst Committed (Bc) and Burst Excess (Be), bytes

   The Ingress Queue holds burst traffic, and the shaper then meters
   traffic out of the egress port according to the SR and Bc/Be
   parameters.  Shapers generally transmit into policers, so the idea is
   for the emitted traffic to conform to the policer's limits.

Top      Up      ToC       Page 33 
6.3.1.1.  Testing Shaper with Stateless Traffic

   Objective:
      Test a shaper by transmitting stateless traffic bursts into the
      shaper ingress port and verifying that the egress traffic is
      shaped according to the shaper traffic profile.

   Test Summary:
      The stateless traffic must be burst into the DUT ingress port and
      not exceed the Ingress Queue.  The burst can be a single burst or
      multiple bursts.  If multiple bursts are transmitted, then the
      transmission interval (Ti) must be large enough so that the SR is
      not exceeded.  An example will clarify single-burst and multiple-
      burst test cases.

      In this example, the shaper's ingress and egress ports are both
      full-duplex Gigabit Ethernet.  The Ingress Queue is configured to
      be 512,000 bytes, the SR = 50 Mbps, and both Bc and Be are
      configured to be 32,000 bytes.  For a single-burst test, the
      transmitting test device would burst 512,000 bytes maximum into
      the ingress port and then stop transmitting.

      If a multiple-burst test is to be conducted, then the burst bytes
      divided by the transmission interval between the 512,000-byte
      bursts must not exceed the SR.  The transmission interval (Ti)
      must adhere to a formula similar to the formula described in
      Section 6.2.1.1 for queues, namely:

         Ti = Ingress Queue * 8 / SR

      For the example from the previous paragraph, the Ti between bursts
      must be greater than 82 milliseconds (512,000 bytes * 8 /
      50,000,000 bps).  This yields an average rate of 50 Mbps so that
      an Ingress Queue would not overflow.

   Test Metrics:
      The metrics defined in Section 4.1 (LP, OOS, PDV, SR, SBB, and
      SBI) SHALL be measured at the egress port and recorded.

   Procedure:
      1. Configure the DUT shaper ingress QL and shaper egress rate
         parameters (SR, Bc, Be).

      2. Configure the tester to generate a stateless traffic burst
         equal to QL and an interval equal to Ti (QL in bits/BB).

Top      Up      ToC       Page 34 
      3. Generate bursts of QL traffic into the DUT, and measure the
         metrics defined in Section 4.1 (LP, OOS, PDV, SR, SBB, and SBI)
         at the egress port and across the entire Td (default 30-second
         duration).

   Reporting Format:
      The Shaper Stateless Traffic individual report MUST contain all
      results for each QL/SR test run.  A recommended format is as
      follows:

      ***********************************************************

      Test Configuration Summary: Tr, Td

      DUT Configuration Summary: Ingress Burst Rate, QL, SR

      The results table should contain entries for each test run,
      as follows (Test #1 to Test #Tr):

      -  LP, OOS, PDV, SR, SBB, and SBI

      ***********************************************************

6.3.1.2.  Testing Shaper with Stateful Traffic

   Objective:
      Test a shaper by transmitting stateful traffic bursts into the
      shaper ingress port and verifying that the egress traffic is
      shaped according to the shaper traffic profile.

   Test Summary:
      To provide a more realistic benchmark and to test queues in
      Layer 4 devices such as firewalls, stateful traffic testing is
      also recommended for the shaper tests.  Stateful traffic tests
      will also utilize the Network Delay Emulator (NDE) from the
      network setup configuration in Section 1.2.

      The BDP of the TCP test traffic must be calculated as described in
      Section 6.2.1.2.  To properly stress network buffers and the
      traffic-shaping function, the TCP window size (which is the
      minimum of the TCP RWND and sender socket) should be greater than
      the BDP, which will stress the shaper.  BDP factors of 1.1 to 1.5
      are recommended, but the values are left to the discretion of the
      tester and should be documented.

Top      Up      ToC       Page 35 
      The cumulative TCP window sizes* (RWND at the receiving end and
      CWND at the transmitting end) equates to the TCP window size* for
      each connection, multiplied by the number of connections.

      *  As described in Section 3 of [RFC6349], the SSB MUST be large
         enough to fill the BDP.

      For example, if the BDP is equal to 256 KB and a connection size
      of 64 KB is used for each connection, then it would require four
      (4) connections to fill the BDP and 5-6 connections (oversubscribe
      the BDP) to stress-test the traffic-shaping function.

      Two types of TCP tests MUST be performed: the Bulk Transfer Test
      and the Micro Burst Test Pattern, as documented in Appendix B.
      The Bulk Transfer Test only bursts during the TCP Slow Start (or
      Congestion Avoidance) state, while the Micro Burst Test Pattern
      emulates application-layer bursting, which may occur any time
      during the TCP connection.

      Other types of tests SHOULD include the following: simple web
      sites, complex web sites, business applications, email, and
      SMB/CIFS file copy (all of which are also documented in
      Appendix B).

   Test Metrics:
      The test results will be recorded per the stateful metrics defined
      in Section 4.2 -- primarily the TCP Test Pattern Execution Time
      (TTPET), TCP Efficiency, and Buffer Delay.

   Procedure:
      1. Configure the DUT shaper ingress QL and shaper egress rate
         parameters (SR, Bc, Be).

      2. Configure the test generator* with a profile of an emulated
         application traffic mixture.

         -  The application mixture MUST be defined in terms of
            percentage of the total bandwidth to be tested.

         -  The rate of transmission for each application within the
            mixture MUST also be configurable.

         *  To ensure repeatable results, the test generator MUST be
            capable of generating precise TCP test patterns for each
            application specified.

Top      Up      ToC       Page 36 
      3. Generate application traffic between the ingress (client side)
         and egress (server side) ports of the DUT, and measure the
         metrics (TTPET, TCP Efficiency, and Buffer Delay) per
         application stream and at the ingress and egress ports (across
         the entire Td, default 30-second duration).

   Reporting Format:
      The Shaper Stateful Traffic individual report MUST contain all
      results for each traffic scheduler and QL/SR test run.  A
      recommended format is as follows:

      ******************************************************************

      Test Configuration Summary: Tr, Td

      DUT Configuration Summary: Ingress Burst Rate, QL, SR

      Application Mixture and Intensities: These are the percentages
      configured for each application type.

      The results table should contain entries for each test run, with
      minimum, maximum, and average per application session, as follows
      (Test #1 to Test #Tr):

      -  Throughput (bps) and TTPET for each application session

      -  Bytes In and Bytes Out for each application session

      -  TCP Efficiency and Buffer Delay for each application session

      ******************************************************************

6.3.2.  Shaper Capacity Tests

   Objective:
      The intent of these scalability tests is to verify shaper
      performance in a scaled environment with shapers active on
      multiple queues on multiple egress physical ports.  These tests
      will benchmark the maximum number of shapers as specified by the
      device manufacturer.

   The following sections provide the specific test scenarios,
   procedures, and reporting formats for each shaper capacity test.

Top      Up      ToC       Page 37 
6.3.2.1.  Single Queue Shaped, All Physical Ports Active

   Test Summary:
      The first shaper capacity test involves per-port shaping with all
      physical ports active.  Traffic from multiple ingress physical
      ports is directed to the same egress physical port.  This will
      cause oversubscription on the egress physical port.  Also, the
      same amount of traffic is directed to each egress physical port.

      The benchmarking methodologies specified in Sections 6.3.1.1
      (stateless) and 6.3.1.2 (stateful) (procedure, metrics, and
      reporting format) should be applied here.  Since this is a
      capacity test, the configuration and report results format (see
      Section 6.3.1) MUST also include:

      Configuration:

      -  The number of physical ingress ports active during the test

      -  The classification marking (DSCP, VLAN, etc.) for each physical
         ingress port

      -  The traffic rate for stateful traffic and the traffic
         rate/mixture for stateful traffic for each physical
         ingress port

      -  The shaped egress port shaper parameters (QL, SR, Bc, Be)

      Report Results:

      -  For each active egress port, the achieved throughput rate and
         shaper metrics for each ingress port traffic stream

      Example:

      -  Egress Port 1: throughput and metrics for ingress streams 1-n

      -  Egress Port n: throughput and metrics for ingress streams 1-n

6.3.2.2.  All Queues Shaped, Single Port Active

   Test Summary:
      The second shaper capacity test is conducted with all queues
      actively shaping on a single physical port.  The benchmarking
      methodology described in the per-port shaping test
      (Section 6.3.2.1) serves as the foundation for this.
      Additionally, each of the SP queues on the egress physical port is
      configured with a shaper.  For the highest-priority queue, the

Top      Up      ToC       Page 38 
      maximum amount of bandwidth available is limited by the bandwidth
      of the shaper.  For the lower-priority queues, the maximum amount
      of bandwidth available is limited by the bandwidth of the shaper
      and traffic in higher-priority queues.

      The benchmarking methodologies specified in Sections 6.3.1.1
      (stateless) and 6.3.1.2 (stateful) (procedure, metrics, and
      reporting format) should be applied here.  Since this is a
      capacity test, the configuration and report results format (see
      Section 6.3.1) MUST also include:

      Configuration:

      -  The number of physical ingress ports active during the test

      -  The classification marking (DSCP, VLAN, etc.) for each physical
         ingress port

      -  The traffic rate for stateful traffic and the traffic
         rate/mixture for stateful traffic for each physical
         ingress port

      -  For the active egress port, each of the following shaper queue
         parameters: QL, SR, Bc, Be

      Report Results:

      -  For each queue of the active egress port, the achieved
         throughput rate and shaper metrics for each ingress port
         traffic stream

      Example:

      -  Egress Port High-Priority Queue: throughput and metrics for
         ingress streams 1-n

      -  Egress Port Lower-Priority Queue: throughput and metrics for
         ingress streams 1-n

Top      Up      ToC       Page 39 
6.3.2.3.  All Queues Shaped, All Ports Active

   Test Summary:
      For the third shaper capacity test (which is a combination of the
      tests listed in Sections 6.3.2.1 and 6.3.2.2), all queues will be
      actively shaping and all physical ports active.

      The benchmarking methodologies specified in Sections 6.3.1.1
      (stateless) and 6.3.1.2 (stateful) (procedure, metrics, and
      reporting format) should be applied here.  Since this is a
      capacity test, the configuration and report results format (see
      Section 6.3.1) MUST also include:

      Configuration:

      -  The number of physical ingress ports active during the test

      -  The classification marking (DSCP, VLAN, etc.) for each physical
         ingress port

      -  The traffic rate for stateful traffic and the traffic
         rate/mixture for stateful traffic for each physical
         ingress port

      -  For each of the active egress ports: shaper port parameters and
         per-queue parameters (QL, SR, Bc, Be)

      Report Results:

      -  For each queue of each active egress port, the achieved
         throughput rate and shaper metrics for each ingress port
         traffic stream

      Example:

      -  Egress Port 1, High-Priority Queue: throughput and metrics for
         ingress streams 1-n

      -  Egress Port 1, Lower-Priority Queue: throughput and metrics for
         ingress streams 1-n

      ...

      -  Egress Port n, High-Priority Queue: throughput and metrics for
         ingress streams 1-n

      -  Egress Port n, Lower-Priority Queue: throughput and metrics for
         ingress streams 1-n

Top      Up      ToC       Page 40 
6.4.  Concurrent Capacity Load Tests

   As mentioned in Section 3 of this document, it is impossible to
   specify the various permutations of concurrent traffic management
   functions that should be tested in a device for capacity testing.
   However, some profiles are listed below that may be useful for
   testing multiple configurations of traffic management functions:

   -  Policers on ingress and queuing on egress

   -  Policers on ingress and shapers on egress (not intended for a flow
      to be policed and then shaped; these would be two different flows
      tested at the same time)

   The test procedures and reporting formats from Sections 6.1, 6.2,
   and 6.3 may be modified to accommodate the capacity test profile.

7.  Security Considerations

   Documents of this type do not directly affect the security of the
   Internet or of corporate networks as long as benchmarking is not
   performed on devices or systems connected to production networks.

   Further, benchmarking is performed on a "black box" basis, relying
   solely on measurements observable external to the DUT/SUT.

   Special capabilities SHOULD NOT exist in the DUT/SUT specifically for
   benchmarking purposes.  Any implications for network security arising
   from the DUT/SUT SHOULD be identical in the lab and in production
   networks.

Top      Up      ToC       Page 41 
8.  References

8.1.  Normative References

   [3GPP2-C_R1002-A]
              3rd Generation Partnership Project 2, "cdma2000 Evaluation
              Methodology", Version 1.0, Revision A, May 2009,
              <http://www.3gpp2.org/public_html/specs/
              C.R1002-A_v1.0_Evaluation_Methodology.pdf>.

   [RFC1242]  Bradner, S., "Benchmarking Terminology for Network
              Interconnection Devices", RFC 1242, DOI 10.17487/RFC1242,
              July 1991, <http://www.rfc-editor.org/info/rfc1242>.

   [RFC2119]  Bradner, S., "Key words for use in RFCs to Indicate
              Requirement Levels", BCP 14, RFC 2119,
              DOI 10.17487/RFC2119, March 1997,
              <http://www.rfc-editor.org/info/rfc2119>.

   [RFC2544]  Bradner, S. and J. McQuaid, "Benchmarking Methodology for
              Network Interconnect Devices", RFC 2544,
              DOI 10.17487/RFC2544, March 1999,
              <http://www.rfc-editor.org/info/rfc2544>.

   [RFC2680]  Almes, G., Kalidindi, S., and M. Zekauskas, "A One-way
              Packet Loss Metric for IPPM", RFC 2680,
              DOI 10.17487/RFC2680, September 1999,
              <http://www.rfc-editor.org/info/rfc2680>.

   [RFC3148]  Mathis, M. and M. Allman, "A Framework for Defining
              Empirical Bulk Transfer Capacity Metrics", RFC 3148,
              DOI 10.17487/RFC3148, July 2001,
              <http://www.rfc-editor.org/info/rfc3148>.

   [RFC4115]  Aboul-Magd, O. and S. Rabie, "A Differentiated Service
              Two-Rate, Three-Color Marker with Efficient Handling of
              in-Profile Traffic", RFC 4115, DOI 10.17487/RFC4115,
              July 2005, <http://www.rfc-editor.org/info/rfc4115>.

   [RFC4689]  Poretsky, S., Perser, J., Erramilli, S., and S. Khurana,
              "Terminology for Benchmarking Network-layer Traffic
              Control Mechanisms", RFC 4689, DOI 10.17487/RFC4689,
              October 2006, <http://www.rfc-editor.org/info/rfc4689>.

   [RFC4737]  Morton, A., Ciavattone, L., Ramachandran, G., Shalunov,
              S., and J. Perser, "Packet Reordering Metrics", RFC 4737,
              DOI 10.17487/RFC4737, November 2006,
              <http://www.rfc-editor.org/info/rfc4737>.

Top      Up      ToC       Page 42 
   [RFC5481]  Morton, A. and B. Claise, "Packet Delay Variation
              Applicability Statement", RFC 5481, DOI 10.17487/RFC5481,
              March 2009, <http://www.rfc-editor.org/info/rfc5481>.

   [RFC6349]  Constantine, B., Forget, G., Geib, R., and R. Schrage,
              "Framework for TCP Throughput Testing", RFC 6349,
              DOI 10.17487/RFC6349, August 2011,
              <http://www.rfc-editor.org/info/rfc6349>.

   [RFC6703]  Morton, A., Ramachandran, G., and G. Maguluri, "Reporting
              IP Network Performance Metrics: Different Points of View",
              RFC 6703, DOI 10.17487/RFC6703, August 2012,
              <http://www.rfc-editor.org/info/rfc6703>.

   [SPECweb2009]
              Standard Performance Evaluation Corporation (SPEC),
              "SPECweb2009 Release 1.20 Benchmark Design Document",
              April 2010, <https://www.spec.org/web2009/docs/design/
              SPECweb2009_Design.html>.

8.2.  Informative References

   [CA-Benchmark]
              Hamilton, M. and S. Banks, "Benchmarking Methodology for
              Content-Aware Network Devices", Work in Progress,
              draft-ietf-bmwg-ca-bench-meth-04, February 2013.

   [CoDel]    Nichols, K., Jacobson, V., McGregor, A., and J. Iyengar,
              "Controlled Delay Active Queue Management", Work in
              Progress, draft-ietf-aqm-codel-01, April 2015.

   [MEF-10.3] Metro Ethernet Forum, "Ethernet Services Attributes
              Phase 3", MEF 10.3, October 2013,
              <https://www.mef.net/Assets/Technical_Specifications/
              PDF/MEF_10.3.pdf>.

   [MEF-12.2] Metro Ethernet Forum, "Carrier Ethernet Network
              Architecture Framework -- Part 2: Ethernet Services
              Layer", MEF 12.2, May 2014,
              <https://www.mef.net/Assets/Technical_Specifications/
              PDF/MEF_12.2.pdf>.

   [MEF-14]   Metro Ethernet Forum, "Abstract Test Suite for Traffic
              Management Phase 1", MEF 14, November 2005,
              <https://www.mef.net/Assets/
              Technical_Specifications/PDF/MEF_14.pdf>.

Top      Up      ToC       Page 43 
   [MEF-19]   Metro Ethernet Forum, "Abstract Test Suite for UNI
              Type 1", MEF 19, April 2007, <https://www.mef.net/Assets/
              Technical_Specifications/PDF/MEF_19.pdf>.

   [MEF-26.1] Metro Ethernet Forum, "External Network Network Interface
              (ENNI) - Phase 2", MEF 26.1, January 2012,
              <http://www.mef.net/Assets/Technical_Specifications/
              PDF/MEF_26.1.pdf>.

   [MEF-37]   Metro Ethernet Forum, "Abstract Test Suite for ENNI",
              MEF 37, January 2012, <https://www.mef.net/Assets/
              Technical_Specifications/PDF/MEF_37.pdf>.

   [PIE]      Pan, R., Natarajan, P., Baker, F., White, G., VerSteeg,
              B., Prabhu, M., Piglione, C., and V. Subramanian, "PIE: A
              Lightweight Control Scheme To Address the Bufferbloat
              Problem", Work in Progress, draft-ietf-aqm-pie-02,
              August 2015.

   [RFC2697]  Heinanen, J. and R. Guerin, "A Single Rate Three Color
              Marker", RFC 2697, DOI 10.17487/RFC2697, September 1999,
              <http://www.rfc-editor.org/info/rfc2697>.

   [RFC2698]  Heinanen, J. and R. Guerin, "A Two Rate Three Color
              Marker", RFC 2698, DOI 10.17487/RFC2698, September 1999,
              <http://www.rfc-editor.org/info/rfc2698>.

   [RFC7567]  Baker, F., Ed., and G. Fairhurst, Ed., "IETF
              Recommendations Regarding Active Queue Management",
              BCP 197, RFC 7567, DOI 10.17487/RFC7567, July 2015,
              <http://www.rfc-editor.org/info/rfc7567>.

Top      Up      ToC       Page 44 
Appendix A.  Open Source Tools for Traffic Management Testing

   This framework specifies that stateless and stateful behaviors SHOULD
   both be tested.  Some open source tools that can be used to
   accomplish many of the tests proposed in this framework are iperf,
   netperf (with netperf-wrapper), the "uperf" tool, Tmix,
   TCP-incast-generator, and D-ITG (Distributed Internet Traffic
   Generator).

   iperf can generate UDP-based or TCP-based traffic; a client and
   server must both run the iperf software in the same traffic mode.
   The server is set up to listen, and then the test traffic is
   controlled from the client.  Both unidirectional and bidirectional
   concurrent testing are supported.

   The UDP mode can be used for the stateless traffic testing.  The
   target bandwidth, packet size, UDP port, and test duration can be
   controlled.  A report of bytes transmitted, packets lost, and delay
   variation is provided by the iperf receiver.

   iperf (TCP mode), TCP-incast-generator, and D-ITG can be used for
   stateful traffic testing to test bulk transfer traffic.  The TCP
   window size (which is actually the SSB), number of connections,
   packet size, TCP port, and test duration can be controlled.  A report
   of bytes transmitted and throughput achieved is provided by the iperf
   sender, while TCP-incast-generator and D-ITG provide even more
   statistics.

   netperf is a software application that provides network bandwidth
   testing between two hosts on a network.  It supports UNIX domain
   sockets, TCP, SCTP, and UDP via BSD Sockets.  netperf provides a
   number of predefined tests, e.g., to measure bulk (unidirectional)
   data transfer or request/response performance
   (http://en.wikipedia.org/wiki/Netperf).  netperf-wrapper is a Python
   script that runs multiple simultaneous netperf instances and
   aggregates the results.

   uperf uses a description (or model) of an application mixture.  It
   generates the load according to the model descriptor.  uperf is more
   flexible than netperf in its ability to generate request/response
   application behavior within a single TCP connection.  The application
   model descriptor can be based on empirical data, but at the time of
   this writing, the import of packet captures is not directly
   supported.

Top      Up      ToC       Page 45 
   Tmix is another application traffic emulation tool.  It uses packet
   captures directly to create the traffic profile.  The packet trace is
   "reverse compiled" into a source-level characterization, called a
   "connection vector", of each TCP connection present in the trace.
   While most widely used in ns2 simulation environments, Tmix also runs
   on Linux hosts.

   The traffic generation capabilities of these open source tools
   facilitate the emulation of the TCP test patterns discussed in
   Appendix B.

Appendix B.  Stateful TCP Test Patterns

   This framework recommends at a minimum the following TCP test
   patterns, since they are representative of real-world application
   traffic (Section 5.2.1 describes some methods to derive other
   application-based TCP test patterns).

   -  Bulk Transfer: Generate concurrent TCP connections whose aggregate
      number of in-flight data bytes would fill the BDP.  Guidelines
      from [RFC6349] are used to create this TCP traffic pattern.

   -  Micro Burst: Generate precise burst patterns within a single TCP
      connection or multiple TCP connections.  The idea is for TCP to
      establish equilibrium and then burst application bytes at defined
      sizes.  The test tool must allow the burst size and burst time
      interval to be configurable.

   -  Web Site Patterns: The HTTP traffic model shown in Table 4.1.3-1
      of [3GPP2-C_R1002-A] demonstrates a way to develop these TCP test
      patterns.  In summary, the HTTP traffic model consists of the
      following parameters:

      -  Main object size (Sm)

      -  Embedded object size (Se)

      -  Number of embedded objects per page (Nd)

      -  Client processing time (Tcp)

      -  Server processing time (Tsp)

Top      Up      ToC       Page 46 
   Web site test patterns are illustrated with the following examples:

   -  Simple web site: Mimic the request/response and object download
      behavior of a basic web site (small company).

   -  Complex web site: Mimic the request/response and object download
      behavior of a complex web site (eCommerce site).

   Referencing the HTTP traffic model parameters, the following table
   was derived (by analysis and experimentation) for simple web site and
   complex web site TCP test patterns:

                             Simple         Complex
    Parameter                Web Site       Web Site
    -----------------------------------------------------
    Main object              Ave. = 10KB    Ave. = 300KB
     size (Sm)               Min. = 100B    Min. = 50KB
                             Max. = 500KB   Max. = 2MB

    Embedded object          Ave. = 7KB     Ave. = 10KB
     size (Se)               Min. = 50B     Min. = 100B
                             Max. = 350KB   Max. = 1MB

    Number of embedded       Ave. = 5       Ave. = 25
     objects per page (Nd)   Min. = 2       Min. = 10
                             Max. = 10      Max. = 50

    Client processing        Ave. = 3s      Ave. = 10s
     time (Tcp)*             Min. = 1s      Min. = 3s
                             Max. = 10s     Max. = 30s

    Server processing        Ave. = 5s      Ave. = 8s
     time (Tsp)*             Min. = 1s      Min. = 2s
                             Max. = 15s     Max. = 30s

   *  The client and server processing time is distributed across the
      transmission/receipt of all of the main and embedded objects.

   To be clear, the parameters in this table are reasonable guidelines
   for the TCP test pattern traffic generation.  The test tool can use
   fixed parameters for simpler tests and mathematical distributions for
   more complex tests.  However, the test pattern must be repeatable to
   ensure that the benchmark results can be reliably compared.

Top      Up      ToC       Page 47 
   -  Interactive Patterns: While web site patterns are interactive to a
      degree, they mainly emulate the downloading of web sites of
      varying complexity.  Interactive patterns are more chatty in
      nature, since there is a lot of user interaction with the servers.
      Examples include business applications such as PeopleSoft and
      Oracle, and consumer applications such as Facebook and IM.  For
      the interactive patterns, the packet capture technique was used to
      characterize some business applications and also the email
      application.

   In summary, an interactive application can be described by the
   following parameters:

   -  Client message size (Scm)

   -  Number of client messages (Nc)

   -  Server response size (Srs)

   -  Number of server messages (Ns)

   -  Client processing time (Tcp)

   -  Server processing time (Tsp)

   -  File size upload (Su)*

   -  File size download (Sd)*

   *  The file size parameters account for attachments uploaded or
      downloaded and may not be present in all interactive applications.

Top      Up      ToC       Page 48 
   Again using packet capture as a means to characterize, the following
   table reflects the guidelines for simple business applications,
   complex business applications, eCommerce, and email Send/Receive:

                     Simple       Complex
                     Business     Business
   Parameter         Application  Application  eCommerce*   Email
   --------------------------------------------------------------------
   Client message    Ave. = 450B  Ave. = 2KB   Ave. = 1KB   Ave. = 200B
    size (Scm)       Min. = 100B  Min. = 500B  Min. = 100B  Min. = 100B
                     Max. = 1.5KB Max. = 100KB Max. = 50KB  Max. = 1KB

   Number of client  Ave. = 10    Ave. = 100   Ave. = 20    Ave. = 10
    messages (Nc)    Min. = 5     Min. = 50    Min. = 10    Min. = 5
                     Max. = 25    Max. = 250   Max. = 100   Max. = 25

   Client processing Ave. = 10s   Ave. = 30s   Ave. = 15s   Ave. = 5s
    time (Tcp)**     Min. = 3s    Min. = 3s    Min. = 5s    Min. = 3s
                     Max. = 30s   Max. = 60s   Max. = 120s  Max. = 45s

   Server response   Ave. = 2KB   Ave. = 5KB   Ave. = 8KB   Ave. = 200B
    size (Srs)       Min. = 500B  Min. = 1KB   Min. = 100B  Min. = 150B
                     Max. = 100KB Max. = 1MB   Max. = 50KB  Max. = 750B

   Number of server  Ave. = 50    Ave. = 200   Ave. = 100   Ave. = 15
    messages (Ns)    Min. = 10    Min. = 25    Min. = 15    Min. = 5
                     Max. = 200   Max. = 1000  Max. = 500   Max. = 40

   Server processing Ave. = 0.5s  Ave. = 1s    Ave. = 2s    Ave. = 4s
    time (Tsp)**     Min. = 0.1s  Min. = 0.5s  Min. = 1s    Min. = 0.5s
                     Max. = 5s    Max. = 20s   Max. = 10s   Max. = 15s

   File size         Ave. = 50KB  Ave. = 100KB Ave. = N/A   Ave. = 100KB
    upload (Su)      Min. = 2KB   Min. = 10KB  Min. = N/A   Min. = 20KB
                     Max. = 200KB Max. = 2MB   Max. = N/A   Max. = 10MB

   File size         Ave. = 50KB  Ave. = 100KB Ave. = N/A   Ave. = 100KB
    download (Sd)    Min. = 2KB   Min. = 10KB  Min. = N/A   Min. = 20KB
                     Max. = 200KB Max. = 2MB   Max. = N/A   Max. = 10MB

   *  eCommerce used a combination of packet capture techniques and
      reference traffic flows as described in [SPECweb2009].

   ** The client and server processing time is distributed across the
      transmission/receipt of all of the messages.  The client
      processing time consists mainly of the delay between user
      interactions (not machine processing).

Top      Up      ToC       Page 49 
   Again, the parameters in this table are the guidelines for the TCP
   test pattern traffic generation.  The test tool can use fixed
   parameters for simpler tests and mathematical distributions for more
   complex tests.  However, the test pattern must be repeatable to
   ensure that the benchmark results can be reliably compared.

   -  SMB/CIFS file copy: Mimic a network file copy, both read and
      write.  As opposed to FTP, which is a bulk transfer and is only
      flow-controlled via TCP, SMB/CIFS divides a file into application
      blocks and utilizes application-level handshaking in addition to
      TCP flow control.

   In summary, an SMB/CIFS file copy can be described by the following
   parameters:

   -  Client message size (Scm)

   -  Number of client messages (Nc)

   -  Server response size (Srs)

   -  Number of server messages (Ns)

   -  Client processing time (Tcp)

   -  Server processing time (Tsp)

   -  Block size (Sb)

   The client and server messages are SMB control messages.  The block
   size is the data portion of the file transfer.

Top      Up      ToC       Page 50 
   Again using packet capture as a means to characterize, the following
   table reflects the guidelines for SMB/CIFS file copy:

                          SMB/CIFS
      Parameter           File Copy
      --------------------------------
      Client message      Ave. = 450B
       size (Scm)         Min. = 100B
                          Max. = 1.5KB

      Number of client    Ave. = 10
       messages (Nc)      Min. = 5
                          Max. = 25

      Client processing   Ave. = 1ms
       time (Tcp)         Min. = 0.5ms
                          Max. = 2

      Server response     Ave. = 2KB
       size (Srs)         Min. = 500B
                          Max. = 100KB

      Number of server    Ave. = 10
       messages (Ns)      Min. = 10
                          Max. = 200

      Server processing   Ave. = 1ms
       time (Tsp)         Min. = 0.5ms
                          Max. = 2ms

      Block               Ave. = N/A
       size (Sb)*         Min. = 16KB
                          Max. = 128KB

      *  Depending upon the tested file size, the block size will be
         transferred "n" number of times to complete the example.  An
         example would be a 10 MB file test and 64 KB block size.  In
         this case, 160 blocks would be transferred after the control
         channel is opened between the client and server.

Top      Up      ToC       Page 51 
Acknowledgments

   We would like to thank Al Morton for his continuous review and
   invaluable input to this document.  We would also like to thank Scott
   Bradner for providing guidance early in this document's conception,
   in the area of the benchmarking scope of traffic management
   functions.  Additionally, we would like to thank Tim Copley for his
   original input, as well as David Taht, Gory Erg, and Toke
   Hoiland-Jorgensen for their review and input for the AQM group.
   Also, for the formal reviews of this document, we would like to thank
   Gilles Forget, Vijay Gurbani, Reinhard Schrage, and Bhuvaneswaran
   Vengainathan.

Authors' Addresses

   Barry Constantine
   JDSU, Test and Measurement Division
   Germantown, MD  20876-7100
   United States

   Phone: +1-240-404-2227
   Email: barry.constantine@jdsu.com


   Ram (Ramki) Krishnan
   Dell Inc.
   Santa Clara, CA  95054
   United States

   Phone: +1-408-406-7890
   Email: ramkri123@gmail.com