Objective: To characterize the ability of a DUT to process Back-to-Back Frames as defined in [RFC 1242
The procedure follows.
From the list of RECOMMENDED
frame sizes (Section 9
of RFC 2544
), select the subset of frame sizes whose Measured Throughput (during prerequisite testing) was less than the Max Theoretical Frame Rate of the DUT/test setup. These are the only frame sizes where it is possible to produce a burst of frames that cause the DUT buffers to fill and eventually overflow, producing one or more discarded frames.
Each trial in the test requires the tester to send a burst of frames (after idle time) with the minimum interframe gap and to count the corresponding frames forwarded by the DUT.
The duration of the trial includes three REQUIRED
The time to send the burst of frames (at the back-to-back rate), determined by the search algorithm.
The time to receive the transferred burst of frames (at the [RFC 2544] Throughput rate), possibly truncated by buffer overflow, and certainly including the latency of the DUT.
At least 2 seconds not overlapping the time to receive the burst (Component 2, above), to ensure that DUT buffers have depleted. Longer times MUST be used when conditions warrant, such as when buffer times >2 seconds are measured or when burst sending times are >2 seconds, but care is needed, since this time component directly increases trial duration, and many trials and tests comprise a complete benchmarking study.
The upper search limit for the time to send each burst MUST
be configurable to values as high as 30 seconds (buffer time results reported at or near the configured upper limit are likely invalid, and the test MUST
be repeated with a higher search limit).
If all frames have been received, the tester increases the length of the burst according to the search algorithm and performs another trial.
If the received frame count is less than the number of frames in the burst, then the limit of DUT processing and buffering may have been exceeded, and the burst length for the next trial is determined by the search algorithm (the burst length is typically reduced, but see below).
Classic search algorithms have been adapted for use in benchmarking, where the search requires discovery of a pair of outcomes, one with no loss and another with loss, at load conditions within the acceptable tolerance or accuracy. Conditions encountered when benchmarking the infrastructure for network function virtualization require algorithm enhancement. Fortunately, the adaptation of Binary Search, and an enhanced Binary Search with Loss Verification, have been specified in Clause 12.3 of [TST009
]. These algorithms can easily be used for Back-to-Back Frame benchmarking by replacing the offered load level with burst length in frames. [TST009
], Annex B describes the theory behind the enhanced Binary Search with Loss Verification algorithm.
There are also promising works in progress that may prove useful in Back-to-Back Frame benchmarking. [BMWG-MLRSEARCH
] and [BMWG-PLRSEARCH
] are two such examples.
Either the [TST009
] Binary Search or Binary Search with Loss Verification algorithms MUST
be used, and input parameters to the algorithm(s) MUST
The tester usually imposes a (configurable) minimum step size for burst length, and the step size MUST
be reported with the results (as this influences the accuracy and variation of test results).
The original Section 26.4
of RFC 2544
definition is stated below:
The back-to-back value is the number of frames in the longest burst that the DUT will handle without the loss of any frames.
On this topic, Section 26.4
of RFC 2544
The trial length MUST be at least 2 seconds and SHOULD be repeated at least 50 times with the average of the recorded values being reported.
Therefore, the Back-to-Back Frame benchmark is the average of burst-length values over repeated tests to determine the longest burst of frames that the DUT can successfully process and buffer without frame loss. Each of the repeated tests completes an independent search process.
In this update, the test MUST
be repeated N times (the number of repetitions is now a variable that must be reported) for each frame size in the subset list, and each Back-to-Back Frame value MUST
be made available for further processing (below).
For each frame size, calculate the following summary statistics for longest Back-to-Back Frame values over the N tests:
Further, calculate the Implied DUT Buffer Time and the Corrected DUT Buffer Time in seconds, as follows:
Implied DUT buffer time =
Average num of Back-to-back Frames / Max Theoretical Frame Rate
The formula above is simply expressing the burst of frames in units of time.
The next step is to apply a correction factor that accounts for the DUT's frame forwarding operation during the test (assuming the simple model of the DUT composed of a buffer and a forwarding function, described in Section 4
Corrected DUT Buffer Time =
Implied DUT |Implied DUT Measured Throughput |
= Buffer Time - |Buffer Time * -------------------------- |
| Max Theoretical Frame Rate |
The "Measured Throughput" is the [RFC 2544] Throughput Benchmark for the frame size tested, as augmented by methods including the Binary Search with Loss Verification algorithm in [TST009] where applicable and MUST be expressed in frames per second in this equation.
The "Max Theoretical Frame Rate" is a calculated value for the interface speed and link-layer technology used, and it MUST be expressed in frames per second in this equation.
The term on the far right in the formula for Corrected DUT Buffer Time accounts for all the frames in the burst that were transmitted by the DUT while the burst of frames was sent in
. So, these frames are not in the buffer, and the buffer size is more accurately estimated by excluding them. If Measured Throughput is not available, an acceptable approximation is the received frame rate (see Forwarding Rate in [RFC 2889
] measured during Back-to-back Frame testing).