The Alternate-Marking technique does not require a strong
synchronization, especially for packet loss and two-way delay
measurement. Only one-way delay measurement requires network devices
to have synchronized clocks.
Color switching is the reference for all the network devices, and the
only requirement to be achieved is that all network devices have to
recognize the right batch along the path.
If the length of the measurement period is L time units, then all
network devices must be synchronized to the same clock reference with
an accuracy of +/- L/2 time units (without considering network
delay). This level of accuracy guarantees that all network devices
consistently match the color bit to the correct block. For example,
if the color is toggled every second (L = 1 second), then clocks must
be synchronized with an accuracy of +/- 0.5 second to a common time
This synchronization requirement can be satisfied even with a
relatively inaccurate synchronization method. This is true for
packet loss and two-way delay measurement, but not for one-way delay
measurement, where clock synchronization must be accurate.
Therefore, a system that uses only packet loss and two-way delay
measurement does not require synchronization. This is because the
value of the clocks of network devices does not affect the
computation of the two-way delay measurement.
4.2. Data Correlation
Data correlation is the mechanism to compare counters and timestamps
for packet loss, delay, and delay variation calculation. It could be
performed in several ways depending on the Alternate-Marking
application and use case. Some possibilities are to:
o use a centralized solution using NMS to correlate data; and
o define a protocol-based distributed solution by introducing a new
protocol or by extending the existing protocols (e.g., see RFC
6374 [RFC6374] or the Two-Way Active Measurement Protocol (TWAMP)
as defined in RFC 5357 [RFC5357] or the One-Way Active Measurement
Protocol (OWAMP) as defined in RFC 4656 [RFC4656]) in order to
communicate the counters and timestamps between nodes.
In the following paragraphs, an example data correlation mechanism is
explained and could be used independently of the adopted solutions.
When data is collected on the upstream and downstream nodes, e.g.,
packet counts for packet loss measurement or timestamps for packet
delay measurement, and is periodically reported to or pulled by other
nodes or an NMS, a certain data correlation mechanism SHOULD be in
use to help the nodes or NMS tell whether any two or more packet
counts are related to the same block of markers or if any two
timestamps are related to the same marked packet.
The Alternate-Marking Method described in this document literally
splits the packets of the measured flow into different measurement
blocks; in addition, a Block Number (BN) could be assigned to each
such measurement block. The BN is generated each time a node reads
the data (packet counts or timestamps) and is associated with each
packet count and timestamp reported to or pulled by other nodes or
NMSs. The value of a BN could be calculated as the modulo of the
local time (when the data are read) and the interval of the marking
When the nodes or NMS see, for example, the same BNs associated with
two packet counts from an upstream and a downstream node,
respectively, it considers that these two packet counts correspond to
the same block, i.e., these two packet counts belong to the same
block of markers from the upstream and downstream nodes. The
assumption of this BN mechanism is that the measurement nodes are
time synchronized. This requires the measurement nodes to have a
certain time synchronization capability (e.g., the Network Time
Protocol (NTP) [RFC5905] or the IEEE 1588 Precision Time Protocol
(PTP) [IEEE-1588]). Synchronization aspects are further discussed in
4.3. Packet Reordering
Due to ECMP, packet reordering is very common in an IP network. The
accuracy of a marking-based PM, especially packet loss measurement,
may be affected by packet reordering. Take a look at the following
Block : 1 | 2 | 3 | 4 | 5 |...
Node R1 : AAAAAAA | BBBBBBB | AAAAAAA | BBBBBBB | AAAAAAA |...
Node R2 : AAAAABB | AABBBBA | AAABAAA | BBBBBBA | ABAAABA |...
Figure 5: Packet Reordering
In Figure 5, the packet stream for Node R1 isn't being reordered and
can be safely assigned to interval blocks, but the packet stream for
Node R2 is being reordered; so, looking at the packet with the marker
of "B" in block 3, there is no safe way to tell whether the packet
belongs to block 2 or block 4.
In general, there is the need to assign packets with the marker of
"B" or "A" to the right interval blocks. Most of the packet
reordering occurs at the edge of adjacent blocks, and they are easy
to handle if the interval of each block is sufficiently large. Then,
it can be assumed that the packets with different markers belong to
the block that they are closer to. If the interval is small, it is
difficult and sometimes impossible to determine to which block a
To choose a proper interval is important, and how to choose a proper
interval is out of the scope of this document. But an implementation
SHOULD provide a way to configure the interval and allow a certain
degree of packet reordering.
5. Applications, Implementation, and Deployment
The methodology described in the previous sections can be applied in
various situations. Basically, the Alternate-Marking technique could
be used in many cases for performance measurement. The only
requirement is to select and mark the flow to be monitored; in this
way, packets are batched by the sender, and each batch is alternately
marked such that it can be easily recognized by the receiver.
Some recent Alternate-Marking Method applications are listed below:
o IP Flow Performance Measurement (IPFPM): this application of the
marking method is described in [COLORING]. As an example, in this
document, the last reserved bit of the Flag field of the IPv4
header is proposed to be used for marking, while a solution for
IPv6 could be to leverage the IPv6 extension header for marking.
o OAM Passive Performance Measurement: In [RFC8296], two OAM bits
from the Bit Index Explicit Replication (BIER) header are reserved
for the Passive performance measurement marking method.
[PM-MM-BIER] details the measurement for multicast service over
the BIER domain. In addition, the Alternate-Marking Method could
also be used in a Service Function Chaining (SFC) domain. Lastly,
the application of the marking method to Network Virtualization
over Layer 3 (NVO3) protocols is considered by [NVO3-ENCAPS].
o MPLS Performance Measurement: RFC 6374 [RFC6374] uses the Loss
Measurement (LM) packet as the packet accounting demarcation
point. Unfortunately, this gives rise to a number of problems
that may lead to significant packet accounting errors in certain
situations. [MPLS-FLOW] discusses the desired capabilities for
MPLS flow identification in order to perform a better in-band
performance monitoring of user data packets. A method of
accomplishing identification is Synonymous Flow Labels (SFLs)
introduced in [SFL-FRAMEWORK], while [SYN-FLOW-LABELS] describes
performance measurements in RFC 6374 with SFL.
o Active Performance Measurement: [ALT-MM-AMP] describes how to
extend the existing Active Measurement Protocol, in order to
implement the Alternate-Marking methodology. [ALT-MM-SLA]
describes an extension to the Cisco SLA Protocol Measurement-Type
An example of implementation and deployment is explained in the next
section, just to clarify how the method can work.
5.1. Report on the Operational Experiment
The method described in this document, also called Packet Network
Performance Monitoring (PNPM), has been invented and engineered in
It is important to highlight that the general description of the
methodology in this document is a consequence of the operational
experiment. The fundamental elements of the technique have been
tested, and the lessons learned from the operational experiment
inspired the formalization of the Alternate-Marking Method as
detailed in the previous sections.
The methodology has been used experimentally in Telecom Italia's
network and is applied to multicast IPTV channels or other specific
traffic flows with high QoS requirements (i.e., Mobile Backhauling
traffic realized with a VPN MPLS).
This technology has been employed by leveraging functions and tools
available on IP routers, and it's currently being used to monitor
packet loss in some portions of Telecom Italia's network. The
application of this method for delay measurement has also been
evaluated in Telecom Italia's labs.
This section describes how the experiment has been executed,
particularly, how the features currently available on existing
routing platforms can be used to apply the method, in order to give
an example of implementation and deployment.
The operational test, described herein, uses the flow-based strategy,
as defined in Section 3. Instead, the link-based strategy could be
applied to a physical link or a logical link (e.g., an Ethernet VLAN
or an MPLS Pseudowire (PW)).
The implementation of the method leverages the available router
functions, since the experiment has been done by a Service Provider
(as Telecom Italia is) on its own network. So, with current router
implementations, only QoS-related fields and features offer the
required flexibility to set bits in the packet header. In case a
Service Provider only uses the three most-significant bits of the
DSCP field (corresponding to IP Precedence) for QoS classification
and queuing, it is possible to use the two least-significant bits of
the DSCP field (bit 0 and bit 1) to implement the method without
affecting QoS policies. That is the approach used for the
experiment. One of the two bits (bit 0) could be used to identify
flows subject to traffic monitoring (set to 1 if the flow is under
monitoring, otherwise, it is set to 0), while the second (bit 1) can
be used for coloring the traffic (switching between values 0 and 1,
corresponding to colors A and B) and creating the blocks.
The experiment considers a flow as all the packets sharing the same
source IP address or the same destination IP address, depending on
the direction. In practice, once the flow has been defined, traffic
coloring using the DSCP field can be implemented by configuring an
access-list on the router output interface. The access-list
intercepts the flow(s) to be monitored and applies a policy to them
that sets the DSCP field accordingly. Since traffic coloring has to
be switched between the two values over time, the policy needs to be
modified periodically. An automatic script is used to perform this
task on the basis of a fixed timer. The automatic script is loaded
on board of the router and automatizes the basic operations that are
needed to realize the methodology.
After the traffic is colored using the DSCP field, all the routers on
the path can perform the counting. For this purpose, an access-list
that matches specific DSCP values can be used to count the packets of
the flow(s) being monitored. The same access-list can be installed
on all the routers of the path. In addition, network flow
monitoring, such as provided by IPFIX [RFC7011], can be used to
recognize timestamps of the first/last packet of a batch in order to
enable one of the alternatives to measure the delay as detailed in
In Telecom Italia's experiment, the timer is set to 5 minutes, so the
sequence of actions of the script is also executed every 5 minutes.
This value has shown to be a good compromise between measurement
frequency and stability of the measurement (i.e., the possibility of
collecting all the measures referring to the same block).
For this experiment, both counters and any other data are collected
by using the automatic script that sends these out to an NMS. The
NMS is responsible for packet loss calculation, performed by
comparing the values of counters from the routers along the flow
path(s). A 5-minute timer for color switching is a safe choice for
reading the counters and is also coherent with the reporting window
of the NMS.
Note that the use of the DSCP field for marking implies that the
method in this case works reliably only within a single management
and operation domain.
Lastly, the Telecom Italia experiment scales up to 1000 flows
monitored together on a single router, while an implementation on
dedicated hardware scales more, but it was tested only in labs for
5.1.1. Metric Transparency
Since a Service Provider application is described here, the method
can be applied to end-to-end services supplied to customers. So it
is important to highlight that the method MUST be transparent outside
the Service Provider domain.
In Telecom Italia's implementation, the source node colors the
packets with a policy that is modified periodically via an automatic
script in order to alternate the DSCP field of the packets. The
nodes between source and destination (included) have to use an
access-list to count the colored packets that they receive and
Moreover, the destination node has an important role: the colored
packets are intercepted and a policy restores and sets the DSCP field
of all the packets to the initial value. In this way, the metric is
transparent because outside the section of the network under
monitoring, the traffic flow is unchanged.
In such a case, thanks to this restoring technique, network elements
outside the Alternate-Marking monitoring domain (e.g., the two
Provider Edge nodes of the Mobile Backhauling VPN MPLS) are totally
unaware that packets were marked. So this restoring technique makes
Alternate Marking completely transparent outside its monitoring
6. Hybrid Measurement
The method has been explicitly designed for Passive measurements, but
it can also be used with Active measurements. In order to have both
end-to-end measurements and intermediate measurements (Hybrid
measurements), two endpoints can exchange artificial traffic flows
and apply Alternate Marking over these flows. In the intermediate
points, artificial traffic is managed in the same way as real traffic
and measured as specified before. So the application of the marking
method can also simplify the Active measurement, as explained in
7. Compliance with Guidelines from RFC 6390
RFC 6390 [RFC6390] defines a framework and a process for developing
Performance Metrics for protocols above and below the IP layer (such
as IP-based applications that operate over reliable or datagram
This document doesn't aim to propose a new Performance Metric but
rather a new Method of Measurement for a few Performance Metrics that
have already been standardized. Nevertheless, it's worth applying
guidelines from [RFC6390] to the present document, in order to
provide a more complete and coherent description of the proposed
method. We used a combination of the Performance Metric Definition
template defined in Section 5.4 of [RFC6390] and the Dependencies
laid out in Section 5.5 of that document.
o Metric Name / Metric Description: as already stated, this document
doesn't propose any new Performance Metrics. On the contrary, it
describes a novel method for measuring packet loss [RFC7680]. The
same concept, with small differences, can also be used to measure
delay [RFC7679] and jitter [RFC3393]. The document mainly
describes the applicability to packet loss measurement.
o Method of Measurement or Calculation: according to the method
described in the previous sections, the number of packets lost is
calculated by subtracting the value of the counter on the source
node from the value of the counter on the destination node. Both
counters must refer to the same color. The calculation is
performed when the value of the counters is in a steady state.
The steady state is an intrinsic characteristic of the marking
method counters because the alternation of color makes the
counters associated with each color still one at a time for the
duration of a marking period.
o Units of Measurement: the method calculates and reports the exact
number of packets sent by the source node and not received by the
o Measurement Point(s) with Potential Measurement Domain: the
measurement can be performed between adjacent nodes, on a per-link
basis, or along a multi-hop path, provided that the traffic under
measurement follows that path. In case of a multi-hop path, the
measurements can be performed both end-to-end and hop-by-hop.
o Measurement Timing: the method has a constraint on the frequency
of measurements. This is detailed in Section 3.2, where it is
specified that the marking period and the guard band interval are
strictly related each other to avoid out-of-order issues. That is
because, in order to perform a measurement, the counter must be in
a steady state, and this happens when the traffic is being colored
with the alternate color. As an example, in the experiment of the
method, the time interval is set to 5 minutes, while other
optimized implementations can also use a marking period of a few
o Implementation: the experiment of the method uses two encodings of
the DSCP field to color the packets; this enables the use of
policy configurations on the router to color the packets and
accordingly configure the counter for each color. The path
followed by traffic being measured should be known in advance in
order to configure the counters along the path and be able to
compare the correct values.
o Verification: both in the lab and in the operational network, the
methodology has been tested and experimented for packet loss and
delay measurements by using traffic generators together with
precision test instruments and network emulators.
o Use and Applications: the method can be used to measure packet
loss with high precision on live traffic; moreover, by combining
end-to-end and per-link measurements, the method is useful to
pinpoint the single link that is experiencing loss events.
o Reporting Model: the value of the counters has to be sent to a
centralized management system that performs the calculations; such
samples must contain a reference to the time interval they refer
to, so that the management system can perform the correct
correlation; the samples have to be sent while the corresponding
counter is in a steady state (within a time interval); otherwise,
the value of the sample should be stored locally.
o Dependencies: the values of the counters have to be correlated to
the time interval they refer to; moreover, because the experiment
of the method is based on DSCP values, there are significant
dependencies on the usage of the DSCP field: it must be possible
to rely on unused DSCP values without affecting QoS-related
configuration and behavior; moreover, the intermediate nodes must
not change the value of the DSCP field not to alter the
o Organization of Results: the Method of Measurement produces
o Parameters: currently, the main parameter of the method is the
time interval used to alternate the colors and read the counters.
8. IANA Considerations
This document has no IANA actions.
9. Security Considerations
This document specifies a method to perform measurements in the
context of a Service Provider's network and has not been developed to
conduct Internet measurements, so it does not directly affect
Internet security nor applications that run on the Internet.
However, implementation of this method must be mindful of security
and privacy concerns.
There are two types of security concerns: potential harm caused by
the measurements and potential harm to the measurements.
o Harm caused by the measurement: the measurements described in this
document are Passive, so there are no new packets injected into
the network causing potential harm to the network itself and to
data traffic. Nevertheless, the method implies modifications on
the fly to a header or encapsulation of the data packets: this
must be performed in a way that doesn't alter the quality of
service experienced by packets subject to measurements and that
preserves stability and performance of routers doing the
measurements. One of the main security threats in OAM protocols
is network reconnaissance; an attacker can gather information
about the network performance by passively eavesdropping on OAM
messages. The advantage of the methods described in this document
is that the marking bits are the only information that is
exchanged between the network devices. Therefore, Passive
eavesdropping on data-plane traffic does not allow attackers to
gain information about the network performance.
o Harm to the Measurement: the measurements could be harmed by
routers altering the marking of the packets or by an attacker
injecting artificial traffic. Authentication techniques, such as
digital signatures, may be used where appropriate to guard against
injected traffic attacks. Since the measurement itself may be
affected by routers (or other network devices) along the path of
IP packets intentionally altering the value of marking bits of
packets, as mentioned above, the mechanism specified in this
document can be applied just in the context of a controlled
domain; thus, the routers (or other network devices) are locally
administered and this type of attack can be avoided. In addition,
an attacker can't gain information about network performance from
a single monitoring point; it must use synchronized monitoring
points at multiple points on the path, because they have to do the
same kind of measurement and aggregation that Service Providers
using Alternate Marking must do.
The privacy concerns of network measurement are limited because the
method only relies on information contained in the header or
encapsulation without any release of user data. Although information
in the header or encapsulation is metadata that can be used to
compromise the privacy of users, the limited marking technique in
this document seems unlikely to substantially increase the existing
privacy risks from header or encapsulation metadata. It might be
theoretically possible to modulate the marking to serve as a covert
channel, but it would have a very low data rate if it is to avoid
adversely affecting the measurement systems that monitor the marking.
Delay attacks are another potential threat in the context of this
document. Delay measurement is performed using a specific packet in
each block, marked by a dedicated color bit. Therefore, a
man-in-the-middle attacker can selectively induce synthetic delay
only to delay-colored packets, causing systematic error in the delay
measurements. As discussed in previous sections, the methods
described in this document rely on an underlying time synchronization
protocol. Thus, by attacking the time protocol, an attacker can
potentially compromise the integrity of the measurement. A detailed
discussion about the threats against time protocols and how to
mitigate them is presented in RFC 7384 [RFC7384].
10.1. Normative References
IEEE, "IEEE Standard for a Precision Clock Synchronization
Protocol for Networked Measurement and Control Systems",
IEEE Std 1588-2008.
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119,
DOI 10.17487/RFC2119, March 1997,
[RFC3393] Demichelis, C. and P. Chimento, "IP Packet Delay Variation
Metric for IP Performance Metrics (IPPM)", RFC 3393,
DOI 10.17487/RFC3393, November 2002,
Bryant, S., Chen, M., Li, Z., Swallow, G., Sivabalan, S.,
Mirsky, G., and G. Fioccola, "RFC6374 Synonymous Flow
Labels", Work in Progress, draft-ietf-mpls-rfc6374-sfl-01,
The previous IETF specifications describing this technique were:
[IP-MULTICAST-PM] and [OPSAWG-P3M].
The authors would like to thank Alberto Tempia Bonda, Domenico
Laforgia, Daniele Accetta, and Mario Bianchetti for their
contribution to the definition and the implementation of the method.
The authors would also thank Spencer Dawkins, Carlos Pignataro, Brian
Haberman, and Eric Vyncke for their assistance and their detailed and
Giuseppe Fioccola (editor)
Via Reiss Romoli, 274
Via Reiss Romoli, 274
Via Reiss Romoli, 274
Via Reiss Romoli, 274
United States of America
6 Hamada St.