4. Functional Guidelines
Supporting functions and related interfaces over the peering point
that enable the multicast transport of the application are listed in
this section. Critical information parameters that need to be
exchanged in support of these functions are enumerated, along with
guidelines as appropriate. Specific interface functions for
consideration are as follows.
4.1. Network Interconnection Transport Guidelines
The term "network interconnection transport" refers to the
interconnection points between the two administrative domains. The
following is a representative set of attributes that the two
administrative domains will need to agree on to support multicast
o Number of peering points.
o Peering point addresses and locations.
o Connection type - Dedicated for multicast delivery or shared with
o Connection mode - Direct connectivity between the two ADs or via
o Peering point protocol support - Multicast protocols that will be
used for multicast delivery will need to be supported at these
points. Examples of such protocols include External BGP (EBGP)
[RFC4760] peering via MP-BGP (Multiprotocol BGP) SAFI-2 [RFC4760].
o Bandwidth allocation - If shared with other services, then there
needs to be a determination of the share of bandwidth reserved for
multicast delivery. See Section 4.1.1 below for more details.
o QoS requirements - Delay and/or latency specifications that need
to be specified in an SLA.
o AD roles and responsibilities - The role played by each AD for
provisioning and maintaining the set of peering points to support
4.1.1. Bandwidth Management
Like IP unicast traffic, IP multicast traffic carried across
non-controlled networks must comply with congestion control
principles as described in [BCP41] and as explained in detail for UDP
IP multicast in [BCP145].
Non-controlled networks (such as the Internet) are networks where
there is no policy for managing bandwidth other than best effort with
a fair share of bandwidth under congestion. As a simplified rule of
thumb, complying with congestion control principles means reducing
bandwidth under congestion in a way that is fair to competing
(typically TCP) flows ("rate adaptive").
In many instances, multicast content delivery evolves from
intra-domain deployments where it is handled as a controlled network
service and does not comply with congestion control principles. It
was given a reserved amount of bandwidth and admitted to the network
so that congestion never occurs. Therefore, the congestion control
issue should be given specific attention when evolving to an
inter-domain peering deployment.
In the case where end-to-end IP multicast traffic passes across the
network of two ADs (and their subsidiaries/customers), both ADs must
agree on a consistent traffic-management policy. If, for example,
AD-1 sources non-congestion-aware IP multicast traffic and AD-2
carries it as best-effort traffic across links shared with other
Internet traffic (subject to congestion), this will not work: under
congestion, some amount of that traffic will be dropped, often
rendering the remaining packets as undecodable garbage clogging up
the network in AD-2; because this traffic is not congestion aware,
the loss does not reduce this rate. Competing traffic will not get
their fair share under congestion, and EUs will be frustrated by the
extremely bad quality of both their IP multicast traffic and other
(e.g., TCP) traffic. Note that this is not an IP multicast
technology issue but is solely a transport-layer / application-layer
issue: the problem would just as likely happen if AD-1 were to send
non-rate-adaptive unicast traffic -- for example, legacy IPTV
video-on-demand traffic, which is typically also non-congestion
aware. Note that because rate adaptation in IP unicast video is
commonplace today due to the availability of ABR (Adaptive Bitrate)
video, it is very unlikely that this will happen in reality with IP
While the rules for traffic management apply whether IP multicast is
tunneled or not, the one feature that can make AMT tunnels more
difficult is the unpredictability of bandwidth requirements across
underlying links because of the way they can be used: with native IP
multicast or GRE tunnels, the amount of bandwidth depends on the
amount of content -- not the number of EUs -- and is therefore easier
to plan for. AMT tunnels terminating in the EU/G, on the other hand,
scale with the number of EUs. In the vicinity of the AMT relay, they
can introduce a very large amount of replicated traffic, and it is
not always feasible to provision enough bandwidth for all possible
EUs to get the highest quality for all their content during peak
utilization in such setups -- unless the AMT relays are very close to
the EU edge. Therefore, it is also recommended that IP multicast
rate adaptation be used, even inside controlled networks, when using
AMT tunnels directly to the EU/G.
Note that rate-adaptive IP multicast traffic in general does not mean
that the sender is reducing the bitrate but rather that the EUs that
experience congestion are joining to a lower-bitrate (S,G) stream of
the content, similar to ABR streaming over TCP. Therefore, migration
from a non-rate-adaptive bitrate to a rate-adaptive bitrate in IP
multicast will also change the dynamic (S,G) join behavior in the
network, resulting in potentially higher performance requirements for
IP multicast protocols (IGMP/PIM), especially on the last hops where
dynamic changes occur (including AMT gateways/relays): in non-rate-
adaptive IP multicast, only "channel change" causes state change, but
in rate-adaptive multicast, congestion also causes state change.
Even though not fully specified in this document, peerings that rely
on GRE/AMT tunnels may be across one or more transit ADs instead of
an exclusive (non-shared, L1/L2) path. Unless those transit ADs are
explicitly contracted to provide other than "best effort" transit for
the tunneled traffic, the tunneled IP multicast traffic must be
rate adaptive in order to not violate BCP 41 across those
4.2. Routing Aspects and Related Guidelines
The main objective for multicast delivery routing is to ensure that
the EU receives the multicast stream from the "most optimal" source
[INF_ATIS_10], which typically:
o Maximizes the multicast portion of the transport and minimizes any
unicast portion of the delivery, and
o Minimizes the overall combined route distance of the network(s).
This routing objective applies to both native multicast and AMT; the
actual methodology of the solution will be different for each.
Regardless, the routing solution is expected to:
o Be scalable,
o Avoid or minimize new protocol development or modifications, and
o Be robust enough to achieve high reliability and to automatically
adjust to changes and problems in the multicast infrastructure.
For both native and AMT environments, having a source as close as
possible to the EU network is most desirable; therefore, in some
cases, an AD may prefer to have multiple sources near different
peering points. However, that is entirely an implementation issue.
4.2.1. Native Multicast Routing Aspects
Native multicast simply requires that the administrative domains
coordinate and advertise the correct source address(es) at their
network interconnection peering points (i.e., BRs). An example of
multicast delivery via a native multicast process across two
administrative domains is as follows, assuming that the
interconnecting peering points are also multicast enabled:
o Appropriate information is obtained by the EU client, who is a
subscriber to AD-2 (see Use Case 3.1). This information is in the
form of metadata, and it contains instructions directing the EU
client to launch an appropriate application if necessary, as well
as additional information for the application about the source
location and the group (or stream) ID in the form of (S,G) data.
The "S" portion provides the name or IP address of the source of
the multicast stream. The metadata may also contain alternate
delivery information, such as specifying the unicast address of
o The client uses the join message with (S,G) to join the multicast
stream [RFC4604]. To facilitate this process, the two ADs need to
do the following:
* Advertise the source ID(s) over the peering points.
* Exchange such relevant peering point information as capacity
* Implement compatible multicast protocols to ensure proper
multicast delivery across the peering points.
4.2.2. GRE Tunnel over Interconnecting Peering Point
If the interconnecting peering point is not multicast enabled and
both ADs are multicast enabled, then a simple solution is to
provision a GRE tunnel between the two ADs; see Use Case 3.2
(Section 3.2). The termination points of the tunnel will usually be
a network engineering decision but generally will be between the BRs
or even between the AD-2 BR and the AD-1 source (or source access
router). The GRE tunnel would allow end-to-end native multicast or
AMT multicast to traverse the interface. Coordination and
advertisement of the source IP are still required.
The two ADs need to follow the same process as the process described
in Section 4.2.1 to facilitate multicast delivery across the peering
4.2.3. Routing Aspects with AMT Tunnels
Unlike native multicast (with or without GRE), an AMT multicast
environment is more complex. It presents a two-layered problem
in that there are two criteria that should be simultaneously met:
o Find the closest AMT relay to the EU that also has multicast
connectivity to the content source, and
o Minimize the AMT unicast tunnel distance.
There are essentially two components in the AMT specification:
AMT relays: These serve the purpose of tunneling UDP multicast
traffic to the receivers (i.e., endpoints). The AMT relay will
receive the traffic natively from the multicast media source and
will replicate the stream on behalf of the downstream AMT
gateways, encapsulating the multicast packets into unicast packets
and sending them over the tunnel toward the AMT gateways. In
addition, the AMT relay may collect various usage and activity
statistics. This results in moving the replication point closer
to the EU and cuts down on traffic across the network. Thus, the
linear costs of adding unicast subscribers can be avoided.
However, unicast replication is still required for each requesting
endpoint within the unicast-only network.
AMT gateway: The gateway will reside on an endpoint; this could be
any type of IP host, such as a Personal Computer (PC), mobile
phone, Set-Top Box (STB), or appliances. The AMT gateway receives
join and leave requests from the application via an Application
Programming Interface (API). In this manner, the gateway allows
the endpoint to conduct itself as a true multicast endpoint. The
AMT gateway will encapsulate AMT messages into UDP packets and
send them through a tunnel (across the unicast-only
infrastructure) to the AMT relay.
The simplest AMT use case (Section 3.3) involves peering points that
are not multicast enabled between two multicast-enabled ADs. An
AMT tunnel is deployed between an AMT relay on the AD-1 side of the
peering point and an AMT gateway on the AD-2 side of the peering
point. One advantage of this arrangement is that the tunnel is
established on an as-needed basis and need not be a provisioned
element. The two ADs can coordinate and advertise special AMT relay
anycast addresses with, and to, each other. Alternately, they may
decide to simply provision relay addresses, though this would not be
an optimal solution in terms of scalability.
Use Cases 3.4 and 3.5 describe AMT situations that are more
complicated, as AD-2 is not multicast enabled in these two cases.
For these cases, the EU device needs to be able to set up an AMT
tunnel in the most optimal manner. There are many methods by which
relay selection can be done, including the use of DNS-based queries
and static lookup tables [RFC7450]. The choice of the method is
implementation dependent and is up to the network operators.
Comparison of various methods is out of scope for this document and
is left for further study.
An illustrative example of a relay selection based on DNS queries as
part of an anycast IP address process is described here for Use
Cases 3.4 and 3.5 (Sections 3.4 and 3.5). Using an anycast
IP address for AMT relays allows all AMT gateways to find the
"closest" AMT relay -- the nearest edge of the multicast topology of
the source. Note that this is strictly illustrative; the choice of
the method is up to the network operators. The basic process is as
o Appropriate metadata is obtained by the EU client application.
The metadata contains instructions directing the EU client to an
ordered list of particular destinations to seek the requested
stream and, for multicast, specifies the source location and the
group (or stream) ID in the form of (S,G) data. The "S" portion
provides the URI (name or IP address) of the source of the
multicast stream, and the "G" identifies the particular stream
originated by that source. The metadata may also contain
alternate delivery information such as the address of the unicast
form of the content to be used -- for example, if the multicast
stream becomes unavailable.
o Using the information from the metadata and, possibly, information
provisioned directly in the EU client, a DNS query is initiated in
order to connect the EU client / AMT gateway to an AMT relay.
o Query results are obtained and may return an anycast address or a
specific unicast address of a relay. Multiple relays will
typically exist. The anycast address is a routable
"pseudo-address" shared among the relays that can gain multicast
access to the source.
o If a specific IP address unique to a relay was not obtained, the
AMT gateway then sends a message (e.g., the discovery message) to
the anycast address such that the network is making the routing
choice of a particular relay, e.g., the relay that is closest to
the EU. Details are outside the scope of this document. See
o The contacted AMT relay then returns its specific unicast IP
address (after which the anycast address is no longer required).
Variations may exist as well.
o The AMT gateway uses that unicast IP address to initiate a
three-way handshake with the AMT relay.
o The AMT gateway provides the (S,G) information to the AMT relay
(embedded in AMT protocol messages).
o The AMT relay receives the (S,G) information and uses it to join
the appropriate multicast stream, if it has not already subscribed
to that stream.
o The AMT relay encapsulates the multicast stream into the tunnel
between the relay and the gateway, providing the requested content
to the EU.
4.2.4. Public Peering Routing Aspects
Figure 6 shows an example of a broadcast peering point.
--+-+---------------+-+-- broadcast peering point LAN
Figure 6: Broadcast Peering Point
A broadcast peering point is an L2 subnet connecting three or more
ADs. It is common in IXPs and usually consists of Ethernet
switch(es) operated by the IXP connecting to BRs operated by the ADs.
In an example setup domain, AD-2a peers with AD-1a and wants to
receive IP multicast from it. Likewise, AD-2b peers with AD-1b and
wants to receive IP multicast from it.
Assume that one or more IP multicast (S,G) traffic streams can be
served by both AD-1a and AD-1b -- for example, because both AD-1a and
AD-1b contact this content from the same content source.
In this case, AD-2a and AD-2b can no longer control which upstream
domain -- AD-1a or AD-1b -- will forward this (S,G) into the LAN.
The AD-2a BR requests the (S,G) from the AD-1a BR, and the AD-2b BR
requests the same (S,G) from the AD-1b BR. To avoid duplicate
packets, an (S,G) can be forwarded by only one router onto the LAN;
PIM-SM / PIM-SSM detects requests for duplicate transmissions and
resolves them via the so-called "assert" protocol operation, which
results in only one BR forwarding the traffic. Assume that this is
the AD-1a BR. AD-2b will then receive unexpected multicast traffic
from a provider with whom it does not have a mutual agreement for
that traffic. Quality issues in EUs behind AD-2b caused by AD-1a
will cause a lot of issues related to responsibility and
In light of these technical issues, we describe, via the following
options, how IP multicast can be carried across broadcast peering
1. IP multicast is tunneled across the LAN. Any of the GRE/AMT
tunneling solutions mentioned in this document are applicable.
This is the one case where a GRE tunnel between the upstream BR
(e.g., AD-1a) and downstream BR (e.g., AD-2a) is specifically
recommended, as opposed to tunneling across uBRs (which are not
the actual BRs).
2. The LAN has only one upstream AD that is sourcing IP multicast,
and native IP multicast is used. This is an efficient way to
distribute the same IP multicast content to multiple downstream
ADs. Misbehaving downstream BRs can still disrupt the delivery
of IP multicast from the upstream BR to other downstream BRs;
therefore, strict rules must be followed to prohibit such a case.
The downstream BRs must ensure that they will always consider
only the upstream BR as a source for multicast traffic: e.g., no
BGP SAFI-2 peerings between the downstream ADs across the peering
point LAN, so that the upstream BR is the only possible next hop
reachable across this LAN. Also, routing policies can be
configured to avoid falling back to using SAFI-1 (unicast) routes
for IP multicast if unicast BGP peering is not limited in the
3. The LAN has multiple upstream ADs, but they are federated and
agree on a consistent policy for IP multicast traffic across the
LAN. One policy is that each possible source is only announced
by one upstream BR. Another policy is that sources are
redundantly announced (the problematic case mentioned in the
example in Figure 6 above), but the upstream domains also provide
mutual operational insight to help with troubleshooting (outside
the scope of this document).
4.3. Back-Office Functions - Provisioning and Logging Guidelines
"Back office" refers to the following:
o Servers and content-management systems that support the delivery
of applications via multicast and interactions between ADs.
o Functionality associated with logging, reporting, ordering,
provisioning, maintenance, service assurance, settlement, etc.
4.3.1. Provisioning Guidelines
Resources for basic connectivity between ADs' providers need to be
provisioned as follows:
o Sufficient capacity must be provisioned to support multicast-based
delivery across ADs.
o Sufficient capacity must be provisioned for connectivity between
all supporting back offices of the ADs as appropriate. This
includes activating proper security treatment for these
back-office connections (gateways, firewalls, etc.) as
Provisioning aspects related to multicast-based inter-domain delivery
are as follows.
The ability to receive a requested application via multicast is
triggered via receipt of the necessary metadata. Hence, this
metadata must be provided to the EU regarding the multicast URL --
and unicast fallback if applicable. AD-2 must enable the delivery of
this metadata to the EU and provision appropriate resources for this
It is assumed that native multicast functionality is available across
many ISP backbones, peering points, and access networks. If,
however, native multicast is not an option (Use Cases 3.4 and 3.5),
o The EU must have a multicast client to use AMT multicast obtained
from either (1) the application source (per agreement with AD-1)
or (2) AD-1 or AD-2 (if delegated by the application source).
o If provided by AD-1 or AD-2, then the EU could be redirected to a
client download site. (Note: This could be an application source
site.) If provided by the application source, then this source
would have to coordinate with AD-1 to ensure that the proper
client is provided (assuming multiple possible clients).
o Where AMT gateways support different application sets, all AD-2
AMT relays need to be provisioned with all source and group
addresses for streams it is allowed to join.
o DNS across each AD must be provisioned to enable a client gateway
to locate the optimal AMT relay (i.e., longest multicast path and
shortest unicast tunnel) with connectivity to the content's
Provisioning aspects related to operations and customer care are as
It is assumed that each AD provider will provision operations and
customer care access to their own systems.
AD-1's operations and customer care functions must be able to see
enough of what is happening in AD-2's network or in the service
provided by AD-2 to verify their mutual goals and operations, e.g.,
to know how the EUs are being served. This can be done in two ways:
o Automated interfaces are built between AD-1 and AD-2 such that
operations and customer care continue using their own systems.
This requires coordination between the two ADs, with appropriate
provisioning of necessary resources.
o AD-1's operations and customer care personnel are provided direct
access to AD-2's systems. In this scenario, additional
provisioning in these systems will be needed to provide necessary
access. The two ADs must agree on additional provisioning to
support this option.
4.3.2. Inter-domain Authentication Guidelines
All interactions between pairs of ADs can be discovered and/or
associated with the account(s) utilized for delivered applications.
Supporting guidelines are as follows:
o A unique identifier is recommended to designate each master
o AD-2 is expected to set up "accounts" (a logical facility
generally protected by credentials such as login passwords) for
use by AD-1. Multiple accounts, and multiple types or partitions
of accounts, can apply, e.g., customer accounts, security
The reason to specifically mention the need for AD-1 to initiate
interactions with AD-2 (and use some account for that), as opposed to
the opposite, is based on the recommended workflow initiated by
customers (see Section 4.4): the customer contacts the content
source, which is part of AD-1. Consequently, if AD-1 sees the need
to escalate the issue to AD-2, it will interact with AD-2 using the
4.3.3. Log-Management Guidelines
content via multicast between pairs of interconnecting ADs can be
improved through the ability to exchange appropriate logs for various
workflows -- troubleshooting, accounting and billing, optimization of
traffic and content transmission, optimization of content and
application development, and so on.
Specifically, AD-1 take over primary responsibility for customer
experience on behalf of the content source, with support from AD-2 as
needed. The application/content owner is the only participant who
has, and needs, full insight into the application level and can map
the customer application experience to the network traffic flows --
which, with the help of AD-2 or logs from AD-2, it can then analyze
The main difference between unicast delivery and multicast delivery
is that the content source can infer a lot more about downstream
network problems from a unicast stream than from a multicast stream:
the multicast stream is not per EU, except after the last
replication, which is in most cases not in AD-1. Logs from the
application, including the receiver side at the EU, can provide
insight but cannot help to fully isolate network problems because of
the IP multicast per-application operational state built across AD-1
and AD-2 (aka the (S,G) state and any other operational-state
features, such as Diffserv QoS).
See Section 7 for more discussion regarding the privacy
considerations of the model described here.
Different types of logs are known to help support operations in AD-1
when provided by AD-2. This could be done as part of AD-1/AD-2
contracts. Note that except for implied multicast-specific elements,
the options listed here are not unique or novel for IP multicast, but
they are more important for services novel to the operators than for
operationally well-established services (such as unicast). We
therefore detail them as follows:
o Usage information logs at an aggregate level.
o Usage failure instances at an aggregate level.
o Grouped or sequenced application access: performance, behavior,
and failure at an aggregate level to support potential
application-provider-driven strategies. Examples of aggregate
levels include grouped video clips, web pages, and software-
o Security logs, aggregated or summarized according to agreement
(with additional detail potentially provided during security
events, by agreement).
o Access logs (EU), when needed for troubleshooting.
o Application logs ("What is the application doing?"), when needed
for shared troubleshooting.
o Syslogs (network management), when needed for shared
The two ADs may supply additional security logs to each other, as
agreed upon in contract(s). Examples include the following:
o Information related to general security-relevant activity, which
may be of use from a protection or response perspective: types and
counts of attacks detected, related source information, related
target information, etc.
o Aggregated or summarized logs according to agreement (with
additional detail potentially provided during security events, by
4.4. Operations - Service Performance and Monitoring Guidelines
"Service performance" refers to monitoring metrics related to
multicast delivery via probes. The focus is on the service provided
by AD-2 to AD-1 on behalf of all multicast application sources
(metrics may be specified for SLA use or otherwise). Associated
guidelines are as follows:
o Both ADs are expected to monitor, collect, and analyze service
performance metrics for multicast applications. AD-2 provides
relevant performance information to AD-1; this enables AD-1 to
create an end-to-end performance view on behalf of the multicast
o Both ADs are expected to agree on the types of probes to be used
to monitor multicast delivery performance. For example, AD-2 may
permit AD-1's probes to be utilized in the AD-2 multicast service
footprint. Alternately, AD-2 may deploy its own probes and relay
performance information back to AD-1.
"Service monitoring" generally refers to a service (as a whole)
provided on behalf of a particular multicast application source
provider. It thus involves complaints from EUs when service problems
occur. EUs direct their complaints to the source provider; the
source provider in turn submits these complaints to AD-1. The
responsibility for service delivery lies with AD-1; as such, AD-1
will need to determine where the service problem is occurring -- in
its own network or in AD-2. It is expected that each AD will have
tools to monitor multicast service status in its own network.
o Both ADs will determine how best to deploy multicast service
monitoring tools. Typically, each AD will deploy its own set of
monitoring tools, in which case both ADs are expected to inform
each other when multicast delivery problems are detected.
o AD-2 may experience some problems in its network. For example,
for the AMT use cases (Sections 3.3, 3.4, and 3.5), one or more
AMT relays may be experiencing difficulties. AD-2 may be able to
fix the problem by rerouting the multicast streams via alternate
AMT relays. If the fix is not successful and multicast service
delivery degrades, then AD-2 needs to report the issue to AD-1.
o When a problem notification is received from a multicast
application source, AD-1 determines whether the cause of the
problem is within its own network or within AD-2. If the cause is
within AD-2, then AD-1 supplies all necessary information to AD-2.
Examples of supporting information include the following:
* Kind(s) of problem(s).
* Starting point and duration of problem(s).
* Conditions in which one or more problems occur.
* IP address blocks of affected users.
* ISPs of affected users.
* Type of access, e.g., mobile versus desktop.
* Network locations of affected EUs.
o Both ADs conduct some form of root-cause analysis for multicast
service delivery problems. Examples of various factors for
* Verification that the service configuration matches the product
* Correlation and consolidation of the various customer problems
and resource troubles into a single root-service problem.
* Prioritization of currently open service problems, giving
consideration to problem impacts, SLAs, etc.
* Conducting service tests, including tests performed once or a
series of tests over a period of time.
* Analysis of test results.
* Analysis of relevant network fault or performance data.
* Analysis of the problem information provided by the customer.
o Once the cause of the problem has been determined and the problem
has been fixed, both ADs need to work jointly to verify and
validate the success of the fix.
4.5. Client Reliability Models / Service Assurance Guidelines
There are multiple options for instituting reliability architectures.
Most are at the application level. Both ADs should work these
options out per their contract or agreement and also with the
multicast application source providers.
Network reliability can also be enhanced by the two ADs if they
provision alternate delivery mechanisms via unicast means.
4.6. Application Accounting Guidelines
Application-level accounting needs to be handled differently in the
application than in IP unicast, because the source side does not
directly deliver packets to individual receivers. Instead, this
needs to be signaled back by the receiver to the source.
For network transport diagnostics, AD-1 and AD-2 should have
mechanisms in place to ensure proper accounting for the volume of
bytes delivered through the peering point and, separately, the number
of bytes delivered to EUs.