Until this point, we have made the assumption that hosts are able to choose the correct source address using some unspecified mechanism. This has allowed us to focus on what the routers in a multihomed site network need to do in order to forward packets to the correct ISP based on source address. Now we look at possible mechanisms for hosts to choose the correct source address. We also look at what role, if any, the routers may play in providing information that helps hosts to choose source addresses.
It should be noted that this section discusses how hosts could select the default source address for new connections. Any connection that already exists on a host is bound to a specific source address that cannot be changed. Section 6.7
discusses the connections preservation issue in more detail.
Any host that needs to be able to send traffic using the uplinks to a given ISP is expected to be configured with an address from the prefix assigned by that ISP. The host will control which ISP is used for its traffic by selecting one of the addresses configured on the host as the source address for outgoing traffic. It is the responsibility of the site network to ensure that a packet with the source address from an ISP is now sent on an uplink to that ISP.
If all of the ISP uplinks are working, then the host's choice of source address may be driven by the desire to load share across ISP uplinks, or it may be driven by the desire to take advantage of certain properties of a particular uplink or ISP (if some information about various path properties has been made available to the host somehow. See [PROV-DOMAINS
] as an example). If any of the ISP uplinks is not working, then the choice of source address by the host can cause packets to get dropped.
How a host selects a suitable source address in a multihomed site is not a solved problem. We do not attempt to solve this problem in this document. Instead we discuss the current state of affairs with respect to standardized solutions and the implementation of those solutions. We also look at proposed solutions for this problem.
An external host initiating communication with a host internal to a PA-multihomed site will need to know multiple addresses for that host in order to communicate with it using different ISPs to the multihomed site (knowing just one address would undermine all benefits of redundant connectivity provided by multihoming). These addresses are typically learned through DNS. (For simplicity, we assume that the external host is single-homed.) The external host chooses the ISP that will be used at the remote multihomed site by setting the destination address on the packets it transmits. For a session originated from an external host to an internal host, the choice of source address used by the internal host is simple. The internal host has no choice but to use the destination address in the received packet as the source address of the transmitted packet.
For a session originated by a host inside the multihomed site, the decision of which source address to select is more complicated. We consider three main methods for hosts to get information about the network. The two proactive methods are Neighbor Discovery Router Advertisements and DHCPv6. The one reactive method we consider is ICMPv6. Note that we are explicitly excluding the possibility of having hosts participate in, or even listen directly to, routing protocol advertisements.
First we look at how a host is currently expected to select the default source and destination addresses to be used for a new connection.
] defines the algorithms that hosts are expected to use to select source and destination addresses for packets. It defines an algorithm for selecting a source address and a separate algorithm for selecting a destination address. Both of these algorithms depend on a policy table. [RFC 6724
] defines a default policy that produces certain behavior.
The rules in the two algorithms in [RFC 6724
] depend on many different properties of addresses. While these are needed for understanding how a host should choose addresses in an arbitrary environment, most of the rules are not relevant for understanding how a host should choose among multiple source addresses in a multihomed environment when sending a packet to a remote host. Returning to the example in Figure 3
, we look at what the default algorithms in [RFC 6724
] say about the source address that internal host H31 should use to send traffic to external host H101, somewhere on the Internet.
There is no choice to be made with respect to destination address. H31 needs to send a packet with D=2001:db8:0:1234::101 in order to reach H101. So H31 has to choose between using S=2001:db8:0:a010::31 or S=2001:db8:0:b010::31 as the source address for this packet. We go through the rules for source address selection in Section 5
of RFC 6724
Rule 1 (Prefer same address) is not useful to break the tie between source addresses because neither one of the candidate source addresses equals the destination address.
Rule 2 (Prefer appropriate scope) is also not useful in this scenario because both source addresses and the destination address have global scope.
Rule 3 (Avoid deprecated addresses) applies to an address that has been autoconfigured by a host using stateless address autoconfiguration as defined in [RFC 4862
]. An address autoconfigured by a host has a preferred lifetime and a valid lifetime. The address is preferred until the preferred lifetime expires, after which it becomes deprecated. A deprecated address is not used if there is a preferred address of the appropriate scope available. When the valid lifetime expires, the address cannot be used at all. The preferred and valid lifetimes for an autoconfigured address are set based on the corresponding lifetimes in the Prefix Information Option in Neighbor Discovery Router Advertisements. In this scenario, a possible tool to control source address selection in this scenario would be for a host to deprecate an address by having routers on that link, R1 and R2 in Figure 3
, send Router Advertisement messages containing PIOs with the Preferred Lifetime value for the deprecated source prefix set to zero. This is a rather blunt tool, because it discourages or prohibits the use of that source prefix for all destinations. However, it may be useful in some scenarios. For example, if all uplinks to a particular ISP fail, it is desirable to prevent hosts from using source addresses from that ISP address space.
Rule 4 (Avoid home addresses) does not apply here because we are not considering Mobile IP.
Rule 5 (Prefer outgoing interface) is not useful in this scenario because both source addresses are assigned to the same interface.
Rule 5.5 (Prefer addresses in a prefix advertised by the next-hop) is not useful in the scenario when both R1 and R2 will advertise both source prefixes. However, this rule may potentially allow a host to select the correct source prefix by selecting a next-hop. The most obvious way would be to make R1 advertise itself as a default router and send PIO for 2001:db8:0:a010::/64, while R2 advertises itself as a default router and sends PIO for 2001:db8:0:b010::/64. We'll discuss later how Rule 5.5 can be used to influence a source address selection in single-router topologies (e.g., when H41 is sending traffic using R3 as a default gateway).
Rule 6 (Prefer matching label) refers to the label value determined for each source and destination prefix as a result of applying the policy table to the prefix. With the default policy table defined in Section 2.1
of RFC 6724
, Label(2001:db8:0:a010::31) = 5, Label(2001:db8:0:b010::31) = 5, and Label(2001:db8:0:1234::101) = 5. So with the default policy, Rule 6 does not break the tie. However, the algorithms in [RFC 6724
] are defined in such a way that non-default address selection policy tables can be used. [RFC 7078
] defines a way to distribute a non-default address selection policy table to hosts using DHCPv6. So even though the application of Rule 6 to this scenario using the default policy table is not useful, Rule 6 may still be a useful tool.
Rule 7 (Prefer temporary addresses) has to do with the technique described in [RFC 4941
] to periodically randomize the interface portion of an IPv6 address that has been generated using stateless address autoconfiguration. In general, if H31 were using this technique, it would use it for both source addresses, for example, creating temporary addresses 2001:db8:0:a010:2839:9938:ab58:830f and 2001:db8:0:b010:4838:f483:8384:3208, in addition to 2001:db8:0:a010::31 and 2001:db8:0:b010::31. So this rule would prefer the two temporary addresses, but it would not break the tie between the two source prefixes from ISP-A and ISP-B.
Rule 8 (Use longest matching prefix) dictates that, between two candidate source addresses, the host selects the one that has longest common prefix length with the destination address. For example, if H31 were selecting the source address for sending packets to H101, this rule would not break the tie between candidate source addresses 2001:db8:0:a101::31 and 2001:db8:0:b101::31 since the common prefix length with the destination is 48. However, if H31 were selecting the source address for sending packets to H41 address 2001:db8:0:a020::41, then this rule would result in using 2001:db8:0:a101::31 as a source (2001:db8:0:a101::31 and 2001:db8:0:a020::41 share the common prefix 2001:db8:0:a000::/58, while for 2001:db8:0:b101::31 and 2001:db8:0:a020::41, the common prefix is 2001:db8:0:a000::/51). Therefore Rule 8 might be useful for selecting the correct source address in some, but not all, scenarios (for example if ISP-B services belong to 2001:db8:0:b000::/59, then H31 would always use 2001:db8:0:b010::31 to access those destinations).
So we can see that of the eight source address selection rules from [RFC 6724
], four actually apply to our basic site multihoming scenario. The rules that are relevant to this scenario are summarized below.
Rule 3: Avoid deprecated addresses.
Rule 5.5: Prefer addresses in a prefix advertised by the next-hop.
Rule 6: Prefer matching label.
Rule 8: Prefer longest matching prefix.
The two methods that we discuss for controlling the source address selection through the four relevant rules above are SLAAC Router Advertisement messages and DHCPv6.
We also consider a possible role for ICMPv6 for getting traffic-driven feedback from the network. With the source address selection algorithm discussed above, the goal is to choose the correct source address on the first try, before any traffic is sent. However, another strategy is to choose a source address, send the packet, get feedback from the network about whether or not the source address is correct, and try another source address if it is not.
We consider four scenarios in which a host needs to select the correct source address. In the first scenario, both uplinks are working. In the second scenario, one uplink has failed. The third scenario is a situation in which one failed uplink has recovered. The last scenario is failure of both (all) uplinks.
It should be noted that [RFC 6724
] only defines the behavior of IPv6 hosts to select default addresses that applications and upper-layer protocols can use. Applications and upper-layer protocols can make their own choices on selecting source addresses. The mechanism proposed in this document attempts to ensure that the subset of source addresses available for applications and upper-layer protocols is selected with the up-to-date network state in mind. The rest of the document discusses various aspects of the default source address selection defined in [RFC 6724
], calling it for the sake of brevity "the source address selection".
Again we return to the topology in Figure 3
. Suppose that the site administrator wants to implement a policy by which all hosts need to use ISP-A to reach H101 at D=2001:db8:0:1234::101. So for example, H31 needs to select S=2001:db8:0:a010::31.
This policy can be implemented by using DHCPv6 to distribute an address selection policy table that assigns the same label to destination addresses that match 2001:db8:0:1234::/64 as it does to source addresses that match 2001:db8:0:a000::/52. The following two entries accomplish this.
Prefix Precedence Label
2001:db8:0:1234::/64 50 33
2001:db8:0:a000::/52 50 33
This requires that the hosts implement [RFC 6724
], the basic source and destination address framework, along with [RFC 7078
], the DHCPv6 extension for distributing a non-default policy table. Note that it does NOT require that the hosts use DHCPv6 for address assignment. The hosts could still use stateless address autoconfiguration for address configuration, while using DHCPv6 only for policy table distribution (see [RFC 8415
]). However this method has a number of disadvantages:
DHCPv6 support is not a mandatory requirement for IPv6 hosts [RFC 8504], so this method might not work for all devices.
Network administrators are required to explicitly configure the desired network access policies on DHCPv6 servers. While it might be feasible in the scenario of a single multihomed network, such approach might have some scalability issues, especially if the centralized DHCPv6 solution is deployed to serve a large number of multihomed sites.
Neighbor Discovery currently has two mechanisms to communicate prefix information to hosts. The base specification for Neighbor Discovery (see [RFC 4861
]) defines the Prefix Information Option (PIO) in the Router Advertisement (RA) message. When a host receives a PIO with the A-flag [RFC 8425
] set, it can use the prefix in the PIO as the source prefix from which it assigns itself an IP address using stateless address autoconfiguration (SLAAC) procedures described in [RFC 4862
]. In the example of Figure 3
, if the site network is using SLAAC, we would expect both R1 and R2 to send RA messages with PIOs with the A-flag set for both source prefixes 2001:db8:0:a010::/64 and 2001:db8:0:b010::/64. H31 would then use the SLAAC procedure to configure itself with 2001:db8:0:a010::31 and 2001:db8:0:b010::31.
Whereas a host learns about source prefixes from PIO messages, hosts can learn about a destination prefix from a Router Advertisement containing a Route Information Option (RIO), as specified in [RFC 4191
]. The destination prefixes in RIOs are intended to allow a host to choose the router that it uses as its first hop to reach a particular destination prefix.
As currently standardized, neither PIO nor RIO options contained in Neighbor Discovery Router Advertisements can communicate the information needed to implement the desired routing policy. PIOs communicate source prefixes, and RIOs communicate destination prefixes. However, there is currently no standardized way to directly associate a particular destination prefix with a particular source prefix.
] proposes a Source Address Dependent Route Information option for Neighbor Discovery Router Advertisements that would associate a source prefix with a destination prefix. The details of [SADR-RA
] might need tweaking to address this use case. However, in order to be able to use Neighbor Discovery Router Advertisements to implement this routing policy, an extension is needed that would allow R1 and R2 to explicitly communicate to H31 an association between S=2001:db8:0:a000::/52 and D=2001:db8:0:1234::/64.
However, Rule 5.5 of the default source address selection algorithm (discussed in Section 6.1
), together with default router preference (specified in [RFC 4191
]) and RIO, can be used to influence a source address selection on a host as described below. Let's look at source address selection on the host H41. It receives RAs from R3 with PIOs for 2001:db8:0:a020::/64 and 2001:db8:0:b020::/64. At that point, all traffic would use the same next-hop (R3 link-local address) so Rule 5.5 does not apply. Now let's assume that R3 supports SADR and has two scoped forwarding tables, one scoped to S=2001:db8:0:a000::/52 and another scoped to S=2001:db8:0:b000::/52. If R3 generates two different link-local addresses for its interface facing H41 (one for each scoped forwarding table, LLA_A and LLA_B), R3 will send two different RAs: one from LLA_A that includes a PIO for 2001:db8:0:a020::/64, and another from LLA_B that includes a PIO for 2001:db8:0:b020::/64. Now it is possible to influence H41 source address selection for destinations that follow the default route by setting the default router preference in RAs. If it is desired that H41 reaches H101 (or any destination in the Internet) via ISP-A, then RAs sent from LLA_A should have the default router preference set to 01 (high priority), while RAs sent from LLA_B should have the preference set to 11 (low). LLA_A would then be chosen as a next-hop for H101, and therefore (per Rule 5.5) 2001:db8:0:a020::41 would be selected as the source address. If, at the same time, it is desired that H61 is accessible via ISP-B, then R3 should include a RIO for 2001:db8:0:6666::/64 in its RA sent from LLA_B. H41 would choose LLA_B as a next-hop for all traffic to H61, and then per Rule 5.5, 2001:db8:0:b020::41 would be selected as a source address.
If in the above-mentioned scenario it is desirable that all Internet traffic leaves the network via ISP-A, and the link to ISP-B is used for accessing ISP-B services only (not as an ISP-A link backup), then RAs sent by R3 from LLA_B should have their Router Lifetime values set to zero and should include RIOs for ISP-B address space. It would instruct H41 to use LLA_A for all Internet traffic but to use LLA_B as a next-hop while sending traffic to ISP-B addresses.
The description of the mechanism above assumes SADR support by the first-hop routers as well as SERs. However, a first-hop router can still provide a less flexible version of this mechanism even without implementing SADR. This could be done by providing configuration knobs on the first-hop router that allow it to generate different link-local addresses and to send individual RAs for each prefix.
The mechanism described above relies on Rule 5.5 of the default source address selection algorithm defined in [RFC 6724
]. [RFC 8028
] states that "A host SHOULD
select default routers for each prefix it is assigned an address in." It also recommends that hosts should implement Rule 5.5. of [RFC 6724
]. Hosts following the recommendations specified in [RFC 8028
] therefore should be able to benefit from the solution described in this document. No standards need to be updated in regards to host behavior.
We now discuss how one might use ICMPv6 to implement the routing policy to send traffic destined for H101 out the uplink to ISP-A, even when uplinks to both ISPs are working. If H31 started sending traffic to H101 with S=2001:db8:0:b010::31 and D=2001:db8:0:1234::101, it would be routed through SER-b1 and out the uplink to ISP-B. SERb1 could recognize that this traffic is not following the desired routing policy and react by sending an ICMPv6 message back to H31.
In this example, we could arrange things so that SERb1 drops the packet with S=2001:db8:0:b010::31 and D=2001:db8:0:1234::101, and then sends to H31 an ICMPv6 Destination Unreachable message with Code 5 (Source address failed ingress/egress policy). When H31 receives this packet, it would then be expected to try another source address to reach the destination. In this example, H31 would then send a packet with S=2001:db8:0:a010::31 and D=2001:db8:0:1234::101, which will reach SERa and be forwarded out the uplink to ISP-A.
However, we would also want it to be the case that SERb1 does not enforce this routing policy when the uplink from SERa to ISP-A has failed. This could be accomplished by having SERa originate a source-prefix-scoped route for (S=2001:db8:0:a000::/52, D=2001:db8:0:1234::/64), and have SERb1 monitor the presence of that route. If that route is not present (because SERa has stopped originating it), then SERb1 will not enforce the routing policy, and it will forward packets with S=2001:db8:0:b010::31 and D=2001:db8:0:1234::101 out its uplink to ISP-B.
We can also use this source-prefix-scoped route originated by SERa to communicate the desired routing policy to SERb1. We can define an EXCLUSIVE flag to be advertised together with the IGP route for (S=2001:db8:0:a000::/52, D=2001:db8:0:1234::/64). This would allow SERa to communicate to SERb that SERb should reject traffic for D=2001:db8:0:1234::/64 and respond with an ICMPv6 Destination Unreachable Code 5 message, as long as the route for (S=2001:db8:0:a000::/52, D=2001:db8:0:1234::/64) is present. The definition of an EXCLUSIVE flag for SADR advertisements in IGPs would require future standardization work.
Finally, if we are willing to extend ICMPv6 to support this solution, then we could create a mechanism for SERb1 to tell the host which source address it should be using to successfully forward packets that meet the policy. In its current form, when SERb1 sends an ICMPv6 Destination Unreachable Code 5 message, it is basically saying, "This source address is wrong. Try another source address." In the absence of a clear indication which address to try next, the host will iterate over all addresses assigned to the interface (e.g., various privacy addresses), which would lead to significant delays and degraded user experience. It would be better if the ICMPv6 message could say, "This source address is wrong. Instead use a source address in S=2001:db8:0:a000::/52".
However, using ICMPv6 for signaling source address information back to hosts introduces new challenges. Most routers currently have software or hardware limits on generating ICMP messages. A site administrator deploying a solution that relies on the SERs generating ICMP messages could try to improve the performance of SERs for generating ICMP messages. However, in a large network, it is still likely that ICMP message generation limits will be reached. As a result, hosts would not receive ICMPv6 back, which in turn leads to traffic blackholing and poor user experience. To improve the scalability of ICMPv6-based signaling, hosts SHOULD
cache the preferred source address (or prefix) for the given destination (which in turn might cause issues in the case of the corresponding ISP uplink failure - see Section 6.3
). In addition, the same source prefix SHOULD
be used for other destinations in the same /64 as the original destination address. The source prefix to the destination mapping SHOULD
have a specific lifetime. Expiration of the lifetime SHOULD
trigger the source address selection algorithm again.
Using ICMPv6 Destination Unreachable Messages with Code 5 to influence source address selection introduces some security challenges, which are discussed in Section 10
As currently standardized in [RFC 4443
], the ICMPv6 Destination Unreachable Message with Code 5 would allow for the iterative approach to retransmitting packets using different source addresses. As currently defined, the ICMPv6 message does not provide a mechanism to communicate information about which source prefix should be used for a retransmitted packet. The current document does not define such a mechanism, but it might be a useful extension to define in a different document. However, this approach has some security implications, such as an ability for an attacker to send spoofed ICMPv6 messages to signal an invalid/unreachable source prefix, causing a DoS-type attack.
So to summarize this section, we have looked at three methods for implementing a simple routing policy where all traffic for a given destination on the Internet needs to use a particular ISP, even when the uplinks to both ISPs are working.
The default source address selection policy cannot distinguish between the source addresses needed to enforce this policy, so a non-default policy table using associating source and destination prefixes using label values would need to be installed on each host. A mechanism exists for DHCPv6 to distribute a non-default policy table, but such solution would heavily rely on DHCPv6 support by the host operating system. Moreover, there is no mechanism to translate desired routing/traffic engineering policies into policy tables on DHCPv6 servers. Therefore using DHCPv6 for controlling the address selection policy table is not recommended and SHOULD NOT
At the same time, Router Advertisements provide a reliable mechanism to influence the source address selection process via PIO, RIO, and default router preferences. As all those options have been standardized by the IETF and are supported by various operating systems, no changes are required on hosts. First-hop routers in the enterprise network need to be capable of sending different RAs for different SLAAC prefixes (either based on scoped forwarding tables or based on preconfigured policies).
SERs can enforce the routing policy by sending ICMPv6 Destination Unreachable messages with Code 5 (Source address failed ingress/egress policy) for traffic sent with the wrong source address. The policy distribution could be automated by defining an EXCLUSIVE flag for the source-prefix-scoped route, which could then be set on the SER that originates the route. As ICMPv6 message generation can be rate limited on routers, it SHOULD NOT
be used as the only mechanism to influence source address selection on hosts. While hosts SHOULD
select the correct source address for a given destination, the network SHOULD
signal any source address issues back to hosts using ICMPv6 error messages.
Now we discuss whether DHCPv6, Neighbor Discovery Router Advertisements, and ICMPv6 can help a host choose the right source address when an uplink to one of the ISPs has failed. Again we look at the scenario in Figure 3
. This time we look at traffic from H31 destined for external host H501 at D=2001:db8:0:5678::501. We initially assume that the uplink from SERa to ISP-A is working and that the uplink from SERb1 to ISP-B is working.
We assume there is no particular routing policy desired, so H31 is free to send packets with S=2001:db8:0:a010::31 or S=2001:db8:0:b010::31 and have them delivered to H501. For this example, we assume that H31 has chosen S=2001:db8:0:b010::31 so that the packets exit via SERb to ISP-B. Now we see what happens when the link from SERb1 to ISP-B fails. How should H31 learn that it needs to start sending packets to H501 with S=2001:db8:0:a010::31 in order to start using the uplink to ISP-A? We need to do this in a way that doesn't prevent H31 from still sending packets with S=2001:db8:0:b010::31 in order to reach H61 at D=2001:db8:0:6666::61.
For this example, we assume that the site network in Figure 3
has a centralized DHCP server and that all routers act as DHCP relay agents. We assume that both of the addresses assigned to H31 were assigned via DHCP.
We could try to have the DHCP server monitor the state of the uplink from SERb1 to ISP-B in some manner and then tell H31 that it can no longer use S=2001:db8:0:b010::31 by setting its valid lifetime to zero. The DHCP server could initiate this process by sending a Reconfigure message to H31 as described in Section 18.3
of RFC 8415
. Or the DHCP server can assign addresses with short lifetimes in order to force clients to renew them often.
This approach would prevent H31 from using S=2001:db8:0:b010::31 to reach a host on the Internet. However, it would also prevent H31 from using S=2001:db8:0:b010::31 to reach H61 at D=2001:db8:0:6666::61, which is not desirable.
Another potential approach is to have the DHCP server monitor the uplink from SERb1 to ISP-B and control the choice of source address on H31 by updating its address selection policy table via the mechanism in [RFC 7078
]. The DHCP server could initiate this process by sending a Reconfigure message to H31. Note that [RFC 8415
] requires that Reconfigure messages use DHCP authentication. DHCP authentication could be avoided by using short address lifetimes to force clients to send Renew messages to the server often. If the host does not obtain its IP addresses from the DHCP server, then it would need to use the Information Refresh Time option defined in [RFC 8415
If the following policy table can be installed on H31 after the failure of the uplink from SERb1, then the desired routing behavior should be achieved based on source and destination prefix being matched with label values.
Prefix Precedence Label
::/0 50 44
2001:db8:0:a000::/52 50 44
2001:db8:0:6666::/64 50 55
2001:db8:0:b000::/52 50 55
The described solution has a number of significant drawbacks, some of them already discussed in Section 6.2.1
DHCPv6 support is not required for an IPv6 host, and there are operating systems that do not support DHCPv6. Besides that, it does not appear that [RFC 7078] has been widely implemented on host operating systems.
[RFC 7078] does not clearly specify this kind of a dynamic use case in which the address selection policy needs to be updated quickly in response to the failure of a link. In a large network, it would present scalability issues as many hosts need to be reconfigured in a very short period of time.
Updating DHCPv6 server configuration each time an ISP's uplink changes its state introduces some scalability issues, especially for mid/large distributed-scale enterprise networks. In addition to that, the policy table needs to be manually configured by administrators, which makes that solution prone to human error.
No mechanism exists for making DHCPv6 servers aware of network topology/routing changes in the network. In general, having DHCPv6 servers monitor network-related events sounds like a bad idea as it requires completely new functionality beyond the scope of the DHCPv6 role.
The same mechanism as discussed in Section 6.2.2
can be used to control the source address selection in the case of an uplink failure. If a particular prefix should not be used as a source for any destination, then the router needs to send an RA with the Preferred Lifetime field for that prefix set to zero.
Let's consider a scenario in which all uplinks are operational, and H41 receives two different RAs from R3: one from LLA_A with a PIO for 2001:db8:0:a020::/64 and the default router preference set to 11 (low), and another one from LLA_B with a PIO for 2001:db8:0:a020::/64, the default router preference set to 01 (high), and a RIO for 2001:db8:0:6666::/64. As a result, H41 uses 2001:db8:0:b020::41 as a source address for all Internet traffic, and those packets are sent by SERs to ISP-B. If SERb1's uplink to ISP-B fails, the desired behavior is that H41 stops using 2001:db8:0:b020::41 as a source address for all destinations but H61. To achieve that, R3 should react to SERb1's uplink failure (which could be detected as the disappearance of scoped route (S=2001:db8:0:b000::/52, D=::/0)) by withdrawing itself as a default router. R3 sends a new RA from LLA_B with the Router Lifetime value set to zero (which means that it should not be used as the default router). That RA still contains a PIO for 2001:db8:0:b020::/64 (for SLAAC purposes) and a RIO for 2001:db8:0:6666::/64 so that H41 can reach H61 using LLA_B as a next-hop and 2001:db8:0:b020::41 as a source address. For all traffic following the default route, LLA_A will be used as a next-hop and 2001:db8:0:a020::41 as a source address.
If all of the uplinks to ISP-B have failed, source addresses from ISP-B address space should not be used. In such a failure scenario, the forwarding table scoped S=2001:db8:0:b000::/52 contains no entries, indicating that R3 can instruct hosts to stop using source addresses from 2001:db8:0:b000::/52 by sending RAs containing PIOs with Preferred Lifetime values set to zero.
Now we look at how ICMPv6 messages can provide information back to H31. We assume again that, at the time of the failure, H31 is sending packets to H501 using (S=2001:db8:0:b010::31, D=2001:db8:0:5678::501). When the uplink from SERb1 to ISP-B fails, SERb1 would stop originating its source-prefix-scoped route for the default destination (S=2001:db8:0:b000::/52, D=::/0) as well as its unscoped default destination route. With these routes no longer in the IGP, traffic with (S=2001:db8:0:b010::31, D=2001:db8:0:5678::501) would end up at SERa based on the unscoped default destination route being originated by SERa. Since that traffic has the wrong source address to be forwarded to ISP-A, SERa would drop it and send a Destination Unreachable message with Code 5 (Source address failed ingress/egress policy) back to H31. H31 would then know to use another source address for that destination and would try with (S=2001:db8:0:a010::31, D=2001:db8:0:5678::501). This would be forwarded to SERa based on the source-prefix-scoped default destination route still being originated by SERa, and SERa would forward it to ISP-A. As discussed above, if we are willing to extend ICMPv6, SERa can even tell H31 what source address it should use to reach that destination. The expected host behavior has been discussed in Section 6.2.3
. Using ICMPv6 would have the same scalability/rate limiting issues that are discussed in Section 6.2.3
. An ISP-B uplink failure immediately makes source addresses from 2001:db8:0:b000::/52 unsuitable for external communication and might trigger a large number of ICMPv6 packets being sent to hosts in that subnet.
It appears that DHCPv6 is not particularly well suited to quickly changing the source address used by a host when an uplink fails, which eliminates DHCPv6 from the list of potential solutions. On the other hand, Router Advertisements provide a reliable mechanism to dynamically provide hosts with a list of valid prefixes to use as source addresses as well as to prevent the use of particular prefixes. While no additional new features are required to be implemented on hosts, routers need to be able to send RAs based on the state of scoped forwarding table entries and to react to network topology changes by sending RAs with particular parameters set.
It seems that the use of ICMPv6 Destination Unreachable messages generated by the SER (or any SADR-capable) routers, together with the use of RAs to signal source address selection errors back to hosts, has the potential to provide a support mechanism. However, scalability issues may arise in large networks when topology suddenly changes. Therefore, it is highly desirable that hosts are able to select the correct source address in the case of uplink failure, with ICMPv6 being an additional mechanism to signal unexpected failures back to hosts.
The current behaviors of different host operating systems upon receipt of an ICMPv6 Destination Unreachable message with Code 5 (Source address failed ingress/egress policy) is not clear to the authors. Information from implementers, users, and testing would be quite helpful in evaluating this approach.
The next logical step is to look at the scenario when a failed uplink on SERb1 to ISP-B comes back up, so the hosts can start using source addresses belonging to 2001:db8:0:b000::/52 again.
The mechanism to use DHCPv6 to instruct the hosts (H31 in our example) to start using prefixes from ISP-B space (e.g., S=2001:db8:0:b010::31 for H31) to reach hosts on the Internet is quite similar to one discussed in Section 6.3.1
and shares the same drawbacks.
Let's look at the scenario discussed in Section 6.3.2
. If the uplink(s) failure caused the complete withdrawal of prefixes from the 2001:db8:0:b000::/52 address space by setting the Preferred Lifetime value to zero, then the recovery of the link should just trigger the sending of a new RA with a non-zero Preferred Lifetime. In another scenario discussed in Section 6.3.2
, the failure of the SERb1 uplink to ISP-B leads to the disappearance of the (S=2001:db8:0:b000::/52, D=::/0) entry from the forwarding table scoped to S=2001:db8:0:b000::/52 and, in turn, causes R3 to send RAs with the Router Lifetime set to zero from LLA_B. The recovery of the SERb1 uplink to ISP-B leads to the reappearance of the scoped forwarding entry (S=2001:db8:0:b000::/52, D=::/0). That reappearance acts as a signal for R3 to advertise itself as a default router for ISP-B address space domain (to send RAs from LLA_B with non-zero Router Lifetime).
It looks like ICMPv6 provides a rather limited functionality to signal back to hosts that particular source addresses have become valid again. Unless the changes in the uplink specify a particular (S,D) pair, hosts can keep using the same source address even after an ISP uplink has come back up. For example, after the uplink from SERb1 to ISP-B had failed, H31 received ICMPv6 Code 5 message (as described in Section 6.3.3
) and allegedly started using (S=2001:db8:0:a010::31, D=2001:db8:0:5678::501) to reach H501. Now when the SERb1 uplink comes back up, the packets with that (S,D) pair are still routed to SERa1 and sent to the Internet. Therefore, H31 is not informed that it should stop using 2001:db8:0:a010::31 and start using 2001:db8:0:b010::31 again. Unless SERa has a policy configured to drop packets (S=2001:db8:0:a010::31, D=2001:db8:0:5678::501) and send ICMPv6 back if the SERb1 uplink to ISP-B is up, H31 will be unaware of the network topology change and keep using S=2001:db8:0:a010::31 for Internet destinations, including H51.
One of the possible options may be using a scoped route with an EXCLUSIVE flag as described in Section 6.2.3
. SERa1 uplink recovery would cause the (S=2001:db8:0:a000::/52, D=2001:db8:0:1234::/64) route to reappear in the routing table. In the absence of that, route packets to H101 are sent to ISP-B (as ISP-A uplink was down) with source addresses from 2001:db8:0:b000::/52. When the route reappears, SERb1 rejects those packets and sends ICMPv6 back as discussed in Section 6.2.3
. Practically, it might lead to scalability issues, which have been already discussed in [6.2.3
] and [6.4.3
Once again, DHCPv6 does not look like a reasonable choice to manipulate the source address selection process on a host in the case of network topology changes. Using Router Advertisement provides the flexible mechanism to dynamically react to network topology changes (if routers are able to use routing changes as a trigger for sending out RAs with specific parameters). ICMPv6 could be considered as a supporting mechanism to signal incorrect source address back to hosts, but it should not be considered as the only mechanism to control the address selection in multihomed environments.
One particular tricky case is a scenario when all uplinks have failed. In that case, there is no valid source address to be used for any external destinations when it might be desirable to have intra-site connectivity.
From the DHCPv6 perspective, uplinks failure should be treated as two independent failures and processed as described in Section 6.3.1
. At this stage, it is quite obvious that it would result in a quite complicated policy table that would need to be explicitly configured by administrators and therefore seems to be impractical.
As discussed in Section 6.3.2
, an uplink failure causes the scoped default entry to disappear from the scoped forwarding table and triggers RAs with zero Router Lifetimes. Complete disappearance of all scoped entries for a given source prefix would cause the prefix to be withdrawn from hosts by setting the Preferred Lifetime value to zero in the PIO. If all uplinks (SERa, SERb1 and SERb2) fail, hosts either lose their default routers and/or have no global IPv6 addresses to use as a source. (Note that 'uplink failure' might mean 'IPv6 connectivity failure with IPv4 still being reachable', in which case, hosts might fall back to IPv4 if there is IPv4 connectivity to destinations). As a result, intra-site connectivity is broken. One of the possible ways to solve it is to use ULAs.
In addition to GUAs, all hosts have ULA addresses assigned, and these addresses are used for intra-site communication even if there is no GUA assigned to a host. To avoid accidental leaking of packets with ULA sources, SADR-capable routers SHOULD
have a scoped forwarding table for ULA source for internal routes but MUST NOT
have an entry for D=::/0 in that table. In the absence of (S=ULA_Prefix; D=::/0), first-hop routers will send dedicated RAs from a unique link-local source LLA_ULA with a PIO from ULA address space, a RIO for the ULA prefix, and Router Lifetime set to zero. The behavior is consistent with the situation when SERb1 lost the uplink to ISP-B (so there is no Internet connectivity from 2001:db8:0:b000::/52 sources), but those sources can be used to reach some specific destinations. In the case of ULA, there is no Internet connectivity from ULA sources, but they can be used to reach other ULA destinations. Note that ULA usage could be particularly useful if all ISPs assign prefixes via DHCP prefix delegation. In the absence of ULAs, upon the failure of all uplinks, hosts would lose all their GUAs upon prefix-lifetime expiration, which again makes intra-site communication impossible.
It should be noted that Rule 5.5 (prefer a prefix advertised by the selected next-hop) takes precedence over the Rule 6 (prefer matching label, which ensures that GUA source addresses are preferred over ULAs for GUA destinations). Therefore if ULAs are used, the network administrator needs to ensure that, while the site has Internet connectivity, hosts do not select a router that advertises ULA prefixes as their default router.
In the case of the failure of all uplinks, all SERs will drop outgoing IPv6 traffic and respond with ICMPv6 error messages. In a large network in which many hosts attempt to reach Internet destinations, the SERs need to generate an ICMPv6 error for every packet they receive from hosts, which presents the same scalability issues discussed in Section 6.3.3
Again, combining SADR with Router Advertisements seems to be the most flexible and scalable way to control the source address selection on hosts.
This section summarizes the scenarios and options discussed above.
While DHCPv6 allows administrators to manipulate source address selection policy tables, this method has a number of significant disadvantages that eliminate DHCPv6 from a list of potential solutions:
It requires hosts to support DHCPv6 and its extension [RFC 7078].
A DHCPv6 server needs to monitor network state and detect routing changes.
The use of policy tables requires manual configuration and might be extremely complicated, especially in the case of a distributed network in which a large number of remote sites are being served by centralized DHCPv6 servers.
Network topology/routing policy changes could trigger simultaneous reconfiguration of large number of hosts, which presents serious scalability issues.
The use of Router Advertisements to influence the source address selection on hosts seem to be the most reliable, flexible, and scalable solution. It has the following benefits:
No new (non-standard) functionality needs to be implemented on hosts (except for RIO support [RFC 4191], which is not widely implemented at the time of this writing).
No changes in RA format.
Routers can react to routing table changes by sending RAs, which would minimize the failover time in the case of network topology changes.
Information required for source address selection is broadcast to all affected hosts in the case of a topology change event, which improves the scalability of the solution (compared to DHCPv6 reconfiguration or ICMPv6 error messages).
To fully benefit from the RA-based solution, first-hop routers need to implement SADR, belong to the SADR domain, and be able to send dedicated RAs per scoped forwarding table as discussed above, reacting to network changes by sending new RAs. It should be noted that the proposed solution would work even if first-hop routers are not SADR-capable but still able to send individual RAs for each ISP prefix and react to topology changes as discussed above (e.g., via configuration knobs).
The RA-based solution relies heavily on hosts correctly implementing the default address selection algorithm as defined in [RFC 6724
]. While the basic, and the most common, multihoming scenario (two or more Internet uplinks, no 'walled gardens') would work for any host supporting the minimal implementation of [RFC 6724
], more complex use cases (such as 'walled garden' and other scenarios when some ISP resources can only be reached from that ISP address space) require that hosts support Rule 5.5 of the default address selection algorithm. There is some evidence that not all host OSes have that rule implemented currently. However, it should be noted that [RFC 8028
] states that Rule 5.5 should be implemented.
The ICMPv6 Code 5 error message SHOULD
be used to complement an RA-based solution to signal incorrect source address selection back to hosts, but it SHOULD NOT
be considered as the standalone solution. To prevent scenarios when hosts in multihomed environments incorrectly identify on-link/off-link destinations, hosts SHOULD
treat ICMPv6 Redirects as discussed in [RFC 8028
The proposed solution is not designed to preserve connection state in the case of an uplink failure. When all uplinks to an ISP go down, all transport connections established to/from that ISP address space will be interrupted (unless the transport protocol has specific multihoming support). That behavior is similar to the scenario of IPv4 multihoming with NAT when an uplink failure causes all connections to be NATed to completely different public IPv4 addresses. While it does sound suboptimal, it is determined by the nature of PA address space: if all uplinks to the particular ISP have failed, there is no path for the ingress traffic to reach the site, and the egress traffic is supposed to be dropped by the ingress filters [BCP38
]. The only potential way to overcome this limitation would be to run BGP with all ISPs and to advertise all site prefixes to all uplinks - a solution that shares all the drawbacks of using the PI address space without sharing its benefits. Networks willing and capable of running BGP and using PI are out of scope of this document.
It should be noted that in the case of IPv4 NAT-based multihoming, uplink recovery could cause connection interruptions as well (unless packet forwarding is integrated with the tracking of existing NAT sessions so that the egress interface for the existing sessions is not changed). However, the proposed solution has the benefit of preserving the existing sessions during and after the restoration of the failed uplink. Unlike the uplink failure event, which causes all addresses from the affected prefix to be deprecated, the recovery would just add new, preferred addresses to a host without making any addresses unavailable. Therefore, connections established to and from those addresses do not have to be interrupted.
While it's desirable for active connections to survive ISP failover events, such events affect the reachability of IP addresses assigned to hosts in sites using PA address space. Unless the transport (or higher-level protocols) is capable of surviving the host renumbering, the active connections will be broken. The proposed solution focuses on minimizing the impact of failover on new connections and on multipath-aware protocols.
Another way to preserve connection state is to use multipath transport as discussed in Section 8.3
In a multihomed environment, each ISP might provide their own list of DNS servers. For example, in the topology shown in Figure 3
, ISP-A might provide H51 2001:db8:0:5555::51 as a recursive DNS server, while ISP-B might provide H61 2001:db8:0:6666::61 as a recursive DNS server (RDNSS). [RFC 8106
] defines IPv6 Router Advertisement options to allow IPv6 routers to advertise a list of RDNSS addresses and a DNS Search List (DNSSL) to IPv6 hosts. Using RDNSS together with 'scoped' RAs as described above would allow a first-hop router (R3 in Figure 3
) to send DNS server addresses and search lists provided by each ISP (or the corporate DNS server addresses if the enterprise is running its own DNS servers. As discussed below, the DNS split-horizon problem is too hard to solve without running a local DNS server).
As discussed in Section 6.5.2
, failure of all ISP uplinks would cause deprecation of all addresses assigned to a host from the address space of all ISPs. If any intra-site IPv6 connectivity is still desirable (most likely to be the case for any mid/large-scale network), then ULAs should be used as discussed in Section 6.5.2
. In such a scenario, the enterprise network should run its own recursive DNS server(s) and provide its ULA addresses to hosts via RDNSS in RAs sent for ULA-scoped forwarding table as described in Section 6.5.2
There are some scenarios in which the final outcome of the name resolution might be different depending on:
which DNS server is used;
which source address the client uses to send a DNS query to the server (DNS split horizon).
There is no way currently to instruct a host to use a particular DNS server from the configured servers list for resolving a particular name. Therefore, it does not seem feasible to solve the problem of DNS server selection on the host (it should be noted that this particular issue is protocol-agnostic and happens for IPv4 as well). In such a scenario, it is recommended that the enterprise run its own local recursive DNS server.
To influence host source address selection for packets sent to a particular DNS server, the following requirements must be met:
The host supports RIO as defined in [RFC 4191].
The routers send RIOs for routes to DNS server addresses.
For example, if it is desirable that host H31 reaches the ISP-A DNS server H51 2001:db8:0:5555::51 using its source address 2001:db8:0:a010::31, then both R1 and R2 should send RIOs containing the route to 2001:db8:0:5555::51 (or covering route) in their 'scoped' RAs, containing LLA_A as the default router address and the PIO for SLAAC prefix 2001:db8:0:a010::/64. In that case, host H31 (if it supports Rule 5.5) would select LLA_A as a next-hop and then choose 2001:db8:0:a010::31 as the source address for packets to the DNS server.
It should be noted that [RFC 6106
] explicitly prohibits using DNS information if the RA Router Lifetime has expired:
An RDNSS address or a DNSSL domain name MUST be used only as long as both the RA router Lifetime (advertised by a Router Advertisement message) and the corresponding option Lifetime have not expired.
Therefore, hosts might ignore RDNSS information provided in ULA-scoped RAs, as those RAs would have Router Lifetime values set to zero. However, [RFC 8106
], which obsoletes RFC 6106
, has removed that requirement.
As discussed above, the DNS split-horizon problem and the selection of the correct DNS server in a multihomed environment are not easy problems to solve. The proper solution would require hosts to support the concept of multiple provisioning domains (PvD, a set of configuration information associated with a network, [RFC 7556