Internet Engineering Task Force (IETF) A. Clemm Request for Comments: 8641 Futurewei Category: Standards Track E. Voit ISSN: 2070-1721 Cisco Systems September 2019 Subscription to YANG Notifications for Datastore Updates
AbstractThis document describes a mechanism that allows subscriber applications to request a continuous and customized stream of updates from a YANG datastore. Providing such visibility into updates enables new capabilities based on the remote mirroring and monitoring of configuration and operational state. Status of This Memo This is an Internet Standards Track document. This document is a product of the Internet Engineering Task Force (IETF). It represents the consensus of the IETF community. It has received public review and has been approved for publication by the Internet Engineering Steering Group (IESG). Further information on Internet Standards is available in Section 2 of RFC 7841. Information about the current status of this document, any errata, and how to provide feedback on it may be obtained at https://www.rfc-editor.org/info/rfc8641.
Copyright Notice Copyright (c) 2019 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License. This document may contain material from IETF Documents or IETF Contributions published or made publicly available before November 10, 2008. The person(s) controlling the copyright in some of this material may not have granted the IETF Trust the right to allow modifications of such material outside the IETF Standards Process. Without obtaining an adequate license from the person(s) controlling the copyright in such materials, this document may not be modified outside the IETF Standards Process, and derivative works of it may not be created outside the IETF Standards Process, except to format it for publication as an RFC or to translate it into languages other than English.
1. Introduction ....................................................4 2. Definitions .....................................................4 3. Solution Overview ...............................................6 3.1. Subscription Model .........................................6 3.2. Negotiation of Subscription Policies .......................7 3.3. On-Change Considerations ...................................8 3.4. Reliability Considerations .................................9 3.5. Data Encodings ............................................10 3.6. Defining the Selection with a Datastore ...................11 3.7. Streaming Updates .........................................12 3.8. Subscription Management ...................................15 3.9. Receiver Authorization ....................................16 3.10. On-Change Notifiable Datastore Nodes .....................18 3.11. Other Considerations .....................................18 4. A YANG Data Model for Management of Datastore Push Subscriptions ..................................................20 4.1. Overview ..................................................20 4.2. Subscription Configuration ................................27 4.3. YANG Notifications ........................................28 4.4. YANG RPCs .................................................29 5. YANG Module for YANG-Push ......................................34 6. IANA Considerations ............................................51 7. Security Considerations ........................................51 8. References .....................................................53 8.1. Normative References ......................................53 8.2. Informative References ....................................55 Appendix A. Subscription Errors ...................................56 A.1. RPC Failures ...............................................56 A.2. Failure Notifications ......................................57 Acknowledgments ...................................................58 Contributors ......................................................58 Authors' Addresses ................................................58
RFC7923]. This document defines a corresponding solution that is built on top of [RFC8639]. Supplementing that work are YANG data model augmentations, extended RPCs, and new datastore-specific update notifications. Transport options provided in [RFC8639] will work seamlessly with this solution. RFC2119] [RFC8174] when, and only when, they appear in all capitals, as shown here.
This document uses the terminology defined in [RFC7950], [RFC8341], [RFC8342], and [RFC8639]. In addition, this document defines the following terms: o Datastore node: A node in the instantiated YANG data tree associated with a datastore. In this document, datastore nodes are often also simply referred to as "objects". o Datastore node update: A data item containing the current value of a datastore node at the time the datastore node update was created, as well as the path to the datastore node. o Datastore subscription: A subscription to a stream of datastore node updates. o Datastore subtree: A datastore node and all its descendant datastore nodes. o On-change subscription: A datastore subscription with updates that are triggered when changes in subscribed datastore nodes are detected. o Periodic subscription: A datastore subscription with updates that are triggered periodically according to some time interval. o Selection filter: Evaluation and/or selection criteria that may be applied against a targeted set of objects. o Update record: A representation of one or more datastore node updates. In addition, an update record may contain which type of update led to the datastore node update (e.g., whether the datastore node was added, changed, or deleted). Also included in the update record may be other metadata, such as a subscription ID of the subscription for which the update record was generated. In this document, update records are often also simply referred to as "updates". o Update trigger: A mechanism that determines when an update record needs to be generated. o YANG-Push: The subscription and push mechanism for datastore updates that is specified in this document.
RFC8639] with capabilities that allow subscribers to subscribe to datastore node updates -- specifically, to specify the update triggers defining when to generate update records as well as what to include in an update record. Key enhancements include: o The specification of selection filters that identify targeted YANG datastore nodes and/or datastore subtrees for which updates are to be pushed. o The specification of update policies that contain conditions that trigger the generation and pushing of new update records. There are two types of subscriptions, distinguished by how updates are triggered: periodic and on-change. * For periodic subscriptions, the update trigger is specified by two parameters that define when updates are to be pushed. These parameters are (1) the period interval with which to report updates and (2) an "anchor-time", i.e., a reference point in time that can be used to calculate at which points in time periodic updates need to be assembled and sent. * For on-change subscriptions, an update trigger occurs whenever a change in the subscribed information is detected. The following additional parameters are included: + "dampening-period": In an on-change subscription, detected object changes should be sent as quickly as possible. However, it may be undesirable to send a rapid series of object changes. Such behavior has the potential to exhaust resources in the publisher or receiver. In order to protect against this type of scenario, a dampening period MAY be used to specify the interval that has to pass before successive update records for the same subscription are generated for a receiver. The dampening period collectively
applies to the set of all datastore nodes selected by a single subscription. This means that when there is a change to one or more subscribed objects, an update record containing those objects is created immediately (when no dampening period is in effect) or at the end of a dampening period (when a dampening period is in fact in effect). If multiple changes to a single object occur during a dampening period, only the value that is in effect at the time when the update record is created is included. The dampening period goes into effect every time the assembly of an update record is completed. + "change-type": This parameter can be used to reduce the types of datastore changes for which updates are sent (e.g., you might only send an update when an object is created or deleted, but not when an object value changes). + "sync-on-start": This parameter defines whether or not a complete "push-update" (Section 3.7) of all subscribed data will be sent at the beginning of a subscription. Such early synchronization establishes the frame of reference for subsequent updates. o An encoding (using anydata) for the contents of periodic and on-change push updates.
Section 3.10. Alternatively, a publisher MAY decide to simply reject an on-change subscription if the scope of the subscription contains objects for which on-change is not supported. In the case of a configured subscription, the publisher MAY suspend the subscription. To avoid flooding receivers with repeated updates for subscriptions containing fast-changing objects or objects with oscillating values, an on-change subscription allows for the definition of a dampening period. Once an update record for a given object is generated, no other updates for this particular subscription will be created until the end of the dampening period. Values sent at the end of the dampening period are the values that are current at the end of the dampening period of all changed objects. Changed objects include those objects that were deleted or newly created during that dampening period. If an object has returned to its original value (or even has been created and then deleted) during the dampening period, that value (and not the interim change) will still be sent. This will indicate that churn is occurring on that object.
On-change subscriptions can be refined to let users subscribe only to certain types of changes. For example, a subscriber might only want object creations and deletions, but not modifications of object values. Putting it all together, the conceptual process for creating an update record as part of an on-change subscription is as follows: 1. Just before a change, or at the start of a dampening period, evaluate any filtering and any access control rules to ensure that a receiver is authorized to view all subscribed datastore nodes (filtering out any nodes for which this is not the case). The result is a set "A" of datastore nodes and subtrees. 2. Just after a change, or at the end of a dampening period, evaluate any filtering and any (possibly new) access control rules. The result is a set "B" of datastore nodes and subtrees. 3. Construct an update record, which takes the form of a YANG Patch record [RFC8072] for going from A to B. 4. If there were any changes made between A and B that canceled each other out, insert into the YANG Patch record the last change made, even if the new value is no different from the original value (since changes that were made in the interim were canceled out). If the changes involve creating a new datastore node and then deleting it, the YANG Patch record will indicate the deletion of the datastore node. Similarly, if the changes involve deleting a new datastore node and then recreating it, the YANG Patch record will indicate the creation of the datastore node. 5. If the resulting YANG Patch record is non-empty, send it to the receiver. Note: In cases where a subscriber wants to have separate dampening periods for different objects, the subscriber has the option to create multiple subscriptions with different selection filters.
Now, there are many reasons why a publisher may at some point no longer be able to fulfill the terms of the subscription, even if the subscription had been initiated in good faith. For example, the volume of datastore nodes may be larger than anticipated, the interval may prove too short to send full updates in rapid succession, or an internal problem may prevent objects from being collected. For this reason, the solution defined in this document (1) mandates that a publisher notify receivers immediately and reliably whenever it encounters a situation in which it is unable to keep the terms of the subscription and (2) provides the publisher with the option to suspend the subscription in such a case. This includes indicating the fact that an update is incomplete as part of a "push-update" or "push-change-update" notification, as well as emitting a "subscription-suspended" notification as applicable. This is described further in Section 3.11.1. A publisher SHOULD reject a request for a subscription if it is unlikely that the publisher will be able to fulfill the terms of that subscription request. In such cases, it is preferable to have a subscriber request a less resource-intensive subscription than to deal with frequently degraded behavior. The solution builds on [RFC8639]. As defined therein, any loss of an underlying transport connection will be detected and result in subscription termination (in the case of dynamic subscriptions) or suspension (in the case of configured subscriptions), ensuring that situations where the loss of update notifications would go unnoticed will not occur. RFC8072]. The YANG Patch operations will describe what needs to be applied to the earlier state reported by the preceding update in order to result in the now-current state. Note that objects referred to in an update are not limited to
configuration data but can include any objects (including operational data), whereas [RFC8072] patches apply only to configuration data in configuration datastores. A publisher indicates the type of change to a datastore node using the different YANG Patch operations: the "create" operation is used for newly created objects (except entries in a user-ordered list), the "delete" operation is used for deleted objects (including in user-ordered lists), the "replace" operation is used when only the object value changes, the "insert" operation is used when a new entry is inserted in a list, and the "move" operation is used when an existing entry in a user-ordered list is moved. However, a patch must be able to do more than just describe the delta from the previous state to the current state. As per Section 3.3, it must also be able to identify whether transient changes have occurred on an object during a dampening period. To support this, it is valid to encode a YANG Patch operation so that its application would result in no change between the previous state and the current state. This indicates that some churn has occurred on the object. An example of this would be a patch that indicates a "create" operation for a datastore node where the receiver believes one already exists or a "replace" operation that replaces a previous value with the same value. Note that this means that the "create" and "delete" errors as described in [RFC8072], Section 2.5 are not errors in the case of YANG-Push (i.e., they are considered valid operations for YANG-Push). RFC6241], Section 6.
o xpath: An "xpath" selection filter is an XPath expression that returns a node set. (XPath is a query language for selecting nodes in an XML document; see [XPATH] for details.) When specified, updates will only come from the selected datastore nodes. These filters are intended to be used as selectors that define which objects are within the scope of a subscription. A publisher MUST support at least one type of selection filter. XPath itself provides powerful filtering constructs, and care must be used in filter definition. Consider an XPath filter that only passes a datastore node when an interface is up. It is up to the receiver to understand the implications of the presence or absence of objects in each update. When the set of selection-filtering criteria is applied for a periodic subscription, these criteria are applied whenever a periodic update record is constructed, and only datastore nodes that pass the filter and to which a receiver has access are provided to that receiver. If the same filtering criteria are applied to an on-change subscription, only the subset of those datastore nodes supporting on-change is provided. A datastore node that doesn't support on-change is never sent as part of an on-change subscription's "push-update" or "push-change-update" (Section 3.7).
A "push-change-update" notification is the most common type of update for on-change subscriptions. The update record in this case contains the set of changes that datastore nodes have undergone since the last notification message. In other words, this indicates which datastore nodes have been created, have been deleted, or have had changes to their values. In cases where multiple changes have occurred over the course of a dampening period and the object has not been deleted, the object's most current value is reported. (In other words, for each object, only one change is reported, not its entire history. Doing so would defeat the purpose of the dampening period.) "push-update" and "push-change-update" are encoded and placed in notification messages and are ultimately queued for egress over the specified transport. Figure 1 provides an example of a notification message for a subscription tracking the operational status of a single Ethernet interface (per [RFC8343]). This notification message is encoded XML [W3C.REC-xml-20081126] over the Network Configuration Protocol (NETCONF) as per [RFC8640]. <notification xmlns="urn:ietf:params:xml:ns:netconf:notification:1.0"> <eventTime>2017-10-25T08:00:11.22Z</eventTime> <push-update xmlns="urn:ietf:params:xml:ns:yang:ietf-yang-push"> <id>1011</id> <datastore-contents> <interfaces xmlns="urn:ietf:params:xml:ns:yang:ietf-interfaces"> <interface> <name>eth0</name> <oper-status>up</oper-status> </interface> </interfaces> </datastore-contents> </push-update> </notification> Figure 1: Push Example
Figure 2 provides an example of an on-change notification message for the same subscription. <notification xmlns="urn:ietf:params:xml:ns:netconf:notification:1.0"> <eventTime>2017-10-25T08:22:33.44Z</eventTime> <push-change-update xmlns="urn:ietf:params:xml:ns:yang:ietf-yang-push"> <id>89</id> <datastore-changes> <yang-patch> <patch-id>0</patch-id> <edit> <edit-id>edit1</edit-id> <operation>replace</operation> <target>/ietf-interfaces:interfaces</target> <value> <interfaces xmlns="urn:ietf:params:xml:ns:yang:ietf-interfaces"> <interface> <name>eth0</name> <oper-status>down</oper-status> </interface> </interfaces> </value> </edit> </yang-patch> </datastore-changes> </push-change-update> </notification> Figure 2: Push Example for an On-Change Notification Message Of note in the above example is the "patch-id" with a value of "0". Per [RFC8072], the "patch-id" is an arbitrary string. With YANG-Push, the publisher SHOULD put into the "patch-id" a counter starting at "0" that increments with every "push-change-update" generated for a subscription. If used as a counter, this counter MUST be reset to "0" any time a resynchronization occurs (i.e., with the sending of a "push-update"). Also, if used as a counter, the counter MUST be reset to "0" after passing a maximum value of "4294967295" (i.e., the maximum value that can be represented using the uint32 data type). Such a mechanism allows easy identification of lost or out-of-sequence update records.
RFC8639] have been enhanced to support datastore subscription negotiation. Also, new error codes have been added that are able to indicate why a datastore subscription attempt has failed, along with new yang-data that MAY be used to include details on input parameters that might result in a successful subsequent RPC invocation. The establishment or modification of a datastore subscription can be rejected for multiple reasons, including a subtree request that is too large or the inability of the publisher to push update records as frequently as requested. In such cases, no subscription is established. Instead, a subscription result that includes the reason for the failure is returned as part of the RPC response. As part of this response, a set of alternative subscription parameters MAY be returned that would likely have resulted in acceptance of the subscription request. The subscriber may consider including such parameters in future subscription attempts. In the case of a rejected request for establishment of a datastore subscription, if there are hints, the hints SHOULD be transported in a yang-data "establish-subscription-datastore-error-info" container inserted into the RPC error response, in lieu of the "establish- subscription-stream-error-info" that is inserted in the case of a stream subscription. Figure 3 shows a tree diagram for "establish-subscription-datastore- error-info". All tree diagrams used in this document follow the notation defined in [RFC8340]. yang-data establish-subscription-datastore-error-info +--ro establish-subscription-datastore-error-info +--ro reason? identityref +--ro period-hint? centiseconds +--ro filter-failure-hint? string +--ro object-count-estimate? uint32 +--ro object-count-limit? uint32 +--ro kilobytes-estimate? uint32 +--ro kilobytes-limit? uint32 Figure 3: "establish-subscription-datastore-error-info" Tree Diagram
Similarly, in the case of a rejected request for modification of a datastore subscription, if there are hints, the hints SHOULD be transported in a yang-data "modify-subscription-datastore-error-info" container inserted into the RPC error response, in lieu of the "modify-subscription-stream-error-info" that is inserted in the case of a stream subscription. Figure 4 shows a tree diagram for "modify-subscription-datastore- error-info". yang-data modify-subscription-datastore-error-info +--ro modify-subscription-datastore-error-info +--ro reason? identityref +--ro period-hint? centiseconds +--ro filter-failure-hint? string +--ro object-count-estimate? uint32 +--ro object-count-limit? uint32 +--ro kilobytes-estimate? uint32 +--ro kilobytes-limit? uint32 Figure 4: "modify-subscription-datastore-error-info" Tree Diagram Figure 5. This includes validating that read access is permitted for any new objects selected since the last notification message was sent to a particular receiver. A publisher MUST silently omit data nodes from the results that the client is not authorized to see. To accomplish this, implementations SHOULD apply the conceptual authorization model of [RFC8341], specifically Section 3.2.4, extended to apply analogously to data nodes included in notifications, not just <rpc-reply> messages sent in response to <get> and <get-config> requests.
+-----------------+ +--------------------+ push-update or --> | datastore node | yes | add datastore node | push-change-update | access allowed? | ---> | to update record | +-----------------+ +--------------------+ Figure 5: Access Control for Push Updates A publisher MUST allow for the possibility that a subscription's selection filter references nonexistent data or data that a receiver is not allowed to access. Such support permits a receiver the ability to monitor the entire lifecycle of some datastore tree without needing to explicitly enumerate every individual datastore node. If, after access control has been applied, there are no objects remaining in an update record, then the effect varies given if the subscription is a periodic or on-change subscription. For a periodic subscription, an empty "push-update" notification MUST be sent, so that clients do not get confused into thinking that an update was lost. For an on-change subscription, a "push-update" notification MUST NOT be sent, so that clients remain unaware of changes made to nodes they don't have read-access for. By the same token, changes to objects that are filtered MUST NOT affect any dampening intervals. A publisher MAY choose to reject an "establish-subscription" request that selects nonexistent data or data that a receiver is not allowed to access. The error identity "unchanging-selection" SHOULD be returned as the reason for the rejection. In addition, a publisher MAY choose to terminate a dynamic subscription or suspend a configured receiver when the authorization privileges of a receiver change or the access controls for subscribed objects change. In that case, the publisher SHOULD include the error identity "unchanging- selection" as the reason when sending the "subscription-terminated" or "subscription-suspended" notification, respectively. Such a capability enables the publisher to avoid having to support continuous and total filtering of a subscription's content for every update record. It also reduces the possibility of leakage of access-controlled objects. If read access into previously accessible nodes has been lost due to a receiver permissions change, this SHOULD be reported as a patch "delete" operation for on-change subscriptions. If not capable of handling such receiver permission changes with such a "delete", publisher implementations MUST force dynamic subscription re-establishment or configured subscription reinitialization so that appropriate filtering is installed.
RFC8343]), small object changes are frequent and meaningless (e.g., a temperature gauge changing 0.1 degrees), or the implementation is not capable of on-change notification for a particular object. In those cases, it will be important for client applications to have a way to identify for which objects on-change notifications are supported and for which ones they are not supported. Otherwise, client applications will have no way of knowing whether they can indeed rely on their on-change subscription to provide them with the change updates that they are interested in. In other words, if implementations do not provide a solution and do not support comprehensive on-change notifiability, clients of those implementations will have no way of knowing what their on-change subscription actually covers. Implementations are therefore strongly advised to provide a solution to this problem. One solution might involve making discoverable to clients which objects are on-change notifiable, specified using another YANG data model. Such a solution is specified in [Yang-Push-Notif-Cap]. Until this solution is standardized, implementations SHOULD provide their own solution.
It is conceivable that, under certain circumstances, a publisher will recognize that it is unable to include in an update record the full set of objects desired per the terms of a subscription. In this case, the publisher MUST act as follows. o The publisher MUST set the "incomplete-update" flag on any update record that is known to be missing information. o The publisher MAY choose to suspend the subscription as per [RFC8639]. If the publisher does not create an update record at all, it MUST suspend the subscription. o When resuming an on-change subscription, the publisher SHOULD generate a complete patch from the previous update record. If this is not possible and the "sync-on-start" option is set to "true" for the subscription, then the full datastore contents MAY be sent via a "push-update" instead (effectively replacing the previous contents). If neither scenario above is possible, then an "incomplete-update" flag MUST be included on the next "push-change-update". Note: It is perfectly acceptable to have a series of "push-change- update" notifications (and even "push-update" notifications) serially queued at the transport layer awaiting transmission. It is not required for the publisher to merge pending update records sent at the same time. On the receiver side, what action to take when a record with an "incomplete-update" flag is received depends on the application. It could simply choose to wait and do nothing. It could choose to resync, actively retrieving all subscribed information. It could also choose to tear down the subscription and start a new one, perhaps with a smaller scope that contains fewer objects.