Internet Engineering Task Force (IETF) M. Bjorklund Request for Comments: 8342 Tail-f Systems Updates: 7950 J. Schoenwaelder Category: Standards Track Jacobs University ISSN: 2070-1721 P. Shafer K. Watsen Juniper Networks R. Wilton Cisco Systems March 2018 Network Management Datastore Architecture (NMDA) Abstract Datastores are a fundamental concept binding the data models written in the YANG data modeling language to network management protocols such as the Network Configuration Protocol (NETCONF) and RESTCONF. This document defines an architectural framework for datastores based on the experience gained with the initial simpler model, addressing requirements that were not well supported in the initial model. This document updates RFC 7950. Status of This Memo This is an Internet Standards Track document. This document is a product of the Internet Engineering Task Force (IETF). It represents the consensus of the IETF community. It has received public review and has been approved for publication by the Internet Engineering Steering Group (IESG). Further information on Internet Standards is available in Section 2 of RFC 7841. Information about the current status of this document, any errata, and how to provide feedback on it may be obtained at https://www.rfc-editor.org/info/rfc8342.
Copyright Notice Copyright (c) 2018 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License. Table of Contents 1. Introduction ....................................................3 2. Objectives ......................................................4 3. Terminology .....................................................5 4. Background ......................................................8 4.1. Original Model of Datastores ...............................9 5. Architectural Model of Datastores ..............................11 5.1. Conventional Configuration Datastores .....................12 5.1.1. The Startup Configuration Datastore (<startup>) ....12 5.1.2. The Candidate Configuration Datastore (<candidate>) ......................................13 5.1.3. The Running Configuration Datastore (<running>) ....13 5.1.4. The Intended Configuration Datastore (<intended>) ..13 5.2. Dynamic Configuration Datastores ..........................14 5.3. The Operational State Datastore (<operational>) ...........14 5.3.1. Remnant Configuration ..............................16 5.3.2. Missing Resources ..................................16 5.3.3. System-Controlled Resources ........................16 5.3.4. Origin Metadata Annotation .........................17 6. Implications on YANG ...........................................18 6.1. XPath Context .............................................18 6.2. Invocation of Actions and RPCs ............................19 7. YANG Modules ...................................................20 8. IANA Considerations ............................................26 8.1. Updates to the IETF XML Registry ..........................26 8.2. Updates to the YANG Module Names Registry .................27 9. Security Considerations ........................................27 10. References ....................................................28 10.1. Normative References .....................................28 10.2. Informative References ...................................29
Appendix A. Guidelines for Defining Datastores ....................31 A.1. Define Which YANG Modules Can Be Used in the Datastore .....31 A.2. Define Which Subset of YANG-Modeled Data Applies ...........31 A.3. Define How Data Is Actualized ..............................31 A.4. Define Which Protocols Can Be Used .........................31 A.5. Define YANG Identities for the Datastore ...................32 Appendix B. Example of an Ephemeral Dynamic Configuration Datastore .............................................32 Appendix C. Example Data ..........................................33 C.1. System Example .............................................34 C.2. BGP Example ................................................37 C.2.1. Datastores .............................................38 C.2.2. Adding a Peer ..........................................38 C.2.3. Removing a Peer ........................................39 C.3. Interface Example ..........................................40 C.3.1. Pre-provisioned Interfaces .............................41 C.3.2. System-Provided Interface ..............................42 Acknowledgments ...................................................43 Authors' Addresses ................................................44 1. Introduction This document provides an architectural framework for datastores as they are used by network management protocols such as the Network Configuration Protocol (NETCONF) [RFC6241], RESTCONF [RFC8040], and the YANG data modeling language [RFC7950]. Datastores are a fundamental concept binding network management data models to network management protocols. Agreement on a common architectural model of datastores ensures that data models can be written in a way that is network management protocol agnostic. This architectural framework identifies a set of conceptual datastores, but it does not mandate that all network management protocols expose all these conceptual datastores. This architecture is agnostic with regard to the encoding used by network management protocols. This document updates RFC 7950 by refining the definition of the accessible tree for some XML Path Language (XPath) context (see Section 6.1) and the invocation context of operations (see Section 6.2). The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all capitals, as shown here.
2. Objectives Network management data objects can often take two different values: the value configured by the user or an application (configuration) and the value that the device is actually using (operational state). These two values may be different for a number of reasons, e.g., system internal interactions with hardware, interaction with protocols or other devices, or simply the time it takes to propagate a configuration change to the software and hardware components of a system. Furthermore, configuration and operational state data objects may have different lifetimes. The original model of datastores required these data objects to be modeled twice in the YANG schema -- as "config true" objects and as "config false" objects. The convention adopted by the interfaces data model [RFC8343] and the IP data model [RFC8344] was to use two separate branches rooted at the root of the data tree: one branch for configuration data objects and one branch for operational state data objects. The duplication of definitions and the ad hoc separation of operational state data from configuration data lead to a number of problems. Having configuration and operational state data in separate branches in the data model is operationally complicated and impacts the readability of module definitions. Furthermore, the relationship between the branches is not machine readable, and filter expressions operating on configuration and on related operational state are different. With the revised architectural model of datastores defined in this document, the data objects are defined only once in the YANG schema but independent instantiations can appear in different datastores, e.g., one for a configured value and another for an operationally used value. This provides a more elegant and simpler solution to the problem. The revised architectural model of datastores supports additional datastores for systems that support more advanced processing chains converting configuration to operational state. For example, some systems support configuration that is not currently used (so-called "inactive configuration") or they support configuration templates that are used to expand configuration data via a common template.
3. Terminology This document defines the following terminology. Some of the terms are revised definitions of terms originally defined in [RFC6241] and [RFC7950] (see also Section 4). The revised definitions are semantically equivalent to the definitions found in [RFC6241] and [RFC7950]. It is expected that the revised definitions provided in this section will replace the definitions in [RFC6241] and [RFC7950] when these documents are revised. o datastore: A conceptual place to store and access information. A datastore might be implemented, for example, using files, a database, flash memory locations, or combinations thereof. A datastore maps to an instantiated YANG data tree. o schema node: A node in the schema tree. The formal definition is provided in RFC 7950. o datastore schema: The combined set of schema nodes for all modules supported by a particular datastore, taking into consideration any deviations and enabled features for that datastore. o configuration: Data that is required to get a device from its initial default state into a desired operational state. This data is modeled in YANG using "config true" nodes. Configuration can originate from different sources. o configuration datastore: A datastore holding configuration. o running configuration datastore: A configuration datastore holding the current configuration of the device. It may include configuration that requires further transformations before it can be applied. This datastore is referred to as "<running>". o candidate configuration datastore: A configuration datastore that can be manipulated without impacting the device's running configuration datastore and that can be committed to the running configuration datastore. This datastore is referred to as "<candidate>". o startup configuration datastore: A configuration datastore holding the configuration loaded by the device into the running configuration datastore when it boots. This datastore is referred to as "<startup>".
o intended configuration: Configuration that is intended to be used by the device. It represents the configuration after all configuration transformations to <running> have been performed and is the configuration that the system attempts to apply. o intended configuration datastore: A configuration datastore holding the complete intended configuration of the device. This datastore is referred to as "<intended>". o configuration transformation: The addition, modification, or removal of configuration between the <running> and <intended> datastores. Examples of configuration transformations include the removal of inactive configuration and the configuration produced through the expansion of templates. o conventional configuration datastore: One of the following set of configuration datastores: <running>, <startup>, <candidate>, and <intended>. These datastores share a common datastore schema, and protocol operations allow copying data between these datastores. The term "conventional" is chosen as a generic umbrella term for these datastores. o conventional configuration: Configuration that is stored in any of the conventional configuration datastores. o dynamic configuration datastore: A configuration datastore holding configuration obtained dynamically during the operation of a device through interaction with other systems, rather than through one of the conventional configuration datastores. o dynamic configuration: Configuration obtained via a dynamic configuration datastore. o learned configuration: Configuration that has been learned via protocol interactions with other systems and that is neither conventional nor dynamic configuration. o system configuration: Configuration that is supplied by the device itself. o default configuration: Configuration that is not explicitly provided but for which a value defined in the data model is used. o applied configuration: Configuration that is actively in use by a device. Applied configuration originates from conventional, dynamic, learned, system, and default configuration.
o system state: The additional data on a system that is not configuration, such as read-only status information and collected statistics. System state is transient and modified by interactions with internal components or other systems. System state is modeled in YANG using "config false" nodes. o operational state: The combination of applied configuration and system state. o operational state datastore: A datastore holding the complete operational state of the device. This datastore is referred to as "<operational>". o origin: A metadata annotation indicating the origin of a data item. o remnant configuration: Configuration that remains part of the applied configuration for a period of time after it has been removed from the intended configuration or dynamic configuration. The time period may be minimal or may last until all resources used by the newly deleted configuration (e.g., network connections, memory allocations, file handles) have been deallocated. The following additional terms are not datastore specific, but they are commonly used and are thus defined here as well: o client: An entity that can access YANG-defined data on a server, over some network management protocol. o server: An entity that provides access to YANG-defined data to a client, over some network management protocol. o notification: A server-initiated message indicating that a certain event has been recognized by the server. o remote procedure call: An operation that can be invoked by a client on a server.
4. Background NETCONF [RFC6241] provides the following definitions: o datastore: A conceptual place to store and access information. A datastore might be implemented, for example, using files, a database, flash memory locations, or combinations thereof. o configuration datastore: The datastore holding the complete set of configuration that is required to get a device from its initial default state into a desired operational state. YANG 1.1 [RFC7950] provides the following refinements when NETCONF is used with YANG (which is the usual case, but note that NETCONF was defined before YANG existed): o datastore: When modeled with YANG, a datastore is realized as an instantiated data tree. o configuration datastore: When modeled with YANG, a configuration datastore is realized as an instantiated data tree with configuration. [RFC6244] defined operational state data as follows: o Operational state data is a set of data that has been obtained by the system at runtime and influences the system's behavior similar to configuration data. In contrast to configuration data, operational state is transient and modified by interactions with internal components or other systems via specialized protocols. Section 4.3.3 of [RFC6244] discusses operational state and mentions, among other things, the option to consider operational state as being stored in another datastore. Section 4.4 of [RFC6244] then concludes that, at the time of its writing, modeling state as distinct leafs and distinct branches is the recommended approach. Implementation experience and requests from operators [OpState-Reqs] [OpState-Modeling] indicate that the datastore model initially designed for NETCONF and refined by YANG needs to be extended. In particular, the notion of intended configuration and applied configuration has developed.
4.1. Original Model of Datastores The following drawing shows the original model of datastores as it is currently used by NETCONF [RFC6241]: +-------------+ +-----------+ | <candidate> | | <startup> | | (ct, rw) |<---+ +--->| (ct, rw) | +-------------+ | | +-----------+ | | | | | +-----------+ | +-------->| <running> |<--------+ | (ct, rw) | +-----------+ | v operational state <--- control plane (cf, ro) ct = config true; cf = config false rw = read-write; ro = read-only boxes denote datastores Figure 1 Note that this diagram simplifies the model: "read-only" (ro) and "read-write" (rw) are to be understood from the client's perspective, at a conceptual level. In NETCONF, for example, support for <candidate> and <startup> is optional, and <running> does not have to be writable. Furthermore, <startup> can only be modified by copying <running> to <startup> in the standardized NETCONF datastore editing model. The RESTCONF protocol does not expose these differences and instead provides only a writable unified datastore, which hides whether edits are done through <candidate>, by directly modifying <running>, or via some other implementation-specific mechanism. RESTCONF also hides how configuration is made persistent. Note that implementations may also have additional datastores that can propagate changes to <running>. NETCONF explicitly mentions so-called "named datastores".
Some observations: o Operational state has not been defined as a datastore, although there were proposals in the past to introduce an operational state datastore. o The NETCONF <get> operation returns the contents of <running> together with the operational state. It is therefore necessary that "config false" data be in a different branch than the "config true" data if the operational state can have a different lifetime compared to configuration or if configuration is not immediately or successfully applied. o Several implementations have proprietary mechanisms that allow clients to store inactive data in <running>. Inactive data is conceptually removed before validation. o Some implementations have proprietary mechanisms that allow clients to define configuration templates in <running>. These templates are expanded automatically by the system, and the resulting configuration is applied internally. o Some operators have reported that it is essential for them to be able to retrieve the configuration that has actually been successfully applied, which may be a subset or a superset of the <running> configuration.
5. Architectural Model of Datastores Below is a new conceptual model of datastores, extending the original model in order to reflect the experience gained with the original model. +-------------+ +-----------+ | <candidate> | | <startup> | | (ct, rw) |<---+ +--->| (ct, rw) | +-------------+ | | +-----------+ | | | | | +-----------+ | +-------->| <running> |<--------+ | (ct, rw) | +-----------+ | | // configuration transformations, | // e.g., removal of nodes marked as | // "inactive", expansion of | // templates v +------------+ | <intended> | // subject to validation | (ct, ro) | +------------+ | // changes applied, subject to | // local factors, e.g., missing | // resources, delays | dynamic | +-------- learned configuration configuration | +-------- system configuration datastores -----+ | +-------- default configuration | | | v v v +---------------+ | <operational> | <-- system state | (ct + cf, ro) | +---------------+ ct = config true; cf = config false rw = read-write; ro = read-only boxes denote named datastores Figure 2
5.1. Conventional Configuration Datastores The conventional configuration datastores are a set of configuration datastores that share exactly the same datastore schema, allowing data to be copied between them. The term is meant as a generic umbrella description of these datastores. If a module does not contain any configuration data nodes and it is not needed to satisfy any imports, then it MAY be omitted from the datastore schema for the conventional configuration datastores. The set of datastores include: o <running> o <candidate> o <startup> o <intended> Other conventional configuration datastores may be defined in future documents. The flow of data between these datastores is depicted in Section 5. The specific protocols may define explicit operations to copy between these datastores, e.g., NETCONF defines the <copy-config> operation. 5.1.1. The Startup Configuration Datastore (<startup>) The startup configuration datastore (<startup>) is a configuration datastore holding the configuration loaded by the device when it boots. <startup> is only present on devices that separate the startup configuration from the running configuration datastore. The startup configuration datastore may not be supported by all protocols or implementations. On devices that support non-volatile storage, the contents of <startup> will typically persist across reboots via that storage. At boot time, the device loads the saved startup configuration into <running>. To save a new startup configuration, data is copied to <startup> via either implicit or explicit protocol operations.
5.1.2. The Candidate Configuration Datastore (<candidate>) The candidate configuration datastore (<candidate>) is a configuration datastore that can be manipulated without impacting the device's current configuration and that can be committed to <running>. The candidate configuration datastore may not be supported by all protocols or implementations. <candidate> does not typically persist across reboots, even in the presence of non-volatile storage. If <candidate> is stored using non-volatile storage, it is reset at boot time to the contents of <running>. 5.1.3. The Running Configuration Datastore (<running>) The running configuration datastore (<running>) is a configuration datastore that holds the current configuration of the device. It MAY include configuration that requires further transformation before it can be applied, e.g., inactive configuration, or template-mechanism- oriented configuration that needs further expansion. However, <running> MUST always be a valid configuration data tree, as defined in Section 8.1 of [RFC7950]. <running> MUST be supported if the device can be configured via conventional configuration datastores. If a device does not have a distinct <startup> and non-volatile storage is available, the device will typically use that non-volatile storage to allow <running> to persist across reboots. 5.1.4. The Intended Configuration Datastore (<intended>) The intended configuration datastore (<intended>) is a read-only configuration datastore. It represents the configuration after all configuration transformations to <running> are performed (e.g., template expansion, removal of inactive configuration) and is the configuration that the system attempts to apply. <intended> is tightly coupled to <running>. Whenever data is written to <running>, the server MUST also immediately update and validate <intended>. <intended> MAY also be updated independently of <running> if the effect of a configuration transformation changes, but <intended> MUST always be a valid configuration data tree, as defined in Section 8.1 of [RFC7950].
For simple implementations, <running> and <intended> are identical. The contents of <intended> are also related to the "config true" subset of <operational>; hence, a client can determine to what extent the intended configuration is currently in use by checking to see whether the contents of <intended> also appear in <operational>. <intended> does not persist across reboots; its relationship with <running> makes that unnecessary. Currently, there are no standard mechanisms defined that affect <intended> so that it would have different content than <running>, but this architecture allows for such mechanisms to be defined. One example of such a mechanism is support for marking nodes as inactive in <running>. Inactive nodes are not copied to <intended>. A second example is support for templates, which can perform transformations on the configuration from <running> to the configuration written to <intended>. 5.2. Dynamic Configuration Datastores The model recognizes the need for dynamic configuration datastores that are, by definition, not part of the persistent configuration of a device. In some contexts, these have been termed "ephemeral datastores", since the information is ephemeral, i.e., lost upon reboot. The dynamic configuration datastores interact with the rest of the system through <operational>. The datastore schema for a dynamic configuration datastore MAY differ from the datastore schema used for conventional configuration datastores. If a module does not contain any configuration data nodes and it is not needed to satisfy any imports, then it MAY be omitted from the datastore schema for the dynamic configuration datastore. 5.3. The Operational State Datastore (<operational>) The operational state datastore (<operational>) is a read-only datastore that consists of all "config true" and "config false" nodes defined in the datastore's schema. In the original NETCONF model, the operational state only had "config false" nodes. The reason for incorporating "config true" nodes here is to be able to expose all operational settings without having to replicate definitions in the data models.
<operational> contains system state and all configuration actually used by the system. This includes all applied configuration from <intended>, learned configuration, system-provided configuration, and default values defined by any supported data models. In addition, <operational> also contains applied configuration from dynamic configuration datastores. The datastore schema for <operational> MUST be a superset of the combined datastore schema used in all configuration datastores, except that configuration data nodes supported in a configuration datastore MAY be omitted from <operational> if a server is not able to accurately report them. Requests to retrieve nodes from <operational> always return the value in use if the node exists, regardless of any default value specified in the YANG module. If no value is returned for a given node, then this implies that the node is not used by the device. The interpretation of what constitutes being "in use" by the system is dependent on both the schema definition and the device implementation. Generally, functionality that is enabled and operational on the system would be considered to be "in use". Conversely, functionality that is neither enabled nor operational on the system is considered not to be "in use"; hence, it SHOULD be omitted from <operational>. <operational> SHOULD conform to any constraints specified in the data model, but given the principal aim of returning "in use" values, it is possible that constraints MAY be violated under some circumstances (e.g., an abnormal value is "in use", the structure of a list is being modified, or remnant configuration (see Section 5.3.1) still exists). Note that deviations SHOULD be used when it is known in advance that a device does not fully conform to the <operational> schema. Only semantic constraints MAY be violated. These are the YANG "when", "must", "mandatory", "unique", "min-elements", and "max-elements" statements; and the uniqueness of key values. Syntactic constraints MUST NOT be violated, including hierarchical organization, identifiers, and type-based constraints. If a node in <operational> does not meet the syntactic constraints, then it MUST NOT be returned, and some other mechanism should be used to flag the error. <operational> does not persist across reboots.
5.3.1. Remnant Configuration Changes to configuration may take time to percolate through to <operational>. During this period, <operational> may contain nodes for both the previous and current configuration, as closely as possible tracking the current operation of the device. Such remnant configuration from the previous configuration persists until the system has released resources used by the newly deleted configuration (e.g., network connections, memory allocations, file handles). Remnant configuration is a common example of where the semantic constraints defined in the data model cannot be relied upon for <operational>, since the system may have remnant configuration whose constraints were valid with the previous configuration and that are not valid with the current configuration. Since constraints on "config false" nodes may refer to "config true" nodes, remnant configuration may force the violation of those constraints. 5.3.2. Missing Resources Configuration in <intended> can refer to resources that are not available or otherwise not physically present. In these situations, these parts of <intended> are not applied. The data appears in <intended> but does not appear in <operational>. A typical example is an interface configuration that refers to an interface that is not currently present. In such a situation, the interface configuration remains in <intended> but the interface configuration will not appear in <operational>. Note that configuration validity cannot depend on the current state of such resources, since that would imply that removing a resource might render the configuration invalid. This is unacceptable, especially given that rebooting such a device would cause it to restart with an invalid configuration. Instead, we allow configuration for missing resources to exist in <running> and <intended>, but it will not appear in <operational>. 5.3.3. System-Controlled Resources Sometimes, resources are controlled by the device and the corresponding system-controlled data appears in (and disappears from) <operational> dynamically. If a system-controlled resource has matching configuration in <intended> when it appears, the system will try to apply the configuration; this causes the configuration to appear in <operational> eventually (if application of the configuration was successful).
5.3.4. Origin Metadata Annotation As configuration flows into <operational>, it is conceptually marked with a metadata annotation [RFC7952] that indicates its origin. The origin applies to all configuration nodes except non-presence containers. The "origin" metadata annotation is defined in Section 7. The values are YANG identities. The following identities are defined: o origin: abstract base identity from which the other origin identities are derived. o intended: represents configuration provided by <intended>. o dynamic: represents configuration provided by a dynamic configuration datastore. o system: represents configuration provided by the system itself. Examples of system configuration include applied configuration for an always-existing loopback interface, or interface configuration that is auto-created due to the hardware currently present in the device. o learned: represents configuration that has been learned via protocol interactions with other systems, including such protocols as link-layer negotiations, routing protocols, and DHCP. o default: represents configuration using a default value specified in the data model, using either values in the "default" statement or any values described in the "description" statement. The default origin is only used when the configuration has not been provided by any other source. o unknown: represents configuration for which the system cannot identify the origin. These identities can be further refined, e.g., there could be separate identities for particular types or instances of dynamic configuration datastores derived from "dynamic". For all configuration data nodes in <operational>, the device SHOULD report the origin that most accurately reflects the source of the configuration that is in use by the system.
In cases where it could be ambiguous as to which origin should be used, i.e., where the same data node value has originated from multiple sources, the "description" statement in the YANG module SHOULD be used as guidance for choosing the appropriate origin. For example: If, for a particular configuration node, the associated YANG "description" statement indicates that a protocol-negotiated value overrides any configured value, then the origin would be reported as "learned", even when a learned value is the same as the configured value. Conversely, if, for a particular configuration node, the associated YANG "description" statement indicates that a protocol-negotiated value does not override an explicitly configured value, then the origin would be reported as "intended", even when a learned value is the same as the configured value. In the case that a device cannot provide an accurate origin for a particular configuration data node, it SHOULD use the origin "unknown". 6. Implications on YANG 6.1. XPath Context This section updates Section 6.4.1 of RFC 7950. If a server implements the architecture defined in this document, the accessible trees for some XPath contexts are refined as follows: o If the XPath expression is defined in a substatement to a data node that represents system state, the accessible tree is all operational state in the server. The root node has all top-level data nodes in all modules as children. o If the XPath expression is defined in a substatement to a "notification" statement, the accessible tree is the notification instance and all operational state in the server. If the notification is defined on the top level in a module, then the root node has the node representing the notification being defined and all top-level data nodes in all modules as children. Otherwise, the root node has all top-level data nodes in all modules as children.
o If the XPath expression is defined in a substatement to an "input" statement in an "rpc" or "action" statement, the accessible tree is the RPC or action operation instance and all operational state in the server. The root node has top-level data nodes in all modules as children. Additionally, for an RPC, the root node also has the node representing the RPC operation being defined as a child. The node representing the operation being defined has the operation's input parameters as children. o If the XPath expression is defined in a substatement to an "output" statement in an "rpc" or "action" statement, the accessible tree is the RPC or action operation instance and all operational state in the server. The root node has top-level data nodes in all modules as children. Additionally, for an RPC, the root node also has the node representing the RPC operation being defined as a child. The node representing the operation being defined has the operation's output parameters as children. 6.2. Invocation of Actions and RPCs This section updates Section 7.15 of RFC 7950. Actions are always invoked in the context of the operational state datastore. The node for which the action is invoked MUST exist in the operational state datastore. Note that this document does not constrain the result of invoking an RPC or action in any way. For example, an RPC might be defined to modify the contents of some datastore.