Internet Engineering Task Force (IETF) E. Haleplidis Request for Comments: 6369 O. Koufopavlou Category: Informational S. Denazis ISSN: 2070-1721 University of Patras September 2011 Forwarding and Control Element Separation (ForCES) Implementation Experience
AbstractThe Forwarding and Control Element Separation (ForCES) protocol defines a standard communication and control mechanism through which a Control Element (CE) can control the behavior of a Forwarding Element (FE). This document captures the experience of implementing the ForCES protocol and model. Its aim is to help others by providing examples and possible strategies for implementing the ForCES protocol. Status of This Memo This document is not an Internet Standards Track specification; it is published for informational purposes. This document is a product of the Internet Engineering Task Force (IETF). It represents the consensus of the IETF community. It has received public review and has been approved for publication by the Internet Engineering Steering Group (IESG). Not all documents approved by the IESG are a candidate for any level of Internet Standard; see Section 2 of RFC 5741. Information about the current status of this document, any errata, and how to provide feedback on it may be obtained at http://www.rfc-editor.org/info/rfc6369. Copyright Notice Copyright (c) 2011 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must
include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License. 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 2 1.1. Document Goal . . . . . . . . . . . . . . . . . . . . . . 3 2. Terminology and Conventions . . . . . . . . . . . . . . . . . 3 3. ForCES Architecture . . . . . . . . . . . . . . . . . . . . . 4 3.1. Pre-Association Setup - Initial Configuration . . . . . . 5 3.2. TML . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 3.3. Model . . . . . . . . . . . . . . . . . . . . . . . . . . 6 3.3.1. Components . . . . . . . . . . . . . . . . . . . . . . 6 3.3.2. LFBs . . . . . . . . . . . . . . . . . . . . . . . . . 9 3.4. Protocol . . . . . . . . . . . . . . . . . . . . . . . . . 10 3.4.1. TLVs . . . . . . . . . . . . . . . . . . . . . . . . . 10 3.4.2. Message Deserialization . . . . . . . . . . . . . . . 13 3.4.3. Message Serialization . . . . . . . . . . . . . . . . 15 4. Development Platforms . . . . . . . . . . . . . . . . . . . . 15 5. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 16 6. Security Considerations . . . . . . . . . . . . . . . . . . . 16 7. References . . . . . . . . . . . . . . . . . . . . . . . . . . 17 7.1. Normative References . . . . . . . . . . . . . . . . . . . 17 7.2. Informative References . . . . . . . . . . . . . . . . . . 17 RFC3654] defines the ForCES requirements, and [RFC3746] defines the ForCES framework. The ForCES protocol works in a master-slave mode in which Forwarding Elements (FEs) are slaves and Control Elements (CEs) are masters. The protocol includes commands for transport of Logical Functional Block (LFB) configuration information, association setup, status, and event notifications, etc. The reader is encouraged to read the Forwarding and Control Element Separation Protocol [RFC5810] for further information. [RFC5812] presents a formal way to define FE LFBs using XML. LFB configuration components, capabilities, and associated events are defined when LFBs are formally created. The LFBs within the Forwarding Element (FE) are accordingly controlled in a standardized way by the ForCES protocol.
The Transport Mapping Layer (TML) transports the protocol messages. The TML is where the issues of how to achieve transport-level reliability, congestion control, multicast, ordering, etc., are handled. It is expected that more than one TML will be standardized. The various possible TMLs could vary their implementations based on the capabilities of underlying media and transport. However, since each TML is standardized, interoperability is guaranteed as long as both endpoints support the same TML. All ForCES protocol layer implementations must be portable across all TMLs. Although more than one TML may be standardized for the ForCES protocol, all ForCES implementations must implement the Stream Control Transmission Protocol (SCTP) TML [RFC5811]. The Forwarding and Control Element Separation Applicability Statement [RFC6041] captures the applicable areas in which ForCES can be used. RFC5810], the Forwarding and Control Element Separation Forwarding Element Model [RFC5812], and the SCTP- Based Transport Mapping Layer (TML) for the Forwarding and Control Element Separation Protocol [RFC5811]. RFC5810]; some of the definitions below are copied from that document. Control Element (CE): A logical entity that implements the ForCES protocol and uses it to instruct one or more FEs on how to process packets. CEs handle functionality such as the execution of control and signaling protocols.
Forwarding Element (FE): A logical entity that implements the ForCES protocol. FEs use the underlying hardware to provide per-packet processing and handling as directed/controlled by one or more CEs via the ForCES protocol. LFB (Logical Functional Block): The basic building block that is operated on by the ForCES protocol. The LFB is a well-defined, logically separable functional block that resides in an FE and is controlled by the CE via the ForCES protocol. The LFB may reside at the FE's data path and process packets or may be purely an FE control or configuration entity that is operated on by the CE. Note that the LFB is a functionally accurate abstraction of the FE's processing capabilities but not a hardware-accurate representation of the FE implementation. LFB Class and LFB Instance: LFBs are categorized by LFB classes. An LFB instance represents an LFB class (or type) existence. There may be multiple instances of the same LFB class (or type) in an FE. An LFB class is represented by an LFB class ID, and an LFB instance is represented by an LFB instance ID. As a result, an LFB class ID associated with an LFB instance ID uniquely specifies an LFB existence. LFB Component: Operational parameters of the LFBs that must be visible to the CEs are conceptualized in the FE model as the LFB components. The LFB components include, for example, flags, single parameter arguments, complex arguments, and tables that the CE can read and/or write via the ForCES protocol. ForCES Protocol: While there may be multiple protocols used within the overall ForCES architecture, the terms "ForCES protocol" and "protocol" refer to the Fp reference points in the ForCES framework [RFC3746]. This protocol does not apply to CE-to-CE communication, FE-to-FE communication, or communication between FE and CE Managers. Basically, the ForCES protocol works in a master-slave mode in which FEs are slaves and CEs are masters. This document defines the specifications for this ForCES protocol.
There are several issues that should concern a developer for the TML: 1. Security. TML must be secure according to the respective RFC. For SCTP, you have to use IPsec. 2. Remote connection. While ForCES is meant to be used locally, both interoperability tests have proven that ForCES can be deployed everywhere that SCTP/IP is available. In both interoperability tests, there were connections between Greece and China, and the performance was very satisfactory. However, in order for the FE and CE to work in a non-local environment, an implementor must ensure that the SCTP-TML ports are forwarded to the CE and/or FE if they are behind NATs; if there is a firewall, it will allow the SCTP ports through. These were identified during the first ForCES interoperability test and documented in the Implementation Report for Forwarding and Control Element Separation [RFC6053].
o Minimum data size. o Maximum data size. If the data size of the component is not variable, then the size is either the minimum or the maximum size, as both should have the same value. Next, some basic functions are in order: o A common constructor. o A common destructor. o Retrieve Component ID. o Retrieve access right property. o Query if it is an optional component. o Get Full Data. o Set Full Data. o Get Sparse Data. o Set Sparse Data. o Del Full Data. o Del Sparse Data. o Get Property. o Set Property. o Get Value. o Set Value. o Del Value. o Get Data. o Clone component.
The Get/Set/Del Full Data, Get/Set/Del Sparse Data, and Get/Set Property functions handle the respective ForCES commands and return the respective TLV, for example, Set Full Data should return a RESULT-TLV. The Get Value, Set Value, and Del Value functions are called from Get Full/Sparse Data, Set Full/Sparse Data, and Del Full/ Sparse Data respectively and provide the interface to the actual values in the hardware, separating the forces handling logic from the interface to the actual values. The Get Data function should return the value of the data only, not in TLV format. The Clone function seems out of place. This function must return a new component that has the exact same values and attributes. This function is useful in array components as described further below. The only requirement is to implement the base atomic data types. Any new atomic datatype can be built as a child of a base data type, which will inherit all the functions and, if necessary, override them. The struct component can then be built. A struct component is a component by itself but consists of a number of atomic components. These atomic components create a static array within the struct. The ID of each atomic component is the array's index. For a struct component, the Clone function must create and return an exact copy of the struct component with the same static array. The most difficult component to be built is the array. The difficulty lies in the actual benefit of the model: you have absolute freedom over what you build. An array is an array of components. In all rows, you have the exact same type of component, either a single component or a struct. The struct can have multiple single components or a combination of single components, structs, arrays, and so on. So, the difficulty lies in how to create a new row, a new component by itself. This is where the Clone function is very useful. For the array, a mother component that can spawn new components exactly like itself is needed. Once a Set command is received, the mother component can spawn a new component if the targeted row does not exist and add it into the array; with the Set Full Data function, the value is set in the recently spawned component, as the spawned component knows how the data is created. In order to distinguish these spawned components from each other and their functionality, some kind of index is required that will also reflect how the actual data of the specific component is stored on the hardware.
Once the basic constructors of all possible components are created, then a developer only has to create LFB components or datatypes as a child of one of the already-created components, and the only thing the developer really needs to add is the three functions of Get Value, Set Value, and Del Value of each component, which is platform dependent. The rest stays the same. Figure 1. The following code creates a part of FEProtocolLFB:
//FEID cui = new Component_uInt(FEPO_FEID, ACCESS_READ_ONLY, FE_id); Components[cui->get_ComponentId()]=cui; //Add component to array list //Current FEHB Policy Value cub = new Component_uByte(FEPO_FEHBPolicy, ACCESS_READ_WRITE, 0); Components[cub->get_ComponentId()]=cub; //Add component to array list //FEIDs for BackupCEs Array cui = new Component_uInt(0, ACCESS_READ_WRITE, 0); ca = new Component_Array(FEPO_BackupCEs, ACCESS_READ_WRITE); ca->AddRow(cui, 1); ca->AddMotherComponent(cui); Components[ca->get_ComponentId()]=ca; //Add component to array list Figure 1: Example Code for Creating Part of FEProtocolLFB The same concept can be applied to handling LFBs as one FE. An FE is a collection of LFBs. Thus, all LFBs can be stored in an array based on the LFB's class id, version, and instance. Then, what is required is an LFBHandler that will handle the array of LFBs. A specific LFB, for example, can be addressed using the following scheme: LFBs[ClassID][Version][InstanceID] Note: While an array can be used in components, capabilities, and events, a hash table or a similar concept is better suited for storing LFBs using the component ID as the hash key with linked lists for collision handling, as the created array can have large gaps if the values of LFB Class ID vary greatly.
Additionally, the protocol also allows two different path approaches to LFB components, and the CE or FE must handle both or even a mix of them, making a generic decoding of the protocol message difficult. Another difficulty also arises from the batching capabilities of the protocol. You can have multiple Operations within a message; you can select more than one LFB to command and more than one component to manipulate. A possible solution is again provided by inheritance. There are two basic components in a protocol message: 1. The common header. 2. The rest of the message. The rest of the message is divided in Type-Length-Value (TLV) units and, in one case, Index-Length-Value (ILV) units. The TLV hierarchy can be seen in Figure 2: Common Header | +---------------+---------------+---------------+ | | | | REDIRECT-TLV LFBselect-TLV ASResult-TLV ASTreason-TLV | | OPER-TLV | | PATH-DATA-TLV ---> Optional KEYINFO-TLV | +-------------+-------------+-------------+ | | | | SPARSEDATA-TLV RESULT-TLV FULLDATA-TLV PATH-DATA-TLV Figure 2: ForCES TLV Hierarchy The above figure shows only the basic hierarchical level of TLVs and does not show batching. Also, this figure does not show the recursion that can occur at the last level of the hierarchy. The figure shows one kind of recursion with a PATH-DATA-TLV within a PATH-DATA-TLV. A FULLDATA-TLV can be within a FULLDATA-TLV and a SPARSEDATA-TLV. The possible combination of TLVs are described in detail in the Forwarding and Control Element Separation Protocol [RFC5810] as well as the data-packing rules.
A TLV's main attributes are: o Type. o Length. o Data. o An array of TLVs. The array of TLVs is the next hierarchical level of TLVs nested in this TLV. A TLV's common function could be: o A basic constructor. o A constructor using data from the wire. o Add a new TLV for next level. o Get the next TLV of next level. o Get a specific TLV of next level. o Replace a TLV of next level. o Get the Data. o Get the Length. o Set the Data. o Set the Length. o Set the Type. o Serialize the header. o Serialize the TLV to be written on the wire. All TLVs inherit these functions and attributes and either override them or create new where it is required.
2. Read the necessary length for this PATH-DATA-TLV, and create the PATH-DATA-TLV from the data of the wire. 3. Add this PATH-DATA-TLV to the OPER-TLV's array of TLVs. 4. Do this until the rest of the OPER-TLV is finished. Here it gets interesting, as the next level of PATH-DATA-TLVs can be one of the following: o PATH-DATA-TLVs. o FULLDATA-TLV. o SPARSEDATA-TLV. o RESULT-TLV. The solution to this difficulty is recursion. If the next TLV is a PATH-DATA-TLV, then the PATH-DATA-TLV that is created uses the same kind of deserialization until it reaches a FULLDATA-TLV or SPARSEDATA-TLV. There can be only one FULLDATA-TLV or SPARSEDATA-TLV within a PATH-DATA-TLV. 1. Read the next 2 shorts(type-length). 2. If the Type is a PATH-DATA-TLV, then repeat the previous algorithm but add the PATH-DATA-TLV to this PATH-DATA-TLV's array of TLVs. 3. Do this until the rest of the PATH-DATA-TLV is finished. 4. If the Type is a FULLDATA-TLV, then create the FULLDATA-TLV from the message and add this to the PATH-DATA-TLV's array of TLVs. 5. If the Type is a SPARSEDATA-TLV, then create the SPARSEDATA-TLV from the message and add this to the PATH-DATA-TLV's array of TLVs. 6. If the Type is a RESULT-TLV, then create the RESULT-TLV from the message and add this to the PATH-DATA-TLV's array of TLVs. If the message is a Query, it must not have any kind of data inside the PATH-DATA-TLV. If the message is a Query Response, then it must have either a RESULT-TLV or a FULLDATA-TLV.
If the message is a Config, it must contain either a FULLDATA-TLV or a SPARSEDATA-TLV. If the message is a Config Response, it must contain a RESULT-TLV. More details regarding message validation can be read in Section 7 of the Forwarding and Control Element Separation Protocol [RFC5810]. Note: When deserializing, implementors must take care to ignore padding of TLVs as all must be 32-bit aligned. The length value in TLVs includes the Type and Length (4 bytes) but does not include padding. Figure 3 provides an initial survey of SCTP support for C/C++ and Java at the present time.
/-------------+-------------+-------------+-------------\ |\ Platform | | | | | ----------\ | Windows | Linux | Solaris | | Language \| | | | +-------------+-------------+-------------+-------------+ | | | | | | C/C++ | Supported | Supported | Supported | | | | | | +-------------+-------------+-------------+-------------+ | | Limited | | | | Java | Third Party | Supported | Supported | | | Not from SUN| | | \-------------+-------------+-------------+-------------/ Figure 3: SCTP Support on Operating Systems A developer should be aware of some limitations regarding Java implementations. Java inherently does not support unsigned types. A workaround can be found in the creation of classes that do the translation of unsigned types to Java types. The problem is that the unsigned long cannot be used as-is in the Java platform. The proposed set of classes can be found in [JavaUnsignedTypes]. RFC3746] and the Forwarding and Control Element Separation Protocol [RFC5810] into account. Also, as specified in the Security Considerations section of the SCTP-Based Transport Mapping Layer (TML) for the Forwarding and Control Element Separation Protocol [RFC5811], transport-level security has to be ensured by IPsec.
[RFC5810] Doria, A., Hadi Salim, J., Haas, R., Khosravi, H., Wang, W., Dong, L., Gopal, R., and J. Halpern, "Forwarding and Control Element Separation (ForCES) Protocol Specification", RFC 5810, March 2010. [RFC5811] Hadi Salim, J. and K. Ogawa, "SCTP-Based Transport Mapping Layer (TML) for the Forwarding and Control Element Separation (ForCES) Protocol", RFC 5811, March 2010. [RFC5812] Halpern, J. and J. Hadi Salim, "Forwarding and Control Element Separation (ForCES) Forwarding Element Model", RFC 5812, March 2010. [RFC6041] Crouch, A., Khosravi, H., Doria, A., Wang, X., and K. Ogawa, "Forwarding and Control Element Separation (ForCES) Applicability Statement", RFC 6041, October 2010. [RFC6053] Haleplidis, E., Ogawa, K., Wang, W., and J. Hadi Salim, "Implementation Report for Forwarding and Control Element Separation (ForCES)", RFC 6053, November 2010. [JavaUnsignedTypes] "Java Unsigned Types", <http://nam.ece.upatras.gr/index.php?q=node/44>. [RFC3654] Khosravi, H. and T. Anderson, "Requirements for Separation of IP Control and Forwarding", RFC 3654, November 2003. [RFC3746] Yang, L., Dantu, R., Anderson, T., and R. Gopal, "Forwarding and Control Element Separation (ForCES) Framework", RFC 3746, April 2004.