7. Using the FE Model in the ForCES Protocol
The actual model of the forwarding plane in a given NE is something
the CE must learn and control by communicating with the FEs (or by
other means). Most of this communication will happen in the post-
association phase using the ForCES protocol. The following types of
information must be exchanged between CEs and FEs via the ForCES
1. FE topology query,
2. FE capability declaration,
3. LFB topology (per FE) and configuration capabilities query,
4. LFB capability declaration,
5. State query of LFB components,
6. Manipulation of LFB components, and
7. LFB topology reconfiguration.
Items 1 through 5 are query exchanges, where the main flow of
information is from the FEs to the CEs. Items 1 through 4 are
typically queried by the CE(s) in the beginning of the post-
association (PA) phase, though they may be repeatedly queried at any
time in the PA phase. Item 5 (state query) will be used at the
beginning of the PA phase, and often frequently during the PA phase
(especially for the query of statistical counters).
Items 6 and 7 are "command" types of exchanges, where the main flow
of information is from the CEs to the FEs. Messages in Item 6 (the
LFB re-configuration commands) are expected to be used frequently.
Item 7 (LFB topology re-configuration) is needed only if dynamic LFB
topologies are supported by the FEs and it is expected to be used
The inter-FE topology (Item 1 above) can be determined by the CE in
many ways. Neither this document nor the ForCES protocol [RFC5810]
document mandates a specific mechanism. The LFB class definition
does include the capability for an FE to be configured with, and to
provide to the CE in response to a query, the identity of its
neighbors. There may also be defined specific LFB classes and
protocols for neighbor discovery. Routing protocols may be used by
the CE for adjacency determination. The CE may be configured with
the relevant information.
The relationship between the FE model and the seven post-association
messages is visualized in Figure 12:
..........-->| CE |
/----\ . +--------+
\____/ FE Model . ^ |
| |................ (1),2 | | 6, 7
| | (off-line) . 3, 4, 5 | |
\____/ . | v
e.g., RFCs ..........-->| FE |
Figure 12: Relationship between the FE model and the ForCES protocol
messages, where (1) is part of the ForCES base protocol, and the
rest are defined by the FE model.
The actual encoding of these messages is defined by the ForCES
protocol [RFC5810] document and is beyond the scope of the FE model.
Their discussion is nevertheless important here for the following
o These PA model components have considerable impact on the FE
model. For example, some of the above information can be
represented as components of the LFBs, in which case such
components must be defined in the LFB classes.
o The understanding of the type of information that must be
exchanged between the FEs and CEs can help to select the
appropriate protocol format and the actual encoding method (such
as XML, TLVs).
o Understanding the frequency of these types of messages should
influence the selection of the protocol format (efficiency
The remaining sub-sections of this section address each of the seven
7.1. FE Topology Query
An FE may contain zero, one, or more external ingress ports.
Similarly, an FE may contain zero, one, or more external egress
ports. In other words, not every FE has to contain any external
ingress or egress interfaces. For example, Figure 13 shows two
cascading FEs. FE #1 contains one external ingress interface but no
external egress interface, while FE #2 contains one external egress
interface but no ingress interface. It is possible to connect these
two FEs together via their internal interfaces to achieve the
complete ingress-to-egress packet processing function. This provides
the flexibility to spread the functions across multiple FEs and
interconnect them together later for certain applications.
While the inter-FE communication protocol is out of scope for ForCES,
it is up to the CE to query and understand how multiple FEs are
inter-connected to perform a complete ingress-egress packet
processing function, such as the one described in Figure 13. The
inter-FE topology information may be provided by FEs, may be hard-
coded into CE, or may be provided by some other entity (e.g., a bus
manager) independent of the FEs. So while the ForCES protocol
[RFC5810] supports FE topology query from FEs, it is optional for the
CE to use it, assuming that the CE has other means to gather such
| +---------+ +------------+ +---------+ |
input| | | | | | output |
---+->| Ingress |-->|Header |-->|IPv4 |---------+--->+
| | port | |Decompressor| |Forwarder| FE | |
| +---------+ +------------+ +---------+ #1 | |
V | +------------+ +----------+ |
| input | | | | output |
+->--+->|Header |-->| Egress |---------+-->
| |Compressor | | port | FE |
| +------------+ +----------+ #2 |
Figure 13: An example of two FEs connected together.
Once the inter-FE topology is discovered by the CE after this query,
it is assumed that the inter-FE topology remains static. However, it
is possible that an FE may go down during the NE operation, or a
board may be inserted and a new FE activated, so the inter-FE
topology will be affected. It is up to the ForCES protocol to
provide a mechanism for the CE to detect such events and deal with
the change in FE topology. FE topology is outside the scope of the
7.2. FE Capability Declarations
FEs will have many types of limitations. Some of the limitations
must be expressed to the CEs as part of the capability model. The
CEs must be able to query these capabilities on a per-FE basis.
Examples are the following:
o Metadata passing capabilities of the FE. Understanding these
capabilities will help the CE to evaluate the feasibility of LFB
topologies, and hence to determine the availability of certain
o Global resource query limitations (applicable to all LFBs of the
o LFB supported by the FE.
o LFB class instantiation limit.
o LFB topological limitations (linkage constraint, ordering, etc.).
7.3. LFB Topology and Topology Configurability Query
The ForCES protocol must provide the means for the CEs to discover
the current set of LFB instances in an FE and the interconnections
between the LFBs within the FE. In addition, sufficient information
should be available to determine whether the FE supports any CE-
initiated (dynamic) changes to the LFB topology, and if so, determine
the allowed topologies. Topology configurability can also be
considered as part of the FE capability query as described in Section
7.4. LFB Capability Declarations
LFB class specifications define a generic set of capabilities. When
an LFB instance is implemented (instantiated) on a vendor's FE, some
additional limitations may be introduced. Note that we discuss only
those limitations that are within the flexibility of the LFB class
specification. That is, the LFB instance will remain compliant with
the LFB class specification despite these limitations. For example,
certain features of an LFB class may be optional, in which case it
must be possible for the CE to determine whether or not an optional
feature is supported by a given LFB instance. Also, the LFB class
definitions will probably contain very few quantitative limits (e.g.,
size of tables), since these limits are typically imposed by the
implementation. Therefore, quantitative limitations should always be
expressed by capability arguments.
LFB instances in the model of a particular FE implementation will
possess limitations on the capabilities defined in the corresponding
LFB class. The LFB class specifications must define a set of
capability arguments, and the CE must be able to query the actual
capabilities of the LFB instance via querying the value of such
arguments. The capability query will typically happen when the LFB
is first detected by the CE. Capabilities need not be re-queried in
case of static limitations. In some cases, however, some
capabilities may change in time (e.g., as a result of adding/removing
other LFBs, or configuring certain components of some other LFB when
the LFBs share physical resources), in which case additional
mechanisms must be implemented to inform the CE about the changes.
The following two broad types of limitations will exist:
o Qualitative restrictions. For example, a standardized multi-
field classifier LFB class may define a large number of
classification fields, but a given FE may support only a subset of
o Quantitative restrictions, such as the maximum size of tables,
The capability parameters that can be queried on a given LFB class
will be part of the LFB class specification. The capability
parameters should be regarded as special components of the LFB. The
actual values of these components may, therefore, be obtained using
the same component query mechanisms as used for other LFB components.
Capability components are read-only arguments. In cases where some
implementations may allow CE modification of the value, the
information must be represented as an operational component, not a
Assuming that capabilities will not change frequently, the efficiency
of the protocol/schema/encoding is of secondary concern.
Much of this restrictive information is captured by the component
property information, and so can be accessed uniformly for all
information within the model.
7.5. State Query of LFB Components
This feature must be provided by all FEs. The ForCES protocol and
the data schema/encoding conveyed by the protocol must together
satisfy the following requirements to facilitate state query of the
o Must permit FE selection. This is primarily to refer to a single
FE, but referring to a group of (or all) FEs may optionally be
o Must permit LFB instance selection. This is primarily to refer to
a single LFB instance of an FE, but optionally addressing of a
group of (or all) LFBs may be supported.
o Must support addressing of individual components of an LFB.
o Must provide efficient encoding and decoding of the addressing
info and the configured data.
o Must provide efficient data transmission of the component state
over the wire (to minimize communication load on the CE-FE link).
7.6. LFB Component Manipulation
The FE model provides for the definition of LFB classes. Each class
has a globally unique identifier. Information within the class is
represented as components and assigned identifiers within the scope
of that class. This model also specifies that instances of LFB
classes have identifiers. The combination of class identifiers,
instance identifiers, and component identifiers is used by the
protocol to reference the LFB information in the protocol operations.
7.7. LFB Topology Reconfiguration
Operations that will be needed to reconfigure LFB topology are as
o Create a new instance of a given LFB class on a given FE.
o Connect a given output of LFB x to the given input of LFB y.
o Disconnect: remove a link between a given output of an LFB and a
given input of another LFB.
o Delete a given LFB (automatically removing all interconnects to/
from the LFB).
8. Example LFB Definition
This section contains an example LFB definition. While some
properties of LFBs are shown by the FE Object LFB, this endeavors to
show how a data plane LFB might be build. This example is a
fictional case of an interface supporting a coarse WDM optical
interface that carries frame relay traffic. The statistical
information (including error statistics) is omitted.
Later portions of this example include references to protocol
operations. The operations described are operations the protocol
needs to support. The exact format and fields are purely
informational here, as the ForCES protocol [RFC5810] document defines
the precise syntax and semantics of its operations.
<?xml version="1.0" encoding="UTF-8"?>
A frame relay frame, with DLCI without
<synopsis>An IP Packet</synopsis>
Information about a single CWDM frequency
<synopsis>state of this frequency</synopsis>
<synopsis>current observed power</synopsis>
Information about circuits on this Frequency
Information about a single Frame Relay Circuit
<synopsis>DLCI of the circuit</synopsis>
<synopsis>state of the circuit</synopsis>
<synopsis>is this the LMI circuit</synopsis>
which input / output port is associated
with this circuit
The possible values of status. Used for both
administrative and operational status
<synopsis>the component is disabled</synopsis>
<synopsis>FE is operatively enabled</synopsis>
<synopsis>The DLCI the frame arrived on</synopsis>
<synopsis>The index of the laser channel</synopsis>
<!-- dummy classid, but needs to be a valid value -->
<synopsis>Fictional LFB for Demonstrations</synopsis>
<component access="read-write" componentID="1">
<synopsis>is this port allowed to function</synopsis>
<component access="read-write" componentID="2">
table of information per CWDM frequency
whether the port over all is operational
how many laser frequencies are there
Total supportable Frame Relay Circuits, across
all laser frequencies
The state of a frequency has changed
<!-- report the new state -->
<synopsis>A new frequency has appeared</synopsis>
A frequency Table entry has been deleted
8.1. Data Handling
This LFB is designed to handle data packets coming in from or going
out to the external world. It is not a full port, and it lacks many
useful statistics, but it serves to show many of the relevant
behaviors. The following paragraphs describe a potential operational
device and how it might use this LFB definition.
Packets arriving without error from the physical interface come in on
a frame relay DLCI on a laser channel. These two values are used by
the LFB to look up the handling for the packet. If the handling
indicates that the packet is LMI, then the output index is used to
select an LFB port from the LMItoFE port group. The packet is sent
as a full frame relay frame (without any bit or byte stuffing) on the
selected port. The laser channel and DLCI are sent as metadata, even
though the DLCI is also still in the packet.
Good packets that arrive and are not LMI and have a frame relay type
indicator of IP are sent as IP packets on the port in the DatatoFE
port group, using the same index field from the table based on the
laser channel and DLCI. The channel and DLCI are attached as
metadata for other use (classifiers, for example).
The current definition does not specify what to do if the frame relay
type information is not IP.
Packets arriving on input ports arrive with the laser channel and
frame relay DLCI as metadata. As such, a single input port could
have been used. With the structure that is defined (which parallels
the output structure), the selection of channel and DLCI could be
restricted by the arriving input port group (LMI vs. data) and port
index. As an alternative LFB design, the structures could require a
1-1 relationship between DLCI and the LFB port, in which case no
metadata would be needed. This would however be quite complex and
noisy. The intermediate level of structure here allows parallelism
between input and output, without requiring excessive ports.
8.1.1. Setting Up a DLCI
When a CE chooses to establish a DLCI on a specific laser channel, it
sends a SET request directed to this LFB. The request might look
T = SET
T = PATH-DATA
Path: flags = none, length = 4, path = 2, channel, 4, entryIdx
DataRaw: DLCI, Enabled(1), false, out-idx
which would establish the DLCI as enabled, with traffic going to a
specific entry of the output port group DatatoFE. (The CE would
ensure that the output port is connected to the right place before
issuing this request.)
The response would confirm the creation of the specified entry. This
table is structured to use separate internal indices and DLCIs. An
alternative design could have used the DLCI as index, trading off
One could also imagine that the FE has an LMI LFB. Such an LFB would
be connected to the LMItoFE and LMIfromFE port groups. It would
process LMI information. It might be the LFB's job to set up the
frame relay circuits. The LMI LFB would have an alias entry that
points to the frame relay circuits table it manages, so that it can
manipulate those entities.
8.1.2. Error Handling
The LFB will receive invalid packets over the wire. Many of these
will simply result in incrementing counters. The LFB designer might
also specify some error rate measures. This puts more work on the
FE, but allows for more meaningful alarms.
There may be some error conditions that should cause parts of the
packet to be sent to the CE. The error itself is not something that
can cause an event in the LFB. There are two ways this can be
One way is to define a specific component to count the error, and a
component in the LFB to hold the required portion of the packet. The
component could be defined to hold the portion of the packet from the
most recent error. One could then define an event that occurs
whenever the error count changes, and declare that reporting the
event includes the LFB field with the packet portion. For rare but
extremely critical errors, this is an effective solution. It ensures
reliable delivery of the notification. And it allows the CE to
control if it wants the notification.
Another approach is for the LFB to have a port that connects to a
redirect sink. The LFB would attach the laser channel, the DLCI, and
the error indication as metadata, and ship the packet to the CE.
Other aspects of error handling are discussed under events below.
8.2. LFB Components
This LFB is defined to have two top-level components. One reflects
the administrative state of the LFB. This allows the CE to disable
the LFB completely.
The other component is the table of information about the laser
channels. It is a variable-sized array. Each array entry contains
an identifier for what laser frequency this entry is associated with,
whether that frequency is operational, the power of the laser at that
frequency, and a table of information about frame relay circuits on
this frequency. There is no administrative status since a CE can
disable an entry simply by removing it. (Frequency and laser power
of a non-operational channel are not particularly useful. Knowledge
about what frequencies can be supported would be a table in the
The frame relay circuit information contains the DLCI, the
operational circuit status, whether this circuit is to be treated as
carrying LMI information, and which port in the output port group of
the LFB traffic is to be sent to. As mentioned above, the circuit
index could, in some designs, be combined with the DLCI.
The capability information for this LFB includes whether the
underlying interface is operational, how many frequencies are
supported, and how many total circuits, across all channels, are
permitted. The maximum number for a given laser channel can be
determined from the properties of the FrameRelayCircuits table. A
GET-PROP on path 2.channel.4 will give the CE the properties of that
FrameRelayCircuits array which include the number of entries used,
the first available entry, and the maximum number of entries
This LFB is defined to be able to generate several events in which
the CE may be interested. There are events to report changes in
operational state of frequencies, and the creation and deletion of
frequency entries. There is an event for changes in status of
individual frame relay circuits. So an event notification of
18.104.22.168 would indicate that there had been a circuit status change
on subscript 11 of the circuit table in subscript 3 of the frequency
table. The event report would include the new status of the circuit
and the DLCI of the circuit. Arguably, the DLCI is redundant, since
the CE presumably knows the DLCI based on the circuit index. It is
included here to show including two pieces of information in an event
As described above, the event declaration defines the event target,
the event condition, and the event report content. The event
properties indicate whether the CE is subscribed to the event, the
specific threshold for the event, and any filter conditions for the
Another event shown is a laser power problem. This event is
generated whenever the laser falls below the specified threshold.
Thus, a CE can register for the event of laser power loss on all
circuits. It would do this by:
T = SET-PROP
Path-TLV: flags=0, length = 2, path = 61.4
Path-TLV: flags = property-field, length = 1, path = 2
Content = 1 (register)
Path-TLV: flags = property-field, length = 1, path = 3
Content = 15 (threshold)
This would set the registration for the event on all entries in the
table. It would also set the threshold for the event, causing
reporting if the power falls below 15. (Presumably, the CE knows
what the scale is for power, and has chosen 15 as a meaningful
If a laser oscillates in power near the 15 mark, one could get a lot
of notifications. (If it flips back and forth between 14 and 15,
each flip down will generate an event.) Suppose that the CE decides
to suppress this oscillation somewhat on laser channel 5. It can do
this by setting the hysteresis property on that event. The request
would look like:
T = SET-PROP
Path-TLV: flags=0, length = 3, path = 61.4.5
Path-TLV: flags = property-field, length = 1, path = 4
Content = 2 (hysteresis)
Setting the hysteresis to 2 suppresses a lot of spurious
notifications. When the level first falls below 10, a notification
is generated. If the power level increases to 10 or 11, and then
falls back below 10, an event will not be generated. The power has
to recover to at least 12 and fall back below 10 to generate another
event. One common cause of this form of oscillation is when the
actual value is right near the border. If it is really 9.5, tiny
changes might flip it back and forth between 9 and 10. A hysteresis
level of 1 will suppress this sort of condition. Many other events
have oscillations that are somewhat wider, so larger hysteresis
settings can be used with those.
9. IANA Considerations
The ForCES model creates the need for a unique XML namespace for
ForCES library definition usage, and unique class names and numeric
9.1. URN Namespace Registration
IANA has registered a new XML namespace, as per the guidelines in RFC
URI: The URI for this namespace is
Registrant Contact: IESG
XML: none, this is an XML namespace
9.2. LFB Class Names and LFB Class Identifiers
In order to have well defined ForCES LFB Classes, and well defined
identifiers for those classes, IANA has created a registry of LFB
class names, corresponding class identifiers, and the document that
defines the LFB class. The registry policy is simply first come,
first served (FCFS) with regard to LFB class names. With regard to
LFB class identifiers, identifiers less than 65536 are reserved for
assignment by IETF Standards-Track RFCs. Identifiers equal to or
above 65536, in the 32-bit class ID space, are available for
assignment on a first come, first served basis. All registry entries
must be documented in a stable, publicly available form.
Since this registry provides for FCFS allocation of a portion of the
class identifier space, it is necessary to define rules for naming
classes that are using that space. As these can be defined by
anyone, the needed rule is to keep the FCFS class names from
colliding with IETF-defined class names. Therefore, all FCFS class
names MUST start with the string "Ext-".
Table 1 tabulates the above information.
IANA has created a registry of ForCES LFB Class Names and the
corresponding ForCES LFB Class Identifiers, with the location of the
definition of the ForCES LFB Class, in accordance with the rules in
the following table.
| LFB Class Name | LFB Class | Place Defined | Description |
| | Identifier | | |
| Reserved | 0 | RFC 5812 | Reserved |
| | | | -------- |
| FE Object | 1 | RFC 5812 | Defines ForCES |
| | | | Forwarding Element |
| | | | information |
| FE Protocol | 2 |  | Defines parameters |
| Object | | | for the ForCES |
| | | | protocol operation |
| | | | -------- |
| IETF defined | 3-65535 | Standards | Reserved for IETF |
| LFBs | | Track RFCs | defined RFCs |
| | | | -------- |
| ForCES LFB | >65535 | Any Publicly | First Come, First |
| Class names | | Available | Served for any use |
| beginning EXT- | | Document | |
10. Authors Emeritus
The following are the authors who were instrumental in the creation
of earlier releases of this document.
Ellen Delganes, Intel Corp.
Lily Yang, Intel Corp.
Ram Gopal, Nokia Research Center
Alan DeKok, Infoblox, Inc.
Zsolt Haraszti, Clovis Solutions
Many of the colleagues in our companies and participants in the
ForCES mailing list have provided invaluable input into this work.
Particular thanks to Evangelos Haleplidis for help getting the XML
12. Security Considerations
The FE model describes the representation and organization of data
sets and components in the FEs. The ForCES framework document
[RFC3746] provides a comprehensive security analysis for the overall
ForCES architecture. For example, the ForCES protocol entities must
be authenticated per the ForCES requirements before they can access
the information elements described in this document via ForCES.
Access to the information contained in the FE model is accomplished
via the ForCES protocol, which is defined in separate documents, and
thus the security issues will be addressed there.
13.1. Normative References
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119, March 1997.
[RFC5810] Doria, A., Ed., Hadi Salim, J., Ed., Haas, R., Ed.,
Khosravi, H., Ed., Wang, W., Ed., Dong, L., Gopal, R., and
J. Halpern, "Forwarding and Control Element Separation
(ForCES) Protocol Specification", RFC 5810, March 2010.
[RFC3688] Mealling, M., "The IETF XML Registry", BCP 81, RFC 3688,
[Schema1] Thompson, H., Beech, D., Maloney, M., and N. Mendelsohn,
"XML Schema Part 1: Structures", W3C REC-xmlschema-1,
http://www.w3.org/TR/xmlshcema-1/, May 2001.