Content for TS 23.288 Word version: 18.4.0

1… 4… 5… 5A… 6… 6.1.3 6.1.4… 6.1.4.4… 6.1.5… 6.1A… 6.1B… 6.1B.2.3… 6.1C 6.2… 6.2.3… 6.2.6… 6.2.6.2 6.2.6.3… 6.2.6.3.3 6.2.6.3.4 6.2.6.3.5 6.2.6.3.6… 6.2.7… 6.2.8… 6.2.9… 6.2.13… 6.2A… 6.2B… 6.2B.3 6.2B.4… 6.2C… 6.2D… 6.2E… 6.2F… 6.3… 6.4… 6.5… 6.6… 6.7… 6.7.3… 6.7.4… 6.7.5… 6.8… 6.9… 6.10… 6.11… 6.12… 6.13… 6.14… 6.16… 6.17… 6.18… 6.19… 6.20… 6.21… 7… 7.4… 7.7… 7.9… 8… 9… 10…

6.2A Procedure for ML Model Provisioning 6.2A.0 General 6.2A.1 ML Model Subscribe/Unsubscribe 6.2A.2 Contents of ML Model Provisioning 6.2A.3 ML Model request
...

6.2A Procedure for ML Model Provisioning |R17| p. 119

6.2A.0 General p. 119

This clause presents the procedure for the ML Model provisioning.

An NWDAF containing AnLF may be locally configured with (a set of) IDs of NWDAFs containing MTLF and the Analytics ID(s) supported by each NWDAF containing MTLF to retrieve trained ML models or may use the NWDAF discovery procedure specified in clause 5.2 for discovering NWDAFs containing MTLF. An NWDAF containing MTLF may determine that further training for an existing ML model is needed when it receives the ML model subscription or the ML model request.

6.2A.1 ML Model Subscribe/Unsubscribe p. 119

The procedure in Figure 6.2A.1-1 is used by an NWDAF service consumer, i.e. an NWDAF containing AnLF to subscribe/unsubscribe at another NWDAF, i.e. an NWDAF containing MTLF, to be notified when ML Model Information on the related Analytics becomes available, using Nnwdaf_MLModelProvision services as defined in clause 7.5. The ML Model Information is used by an NWDAF containing AnLF to derive analytics. The service is also used by an NWDAF to modify existing ML Model Subscription(s). An NWDAF can be at the same time a consumer of this service provided by other NWDAF(s) and a provider of this service to other NWDAF(s).

Figure 6.2A.1-1: ML Model for analytics subscribe/unsubscribe

Step 1.

The NWDAF service consumer (i.e. an NWDAF containing AnLF) subscribes to, modifies, or cancels subscription for a (set of) trained ML Model(s) associated with a/an (set of) Analytics ID(s) by invoking the Nnwdaf_MLModelProvision_Subscribe / Nnwdaf_MLModelProvision_Unsubscribe service operation. The parameters that can be provided by the NWDAF service consumer are listed in clause 6.2A.2. The service consumer optionally indicates its support for multiple ML models if available.

When a subscription for a trained ML model associated with an Analytics ID is received, the NWDAF containing MTLF may:

determine whether existing trained ML Model(s) can be used for the subscription; or
determine whether triggering further training for the existing trained ML models is needed for the subscription.

If the NWDAF containing MTLF determines that further training is needed, this NWDAF may initiate data collection from NFs, (e.g. AMF/DCCF/ADRF), UE Application (via AF) or OAM as described in clause 6.2, to generate the ML model.

If the service invocation is for a subscription modification or subscription cancelation, the NWDAF service consumer includes an identifier (Subscription Correlation ID) to be modified in the invocation of Nnwdaf_MLModelProvision_Subscribe.

Step 2.

If the NWDAF service consumer subscribes to a (set of) trained ML model(s) associated to a (set of) Analytics ID(s), the NWDAF containing MTLF notifies the NWDAF service consumer with:

a set of pair(s) of unique ML Model Identifier and ML Model Information associated with each Analytics ID requested by the service consumer.

by invoking Nnwdaf_MLModelProvision_Notify service operation. The content of trained ML Model Information that can be provided by the NWDAF containing MTLF is specified in clause 6.2A.2.

The NWDAF containing MTLF also invokes the Nnwdaf_MLModelProvision_Notify service operation to notify an available re-trained ML model when the NWDAF containing MTLF determines that the previously provided trained ML Model required re-training at step 1.

When the step 1 is for a subscription modification (i.e. including Subscription Correlation ID), the NWDAF containing MTLF may provide either a new trained ML model different to the previously provided one, or re-trained ML model by invoking Nnwdaf_MLModelProvision_Notify service operation.

6.2A.2 Contents of ML Model Provisioning p. 120

The consumers of the ML model provisioning services (i.e. an NWDAF containing AnLF) as described in clause 7.5 and clause 7.6 may provide the input parameters as listed below:

Information of the analytics for which the requested ML model is to be used, including:
- A list of Analytics ID(s): identifies the analytics for which the ML model is used.
- [OPTIONAL] NF consumer information: identifies the vendor of NWDAF containing AnLF.
  
  NOTE 1:
  NF consumer information such as Vendor ID is defined in Stage 3.
- [OPTIONAL] Use case context: indicates the context of use of the analytics to select the most relevant ML model ML model.
  
  NOTE 2:
  The NWDAF containing MTLF can use the parameter "Use case context" to select the most relevant ML model, when several ML models are available for the requested Analytics ID(s). The values of this parameter are not standardized.
- [OPTIONAL] ML Model Interoperability Information. This is vendor-specific information that conveys, e.g., requested model file format, model execution environment, etc. The encoding, format, and value of ML Model Interoperable Information is not specified since it is vendor specific information, and is agreed between vendors, if necessary for sharing purposes.
- [OPTIONAL] ML Model Filter Information: enables the NWDAF containing MTLF to select which ML model for the analytics is requested, e.g. S-NSSAI, Area of Interest. Parameter types in the ML Model Filter Information are the same as parameter types in the Analytics Filter Information which are defined in procedures.
- [OPTIONAL] Target of ML Model Reporting: indicates the object(s) for which ML model is requested, e.g. specific UEs, a group of UE(s) or any UE (i.e. all UEs).
- [OPTIONAL] Requested representative ratio: a minimum percentage of UEs in the group whose data is a non-empty set and can be used in the model training when the Target of ML Model Reporting is a group of UEs.
- ML Model Reporting Information with the following parameters:
  - (Only for Nnwdaf_MLModelProvision_Subscribe) ML Model Reporting Information Parameters as per Event Reporting Information Parameter defined in Table 4.15.1-1, TS 23.502.
- [OPTIONAL] ML Model Target Period: indicates time interval [start, end] for which ML model for the Analytics is requested. The time interval is expressed with actual start time and actual end time (e.g. via UTC time).
- [OPTIONAL] Inference Input Data information: contains information about various settings that are expected to be used by AnLF during inferences such as:
  - the "Input Data" that are expected be used, each of them optionally accompanied by metrics that show the granularity with which this data will be used (i.e., a sampling ratio, the maximum number of input values, and/or a maximum time interval between the samples of this input data).
    
    NOTE 3:
    This can be a subset of the possible Input Data specified for a certain analytics type.
  - the data sources that are expected to be used as a list of NF instance (or NF set) identifiers.
- A Notification Target Address (+ Notification Correlation ID) as defined in clause 4.15.1 of TS 23.502, allowing to correlate notifications received from the NWDAF containing MTLF with this subscription.
[OPTIONAL] Indication of supporting multiple ML models.
[OPTIONAL] Accuracy level(s) of Interest.
[OPTIONAL] ML Model Monitoring Information:

NOTE 4:
Multiple ML models Filter Information are composed by Accuracy level(s) of Interest and Number of ML model(s).
[OPTIONAL] Time when model is needed: indicates the latest time when the consumer expects to receive the ML model(s).
[OPTIONAL] ML Model Monitoring Information:
- [OPTIONAL] ML Model metric: i.e. ML Model Accuracy.
- [OPTIONAL] ML model monitoring reporting mode: such as Accuracy reporting interval or pre-determined status. Depending on the reporting mode, the NWDAF containing MTLF reports the model accuracy to NWDAF containing AnLF either periodically or when the ML model accuracy is crossing an ML Model Accuracy threshold, i.e. the accuracy either becomes higher or lower than the ML Model Accuracy threshold.
- [OPTIONAL] ML Model Accuracy Threshold: indicating the accuracy threshold of the ML Model requested by the consumer (as a kind of pre-determined status). It also can be used as an indication that the MTLF is triggered to execute the accuracy monitoring operations for the ML Model provisioned to AnLF.
- [OPTIONAL] DataSetTag and ADRF ID if available: indicates the inference data (including input data, prediction and the ground truth data at the time which the prediction refers to) stored in ADRF which can be used by MTLF to retrain or reprovision of the ML model.
- [OPTIONAL] ML Model Identifier: indicates the Model that the data corresponding to the DataSetTag is related to (in the case of subscription modification).

The NWDAF containing MTLF provides to the consumer of the ML model provisioning service operations as described in clause 7.5 and clause 7.6, the output information as listed below:

(Only for Nnwdaf_MLModelProvision_Notify) The Notification Correlation Information.
For each Analytics ID requested by the service consumer, a set of pair (s) of unique ML Model identifier and the following information.
- ML Model Information, which includes:
  - the ML model file address (e.g. URL or FQDN); or
  - ADRF (Set) ID.
    When ADRF (Set) ID is provisioned, a Storage Transaction ID may also be provisioned.
- [OPTIONAL] ML model degradation indicator: indicates whether the provided ML model is degraded.
- [OPTIONAL] Validity period: indicates time period when the provided ML Model Information applies.
- [OPTIONAL] Spatial validity: indicates Area where the provided ML Model Information applies.
  
  NOTE 5:
  Spatial validity and Validity period are determined by MTLF internal logic and it is a subset of AoI if provided in ML Model Filter Information and of ML Model Target Period, respectively.
- [OPTIONAL] ML model representative ratio: indicating the percentage of UEs in the group whose data is used in the ML model training when the Target of ML Model Reporting is a group of UEs.
- [OPTIONAL] Training Input Data Information: contains information about various settings that have been used by MTLF during training, such as:
  - the "Input Data" that have been used, each of them optionally accompanied by metrics that show the data characteristics and granularity with which this data has been used (i.e. a sampling ratio, the maximum number of input values and/or a maximum time interval between the samples of this input data, data range including maximum and minimum values, mean and standard deviation and data distribution when applicable) and the time, i.e. timestamp and duration, when this data was obtained.
  - the data sources related to the "Input Data" that were used for ML model training, which have been identified by a list of NF instance (or NF set) identifiers.
    
    NOTE 6:
    This can be a subset of the possible Input Data specified for a certain analytics type.
    
    NOTE 7:
    Data source information enables ML Model selection when different models are available for an Analytics ID, or it enables a consumer to avoid selecting a ML model that used data from a specific data source at a particular time or used data characterized by specific data characteristics.
- [OPTIONAL] ML Model Accuracy Information: indicates the accuracy of the ML model if ML Model accuracy threshold is requested, which includes:
  - the accuracy value of the ML model.
  - [OPTIONAL] ML model metric (i.e. ML Model Accuracy).

6.2A.3 ML Model request p. 122

The procedure in Figure 6.2A.3-1 is used by an NWDAF service consumer, i.e. an NWDAF containing AnLF to request and get from another NWDAF, i.e. an NWDAF containing MTLF ML Model Information, using Nnwdaf_MLModelInfo services as defined in clause 7.6. The ML Model Information is used by an NWDAF containing AnLF to derive analytics. An NWDAF can be at the same time a consumer of this service provided by other NWDAF(s) and a provider of this service to other NWDAF(s).

Reproduction of 3GPP TS 23.288, Fig. 6.2A.3-1: ML model Request

Figure 6.2A.3-1: ML model Request

Step 1.

The NWDAF service consumer (i.e. an NWDAF containing AnLF) requests a (set of) ML Model(s) associated with a/an (set of) Analytics ID(s) by invoking Nnwdaf_MLModelInfo_Request service operation. The parameters that can be provided by the NWDAF Service Consumer are listed in clause 6.2A.2. The service consumer optionally indicates its support for multiple ML models if available.

When a request to an ML Model Information for the Analytics is received, the NWDAF containing MTLF may:

determine whether existing trained ML Model(s) can be used for the request; or
determine whether triggering further training for the existing trained ML models is needed for the request.

Step 2.

The NWDAF containing MTLF responds to the NWDAF service consumer by invoking Nnwdaf_MLModelInfo_Request response service operation including:

a set of pair(s) of unique ML Model identifier and the ML Model Information for each Analytics ID that the NWDAF service consumer requests.

The content of ML Model Information that can be provided by the NWDAF containing MTLF is specified in clause 6.2A.2.