Tech-invite3GPPspaceIETFspace
21222324252627282931323334353637384‑5x

Content for  TS 23.482  Word version:  19.3.0

Top   Top   Up   Prev   Next
0…   5…   8…   8.3…   8.6…   8.7…   8.9…   8.11…   8.13…   8.14…   8.15…   8.19…   8.23…   9…   9.3…   9.4…   A…   B   C…

 

8.19  ML Model Training Capability Evaluationp. 106

This clause describes procedure for supporting ML model training capability evaluation for FL (e.g., HFL, VFL).

8.19.1  Generalp. 106

The following clauses specify procedures, information flows and APIs to support ML model training capability evaluation for FL (e.g., HFL, VFL). The ML model training capability result can be used by the AIMLE server to select FL members for FL training process (e.g. HFL, VFL).

8.19.2  Procedure for ML model training capability evaluationp. 106

Pre-conditions:
  1. AIMLE server determines to use FL (e.g., HFL, VFL) training.
  2. AIMLE Server knows the information of the FL (e.g., HFL, VFL) members (e.g., AIMLE Client which is deployed to a UE).
Copy of original 3GPP image for 3GPP TS 23.482, Fig. 8.19.2-1: Procedure for ML model training capability evaluation
Up
Step 1.
The AIMLE Server sends ML model training capability evaluation request to the FL members (AIMLE Clients which are deployed on UEs). The request message includes information as described in Table 8.19.3.1-1.
Step 2.
The FL members evaluate their capability and availability to join the FL training process. The FL members (AIMLE Clients which are deployed on UEs) run the test task contained in the request in step 1 and determine if it can join the FL process. For VFL, as part of the test task, data alignment between the datasets of the different domains are determined. The VAL server may also provide data labels for the data alignment.
Step 3.
The FL members (AIMLE Clients which are deployed on UEs) send ML model training capability evaluation response to the AIMLE Server. The response message contains the information as described in Table 8.19.3.2-1.
Up

8.19.3  Information flowsp. 107

8.19.3.1  ML model training capability evaluation requestp. 107

Table 8.19.3.1-1 shows the request sent by AIMLE server to FL (e.g., HFL, VFL) members (AIMLE Clients which are deployed on UEs) for ML model training capability evaluation.
Information element Status Description
Requestor identifierMThe identifier of the requestor.
Available timeORequirement on available time for supporting FL operations.
Test taskOThe task for test ML model training capability.
AI/ML model and model parameter(s)OInformation about the AI/ML model and model parameters for use in FL training. In VFL, AI/ML models for different data domains are provided.
Requirement on datasetORequirements on dataset for FL training.
> Common feature ID(s)O
(NOTE 1)
Identifier(s) of the required features common to the dataset of the different data domains.
> Data domain feature ID listsO
(NOTE 1)
List of features for each data domain(s) of the datasets at the UE.
> Data sourceO
(NOTE 2)
Data source for the FL training.
NOTE 1:
If Requirement on dataset is provided, at least one of these IEs shall be present when for VFL.
NOTE 2:
If Requirement on dataset is provided, the IE shall be present when for HFL.
Up

8.19.3.2  ML model training capability evaluation responsep. 107

Table 8.19.3.2-1 shows the response sent by FL members (AIMLE Clients which are deployed on UEs) to AIMLE server for the ML model training capability evaluation.
Information element Status Description
StatusM The status for the evaluation: success, fail.
  • Success means join the FL training process.
  • Fail means not join the FL training process.
Test resultO The test result of the ML model training capability evaluation. The "test result" shall be provided when the "status" is "success".
Fail reasonO The reason of the ML model training capability evaluation fail. The "fail reason" shall be provided when the "status" is "fail".
Up

8.20  AIML service operations control and management procedurep. 107

8.20.1  Generalp. 107

The control and management of the AIML services is an essential requirement for the applications to manage the AIML services like model training, inference, discovery etc.

8.20.2  AIML service operations control and management procedurep. 108

Copy of original 3GPP image for 3GPP TS 23.482, Fig. 8.20.1-1: AIML service operations control and management procedure
Up
Step 1.
The VAL server sends AIML service operations control and management request to the AIML Enablement server as per the Table 8.20.3.1-1.
Step 2-3.
The AIMLE server determines the required service operation mode to manage the AIML service operation lifecycle based on the AIML service operation mode, AIML service operation information. For example, the AIMLE server can perform client discovery/selection using the procedure defined in clause 9.3 and model training using the procedure defined in clause 8.3. Based on the AIML client identifier the AIMLE server sends the AIML Enablement client service operation request as per Table 8.20.3.3-1.
The AIML service operation mode includes start and stop operation. Start indicates the initiation of the AIML service and stop indicates termination of the AIML service.
The AIML Enablement client receiving the AIML service operation mode performs the service operation mode for the AIML service operation. The AIMLE client can configure and monitor the AIML service operation as per the AIML service operation mode configuration. Based on the AIML service operation mode status reporting configuration (periodic or event-triggered), the AIML client reports the service operation mode status to the AIML Enablement server. The AIML Enablement client sends a response indicating the success or failure of the AIML Enablement client service operation as per Table 8.20.3.4-1.
Step 4.
The AIML Enablement server provides the AIML service operations control and management response to the VAL server. The message includes AIML service operation ID and the reporting status of the AIML service operation.
Up

8.20.3  Information flowsp. 108

8.20.3.1  AIML service operations control and management requestp. 108

Table 8.20.3.1-1 shows the request sent by an AIML service consumer (e.g., VAL server) to an AIMLE server for the AIML service operations control and management request.
Information element Status Description
Requestor identityMThe identifier of the requestor (e.g., VAL server).
VAL service identifierOAn identifier for the VAL service associated with the requestor.
AIMLE client identifier(s)O
(NOTE)
Indicates the identifier(s) of AIML enablement client(s).
AIMLE client set identifierO
(NOTE)
An identifier for the AIMLE client set.
AIML service operation identifierOIndicates the AIML service operation identifier to identify the AIML service. (e.g., model training id, ml task id).
AIML service operation informationOIndicates AIML service operation information. It includes AIML service model container, URI of the model to fetch the model from a repository, AIML service aggregator URI to send model updates, AIML service operation optimization assistance like maximum convergence time.
AIML service operation modeMIndicates the required AIMLE service operation modes like start, stop. The start mode defines the initiation of the AIML service. The stop mode is defined to stop the AIML operation.
AIML service operation mode configurationOIndicates the configuration of the AIML service operation modes. It includes network utilization (like stop the AIML service when latency is worse than x milliseconds, time limit threshold (like stop the AIML service after 24 hours), model performance (like stop the AIML service when model accuracy is 99% achieved).
AIML service operation mode status reportingOIndicates the reporting configuration of the AIML service operation status like periodic (e.g. time interval) or event based (e.g., transition of AIML service operation from stop to start).
NOTE:
One of the information elements is present.
Up

8.20.3.2  AIML service operations control and management responsep. 109

Table 8.20.3.2-1 shows the request sent by an AIML Enablement server to an AIML service consumer (e.g., VAL server) for the AIML service operations control and management response.
Information element Status Description
VAL service identifierOAn identifier for the VAL service associated with the requestor.
AIML service operation IDMAn identifier to identify the AIML service operation ID.
AIML service operation mode report statusMIndicates the current state of AIMLE service operation. E.g., start, stop.
Up

8.20.3.3  AIML Enablement client service operation requestp. 109

Table 8.20.3.3-1 shows the request sent by an AIML Enablement server to an AIML Enablement client for the AIML Enablement client service operation request.
Information element Status Description
Requestor identityMThe identifier of the requestor (e.g. AIML service consumer).
VAL service identifierOAn identifier for the VAL service associated with the requestor.
AIML service operation IDMAn identifier to identify the AIML service operation ID.
AIML service operation modeMIndicates the required AIMLE service operation modes like start, stop.
AIML service operation informationOIndicates AIML service operation information. It includes AIML service model container, URI of the model to fetch the model from a repository, AIML service aggregator URI to send model updates, AIML service operation optimization assistance like maximum convergence time.
AIML service operation mode configurationOIndicates the configuration of the AIML service operation modes. It includes network utilization (like stop the AIML service when latency is worse than x milliseconds, time limit threshold (like stop the AIML service after 24 hours), model performance (like stop the AIML service when model accuracy is 99% achieved).
AIML service operation mode status reportingOIndicates the reporting configuration of the AIML service operation status like periodic (e.g. time interval) or event based (e.g. transition of AIML service operation from stop to start).
Up

8.20.3.4  AIML Enablement client service operation responsep. 110

Table 8.20.3.4-1 shows the request sent by an AIML Enablement client to an AIML Enablement server for the AIML Enablement client service operation response or update response.
Information element Status Description
VAL service identifierOAn identifier for the VAL service associated with the requestor.
AIML service operation IDMAn identifier to identify the AIML service operation ID.
AIML service operation mode statusMIndicates the current state of AIMLE service operation. Possible values start, stop.
Up

8.21  ML model updatep. 110

8.21.1  Generalp. 110

This clause provides the procedures to support ML model re-training and update when model performance degradation is observed by the AIML enablement layer. The model update procedure also supports using an existing model to re-train the model using Transfer Learning. Additionally, if the degraded model is related to other models due to e.g., Transfer Learning, the AIMLE server may trigger the update of those related models as well.

8.21.2  Procedurep. 110

Figure 8.21.2-1 depicts the procedure where the AIML enablement capability can trigger model update upon detecting model performance degradation.
Pre-conditions:
  1. The AIMLE Server has provided a ML model to the AIMLE Consumer. The AIMLE consumer can be a VAL server, AIMLE client, or ADAE server.
  2. The AIMLE Consumer detects a performance degradation of the ML model. If the consumer is an ADAE server, performance degradation may be detected as described in clause 8.17 in TS 23.436.
Copy of original 3GPP image for 3GPP TS 23.482, Fig. 8.21.2-1: Support for ML model update
Figure 8.21.2-1: Support for ML model update
(⇒ copy of original 3GPP image)
Up
Step 1.
The AIMLE Consumer sends an ML model update request to the AIMLE Server that includes the ID of the model and performance degradation information.
Step 2.
Based on the performance degradation information, the AIMLE Server determines whether to update the ML model. If the AIMLE Server does not update the model, steps 3 and 4 are skipped.
Step 3.
The AIMLE Server retrieves the ML model information from the ML Repository as described in clause 8.11.3. The AIMLE Server may also perform ML model discovery to determine whether an existing ML model stored by the ML Repository can be used to replace the degraded model or train the new model (e.g., using Transfer Learning). If an existing model can be used to replace the degraded model, step 4 is skipped, and the identified model is provided in step 5.
The AIMLE Server can also discover models that are related due to Transfer Learning or the use of the same training data, to identify additional models that may require an update.
Step 4.
The AIMLE Server performs ML model re-training, which corresponds to the ML model training procedure as described in clause 8.3. The updated model is stored in the ML repository once re-training is complete.
If the degraded model is linked to other models (e.g., due to Transfer Learning, or the same training data has been used), the AIMLE Server may trigger the re-training and update of those related models.
Step 5.
The AIMLE Server provides the updated ML model to the AIMLE Consumer either by sending it directly, or by providing endpoint information to retrieve it from the ML Repository.
Up

8.21.3  Information flowsp. 111

8.21.3.1  ML model update requestp. 111

Table 8.21.3.1-1 details the ML model update request IEs.
Information element Status Description
Requestor IdentityMThe identity of the AIMLE Consumer sending the request.
ML model IDMProvides the ID of ML model for which the performance degradation has been detected.
Performance degradation informationOProvides details about the detected performance degradation, such as the time, instances, or information on the degraded metrics (e.g. accuracy, recall, F1score).
ML model retrieval endpointOThe endpoint (e.g., URL, URI, IP address) where the ML model file can be retrieved.
Up

8.21.3.2  ML model update responsep. 112

Table 8.21.3.2-1 details the ML model update response IEs.
Information element Status Description
Successful responseO
(NOTE 1)
Indicates that the model has been updated.
> ML modelO
(NOTE 2)
Provides the updated ML model.
> ML model retrieval endpointO
(NOTE 2)
The endpoint (e.g., URL, URI, IP address) where the ML model file can be retrieved.
> ML model informationO Provides information of the ML model, specified in Table 8.11.4.1-2.
Failure responseO
(NOTE 1)
Indicates that the request has failed.
> CauseOIndicates the failure cause.
NOTE 1:
Only one of these information elements shall be provided.
NOTE 2:
At least one of these information elements shall be provided.
Up

8.22  ML model performance monitoringp. 112

8.22.1  Generalp. 112

The following clauses specify procedures, information flows, and APIs for ML model performance monitoring and potential degradation detection.

8.22.2  Procedurep. 112

Pre-conditions:
  1. One or more AIMLE services (at the AIMLE server or clients) using the given ML model are ongoing.
Copy of original 3GPP image for 3GPP TS 23.482, Fig. 8.22.2-1: ML model performance monitoring
Figure 8.22.2-1: ML model performance monitoring
(⇒ copy of original 3GPP image)
Up
Step 1.
The VAL server sends an ML model performance monitoring subscription request to AIMLE server, requesting to assist in monitoring the ML model performance. This request consists of ML model information and optionally the AIML service information, etc.
Step 2.
The AIMLE server checks whether the VAL server is authorized to perform the ML model performance monitoring request.
Step 3.
If the VAL server is authorized, AIMLE server returns the success response, otherwise a failure response indication the reason for failure.
Step 4.
The AIMLE server identifies the AIMLE services which are utilizing the requested ML model. Such AIMLE services can be an ML model training service or an HFL service at the AIMLE server or clients.
The identification of the AIMLE services may be performed via fetching ML model information from the ML repository using ML model management procedure as in clause 8.11.3.
Then, the AIMLE server then starts monitoring the AIMLE service performance (e.g. accuracy, a KPI or QoS metric related to the AIML operation). This step includes receiving information from one or more AIMLE clients performing an operation based on the target ML model (e.g., based on step 7 of procedure in clause 8.12.2 or step 6 of procedure in clause 8.15.2), with an expected or experienced deviation of the required performance of the AIMLE service.
Step 5.
The AIMLE server detects an expected ML model degradation (e.g., model drift, data drift) based on the deviation of the performance of the AIMLE service as indicated in step 4.
Step 6.
The AIMLE server based on the expected ML model degradation, it may also indicate and execute a trigger action to ensure meeting the AIMLE service requirement.
Such trigger action may be either an adaptation of the AIMLE service, such as training of a new ML model for the AIMLE by the same or a different AIMLE client, or re-training of the ML model by the same or different AIMLE client; or termination of the AIMLE service and initiating a new AIMLE service with a new ML model.
Step 7.
The AIMLE server notifies the VAL server on the expected ML model degradation and if requested the triggered adaptation of the AIMLE service.
Up

8.22.3  Information flowsp. 114

8.22.3.1  ML model performance monitoring subscription requestp. 114

Table 8.22.3.1-1 shows the request sent by a VAL server to an AIMLE server for the ML model performance monitoring.
Information element Status Description
Requestor identityMThe identifier of the requestor (e.g., VAL server).
ML model identifierMThe identifier of the ML model for which the monitoring applies.
Notification endpointMThe notification endpoint (e.g. URL/URI/IP address) where the notifications should be sent to.
AIML operation informationOThe AIMLE operation (ML model training, HFL, VFL, TL) for which the ML model is used.
> VAL service IDOThe VAL service identifier of the AIMLE service using the ML model (if known by the requestor).
> AIMLE client ID(s)OThe identifier(s) of the AIMLE client(s) training the ML model (if known by the requestor).
> AIMLE service KPIOOne or more KPIs for the AIMLE service performance (latency, accuracy, etc).
Monitoring report configurationMThe reporting configuration for the monitoring service (thresholds for triggering a monitoring event, e.g. minimum accuracy, delay, whether the reporting is one time or periodical or event-triggered).
Area of interestOThe geographical or service area for which the monitoring applies.
Time validityOThe time validity for the monitoring subscription.
Trigger Action requirementOThis requirement identifies policies for triggering an action based on a monitoring event (e.g. if degradation is detected, to train a new model or re-selecting AIMLE clients).
Up

8.22.3.2  ML model performance monitoring subscription responsep. 114

Table 8.22.3.2-1 shows the response sent by the AIMLE server to the VAL server for the ML model performance monitoring subscription.
Information element Status Description
Successful responseO
(NOTE)
Indicates that the ML model performance monitoring request was successful.
> Subscription IDMSubscription identifier corresponding to the subscription.
> Expiration timeOIndicates the expiration time of the subscription. To maintain an active subscription, a subscription update is required before the expiration time.
Failure responseO
(NOTE)
Indicates that the request has failed.
> CauseOThe cause for the request failure.
NOTE:
Only one of these information elements shall be present.
Up

8.22.3.3  ML model performance monitoring notifyp. 114

Table 8.22.3.3-1 shows the notification sent by the AIMLE server to the VAL server for the ML model performance degradation.
Information element Status Description
Subscription IDMSubscription identifier corresponding to the subscription.
ML model IDMIdentity of the ML model.
ML model degradation indicationMIdentifies the degradation of the ML model.
> ML model degradation parametersOThe performance metrics which are expected to be degraded (F1-score, recall, precision, accuracy).
> CauseOThe cause for the degradation of the ML model.
Trigger ActionO The trigger action, which is notified, and may be one of the following:
  • the adaptation of the AIMLE service, such as training of a new ML model for the AIMLE by the same or a different AIMLE client,
  • the re-training of the ML model by the same or different AIMLE client,
  • the termination of the AIMLE service and initiating a new AIMLE service with a new ML model.
Up

Up   Top   ToC