Content for TR 33.916 Word version: 17.0.0
7 Evaluation and SCAS instantiation
7.1 Security Assurance Specification instantiation documents creation
7.2 Evaluation and evaluation report
7.3 Self-declaration
7.4 Partial compliance and use of SECAM requirements in network product development cycle
7.5 Comparison between two SECAM evaluations
7.6 The evaluation of a new version
7 Evaluation and SCAS instantiation p. 28
7.1 Security Assurance Specification instantiation documents creation p. 28
The SCAS instantiation consists of a set of documents provided by the Vendor to give test laboratories and operators the relevant information to understand the critical parts of the network product to be evaluated. The SCAS instantiation provides a concrete mapping of the "theoretical" assets and security requirements of the SCAS into "real" assets and components supporting the security requirements of the network product being evaluated.
The SCAS instantiation is a set of documents and is not expected to have a fixed structure. This will allow vendors to maximize the reuse of existing documentation.
The content of the SCAS instantiation is however defined and it contains details on:
- Network Product description (e.g. software version, documentation version).
- Scope of evaluation.
- Mapping of SCAS security requirements to the network product and assets in the network product.
- References to the applicable document versions containing operational guidance in the documentation of the network product.
- Information needed to start the Security Compliance Testing, including Basic Vulnerability Testing.
- Details of licenses that are required for the product to operate in the scope of evaluation (if relevant).
7.2 Evaluation and evaluation report p. 28
7.2.1 Network product development process and network product lifecycle management p. 28
The security relevant part of the Vendor network product development and network product lifecycle management process is evaluated during an accreditation administrated by the SECAM Accreditation Body prior to any network product evaluation. During a network product evaluation, the accredited test laboratory validates that the accredited process was used for the network product under consideration. To support this evaluation, the vendor provides the following documents to the accredited test laboraty and, if requested, to the operator:
- The evidence of the vendor network product development and network product lifecycle management process accreditation by the SECAM Accreditation Body.
- The Vendor Network Product Development and network product lifecycle management process self-evaluation report for the network product under evaluation containing:
- a rationale showing that the generic accredited security relevant part of the process was effectively applied during the development of the network product under evaluation (free-form).
7.2.2 SCAS instantiation evaluation p. 29
7.2.2.1 Overview p. 29
SCAS instantiation evaluation is to check whether a SCAS instantiation written by a vendor is a correct instantiation of the SCAS of the network product class and whether it is a good basis for evaluating the network product.
The accredited evaluator (vendor or third-party evaluator) for security compliance testing is responsible for SCAS instantiation evaluation before it is used to evaluate network product. The evaluator confirms at least that the SCAS being instantiated for a given 3GPP network product and the network product for evaluation are consistent.
7.2.2.2 Content p. 29
7.2.2.2.1 Scope of the evaluation p. 29
7.2.2.2.1.1 Overview p. 29
A given network product from a vendor might be packaged in different ways for each commercial transaction to address the tailored request from operators.
SECAM evaluations are conducted for a particular packaging of the network product. One objective in SECAM is to ensure maximum reusability of evaluation results of the evaluation of a particular package while still providing a clear and comprehensive description of the boundaries of what was evaluated. In practice to maximize the reuse, the vendor is likely to have the most commonly sold package of its network product evaluated.
A clear definition of the boundaries of what was evaluated ensures this reusability but also prevents a false perception of what was security tested as additional components are facing well-defined interfaces.
Consequently, in the scope of evaluation of the SCAS instantiation document, the vendor provides a clear description of the network product that will be tested, i.e. a description of the version of the network product in the scope of SCAS.
In particular, the network product description does not contain security requirements or functions, but a logical and physical perimeter for the evaluation. In particular the definition of the network product describes its content in terms of high level description of the components and external interfaces. This description of the network product provides:
- All components mandated by network product class definition in SCAS and implemented by the network product.
- All external communication interfaces of the network product. Details (including protocols, ports, services and purpose) for each of the external communication interfaces of the network product that allow communications between functions inside and outside the network product.
"Requirement: The product shall include a security audit function, accessible only by a user having the role admin X, logged through SSH on the server.
Test case:
In this case it is clear what, from where to test and how to test (physical port of the network product where the SSH server is listening).
- the tester shall connect as the admin user through SSH and verify that he can access the audit
- the tester shall verify that a user without admin rights cannot access the audit using the same connection
- the tester shall verify that no other means exist to access the audit except a SSH session".
7.2.2.2.1.2 Adapting the SCAS instantiation to special circumstances p. 30
A network product may need to adapt the SCAS instantiation to its own circumstances.
E.g. this could happen when the network product only partially implements a network product class, for which a SCAS exists. In such cases where there is no fully fitting SCAS for a SECAM evaluation the derivation in the instantiated SCAS might need some special adaptation. The possibility for adaptation is also useful to avoid that SCAS creation and Network Product Class scoping get too complex and have to cover a multitude of parallel versions with very small differences.
A SCAS instantiation might also need to be adapted when a gap is discovered in an existing SCAS, e.g. due to a newly published vulnerability, and the network product evaluation cannot wait until 3GPP has closed this gap.
7.2.2.2.1.3 Exclusion of components p. 30
The SCAS instantiation does not exclude a component from testing on the grounds that it was already evaluated under another scheme, different from SECAM, unless this SCAS allows it explicitly to refer to the certificate obtained under this different scheme for a given set of tests.
No component can be excluded from evaluation on the grounds that it was not developed by vendor itself and that it is an outsourced or a 3rd party component.
7.2.2.2.2 Mapping of SCAS security requirements to the network product and assets in the network product p. 30
The SCAS instantiation will provide:
- A concrete mapping of the SCAS "theoretical" assets to "real" assets on the network product.
- A concrete mapping of the SCAS security requirements to the high-level components supporting these functions.
-
all assets from SCAS are present in the SCAS instantiation,
EXAMPLE 1:
The SCAS instantiation does not decide, against the SCAS, that some assets need no protection because of physical deployment site protection.
-
if SCAS instantiation introduces new assets they are considered assets to be protected in a manner consistent with SCAS,
EXAMPLE 2:
If the SCAS instantiation uses two admin roles instead of a single one in the generic SCAS, both have their credentials protected consistently.
-
the SCAS instantiation does not waive threats identified in the SCAS.
EXAMPLE 3:
The SCAS instantiation does not claim that a threat from the SCAS is not applicable under the assumption that more organizational control is performed during administrators' recruitment.
7.2.2.2.3 Operational guidance documents and configuration of the network product for evaluation p. 31
Operational guidance documents are part of the documentation created by the vendor and are part of the SCAS instantiation documentation (see clause 7.2.2 for details on SCAS instantiation evaluation). This documentation contains the information on how to initialize, configure and operate the network product so that SECAM security requirements are met. To achieve security, it is necessary to align the network product and the content of the "operational guidance documents".
E.g. this documentation could be a user manual indicating to the administrator:
- By default, the network product is provisioned with root password "XXXX"
- The network product will NOT be able to operate as long as this password in not changed using procedure Y
- The minimum password length is 12 characters for secure operation, at least 12 characters password SHALL be chosen
- vendor or operator staff during initial setup of the network product.
- vendor or operator staff during operation of the network product.
- vendor or operator staff during maintenance or upgrade of the network product.
- evaluators during SECAM compliance and vulnerability evaluations to install a representative test bed.
7.2.2.2.4 Information needed to execute the required tests for SCT and BVT activities p. 31
Information needed to execute the required tests for the Security Compliance Testing:
The compliance tester assesses whether the SCAS instantiation contains enough information to:
- execute the test cases;
- determine whether the tests completely and accurately cover the SCAS.
Information needed to execute the required tests for the Basic Vulnerability Testing:
The basic vulnerability tester assesses whether the SCAS instantiation contains enough information to:
- determine the tools to be used in the Basic Vulnerability Testing;
- execute the test cases;
- determine whether all open ports are explicitly documented;
- determine the scope of vulnerability scanning to reflect the SCAS requirements.
7.2.2.3 Process p. 32
The usage and update of this set of documents during a SECAM evaluation is described in Figure 7.2.2.3-1 below.
Figure 7.2.2.3-1: Overview of the SCAS instantiation documents evolution during a SECAM evaluation
(⇒ copy of original 3GPP image)
(⇒ copy of original 3GPP image)
Step1 is the initial production by the vendor of the required documentation and its update if required by step 2. It is outside of the scope of SECAM to describe this task.
Step 2 is the SCAS instantiation evaluation to check whether an SCAS instantiation written by a vendor is a correct instantiation of the SCAS of the network product class and whether it is a good basis for evaluating the network product.
Step 3 is about performing the SCT and BVT testing tasks as described in the present document. which will use this instantiation documentation as input. The evaluation does not start (neither SCT nor BVT) as long as steps 1 and 2 are not completed.
Further documentation is produced during step 3 . During step 3 for example, the SCT and BVT testers will describe the concrete test bed used for testing as well as "instantiated test cases" (i.e. the description of the concrete test case on the network product corresponding to the generic SCAS test case). At the end of step 3, the SCAS instantiation documentation as well as the SCT and BVT documentation is an output document provided to the operator. These documents are described in clauses 7.2.3 and 7.2.4.
After step 3, the SCAS instantiation documentation as well as the SCT and BVT documentation produced in step 3 are given to the operator for its final review and final security acceptance decision.
7.2.3 Security compliance testing p. 34
7.2.3.1 Inputs p. 34
The test bed configured according to the documentation that was produced in step 3 of clause 7.2.2.3.
7.2.3.2 Outputs p. 34
In the end of Security Compliance tests, the tester delivers a Security Compliance Testing report which includes:
- a declaration about who carried out the tests;
- network products/features tested and reasons for not testing where applicable:
- in particular, copies of other Security Compliance related third party certificates and test reports of previous evaluation (internal and/or third party), if appropriate and available;
-
a description of the testbed used for the tests, which
- are accurate,
- make the test bed reproducible (non ambiguous),
- are representative of real-life network product deployment;
- the test tools and vectors used for the tests;
- a rationale which demonstrates that the tests cover the SCAS test cases;
- the test procedure followed in practice (following SCAS test cases) and results (following SCAS output format indications).
7.2.3.3 Activities p. 34
The security compliance of a network product is its compliance to a defined set of security requirements. The security requirements set will be provided in the SCAS. The test case describes the validation technique to be used by the Security Compliance Test laboratories as well as the expected outputs to provide in the evaluation report.
Security compliance test laboratories execute the tests contained in the 3GPP SCAS for the evaluated network product as described in the test cases, collect evaluation evidences and include them in the final security compliance testing report (see clause 7.2.3.2 above for details of outputs).
7.2.4 Basic Vulnerability Testing p. 34
Basic Vulnerability Testing activities consist of requirements for running automated Free and Open Source Software (FOSS) and Commercial off-the-shelf (COTS) security testing tools against the external interfaces of a Network Product. Such tools or equivalent alternatives are likely available to all kind of attackers.
This activity covers at least three aspects: Port Scanning, Vulnerability Scanner by the use of Vulnerability scanners and robustness/fuzz testing. The tester delivers a Basic Vulnerability Testing report which includes:
- the test procedures (following SCAS);
- the test results (following SCAS output format indications).
7.3 Self-declaration p. 35
After the evaluation process is finished, the vendors review all the evaluation results of the product and give a declaration of their product. In the self-declaration, vendors should:
give a short summary and conclusion of all the evaluation reports;
declare all tests conducted by the vendors are correctly carried out and all the documents provided by the vendors are authentic without intentional deception.
7.4 Partial compliance and use of SECAM requirements in network product development cycle p. 35
The vendor is likely to integrate SECAM requirements and test cases in its continuous development process. During this phase, a given network product might fail fully or partially some of the SECAM compliance and/or vulnerability test. The process of how and when vendor choose to fix or not to fix this network product before the final evaluation is under vendor's responsibility and is outside of SECAM scope.
SECAM scheme describes the final evaluation for the final network product version expected to be bought by operators. SECAM encourages vendors to aim at a full compliance of all SECAM requirements which should represent a minimum baseline. However, the final network product might still only partially fulfil SECAM requirements. This partial compliance will be documented in the test results in the evaluation report. The final security acceptance decision is under operators' control which might accept partially compliant products. This choice is under operators' responsibility and is outside of SECAM scope.
7.5 Comparison between two SECAM evaluations p. 35
SECAM evaluation considers a given version of a network product. SECAM documents don't describe or evaluate the improvement between two evaluations of the same version of the network product.
7.6 The evaluation of a new version p. 35
After a network product completes a SECAM evaluation by a test laboratory, the vendor may upgrade the network product e.g. as a result of modified features, remediation of vulnerabilities etc. A re-evaluation of the new version of the network product can be carried out in two ways as described below:
- evaluation as per a new product by using all the test cases as defined in the SCAS's;
- the vendor provides a detailed list of the updates to the product and provides a list of related test cases that will verify the new version of the product. Once the vendor and the test laboratory agree on the updated test plan, the evaluation takes place.
- the list of updates to the network product.
- Test cases that were executed in the evaluation.
- The test results from the evaluation.
