diff --git a/index.rst b/index.rst index 619486c..6203d2d 100644 --- a/index.rst +++ b/index.rst @@ -56,9 +56,9 @@ :glob: testing/about_system_tests + testing/integration_tests testing/aether-roc-tests - testing/system-tests - testing/acceptance_specification + .. toctree:: :maxdepth: 2 diff --git a/intro.rst b/intro.rst index 2c8edc2..f7853db 100644 --- a/intro.rst +++ b/intro.rst @@ -36,10 +36,10 @@ Other Aether guides available on this site include: * :doc:`Test Automation `: Learn how Aether components are tested. -Note that Aether was originally deployed as a centrally-manged, +Note that Aether was originally deployed as a centrally-managed, ONF-operated cloud service, with the expectation that organizations would participate in Aether by connecting their edge site to this -operational deployment.\ [#]_ That service is now being deprecated in +operational deployment.\ [#]_ That service has now been deprecated in favor of users bringing up their own Aether sites using :doc:`OnRamp `. @@ -52,23 +52,12 @@ favor of users bringing up their own Aether sites using :doc:`OnRamp Aether Components ------------------------ -Aether uses components from several ONF projects. Information about -these projects can be found at the following sites: +Aether builds on two main subsystems: SD-Core (a cloud native Mobile +Core) and SD-RAN (an O-RAN compliant, software-based RAN). +Additional documentation for each is available at: -* SD-Core - - * `SD-Core Website `_ - * :doc:`SD-Core Documentation ` - -* SD-RAN - - * `SD-RAN Website `_ - * :doc:`SD-RAN Documentation ` - -* SD-Fabric - - * `SD-Fabric Website `_ - * :doc:`SD-Fabric Documentation ` +* :doc:`SD-Core Documentation ` +* :doc:`SD-RAN Documentation ` More information about 5G and Aether's architecture can be found in the :doc:`Private 5G: A Systems Approach ` book. diff --git a/testing/about_system_tests.rst b/testing/about_system_tests.rst index 40a37d4..0caa603 100644 --- a/testing/about_system_tests.rst +++ b/testing/about_system_tests.rst @@ -5,44 +5,23 @@ About ===== -The goal and objective of Aether test automation is to build a framework that -provides highly scalable and low maintenance code which will help cover various -categories of tests. Framework includes libraries and tools that allows both -component level and integration level tests. Robot Framework will be used for -covering integration tests. Component level test coverage have been -accomplished by leveraging the existing test frameworks that were developed in -their respective projects. Component level tests include tests for SD-Fabric, PDP, -SD-CORE areas. For detailed information on component tests, please see their -respective documentation pages. +The Aether Project is transitioning to a new automated testing regime, +consistent with its transition from being a managed service to being +a platform anyone can deploy. Information about the previous framework +can be found in the `Version 2.1 Guide +`__, -Test Frameworks ---------------- +The major changes include: -The following diagram outlines each Aether core component, followed by an online -of the test frameworks used: +* Adopting gNBsim (in place of NG40) as the primary test engine for SD-Core. +* Developing new integration tests for a range of configuration options. +* Dropping support for SD-Fabric tests. -.. image:: images/aether-overview-diagram.png - :width: 600 - :alt: Aether Overview Diagram +For detailed information on component-level tests, please see their +respective Guides: -.. list-table:: Test Frameworks - :widths: 5 3 - :header-rows: 1 +* :doc:`SD-Core Documentation ` +* :doc:`SD-RAN Documentation ` - * - Aether Core Component - - Test Framework - * - SD-Core - - `Robot Framework `_, `NG40/NG4T `_ - * - SD-Fabric - - `TestON `_ - * - ROC - - `Robot Framework `_, `Selenium `_ - * - SD-RAN - - `Robot Framework `_ - -Test Automation ---------------- - -`Jenkins `_ is the primary automation server that is -used to help trigger our automated tests. All Aether Jenkins jobs are -created and run on the Aether Jenkins instance. +More information about interface testing is include in the :doc:`API +Tests Section ` of this Guide. diff --git a/testing/acceptance_specification.rst b/testing/acceptance_specification.rst deleted file mode 100644 index 32617ea..0000000 --- a/testing/acceptance_specification.rst +++ /dev/null @@ -1,332 +0,0 @@ -.. - SPDX-FileCopyrightText: © 2020 Open Networking Foundation - SPDX-License-Identifier: Apache-2.0 - -Acceptance Specification -======================== - -Objectives ----------- - -The purpose of this document is to create an end-user test object list (TOL) -for Aether Connected Edge (ACE). - -This document will focus on the connectivity services end-user testing. In the -future, this document will extend to other services offered through ACE. - -The Automated continuous testing framework for the platform is out of the scope of this document. - -Integration Test (eNB-LTE Core) -------------------------------- - -Before we start to test End-to-End connectivity, we have to check the -connection (called S1-MME/S1-C interface) between eNB in an edge and MME in a -public cloud. - -In order to verify this connectivity, the following test cases should be passed. - -Note that all the following test/verification cases have some assumptions: - -1. eNB is connected to the Fabric switch; -2. eNB has correct network configurations; -3. eNB has correct ID configurations provided by the ONF PMFE team. - -IT-TOL01 Fabric Test 1: the connectivity test within the edge -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -In order to test the fabric test, please see the following steps: - - -+----------------------------------------------+------------------------------------+ -|Steps |Expected Outcome | -+----------------------------------------------+------------------------------------+ -|1. Access to the eNB* (through SSH or Web GUI)|Able to get ICMP reply(PING reply) | -| |from the GCP node. | -| | | -|2. Ping to "10.168.0.6" eNB S1-MME/S1-C |( )YES ( )NO | -| interface IP address* | | -| |Comments: | -+----------------------------------------------+------------------------------------+ - -.. note:: - it depends on the eNB device. Some eNBs have a single network interface for the management network, S1-U, and S1-C, - while other eNBs have separate interfaces. The former eNB type has a single IP address, - and the later eNB type has multiple IP addresses for each interface. - -IT-TOL02 Fabric Test 2: the connectivity test between the eNB and the public cloud -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -In order to test the fabric test, please see the following steps: - - -+----------------------------------------------+------------------------------------+ -|Steps |Expected Outcome | -+----------------------------------------------+------------------------------------+ -|1. Access to the eNB* (through SSH or Web GUI)|Able to get ICMP reply(PING reply) | -| |from the GCP node. | -| | | -|2. Ping to "10.168.0.6" |( )YES ( )NO | -| | | -| |Comments: | -+----------------------------------------------+------------------------------------+ - - - -.. note:: - it also depends on the eNB device. Some eNBs allow us to SSH into their device, other eNBs provide the PING tool through Web GUI. - Of course, some eNBs do not support both. In that case, it is okay to skip this test case. - - -IT-TOL03 SCTP Connection Verification -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -In order to verify the SCTP connection between MME and eNB, please see the following steps: - - -+----------------------------------------------+------------------------------------+ -|Steps |Expected Outcome | -+----------------------------------------------+------------------------------------+ -|1. SSH to the gateway device* for the eNB |Able to see Heart Beat | -| S1-MME/S1-C traffic |messages | -| | | -|2. Capture the traffic with Wireshark |( )YES ()NO | -| or the command: `sudo tcpdump -i any sctp` | | -| |Comments: | -+----------------------------------------------+------------------------------------+ - -Capture the traffic with Wireshark or the command: `sudo tcpdump -i any sctp` - -.. note:: - the eNB should have the gateway IP address for the S1-MME/S1-C traffic. - You can SSH there with the gateway IP address in the eNB and capture the traffic. - Normally, the gateway device can be one of those devices: (i) VPN router; (ii) Firewall device in between VPN router and the edge; - (iii) one of edge servers. - -IT-TOL04 PLMN Verification -^^^^^^^^^^^^^^^^^^^^^^^^^^ - -In order to verify the TAC number, please see the following steps: - - -Able to see the correct MCC and MNC values* - -+-------------------------------------------------+------------------------------------+ -|Steps |Expected Outcome | -+-------------------------------------------------+------------------------------------+ -|1. SSH to the gateway device for the eNB |Able to see the correct MCC and MNC | -| S1-MME/S1-C traffic |values | -| | | -|2. Start to capture the traffic with Wireshark |( )YES ()NO | -| or the command: `sudo tcpdump -i any sctp | | -| -w FileName.pcap` |Comments: | -|3. Reboot eNB | | -| | | -|4. Wait until FileName.pcap has `S1SetupRequest` | | -| S1SetupResponse messages | | -| | | -|5. Stop the packet capturing and open | | -| the FileName.pcap | | -| | | -|6. Find out the S1SetupRequest message and | | -| open the detailed packet information | | -| | | -|7. Go to "Item 2: id-SupportedTAs" section | | -| and check "MACC and "MNC" values | | -+-------------------------------------------------+------------------------------------+ - -Example (the MCC is 315 and MNC is 010) - -.. image:: images/it-tol04.png - :width: 840 - :height: 840 - :alt: Example (the MCC is 315 and MNC is 010) - -IT-TOL05 TAC Number Verification -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - - -+-------------------------------------------------+------------------------------------+ -|Steps |Expected Outcome | -+-------------------------------------------------+------------------------------------+ -|1. SSH to the gateway device for the eNB |Able to see the correct TAC number | -| S1-MME/S1-C traffic | | -| | | -|2. Start to capture the traffic with Wireshark |( )YES ()NO | -| or the command: `sudo tcpdump -i any sctp | | -| -w FileName.pcap` |Comments: | -|3. Reboot eNB | | -| | | -|4. Wait until FileName.pcap has `S1SetupRequest` | | -| S1SetupResponse messages | | -| | | -|5. Stop the packet capturing and open | | -| the FileName.pcap | | -| | | -|6. Find out the S1SetupRequest message and | | -| open the detailed packet information | | -| | | -|7. Go to "Item 0: id-SupportedTAs" section | | -| and check tAC " | | -+-------------------------------------------------+------------------------------------+ - -.. note:: - if you already captured packets in IT-TOL03, you can skip steps from 1 to 5. - Just you can check the expected outcome with the file you captured at IT-TOL03. - -Example (the TAC number is 19) - -.. image:: images/it-tol05.png - :width: 840 - :height: 840 - :alt: Example (the TAC number is 19) - -IT-TOL06 eNB Verification -^^^^^^^^^^^^^^^^^^^^^^^^^ - -In order to test the eNB, please see the following steps: - -+-------------------------------------------------+------------------------------------+ -|Steps |Expected Outcome | -+-------------------------------------------------+------------------------------------+ -|1. SSH to the gateway device for the eNB |Able to see the correct eNBID | -| S1-MME/S1-C traffic | | -| | | -|2. Start to capture the traffic with Wireshark |( )YES ()NO | -| or the command: `sudo tcpdump -i any sctp | | -| -w FileName.pcap` |Comments: | -|3. Reboot eNB | | -| | | -|4. Wait until FileName.pcap has `S1SetupRequest` | | -| S1SetupResponse messages | | -| | | -|5. Stop the packet capturing and open | | -| the FileName.pcap | | -| | | -|6. Find out the S1SetupRequest message and | | -| open the detailed packet information | | -| | | -|7. Go to "Item 0: id-Global-eNB-ID" section | | -| and check "eNB-ID: macroENB-ID" | | -+-------------------------------------------------+------------------------------------+ - -.. note:: - if you already captured packets in IT-TOL03, you can skip steps number 1 to 5. - Just you can check the expected outcome with the file you captured at IT-TOL03. - -Example (the eNB ID is 19) - -.. image:: images/it-tol06.png - :width: 840 - :height: 840 - :alt: Example (the eNB ID is 19) - -Connectivity Services ---------------------- - -Aether provides only data connectivity for end-user devices and systems. -So the voice service over LTE is not available. However, users can use -any OTT services over the Aether network for voice connectivity. - -The test specifications are only covering the data connectivity focused tests. - - -CS-TOL01 Device Attach/Connect -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -To test device can attach to Aether network - - -+----------------------------------------------+------------------------------------+ -|Steps |Expected Outcome | -+----------------------------------------------+------------------------------------+ -|1. Turn off the mobile device |Able to attach the device and | -| |connect to the internet/Aether | -|2. Turn on the mobile device |Network | -| | | -|3. Check whether the device is showing |( )YES ( )NO | -| connected on the status, depending on | | -| the device it will show "Aether" or | | -| "MCCMNC" format. | | -|4. Browse http://www.google.com/? |( )YES ( )NO | -| From the device web browser | | -| |Comments: | -+----------------------------------------------+------------------------------------+ - -CS-TOL02 Device Detach/Disconnect -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -To test device can detach/disconnected by user initiation - - -+----------------------------------------------+------------------------------------+ -|Steps |Expected Outcome | -+----------------------------------------------+------------------------------------+ -|1. Make sure the device is connected to Aether|Able to detach the device and | -| |disconnect from the internet/Aether | -|2. Deselect the network (or forget the network|Network | -| , depending on device configuration) | | -|3. Try to browse http://www.google.com/? |( )YES ( )NO | -| From your web browser | | -| |Comments: | -+----------------------------------------------+------------------------------------+ - - -CS-TOL03 Bandwidth Test - Internet -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -To test bandwidth available to a mobile device over Aether network. - -Please note the following, the bandwidth test depends on the eNB hardware, -your local breakout bandwidth, and the overall radio environment. -If you face an unexpected result, please explain it in the comment section in the outcome column. - - -+----------------------------------------------+------------------------------------+ -|Steps |Expected Outcome | -+----------------------------------------------+------------------------------------+ -|1. Open Speedtest app from your mobile device |Expected Bandwidth/Throughput | -| |observed | -| | | -|2. Run Speedtest 3 times, take the average as |( )YES ( )NO | -| the final result | | -| |Comments: | -+----------------------------------------------+------------------------------------+ - - -CS-TOL04 Bandwidth Test - Edge Application -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -To test bandwidth available to a mobile device over Aether network. - -Please note the following, the bandwidth test depends on the eNB hardware, -your local breakout bandwidth, and the overall radio environment. If you face an unexpected result, -please explain it in the comment section in the outcome column. - - -+----------------------------------------------+------------------------------------+ -|Steps |Expected Outcome | -+----------------------------------------------+------------------------------------+ -|1. Initiate FTP Download from a local server |Expected Bandwidth/Throughput | -| (same location) connected to the enterprise|observed | -| network (through local breakout) | | -| | | -|2. Download 3 times, take the average as the |( )YES ( )NO | -| final result | | -| |Comments: | -+----------------------------------------------+------------------------------------+ - - - -Monitoring Services -------------------- - -ACE uses the Grafana dashboard for monitoring services. -Each ACE will be provided with Read-Only Access to our centralized monitoring platform. - - -Application Services --------------------- - -Aether uses Rancher to onboard applications to ACE. -Each ACE host will be provided with access to rancher to onboard applications on their ACE cluster. - diff --git a/testing/aether-roc-tests.rst b/testing/aether-roc-tests.rst index 1648ecd..9344ab4 100644 --- a/testing/aether-roc-tests.rst +++ b/testing/aether-roc-tests.rst @@ -1,9 +1,9 @@ -ROC Testing +API Tests =========== -The REST API and the GUI of the Aether ROC is tested utilizing the Robot Framework. -The tests are located inside the aether-system-tests repository and they are run nightly using -a Jenkins job. +The REST API and GUI of Aether is tested with the Robot Framework. +The tests are located inside the ``aether-system-tests`` repository +and they are run nightly using a Jenkins job. Development Prerequisites ------------------------- @@ -24,7 +24,7 @@ Finally, the ROC GUI tests are running on the Firefox browser, so it is necessary to have the Firefox browser and the Firefox web driver (``geckodriver``) installed on the system in order to run these tests. -Running the ROC API tests +Running the ROC API Tests ------------------------- Follow the steps below to access the ROC API: @@ -110,7 +110,7 @@ Each test file corresponds to one of the Aether 2.0.0 models. This will generate test reports and logs in the ``results`` directory. -Running the ROC GUI tests +Running the ROC GUI Tests ------------------------- We test the ROC GUI by installing the ROC with keycloak-dev.onlab.us. diff --git a/testing/images/aether-overview-diagram.png b/testing/images/aether-overview-diagram.png deleted file mode 100644 index ae62e34..0000000 Binary files a/testing/images/aether-overview-diagram.png and /dev/null differ diff --git a/testing/images/it-tol04.png b/testing/images/it-tol04.png deleted file mode 100644 index 0401420..0000000 Binary files a/testing/images/it-tol04.png and /dev/null differ diff --git a/testing/images/it-tol05.png b/testing/images/it-tol05.png deleted file mode 100644 index a71812a..0000000 Binary files a/testing/images/it-tol05.png and /dev/null differ diff --git a/testing/images/it-tol06.png b/testing/images/it-tol06.png deleted file mode 100644 index ac92662..0000000 Binary files a/testing/images/it-tol06.png and /dev/null differ diff --git a/testing/integration_tests.rst b/testing/integration_tests.rst new file mode 100644 index 0000000..af4d35e --- /dev/null +++ b/testing/integration_tests.rst @@ -0,0 +1,57 @@ +.. + SPDX-FileCopyrightText: © 2023 Open Networking Foundation + SPDX-License-Identifier: Apache-2.0 + +Integration Tests +=================== + +A set of integration tests run daily to validate various +configurations of Aether, corresponding to the set of supported +:doc:`OnRamp Blueprints `. The tests are managed +by Jenkins, and can be monitored using the following +`Dashboard `__. +The following summarizes the current set of tests. + +Basic Functionality +---------------------- + +These tests validate the base components, configured with (``AMP``) or +without the Aether Management Plane; running on either a single server +(``QuickStart``) or two servers (``2server``); configured with the +officially released Helm Charts (``default-charts``) or the most +recently published charts (``latest-charts``); and deployed on Ubuntu +``20.04`` or ``22.04``. + +* ``AetherOnRamp_QuickStart_20.04_default-charts`` +* ``AetherOnRamp_QuickStart_22.04_default-charts`` +* ``AetherOnRamp_QuickStart_20.04_latest-charts`` +* ``AetherOnRamp_QuickStart_22.04_latest-charts`` +* ``AetherOnRamp_QuickStart_20.04_AMP`` +* ``AetherOnRamp_QuickStart_22.04_AMP`` +* ``AetherOnRamp_2servers_20.04_default-charts`` +* ``AetherOnRamp_2servers_22.04_default-charts`` + +Advanced Functionality +---------------------------- + +These tests validate blueprints that incorporate additional +functionality, including being configured with alternative RANs +(``Physical-ENB``, ``Physical-GNB``, ``SD-RAN``, ``UERANSIM``) and +with multiple UPF pods (``Multi-UPF``). + +* ``AetherOnRamp_2servers_20.04_UERANSIM`` +* ``AetherOnRamp_QuickStart_20.04_UERANSIM`` +* ``AetherOnRamp_2servers_Multi-UPF`` +* ``AetherOnRamp_QuickStart_Multi-UPF`` +* ``AetherOnRamp_QuickStart_SD-RAN`` +* ``AetherOnRamp_Physical-ENB`` +* ``AetherOnRamp_Physical-GNB`` + +Testing In-Depth +------------------------- + +Although still a work-in-progress, we also plan for additional +in-depth tests, including automated testing of the :doc:`Aether API +`. + +* ``AetherOnRamp_QuickStart_API-Test`` diff --git a/testing/system-tests.rst b/testing/system-tests.rst deleted file mode 100644 index 0c64a5d..0000000 --- a/testing/system-tests.rst +++ /dev/null @@ -1,161 +0,0 @@ -System Testing -============== - -The Aether system is tested using Robot Framework. -The tests are located inside the aether-system-tests repository and they are run nightly using a Jenkins job. - -Test Structure --------------- - -At a high level, functional tests are ran by: - -- Configuring ROC with various slices required for tests using pre-defined input data. -- ROC updates using pre-defined input data with patch updates built with the test framework. -- Control plane and data plane validated using NG40. - -5G ---- -Functional testing includes multiple slice creations, enable/disable of device groups, -QoS validations, rate limiting tests (at UE, slice, application), application filtering tests, container restart tests. - -4G ---- -Functional testing includes multiple slice creations, enable/disable of device groups, -QoS validations, rate limiting tests (at UE, slice, application), application filtering tests, container restart tests. - -Development Prerequisites -------------------------- - -Deploying each component is done with the use of Helm. -Instructions can be found on `this page `_. - -The following steps are needed to deploy the Aether system: - -- If applicable, remove ROC, CRDs, and other resources. Fetch Kubernetes config. -- Deploy SD-Core components -- Deploy SD-Fabric components -- Deploy ROC -- Wait for pods to be up -- Sync with SD-Core adapter - -Running Tests -------------- - -System tests are executed in the following Jenkins views: - -`Automated Test Jobs (Nightly) `_ - -Tests are executed in the following order after deployment: - -- Functional -- Failure/Restart -- Cleanup - -Test Setup ----------- - -.. code-block:: shell - - # Set up virtual environment - cd aether-system-tests/ - make ast-venv - source ast-venv/bin/activate; set -u; - -Running Tests (4G) ------------------- - -Functional, Failure/Restart, and Cleanup are executed similarly. -Replace `${TEST_FILE}` with the test file to execute. -This runs all tests in the suite. Add `-i ${TEST_TAG}` to robot arguments to run a specific test case. - -.. code-block:: shell - - cd ${WORKSPACE_DIR} - robot -d ${WORKSPACE_DIR}/${TEST_TYPE}/robotlogs \ - --debugfile ${WORKSPACE_DIR}/functionality/robotlogs/${TEST_FILE}_debug \ - -o ${log_xml} \ - -l ${log_html} \ - -r ${report_html} \ - -e notready \ - -v ACC_CONTEXT:${accContext} \ - -v AMP_CONTEXT:${rocContext} \ - -v ACE_CONTEXT:${aceContext} \ - -v UPF_NAMESPACE:${upfNamespace} \ - -v UPF_NAMESPACE2:${upfNamespace2} \ - -v UPF_NAMESPACE3:${upfNamespace3} \ - -v PFCP_NAMESPACE:${pfcpNamespace} \ - -v CORE_TYPE:${coreType} \ - -v UPF_TYPE:${upfType} \ - -v UPF_TYPE_NAME:${upfTypeName}\ - -v NG40_HOST:${ng40Host} \ - -v NG40_USER:${ng40User} \ - -v NG40RAN_DIR:${ng40RanDir} \ - -v NG40TEST_DIR:${ng40TestDir} \ - -v CSV_DIR:${ng40CsvDir} \ - -v PCAP_DIR:${WORKSPACE_DIR}/functionality/${pcapDir} \ - -v NG40_LOG_DIR:${WORKSPACE_DIR}/functionality/${ng40LogDir} \ - -v CDR_LOG_DIR:${WORKSPACE_DIR}/functionality/${cdrLogDir} \ - -v CONTAINER_LOG_DIR:${WORKSPACE_DIR}/functionality/${containerLogDir} \ - -v POD_NAME:${pod} \ - aether-system-tests/system-tests/tests/4G/functional/${TEST_FILE}.robot || true - -Variables ---------- - -| `${WORKSPACE_DIR}` - Current workspace directory for Aether System Tests. Used for output logs and PCAPs. -| `${TEST_TYPE}` - Directories for logs for each test type Ex: functional, failure, cleanup. -| `${accContext}` - SD-Core context. -| `${rocContext}` - ROC context. -| `${aceContext}` - SD-Fabric and UPF context. -| `${upfNamespace}` - UPF namespace used in tests. Ex: `aether-sdcore-upf1`. -| `${pfcpNamespace}` - PFCP namespace used in tests. Ex: `tost`. -| `${coreType}` - 4G or 5G. -| `${upfType}` - UPF type used in tests. Ex: `bess`. Can also be `${None}` or empty. -| `${upfTypeName}` - UPF type name used in tests. - This affects which input files are used. Ex: `p4`. - Can also be `${None}` or empty. -| `${ng40Host}` - IP address for NG40 host. -| `${ng40User}` - NG40 User. -| `${ng40RanDir}` - NG40. RAN directory. -| `${ng40CsvDir}` - NG40 CSV directory. -| `${ng40TestDir}` - NG40 tests directory. -| `${pcapDir}` - Saved PCAP directory. -| `${cdrLogDir}` - NG40 CDR directory. -| `${containerLogDir}` - Log directory for each deployed component. -| `${pod}` - Pod name used in tests. This affects which input files are used. Ex: `qa`, `qa2`. - -Running Tests (5G) ------------------- -5G tests are executed similar to 4G, except using the tests located in the 5G directory. - -.. code-block:: shell - - cd ${WORKSPACE_DIR} - robot -d ${WORKSPACE_DIR}/${TEST_TYPE}/robotlogs \ - --debugfile ${WORKSPACE_DIR}/functionality/robotlogs/${TEST_FILE}_debug \ - -o ${log_xml} \ - -l ${log_html} \ - -r ${report_html} \ - -e notready \ - -v ACC_CONTEXT:${accContext} \ - -v AMP_CONTEXT:${rocContext} \ - -v ACE_CONTEXT:${aceContext} \ - -v UPF_NAMESPACE:${upfNamespace} \ - -v UPF_NAMESPACE2:${upfNamespace2} \ - -v UPF_NAMESPACE3:${upfNamespace3} \ - -v PFCP_NAMESPACE:${pfcpNamespace} \ - -v CORE_TYPE:${coreType} \ - -v UPF_TYPE:${upfType} \ - -v UPF_TYPE_NAME:${upfTypeName}\ - -v NG40_HOST:${ng40Host} \ - -v NG40_USER:${ng40User} \ - -v NG40RAN_DIR:${ng40RanDir} \ - -v NG40TEST_DIR:${ng40TestDir} \ - -v CSV_DIR:${ng40CsvDir} \ - -v PCAP_DIR:${WORKSPACE_DIR}/functionality/${pcapDir} \ - -v NG40_LOG_DIR:${WORKSPACE_DIR}/functionality/${ng40LogDir} \ - -v CDR_LOG_DIR:${WORKSPACE_DIR}/functionality/${cdrLogDir} \ - -v CONTAINER_LOG_DIR:${WORKSPACE_DIR}/functionality/${containerLogDir} \ - -v POD_NAME:${pod} \ - aether-system-tests/system-tests/tests/5G/functional/${TEST_FILE}.robot || true -