Release 0.4.1

.gitignore

@@ -3,11 +3,13 @@
 **/target
 *.log
 out/
+pkits-testsuite/src/site/markdown/
 
 docs/configs/tuTestObjects/
 docs/dev/
 LICENSE_PLACEHOLDER
 Pipeline.CI.Jenkinsfile
 Pipeline.DeployGitHub.Jenkinsfile
 Pipeline.InternalRelease.Jenkinsfile
+Pipeline.CodeCheck.Jenkinsfile
 

LICENSE

@@ -1,4 +1,4 @@
-Copyright (c) 2022 gematik GmbH
+Copyright (c) 2023 gematik GmbH
 
 Licensed under the Apache License, Version 2.0 (the License);
 you may not use this file except in compliance with the License.

README.md

@@ -13,9 +13,10 @@ development is still ongoing [see Todo section](./README.md#todo)
 
 ```console 
 $ git clone https://github.com/gematik/app-PkiTestsuite
-$ cp <UserDefinedConfigfile>.yml pkits/config/pkits.yml (examples see: /docs/config/inttest/)
-$ ./initialTslAndVa.sh (generates VA and TSL in ./out for import in test object)
-$ ./checkInitalState.sh (acquires TSL sequence number from test object and does smoke test)
+$ cp <UserDefinedConfigfile>.yml ./config/pkits.yml (examples see: /docs/config/inttest/)
+$ ./initialTslAndTa.sh (generates VA and TSL in ./out for import in test object)
+$ # The test object has to be started and accessible from now on.
+$ ./checkInitialState.sh (acquires TSL sequence number from the test object by analysing a tsl download request and applying a use case)
 $ ./startApprovalTest.sh (chose tests that shall be executed from allTests.txt)
 $ # Testreport can be found in ./out directory.
 ```
@@ -91,30 +92,35 @@ the fly. If you use your own test data make sure that issuing certificates are a
 ### Initial TSL and Trust Anchor
 
 For the configuration of the test object it is necessary to initialize it with a trust space
-compatible to the test suite. For this, a convenient script ist provided by the test suite:
-By executing `initialTslAndVa.sh` an initial TSL and the corresponding trust anchor are written to
-the `./out` directory for import into the test object.
+compatible to the test suite. For this, a convenient script is provided by the test suite:
+By executing `initialTslAndTa.sh` an initial TSL and the corresponding trust anchor are written to
+the `./out` directory for import into the test object. This TSL contains the TU trust store as well,
+this means, that the test object can be used during the pki tests by other services as well.
 
 ## Test Execution
 
-Tests are executed via the script `./startApprovalTest.sh`. It will compile all modules and run the
-approval test classes via maven-failsafe-plugin. Furthermore,
+The test suite expects a test object that is running and accessible over the configured IP address
+and port (see [Configuration](./README.md#configuration)). Tests are executed via the
+script `./startApprovalTest.sh`. It will compile all modules
+and run the approval test classes via maven-failsafe-plugin. Furthermore,
 the [OCSP responder](./README.md#3-ocsp-responder)
-and [TSL provider](./README.md#2-tsl-provider) are started at the configured sockets. Logs are
-written to the `./out/logs` directory. Afterwards a test report is generated in
-the `./out/testreport` directory.
+and [TSL provider](./README.md#2-tsl-provider) are started at the configured sockets (if they are
+not deactivated). Logs are written to the `./out/logs` directory. Afterwards a test report is
+generated in the `./out/testreport` directory.
 
 ### Smoke Test
 
-In oder to make a quick check if everything is set up correctly, and to initialize the test suite
-with the tsl sequence number set in the test object, we implemented a script that runs
-an initial test: `./checkInitialState.sh`. Within a TSL download is checked and afterwards a use
-case is triggered with a valid certificate. OCSP requests are expected and answered correctly.
+In oder to make a quick check if everything is set up correctly, the test object can be reach by the
+test suite, and to initialize the test suite with the tsl sequence number set in the test object, we
+implemented a script that runs an initial test: `./checkInitialState.sh`. Within a TSL download by
+the test object is exacted and afterwards a use case is triggered with a valid certificate. OCSP
+requests are expected and answered correctly as well. Therefor a configured test object has to be up
+and running and accessible by the testsuite and its components.
 
 ### Selecting Specific Tests
 
 Besides executing all tests, it is possible to select or exclude specific tests for execution.
-This is done via the file `allTest.txt`.
+This is done via the file `allTests.txt`.
 The file lists test classes `CertificateApprovalTestsIT`, `OcspApprovalTestsIT`
 , `TslApprovalTestsIT`, `TslITSignerApprovalTestIT` and all tests defined in the test classes.
 Test classes as well as separate tests can be marked with `+` or `-`.

ReleaseNotes.md

@@ -2,6 +2,21 @@
 
 # Release notes PKI Test Suite
 
+## Release 0.4.1
+
+- add test cases checking number of retries for primary and backend endpoints for TSL download
+- add test cases in the context of Trust Anchor Change verification - the tests are set to run as the last
+- add test cases in the context of TSL signer certificate verification
+- add test cases in the context of TSL approval verification
+- force trust anchor change tests to run as last
+- add documentation of all AFOs and correspondings test cases
+  files: [AFOs description](./docs/afoCoverage_afoDescriptions.txt),
+  [AFOs to tests mapping](./docs/afoCoverage_afoToTests.txt),
+  [tests to AFOs mapping](./docs/afoCoverage_testToAfos.txt)
+- introduce `externalStartup` for the case when OcspResponder and TslProvider are started externally (not by the test
+  suite)
+- integrate logs and configuration into PDF report
+
 ## Release 0.3.1
 
 - add kim client module certificates

allTest.txt → allTests.txt

@@ -26,11 +26,23 @@
 	verifyOcspResponseWithNullParameterInCertId        Test OCSP response with null parameter in CertId
 
 +	de.gematik.pki.pkits.testsuite.approval.TslApprovalTestsIT
-	verifyTslDownloadCompression  Test compression of TSL download
+	verifyForBadCertificateOfTSPService                    Test bad CA certificate is not extractable from TSL
+	verifyForUnspecifiedCertificateOfTSPService            Test proper handling of unspecified CA certificate in TSL
+	verifyForUnspecifiedServiceTypeIdentifierOfTSPService  Test CA certificate with ServiceTypeIdentifier "unspecified" in TSL
+	verifyForWrongServiceInfoExtCertificateOfTSPService    Test CA certificate with missing service information extension in TSL
+	verifyIrregularDifferencesBetweenCurrentAndNewTsls     Test TSL service does not provide updated TSL
+	verifyRetryFailingTslDownload                          Test TSL download not possible
+	verifyRevokedCaCertificateInTsl                        Test CA certificate in TSL is revoked and EE certificate is issued earlier.
+	verifyRevokedCaCertificateInTslLater                   Test CA certificate in TSL is revoked and EE certificate is issued later.
+	verifyTslDownloadCompression                           Test compression of TSL download
+	verifyTslSignatureInvalid                              Test TSL signature invalid - "to be signed block" with integrity violation
+	verifyUpdateTrustStoreInTestObject                     Test update of TSL with different XML format (pretty print)
+	verifyUseBackupTslDownload                             Test TSL download on primary endpoint not possible
 
 +	de.gematik.pki.pkits.testsuite.approval.TslSignerApprovalTestsIT
 	verifyMissingOcspSignerInTslForTslSignerCert                    Test missing OCSP signer in TSL for TSL signer certificate
 	verifyOcspResponseTslSignerCertInvalidCertHash                  Test OCSP response of TSL signer certificate with invalid CertHash
+	verifyOcspResponseTslSignerCertInvalidCertId                    Test invalid cert id in OCSP response for TSL signer cert
 	verifyOcspResponseTslSignerCertMissingCertHash                  Test OCSP response of TSL signer certificate with missing CertHash
 	verifyOcspResponseTslSignerCertMissingNextUpdate                Test OCSP response of TSL signer certificate with missing nextUpdate
 	verifyOcspResponseTslSignerCertNextUpdatePastOutOfTolerance     Test OCSP response of TSL signer certificate with nextUpdate in past out of tolerance
@@ -47,3 +59,14 @@
 	verifyOcspResponseTslSignerCertVariousStatusAndResponseBytes    Test various status of OCSP responses of TSL signer certificate with and without response bytes
 	verifyOcspResponseTslSignerCertWithNullParameterInCertId        Test OCSP response of TSL signer certificate with null parameter in CertId
 	verifyOcspResponseWithInvalidSignatureForTslSignerCert          Test invalid OCSP response signature for TSL signer certificate
+	verifyTslSignerCertBroken                                       Test TSL signer certificate is broken
+	verifyTslSignerCertExpired                                      Test TSL signer certificate that is expired
+	verifyTslSignerCertInvalidKeyUsageAndExtendedKeyUsage           Test TSL signer certificates with invalid key usage and extended key usage
+	verifyTslSignerCertNotYetValid                                  Test TSL signer certificate that is not yet valid - notBefore is in the future
+
++	de.gematik.pki.pkits.testsuite.approval.TslVaApprovalTestsIT
+	verifyHandlingOfStatusStartingTimeAndOverwriteOfAnnouncedTrustAnchors  Test overwrite behaviour and proper handling of StatusStartingTime of announced trust anchors
+	verifyMultipleAnnouncedTrustAnchorsInTsl                               Test multiple announced trust anchors in single TSL
+	verifyNewTrustAnchorInvalidTime                                        Test updating trust anchor with certificates that have invalid times
+	verifyNewTrustAnchorsIsBroken                                          Test for an announced broken trust anchor and cannot be extracted
+	verifyUpdateTrustAnchor                                                Test updating trust anchor

docs/afoCoverage_afoDescriptions.txt

@@ -0,0 +1,48 @@
+A_17124       ECDSA cipher suites for TLS
+A_17700       TSL-Auswertung ServiceTypeIdentifier "unspecified"
+GS-A_4357     ECDSA algorithms - Tab_KRYPT_002a
+GS-A_4357     RSA algorithms - Tab_KRYPT_002
+GS-A_4384     RSA cipher suites for TLS
+GS-A_4642     TUC_PKI_001: Periodische Aktualisierung TI-Vertrauensraum - Schritt 2
+GS-A_4642     TUC_PKI_001: Periodische Aktualisierung TI-Vertrauensraum - Schritt 3
+GS-A_4642     TUC_PKI_001: Periodische Aktualisierung TI-Vertrauensraum - Schritt 4
+GS-A_4642     TUC_PKI_001: Periodische Aktualisierung TI-Vertrauensraum - Schritt 5
+GS-A_4642     TUC_PKI_001: Periodische Aktualisierung TI-Vertrauensraum - Schritt 6
+GS-A_4642     TUC_PKI_001: Prüfung der Aktualität der TSL - Schritt 2
+GS-A_4643     TUC_PKI_013: Import TI-Vertrauensanker aus TSL
+GS-A_4643     TUC_PKI_013: Import TI-Vertrauensanker aus TSL - Schritt 1
+GS-A_4643     TUC_PKI_013: Import TI-Vertrauensanker aus TSL - Schritt 2
+GS-A_4643     TUC_PKI_013: Import TI-Vertrauensanker aus TSL - Schritt 4
+GS-A_4643     TUC_PKI_013: Import TI-Vertrauensanker aus TSL - Schritt 6
+GS-A_4647     TUC_PKI_016: Download der TSL-Datei - Schritt 3 und 4
+GS-A_4648     Prüfung der Aktualität der TSL - Schritt 4
+GS-A_4648     TUC_PKI_019: Prüfung der Aktualität der TSL - Schritt 1
+GS-A_4648     TUC_PKI_019: Prüfung der Aktualität der TSL - Schritt 3
+GS-A_4648     TUC_PKI_019: Prüfung der Aktualität der TSL - Schritt 6
+GS-A_4649     TUC_PKI_020: XML-Dokument validieren
+GS-A_4650     TUC_PKI_011: Prüfung des TSL-Signer-Zertifikates - Schritt 2
+GS-A_4650     TUC_PKI_011: Prüfung des TSL-Signer-Zertifikates - Schritt 3
+GS-A_4652     TUC_PKI_018: Zertifikatsprüfung in der TI - Schritt 5
+GS-A_4652     TUC_PKI_018: Zertifikatsprüfung in der TI - Schritt 5a
+GS-A_4652     TUC_PKI_018: Zertifikatsprüfung in der TI - negative cases
+GS-A_4652     TUC_PKI_018: Zertifikatsprüfung in der TI - positive cases
+GS-A_4653     TUC_PKI_002: Gültigkeitsprüfung des Zertifikats
+GS-A_4657     TUC_PKI_006: OCSP check - step 4c
+GS-A_4657     TUC_PKI_006: OCSP check - step 6b
+GS-A_4657     TUC_PKI_006: OCSP-Abfrage - Schritt 1
+GS-A_4657     TUC_PKI_006: OCSP-Abfrage - Schritt 4c
+GS-A_4657     TUC_PKI_006: OCSP-Abfrage - Schritt 5a
+GS-A_4657     TUC_PKI_006: OCSP-Abfrage - Schritt 5a1
+GS-A_4657     TUC_PKI_006: OCSP-Abfrage - Schritt 6
+GS-A_4657     TUC_PKI_006: OCSP-Abfrage - Schritt 6a
+GS-A_4657     TUC_PKI_006: OCSP-Abfrage - Schritt 6b
+GS-A_4657     TUC_PKI_006: OCSP-Abfrage - Schritt 7b
+GS-A_4657     TUC_PKI_006: OCSP-Abfrage - Schritt 7c
+GS-A_4657     TUC_PKI_006: OCSP-Abfrage - Schritt 8b und 8c
+GS-A_4663     Zertifikats-Prüfparameter für den TLS-Handshake - negative cases
+GS-A_4663     Zertifikats-Prüfparameter für den TLS-Handshake - positive cases
+GS-A_4749     TUC_PKI_007: Prüfung Zertifikatstyp - Schritt 8
+GS-GS-A_4651  TUC_PKI_012: XML-Signatur-Prüfung
+RFC 5280      4.1.1.2. signatureAlgorithm
+RFC 6960      4.2.1. ASN.1 Specification of the OCSP Response
+TIP1-A_5120   Clients des TSL-Dienstes: HTTP-Komprimierung unterstützen