Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test: reduce parameter selection #5161

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

johubertj
Copy link
Contributor

@johubertj johubertj commented Mar 4, 2025

Description of changes:

  1. Keep ALL_TEST_CERTS as is
  2. only test_happy_path.py will use ALL_TEST_CERTS (preserve coverage)
  3. all other tests will use MINIMAL_TEST_CERTS
  4. add ECDSA_521 to MINIMAL_TEST_CERTS to ensure coverage over all key types.

Problem:

This PR reduces the number of certificates used in integration tests, limiting them to only the minimal required set, except for test_happy_path.py, which retains all certs for comprehensive coverage. This change optimizes test execution time.

After making the above changes, tests take 9 seconds to complete!

   9781 test_buffered_send.py
   2588 test_signature_algorithms.py
   2000 test_happy_path.py
   1999 test_external_psk.py
   1724 test_session_resumption.py
   1682 test_client_authentication.py
   1525 test_version_negotiation.py
   1275 test_renegotiate.py
   1275 test_npn.py
   1144 test_early_data.py
    864 test_cross_compatibility.py
    576 test_ocsp.py
    300 test_dynamic_record_sizes.py
    271 test_record_padding.py
    127 test_hello_retry_requests.py
     88 test_sni_match.py
     75 test_sslyze.py
     72 test_fragmentation.py
     28 test_renegotiate_apache.py
     15 test_pq_handshake.py
      8 test_serialization.py
      7 test_key_update.py
      1 test_well_known_endpoints.py
      1 test_sslv2_client_hello.py
      1 27426/240090 tests collected (212664 deselected) in 9.76s

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

@github-actions github-actions bot added the s2n-core team label Mar 4, 2025
@johubertj johubertj requested review from jmayclin and dougch March 4, 2025 23:21
@johubertj johubertj marked this pull request as ready for review March 4, 2025 23:33
Copy link
Contributor

@dougch dougch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Neat, spot checked one of the aarch64 jobs, run time reduced from 35min to 19min.

@jmayclin
Copy link
Contributor

jmayclin commented Mar 5, 2025

Could you include the diff in the collected tests in the PR description? before/after. test_happy_path.py seems oddly low ...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants