Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(test): Test_Run (aws) fluky test #4874

Closed
nikpivkin opened this issue Jul 27, 2023 · 1 comment · Fixed by #4884
Closed

fix(test): Test_Run (aws) fluky test #4874

nikpivkin opened this issue Jul 27, 2023 · 1 comment · Fixed by #4884
Assignees
Labels
kind/testing Categorizes issue or PR as related to a unit/integration test. scan/misconfiguration Issues relating to misconfiguration scanning
Milestone

Comments

@nikpivkin
Copy link
Contributor

Part of the Test_Run tests are falling.

--- FAIL: Test_Run (83.11s)
    --- FAIL: Test_Run/fail_without_region (31.95s)
    --- FAIL: Test_Run/fail_without_creds (46.19s)
    --- FAIL: Test_Run/try_to_call_aws_if_cache_is_expired (4.90s)
    --- PASS: Test_Run/succeed_with_cached_infra (0.01s)
    --- PASS: Test_Run/custom_rego_rule_with_passed_results (0.02s)
    --- PASS: Test_Run/compliance_report_summary (0.01s)
    --- PASS: Test_Run/scan_an_unsupported_service (0.00s)
    --- PASS: Test_Run/scan_every_service (0.02s)
    --- PASS: Test_Run/skip_certain_services_and_include_specific_services (0.01s)
    --- PASS: Test_Run/only_skip_certain_services_but_scan_the_rest (0.01s)
    --- PASS: Test_Run/fail_-_service_specified_to_both_include_and_exclude (0.00s)
FAIL
        github.com/aquasecurity/trivy/pkg/cloud/aws/commands    coverage: 85.6% of statements
FAIL    github.com/aquasecurity/trivy/pkg/cloud/aws/commands    84.975s

I found that this happened after configuring the default aws profile (aws configure command).

cat ~/.aws/config
[default]
aws_access_key_id=***
aws_secret_access_key=***
region = eu-central-1

If the region or account is not passed, then Trivy loads the default config for aws.

My suggestion: set the AWS_PROFILE environment variable in the test or add a new profile flag (similar to the aws cli) to use the passed profile.

@nikpivkin nikpivkin added the kind/testing Categorizes issue or PR as related to a unit/integration test. label Jul 27, 2023
@simar7 simar7 changed the title Test_Run (aws) fluky test fix(test): Test_Run (aws) fluky test Jul 27, 2023
@simar7
Copy link
Member

simar7 commented Jul 27, 2023

If these tests are picking up external configuration then they are not unit tests at that point. We need to make sure that they don't do so by either testing against localstack or by passing in mocks where needed.

@knqyf263 knqyf263 added this to the v0.44.1 milestone Aug 2, 2023
@simar7 simar7 added the scan/misconfiguration Issues relating to misconfiguration scanning label Aug 10, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/testing Categorizes issue or PR as related to a unit/integration test. scan/misconfiguration Issues relating to misconfiguration scanning
Projects
Archived in project
Development

Successfully merging a pull request may close this issue.

3 participants