Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Text File Busy #441

Open
EricAverittTMNA opened this issue Dec 19, 2024 · 3 comments
Open

Text File Busy #441

EricAverittTMNA opened this issue Dec 19, 2024 · 3 comments

Comments

@EricAverittTMNA
Copy link

In my current setup, I have Trivy scanning Terraform repos and checking for any vulnerabilities. There are a lot of terraform repos and to make things common, I am using a reusable workflow that will scan a terraform repo via the Trivy Action. During the scan, there are different environments that have different configurations so the scan results will differ based on the terraform but around 20+ environments are scanned at the same time.

This worked without any issues in Trivy Action 0.24.0 however after updating recently to 0.28.0 I am intermittently receiving an error that says "Text file busy". Simply rerunning the failed jobs causes the GitHub CI to succeed however I have to do this almost every time and the environment that fails is different almost every time.

Something has changed with the entrypoint.sh file such that my jobs continue to fail periodically. There has not been any update to the entrypoint.sh file since 0.28.0 so I have not updated to 0.29.0 just yet. Can anyone assist with this issue?

CI Preview Setup with failed jobs:
image

GitHub Failed Job Step Description: (Sensitive information redacted)

Run aquasecurity/[email protected]
Run aquasecurity/[email protected]
Run echo "dir=$HOME/.local/bin/trivy-bin" >> $GITHUB_OUTPUT
Run actions/cache@v4
Cache Size: ~36 MB (37525339 B)
/usr/bin/tar -xf /mnt/vol1/actions-runner01/_work/_temp/8b[24](https://github.com/org/repo/actions/runs/12403531563/job/34627178933?pr=745#step:13:25)82cc-ca20-40e8-b248-861cf35c7862/cache.tzst -P -C /mnt/vol1/actions-runner01/_work/reponame/reponame --use-compress-program unzstd
Cache restored successfully
Cache restored from key: trivy-binary-v0.56.1-Linux-X[64](https://github.com/org/reponame/actions/runs/12403531563/job/34627178933?pr=745#step:13:68)
Run echo /home/ec2-user/.local/bin/trivy-bin >> $GITHUB_PATH
Run echo "date=$(date +'%Y-%m-%d')" >> $GITHUB_OUTPUT
Run actions/cache@v4
Cache Size: ~0 MB (177764 B)
/usr/bin/tar -xf /mnt/vol1/actions-runner01/_work/_temp/918fe2bc-a25e-4124-9105-9bd050fa0dff/cache.tzst -P -C /mnt/vol1/actions-runner01/_work/reponame/reponame --use-compress-program unzstd
Cache restored successfully
Cache restored from key: cache-trivy-2024-12-18
Run echo "$GITHUB_ACTION_PATH" >> $GITHUB_PATH
Run # Note: There is currently no way to distinguish between undefined variables and empty strings in GitHub Actions.
Run entrypoint.sh
  entrypoint.sh
  shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
  env:
    AWS_REGION: us-east-1
    TARGET_ASSUME_ROLE_NAME: #####
    AWS_DEFAULT_REGION: us-east-1
    AWS_ACCESS_KEY_ID: ***
    AWS_SECRET_ACCESS_KEY: ***
    AWS_SESSION_TOKEN: ***
    TERRAFORM_CLI_PATH: /mnt/vol1/actions-runner01/_work/_temp/5f930761-3e63-4d16-880b-a67c6451497a
    TRIVY_EXIT_CODE: 0
    TRIVY_IGNORE_UNFIXED: true
    TRIVY_SEVERITY: CRITICAL,HIGH,MEDIUM
    TRIVY_OUTPUT: trivy.txt
    TRIVY_QUIET: true
    INPUT_SCAN_TYPE: config
    INPUT_IMAGE_REF: 
    INPUT_SCAN_REF: env/cm/ty/na
    INPUT_TRIVYIGNORES: 
    INPUT_GITHUB_PAT: 
    INPUT_LIMIT_SEVERITIES_FOR_SARIF: 
    TRIVY_CACHE_DIR: /mnt/vol1/actions-runner01/_work/reponame/reponame/.cache/trivy
Running Trivy with options: trivy config env/cm/ty/na
/mnt/vol1/actions-runner01/_work/_actions/aquasecurity/trivy-action/0.28.0/entrypoint.sh: line 44: /home/ec2-user/.local/bin/trivy-bin/trivy: Text file busy
Error: Process completed with exit code 126.

The reusable workflow that contains the trivy action code:

      # Check With Trivy
      - name: Aqua Security Trivy
        uses: aquasecurity/[email protected]
        with:
          scan-type: "config"
          hide-progress: true
          ignore-unfixed: true
          exit-code: "0"
          severity: "CRITICAL,HIGH,MEDIUM"
          output: trivy.txt
          scan-ref: ${{ matrix.target_directory }}

      - name: Publish Trivy Output to Summary
        if: ${{ always() }}
        run: |
          if [[ -s trivy.txt ]]; then
            {
              echo "###  ${{ matrix.target_directory}} Security Output"
              echo "<details><summary>Click to expand</summary>"
              echo ""
              echo '```terraform'
              cat trivy.txt
              echo '```'
              echo "</details>"
            } >> $GITHUB_STEP_SUMMARY
          fi
@EricAverittTMNA
Copy link
Author

As an update, I have also tried to do random file names instead of using trivy.txt but still have received the same message about text file being busy.

      - name: Setup Unique Trivy File Name
        run: |
          echo TRIVY_FILE_NAME=$(cat /dev/urandom | tr -cd 'a-f0-9' | head -c 32).txt >> $GITHUB_ENV

      # Check With Trivy
      - name: Aqua Security Trivy
        uses: aquasecurity/[email protected]
        with:
          scan-type: "config"
          hide-progress: true
          ignore-unfixed: true
          exit-code: "0"
          severity: "CRITICAL,HIGH,MEDIUM"
          output: ${{ env.TRIVY_FILE_NAME }}
          scan-ref: ${{ matrix.target_directory }}

      - name: Publish Trivy Output to Summary
        if: ${{ always() }}
        run: |
          if [[ -s ${{ env.TRIVY_FILE_NAME }} ]]; then
            {
              echo "###  ${{ matrix.target_directory}} Security Output"
              echo "<details><summary>Click to expand</summary>"
              echo ""
              echo '```terraform'
              cat ${{ env.TRIVY_FILE_NAME }}
              echo '```'
              echo "</details>"
            } >> $GITHUB_STEP_SUMMARY
          fi

These jobs run on self-hosted runners (AWS EC!2 instances). Could the reason for the failure being too many parallel jobs are trying to use trivy from the same location /home/ec2-user/.local/bin/trivy-bin/trivy at the same time?

@EricAverittTMNA
Copy link
Author

Another update is I downgraded to [email protected] and had no issues with my CI run.
With [email protected], it fails every time with a random environment without fail.
Testing the upgrade to [email protected], failed with the same text file busy error.

Since the change in this PR for entrypoint.sh my CI runs will fail at least once every time: #399

Would appreciate some support regarding this issue as I don't believe it's due to any setup/configuration issues on my end but rather something that isn't working properly in entrypoint.sh.

@lbvffvbl
Copy link

Hi @EricAverittTMNA Try to add env like TRIVY_CACHE_DIR: /tmp/trivy-cache-${{ github.run_id }}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants