[FEATURE] Provide better error information when something fails

[jmservera]

Is your feature request related to a problem? Please describe.

I started using crda in a project and I love what I can get from it, but I'm getting random errors that I'm unable to debug because they are quite hard to reproduce (they appear randomly without a clear cause); furthermore, the error messages are not really clear about what's happening.

Describe the solution you'd like

I would like to have better error messages to know when the action is failing and why.

Describe alternatives you've considered

Are there any throttling limits for the scans that we should know?

Additional context

I provide here some examples, this is my workflow:

name: Scan Java project
on:
  push:
  
jobs:
  test:
    strategy:
      fail-fast: false
      matrix:
        os: [ "macos-latest", "ubuntu-20.04" ] # removed (, "windows-latest") because got a lot of timeout failures
        # Test once with explicit manifest_file, and once without
        manifest_file: [ "pom.xml", "" ]
    runs-on: ${{ matrix.os }}
    permissions: write-all
      # todo: find permissions
    defaults:
      run:
        shell: bash
    name: Analyse Java project
    steps:
      - name: Checkout action
        uses: actions/checkout@v3

      - name: Setup Java
        uses: actions/setup-java@v2
        with:
          distribution: temurin
          java-version: 11
          cache: maven

      - name: Install CRDA
        uses: redhat-actions/[email protected]
        with:
          source: github
          github_pat: ${{ github.token }}
          crda: "latest"

      - name: CRDA Scan
        id: crda_scan
        uses: redhat-actions/[email protected]
        with:
          manifest_file: ${{ matrix.manifest_file }}
          snyk_token: ${{ secrets.SNYK_KEY }}
          fail_on: never

      - name: Print JSON analysis report
        if: ${{ always() && steps.crda_scan.outputs.crda_report_json }}
        run: cat ${{ steps.crda_scan.outputs.crda_report_json }}

      - name: Print SARIF analysis report
        if: ${{ always() && steps.crda_scan.outputs.crda_report_sarif }}
        run: cat ${{ steps.crda_scan.outputs.crda_report_sarif }}

      - name: Print Report Link
        run: echo ${{ steps.crda_scan.outputs.report_link }}

      - name: Print Artifact Name
        run: echo ${{ steps.crda_scan.outputs.artifact_name }}

And sometimes all the runs fail, other times a few complete and others fail, this one failed only in the macos case without the pom.xml specified, complaining about an invalid '<' character, but the same action ran correctly only repeating the run (Re-run failed jobs):

Run redhat-actions/[email protected]
Working directory is /Users/runner/work/DevSecOpsWorkshop-jm/DevSecOpsWorkshop-jm
crda version
/usr/local/bin/git rev-parse HEAD
e0[19](https://[redacted]/DevSecOpsWorkshop-jm/actions/runs/3330075886/jobs/5508676708#step:5:21)9b383358e6ca316cd737d56931074f0f686e
Ref to analyze is "refs/heads/master"
Commit to analyze is "e0199b383358e6ca316cd737d56931074f0f686e"
"manifest_directory" not provided. Using working directory "/Users/runner/work/DevSecOpsWorkshop-jm/DevSecOpsWorkshop-jm"
"manifest_file" input not provided. Auto-detecting manifest file
🔍 Looking for manifest in "/Users/runner/work/DevSecOpsWorkshop-jm/DevSecOpsWorkshop-jm"
Manifest file is pom.xml
Dependencies will be installed using Maven strategy
⬇️ Installing dependencies...
mvn -ntp -B package
✅ Finished installing dependencies
🖊️ Setting consent_telemetry to false.
/Users/runner/work/_temp/openshift-bin/crda config set consent_telemetry false
successfully set configuration value
🔐 Authenticating with the provided Snyk Token.
/Users/runner/work/_temp/openshift-bin/crda auth --snyk-token=***
*** Suppressing command output
Successfully Registered. 

crda_key:***

This key is confidential, Please keep it safe!

✅ Successfully authenticated with the provided Snyk Token.
/Users/runner/work/_temp/openshift-bin/crda analyse pom.xml --verbose --client=gh-actions
Analysing your Dependency Stack! Please wait...
invalid character '<' looking for beginning of value
Error: crda exited with code 1
invalid character '<' looking for beginning of value

In another run I got this one (just the error msg), I got a timeout but I don't know for which server or where is it connecting:

[...]
Adding labels "CRDA Scan Failed" to pull request
Error: crda exited with code 1
2:09PM ERR Stack Analyses Get Request Failed with status code 408.  Please retry after sometime. If issue persists, Please raise at https://github.com/fabric8-analytics/cli-tools/issues."
message from server: Stack analysis request e08dce85b9784c6db2ced5e8ebd8d622 has timed out. Please retry with a new analysis.
[...]

This one looks that it couldn't authenticate for some reason, but got an exit code 1 only:

[...]
🔐 Authenticating with the provided Snyk Token.
/home/runner/work/_temp/openshift-bin/crda auth --snyk-token=***
*** Suppressing command output
Adding labels "CRDA Scan Failed" to pull request
Error: crda exited with code 1
Usage:
  crda auth [flags]

Flags:
  -h, --help                help for auth
  -t, --snyk-token string   Authenticate with Snyk Token to unlock Verbose stack analyses.

Global Flags:
  -m, --client string   Client [tekton/jenkins/gh-actions] (default "terminal")
  -d, --debug           Sets Log level to Debug.
  -c, --no-color        Toggle colors in output.



/usr/bin/git checkout master
Switched to branch 'master'
Your branch is up to date with 'origin/master'.
/usr/bin/git remote remove remote-8
/usr/bin/git branch -D pr-8
Deleted branch pr-8 (was a9989a3).

Thanks for the great work!

[jmservera]

I've dag a bit deeper and it seems that the openshift.io service is not very stable (at least today), and it will probably be the cause of all these random errors. Would it be possible to better indicate that there was a connection error instead of getting all these strange errors?
Thx