Add OpenSSF scorecard

[planetf1]

I suggest we generate OpenSSF Scorecards

We are offering assets in the security space. In addition to assurance of those assets in algorithmic terms & testing, there are additional criteria relating to the management of the project in github, packaging, dependencies, workflows, contributor diversity, and use of various tools.

scorecards are becoming more discussed as we all worry about supply-chain security, and some organizations are using them as criteria as to which projects can be used.

The tests can be done automatically in a github action to at least generate a local report - can consider later how to share further.

I think by doing this we add credibility (including in being more closely aligned to openssf) - even though initially we will likely fail on multiple criteria, but it gives us a best-practice list to work to.

Note - See also pq-code-package/tsc#14, but since we have code out there with open-quantum-safe, how about adding it here? If useful, we can consider extending it to other relevant repos - oqs-provider comes to mind, perhaps others/all. I'd be happy to propose a PR/take this on, and it should have very little risk/impact

[baentsch]

I'd be happy to propose a PR/take this on

Looking forward to it. We have something that sounds similar, also from IBM, called CBOM, so I'd suggest you target the same directory with your PR.

[planetf1]

Experimented (I've used it before on other projects) with this in a fork - the results aren't valid as such, but I'll create a PR

See docs at https://github.com/ossf/scorecard

Adding a default .github/workflows/scorecard.yml

# This workflow uses actions that are not certified by GitHub. They are provided
# by a third-party and are governed by separate terms of service, privacy
# policy, and support documentation.

name: Scorecard supply-chain security
on:
  # For Branch-Protection check. Only the default branch is supported. See
  # https://github.com/ossf/scorecard/blob/main/docs/checks.md#branch-protection
  branch_protection_rule:
  # To guarantee Maintained check is occasionally updated. See
  # https://github.com/ossf/scorecard/blob/main/docs/checks.md#maintained
  schedule:
    - cron: '29 9 * * 4'
  push:
    branches: [ "main" ]

# Declare default permissions as read only.
permissions: read-all

jobs:
  analysis:
    name: Scorecard analysis
    runs-on: ubuntu-latest
    permissions:
      # Needed to upload the results to code-scanning dashboard.
      security-events: write
      # Needed to publish results and get a badge (see publish_results below).
      id-token: write
      # Uncomment the permissions below if installing in a private repository.
      # contents: read
      # actions: read

    steps:
      - name: "Checkout code"
        uses: actions/checkout@93ea575cb5d8a053eaa0ac8fa3b40d7e05a33cc8 # v3.1.0
        with:
          persist-credentials: false

      - name: "Run analysis"
        uses: ossf/scorecard-action@e38b1902ae4f44df626f11ba0734b14fb91f8f86 # v2.1.2
        with:
          results_file: results.sarif
          results_format: sarif
          # (Optional) "write" PAT token. Uncomment the `repo_token` line below if:
          # - you want to enable the Branch-Protection check on a *public* repository, or
          # - you are installing Scorecard on a *private* repository
          # To create the PAT, follow the steps in https://github.com/ossf/scorecard-action#authentication-with-pat.
          # repo_token: ${{ secrets.SCORECARD_TOKEN }}

          # Public repositories:
          #   - Publish results to OpenSSF REST API for easy access by consumers
          #   - Allows the repository to include the Scorecard badge.
          #   - See https://github.com/ossf/scorecard-action#publishing-results.
          # For private repositories:
          #   - `publish_results` will always be set to `false`, regardless
          #     of the value entered here.
          publish_results: true

      # Upload the results as artifacts (optional). Commenting out will disable uploads of run results in SARIF
      # format to the repository Actions tab.
      - name: "Upload artifact"
        uses: actions/upload-artifact@3cea5372237819ed00197afe530f5a7ea3e805c8 # v3.1.0
        with:
          name: SARIF file
          path: results.sarif
          retention-days: 5

      # Upload the results to GitHub's code scanning dashboard.
      - name: "Upload to code-scanning"
        uses: github/codeql-action/upload-sarif@17573ee1cc1b9d061760f3a006fc4aac4f944fd5 # v2.2.4
        with:
          sarif_file: results.sarif

We end up with an action that should publish it's findings to the 'security' tab

For example my fork reports the following. Note that quite a few of these findings are DUE to running it on a fork (ie without branch protections etc) so not representative of what we'll find:

Some are relevant, and we'll see actions recommended ie:

Analysis, and remediation/filtering of the findings is a second pass.

Note that to view security->code scanning alerts requires 'write' access on the repository - I'd be happy to help handle the results, but would need access.

[baentsch]

Thanks for the initial information.

I'd be happy to help handle the results, but would need access.

Can you please explain why the results must be posted with write permissions? Wouldn't it be sensible --particularly at the start-- to have possible findings reported in the CI run logs only & then work through them and resolve them (before posting initial, un-vetted results causing unnecessary "alarm-ism")? Next would be adding them to the weekly CT tests and when we're sure this delivers reliable and actionable results we should add to repo-writing automatic public reporting... Plausible chain of events?

Write access to the repo will follow automatically when you have submitted sufficient numbers of accepted PRs.

[planetf1]

Thanks for the comments in PR #27 - I've rebased, to refresh the results. Currently these are only found in the sarif file attached to the action.

The scan is currently reporting

• We are using Github actions that are not 'pinned' by version, ie by specifying the hash. This is regarded as unsafe as it prevents reproducibility. More details. My proposal is to identify the current versions in use, and pin at these versions. Making this change should be safe. but will require us to periodically review the versions in use and update as needed

• Similar to the above, in one one (unix.yaml in upstreamcheck, where we do a pip install from the requirements file - It does specify versions, but for the above reason the check suggests hashes to improve supply chain security

• each github action also needs to specify permissions (see here ). I can take a pass at this, there is some risk of making an error (albeit easily backed-off) if any permissions added are too constrained. Hopefully we can mitigate this through careful review

Would you be ok with this direction @baentsch ? I would be inclined to do the pinning in one PR, and the permissions change in a different one just to separate the concerns.

In terms of the openssf PR itself, this could be rerun in draft once the above are merged.

[baentsch]

Would you be ok with this direction @baentsch ? I would be inclined to do the pinning in one PR, and the permissions change in a different one just to separate the concerns.

In terms of the openssf PR itself, this could be rerun in draft once the above are merged.

All sounds very plausible, @planetf1. Thanks for the explanations & looking forward to the PRs.

[planetf1]

I've updated the PR, and also included the changes for pinning versions within the PR (so that I can build together)

1. github actions

I used https://github.com/mheap/pin-github-action to update the github workflows to include the SHA.

ie

npm install -g pin-github-action  

This can then be used to pin the SHAs using

pin-github-action .github/workflows/android.yml 

or a wildcard can be used for the files.

This of course should be a development activity rather than CI, since the intent here is to ensure that it is under control of the dev process.

2. pip requirements

For our 'copy_from_upstream' script we use python. A requirements.txt file defines dependencies, and has the same issue reported by ossf scorecard - we need to pin

There's a useful package at https://pypi.org/project/hashin/ which can be used to help this migration

cat requirements.txt | while read line
do
hashin $line
done

This resulted in the somewhat verbose requirements.txt with full hash specs.

Both of these tools are MIT licensed, and I'm thinking it's worth adding pointers/docs somewhere in the liboqs docs? This would apply to all the repos - any tips on where ?