[NFC] Spurious linting fixes and checks (#104)

* Add lint to detect trailing whitespace. Also detect trailing empty lines at end of file. Signed-off-by: Mihai Maruseac <[email protected]> * Remove trailing whitespace, white lines. Signed-off-by: Mihai Maruseac <[email protected]> * Don't run CI twice. Having triggers for both push and PR without additional constraints causes some CI to run twice. Fix it so it only runs when PRs get created or updated. Signed-off-by: Mihai Maruseac <[email protected]> * Ensure only Windows files are CRLF ended. Add `.gitattributes` to force all files be LF. Will do another commit to fix those files that are already using the wrong line endings. Signed-off-by: Mihai Maruseac <[email protected]> * Convert files to LF line endings. Signed-off-by: Mihai Maruseac <[email protected]> --------- Signed-off-by: Mihai Maruseac <[email protected]>
sigstore · Jan 12, 2024 · cc76d0a · cc76d0a
1 parent aa94e08
commit cc76d0a
Show file tree

Hide file tree

Showing 12 changed files with 197 additions and 161 deletions.
diff --git a/.gitattributes b/.gitattributes
@@ -0,0 +1 @@
+* text=auto
diff --git a/.github/workflows/codeql.yml b/.github/workflows/codeql.yml
@@ -65,4 +65,3 @@ jobs:
       uses: github/codeql-action/analyze@f9a7c6738f28efb36e31d49c53a201a9c5d6a476 # v2.14.2
       with:
         category: "/language:${{matrix.language}}"
-
diff --git a/.github/workflows/dependency_review.yml b/.github/workflows/dependency_review.yml
@@ -13,7 +13,10 @@
 # limitations under the License.
 
 name: 'Dependency Review'
-on: [pull_request]
+on:
+  pull_request:
+    branches: [main]
+    types: [opened, synchronize]
 
 permissions:
   contents: read

diff --git a/.github/workflows/lint.yml b/.github/workflows/lint.yml
@@ -1,6 +1,9 @@
 name: Lint
 
-on: [push, pull_request]
+on:
+  pull_request:
+    branches: [main]
+    types: [opened, synchronize]
 
 permissions: read-all
 
@@ -17,3 +20,27 @@ jobs:
           python-version: "3.11"
       - name: flake8 Lint
         uses: py-actions/flake8@84ec6726560b6d5bd68f2a5bed83d62b52bb50ba # v2.3.0
+      - name: Detect empty lines at end of file and trailing whitespace
+        run: |
+          set -euxo pipefail  # No -x here!
+          failed=0
+          # First, check for empty files at end
+          for file in $(git ls-files --eol | grep 'i/[cr]*lf' | awk '{print $4}'); do
+            lines=$(tac "$file" | awk 'NF{exit};END{print NR?NR-1:0}')
+            if [[ $lines -ne 0 ]]; then
+              line=$(wc -l "$file" | cut -d' ' -f1)
+              echo "::error file=$file,line=$line::File $file has $lines empty lines at end. Please remove."
+              failed=$((failed + 1))
+            fi
+          done
+          # Next, check for files with whitespace at end of line. Remove CRLF files.
+          for file in $(git ls-files --eol | grep 'i/lf' | awk '{print $4}'); do
+            for line in $(grep -n '[[:space:]]$' "$file" | cut -d: -f1); do
+              echo "::error file=$file,line=$line::File $file has trailing whitespace at line $line. Please remove."
+              failed=$((failed + 1))
+            done
+          done
+          if [[ $failed -ne 0 ]]; then
+            echo "::error Found $failed whitespace errors, failing"
+            exit 1
+          fi
diff --git a/.github/workflows/pin_deps.yml b/.github/workflows/pin_deps.yml
@@ -1,98 +1,98 @@
-name: Pin dependencies
-on:
-  workflow_dispatch:
-  schedule:
-    - cron: '0 0 * * TUE' # run every Tuesday at midnight
-
-permissions: {}
-
-defaults:
-  run:
-    shell: bash
-
-jobs:
-  pin:
-    name: Generate dependency lock
-    runs-on: ${{ matrix.os }}
-    strategy:
-      fail-fast: false # Don't cancel other jobs if one fails
-      matrix:
-        os: [ubuntu-latest, macos-latest, windows-latest]
-    steps:
-    - uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 # v4.1.1
-    - uses: actions/setup-python@0a5c61591373683505ea898e09a3ea4f39ef2b9c # v5.0.0
-      with:
-        python-version: 3.11
-        cache: pip
-        cache-dependency-path: |
-          model_signing/install/requirements_${{ runner.os }}.txt
-          model_signing/install/requirements_test_${{ runner.os }}.txt
-          slsa_for_models/install/requirements_${{ runner.os }}.txt
-    - name: Create an empty virtualenv and install `pip-tools`
-      run: |
-        set -exuo pipefail
-        python -m venv venv
-        .github/workflows/scripts/venv_activate.sh
-        pip install pip-tools
-        pip list  # For debugging
-    - name: Use `pip-compile` to generate all freeze files
-      run: |
-        set -exuo pipefail
-        .github/workflows/scripts/venv_activate.sh
-        pip-compile --upgrade --generate-hashes --strip-extras --output-file=model_signing/install/requirements_${{ runner.os }}.txt model_signing/install/requirements.in
-        pip-compile --upgrade --generate-hashes --strip-extras --output-file=model_signing/install/requirements_test_${{ runner.os }}.txt model_signing/install/requirements_test.in
-        pip-compile --upgrade --generate-hashes --strip-extras --output-file=slsa_for_models/install/requirements_${{ runner.os }}.txt slsa_for_models/install/requirements.in
-    - name: Test freeze file (for model signing)
-      run: |
-        set -exuo pipefail
-        rm -rf venv  # Need clean sandbox
-        python -m venv venv
-        .github/workflows/scripts/venv_activate.sh
-        pip install -r model_signing/install/requirements_${{ runner.os }}.txt
-        pip list  # For debugging
-    - name: Test freeze file (for testing model signing)
-      run: |
-        set -exuo pipefail
-        rm -rf venv  # Need clean sandbox
-        python -m venv venv
-        .github/workflows/scripts/venv_activate.sh
-        pip install -r model_signing/install/requirements_test_${{ runner.os }}.txt
-        pip list  # For debugging
-    - name: Test freeze file (for SLSA for models)
-      run: |
-        set -exuo pipefail
-        rm -rf venv  # Need clean sandbox
-        python -m venv venv
-        .github/workflows/scripts/venv_activate.sh
-        pip install -r slsa_for_models/install/requirements_${{ runner.os }}.txt
-        pip list  # For debugging
-    - name: Upload freeze files
-      uses: actions/upload-artifact@c7d193f32edcb7bfad88892161225aeda64e9392 # v4.0.0
-      with:
-        name: freeze-files-${{ matrix.os }}
-        path: ./*/install/requirements*${{ runner.os }}*txt
-
-  # Separate PR creation job to make sure it creates only one single PR with
-  # all changed files, eliminate race-conditions and restrict permissions only
-  # to this specific job.
-  create-pr:
-    needs: [pin]
-    runs-on: ubuntu-latest
-    permissions:
-      contents: write
-      pull-requests: write
-    steps:
-    - uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 # v4.1.1
-    - uses: actions/download-artifact@6b208ae046db98c579e8a3aa621ab581ff575935 # v4.1.1
-      with:
-        path: .
-        merge-multiple: true
-    - name: Create dependent PR with dependency changes
-      uses: peter-evans/create-pull-request@153407881ec5c347639a548ade7d8ad1d6740e38 # v5.0.2
-      with:
-        title: "Update frozen python dependencies"
-        commit-message: "Bump frozen dependencies"
-        committer: "Mihai Maruseac (automated) <[email protected]>"
-        author: "Mihai Maruseac (automated) <[email protected]>"
-        signoff: true
-        delete-branch: true
+name: Pin dependencies
+on:
+  workflow_dispatch:
+  schedule:
+    - cron: '0 0 * * TUE' # run every Tuesday at midnight
+
+permissions: {}
+
+defaults:
+  run:
+    shell: bash
+
+jobs:
+  pin:
+    name: Generate dependency lock
+    runs-on: ${{ matrix.os }}
+    strategy:
+      fail-fast: false # Don't cancel other jobs if one fails
+      matrix:
+        os: [ubuntu-latest, macos-latest, windows-latest]
+    steps:
+    - uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 # v4.1.1
+    - uses: actions/setup-python@0a5c61591373683505ea898e09a3ea4f39ef2b9c # v5.0.0
+      with:
+        python-version: 3.11
+        cache: pip
+        cache-dependency-path: |
+          model_signing/install/requirements_${{ runner.os }}.txt
+          model_signing/install/requirements_test_${{ runner.os }}.txt
+          slsa_for_models/install/requirements_${{ runner.os }}.txt
+    - name: Create an empty virtualenv and install `pip-tools`
+      run: |
+        set -exuo pipefail
+        python -m venv venv
+        .github/workflows/scripts/venv_activate.sh
+        pip install pip-tools
+        pip list  # For debugging
+    - name: Use `pip-compile` to generate all freeze files
+      run: |
+        set -exuo pipefail
+        .github/workflows/scripts/venv_activate.sh
+        pip-compile --upgrade --generate-hashes --strip-extras --output-file=model_signing/install/requirements_${{ runner.os }}.txt model_signing/install/requirements.in
+        pip-compile --upgrade --generate-hashes --strip-extras --output-file=model_signing/install/requirements_test_${{ runner.os }}.txt model_signing/install/requirements_test.in
+        pip-compile --upgrade --generate-hashes --strip-extras --output-file=slsa_for_models/install/requirements_${{ runner.os }}.txt slsa_for_models/install/requirements.in
+    - name: Test freeze file (for model signing)
+      run: |
+        set -exuo pipefail
+        rm -rf venv  # Need clean sandbox
+        python -m venv venv
+        .github/workflows/scripts/venv_activate.sh
+        pip install -r model_signing/install/requirements_${{ runner.os }}.txt
+        pip list  # For debugging
+    - name: Test freeze file (for testing model signing)
+      run: |
+        set -exuo pipefail
+        rm -rf venv  # Need clean sandbox
+        python -m venv venv
+        .github/workflows/scripts/venv_activate.sh
+        pip install -r model_signing/install/requirements_test_${{ runner.os }}.txt
+        pip list  # For debugging
+    - name: Test freeze file (for SLSA for models)
+      run: |
+        set -exuo pipefail
+        rm -rf venv  # Need clean sandbox
+        python -m venv venv
+        .github/workflows/scripts/venv_activate.sh
+        pip install -r slsa_for_models/install/requirements_${{ runner.os }}.txt
+        pip list  # For debugging
+    - name: Upload freeze files
+      uses: actions/upload-artifact@c7d193f32edcb7bfad88892161225aeda64e9392 # v4.0.0
+      with:
+        name: freeze-files-${{ matrix.os }}
+        path: ./*/install/requirements*${{ runner.os }}*txt
+
+  # Separate PR creation job to make sure it creates only one single PR with
+  # all changed files, eliminate race-conditions and restrict permissions only
+  # to this specific job.
+  create-pr:
+    needs: [pin]
+    runs-on: ubuntu-latest
+    permissions:
+      contents: write
+      pull-requests: write
+    steps:
+    - uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 # v4.1.1
+    - uses: actions/download-artifact@6b208ae046db98c579e8a3aa621ab581ff575935 # v4.1.1
+      with:
+        path: .
+        merge-multiple: true
+    - name: Create dependent PR with dependency changes
+      uses: peter-evans/create-pull-request@153407881ec5c347639a548ade7d8ad1d6740e38 # v5.0.2
+      with:
+        title: "Update frozen python dependencies"
+        commit-message: "Bump frozen dependencies"
+        committer: "Mihai Maruseac (automated) <[email protected]>"
+        author: "Mihai Maruseac (automated) <[email protected]>"
+        signoff: true
+        delete-branch: true
diff --git a/.github/workflows/scorecard.yml b/.github/workflows/scorecard.yml
@@ -70,4 +70,3 @@ jobs:
         uses: github/codeql-action/upload-sarif@17573ee1cc1b9d061760f3a006fc4aac4f944fd5 # v2.2.4
         with:
           sarif_file: results.sarif
-
diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
@@ -30,4 +30,3 @@ use GitHub pull requests for this purpose.
 Contributions made by corporations are covered by a different agreement than the
 one above, the [Software Grant and Corporate Contributor License
 Agreement](https://cla.developers.google.com/about/google-corporate).
-
diff --git a/README.md b/README.md
@@ -18,11 +18,10 @@
 ## Overview
 
 There is currently significant growth in the number of ML-powered applications.
-This brings benefits, but it also provides grounds for attackers to exploit 
-unsuspecting ML users. This
-is why Google launched the [Secure AI Framework (SAIF)][saif] to establish industry
-standards for creating trustworthy and responsible AI applications. The first
-principle of SAIF is to
+This brings benefits, but it also provides grounds for attackers to exploit
+unsuspecting ML users. This is why Google launched the [Secure AI Framework
+(SAIF)][saif] to establish industry standards for creating trustworthy and
+responsible AI applications. The first principle of SAIF is to
 
 > Expand strong security foundations to the AI ecosystem
 

diff --git a/model_signing/benchmarks/run.sh b/model_signing/benchmarks/run.sh
@@ -157,7 +157,7 @@ model_init() {
 run "${model_name}" "${model_path}" model_init
 
 
-echo 
+echo
 echo "===== RESULTS ======"
 # NOTE: Requires bash >= 4.4.
 echo "results:" "${!results[@]}"
Original file line number	Diff line number	Diff line change
Expand Up		@@ -65,4 +65,3 @@ jobs:
		uses: github/codeql-action/analyze@f9a7c6738f28efb36e31d49c53a201a9c5d6a476 # v2.14.2
		with:
		category: "/language:${{matrix.language}}"
Original file line number	Diff line number	Diff line change
Expand Up		@@ -70,4 +70,3 @@ jobs:
		uses: github/codeql-action/upload-sarif@17573ee1cc1b9d061760f3a006fc4aac4f944fd5 # v2.2.4
		with:
		sarif_file: results.sarif
Original file line number	Diff line number	Diff line change
Expand Up		@@ -30,4 +30,3 @@ use GitHub pull requests for this purpose.
		Contributions made by corporations are covered by a different agreement than the
		one above, the [Software Grant and Corporate Contributor License
		Agreement](https://cla.developers.google.com/about/google-corporate).