Pre‐commit integration

Introduction

List of common usage of pre-commit extensions (in repos tag).

Integrate in the CI

pre-commit.ci has some limitations (no http request, only Python and JavaScript is supported, no whell dependencies update) It I use Renovate ad CI to do that.

Add your project to the Geo GHCI GitHub application to automatically be patched when there is a diff.

Renovate config to update the additional_dependencies that should be followed by # npm or # pypi to determine the data source.

{
  regexManagers: [
    /** Do updates on pre-commit additional dependencies */
    {
      fileMatch: ['^\\.pre\\-commit\\-config\\.yaml$'],
      matchStrings: [" +- '?(?<depName>[^' @=]+)(@|==)(?<currentValue>[^' @=]+)'? # (?<datasource>.+)"],
    }
  ]
}

pre-commit should be added in ci/requirements.txt.

Workflow:

      # Checkout
      - uses: actions/checkout@v4
        with:
          fetch-depth: 0

      # Install pre-commit from ci/requirements.txt
      - run: echo "${HOME}/.local/bin" >> ${GITHUB_PATH}
      - run: python3 -m pip install --user --requirement=ci/requirements.txt

      # Run pre-commit with cache and patch uploaf for the GHCI GitHub application
      - uses: actions/cache@v4
        with:
          path: ~/.cache/pre-commit
          key: pre-commit-${{ hashFiles('.pre-commit-config.yaml') }}
          restore-keys: "pre-commit-${{ hashFiles('.pre-commit-config.yaml') }}\npre-commit-"
      - run: pre-commit run --all-files
        env:
          SKIP: poetry-lock,pipenv-lock,helm-lock
      - run: git diff --exit-code --patch > /tmp/pre-commit.patch || true
        if: failure()
      - uses: actions/upload-artifact@v4
        with:
          name: Apply pre-commit fix.patch
          path: /tmp/pre-commit.patch
          retention-days: 1
        if: failure()

Common config

Common

  - repo: https://github.com/pre-commit/pre-commit-hooks
    rev: v4.5.0
    hooks:
      - id: check-merge-conflict
      - id: check-ast
      - id: debug-statements
      - id: check-toml
      - id: check-yaml
      - id: check-json
      - id: end-of-file-fixer
      - id: trailing-whitespace
      - id: mixed-line-ending

Secrets

Ensure that you don't commit any secret...

  - repo: https://github.com/pre-commit/pre-commit-hooks
    rev: v4.5.0
    hooks:
      - id: detect-private-key
  - repo: https://github.com/sirwart/ripsecrets
    rev: v0.1.7
    hooks:
      - id: ripsecrets

Copyright

Useful when you should have the copyright on all the files

  - repo: https://github.com/sbrunner/hooks
    rev: 0.7.0
    hooks:
      - id: copyright

Commons CI files checks

  - repo: https://github.com/python-jsonschema/check-jsonschema
    rev: 0.27.4
    hooks:
      - id: check-github-workflows
      - id: check-renovate
        additional_dependencies:
          - pyjson5==1.6.5 # pypi
      - id: check-github-actions
      - id: check-jsonschema
        name: Check GitHub Workflows set timeout-minutes
        files: ^\.github/workflows/[^/]+$
        types:
          - yaml
        args:
          - --builtin-schema
          - github-workflows-require-timeout

Poetry

The first one check that the lock is up-to-date regarding the meta hash. The second, if it's out of sync, he will apply the lock (with the --no-update option).

  - repo: https://github.com/sbrunner/hooks
    rev: 0.7.0
    hooks:
      - id: poetry-check
        additional_dependencies:
          - poetry==1.7.1 # pypi
      - id: poetry-lock
        additional_dependencies:
          - poetry==1.7.1 # pypi

Codespell

  - repo: https://github.com/codespell-project/codespell
    rev: v2.2.6
    hooks:
      - id: codespell
        exclude: |-
          (?x)^(
            (.*/)?package-lock\.json$
            |geoportal/tests/data/dem4\..*
            |(.*/)?poetry\.lock$
          )
        args:
          - --ignore-words=.github/spell-ignore-words.txt

Prettier

JavaScript, HTML, CSS, JSON, YAML, ...

  - repo: https://github.com/pre-commit/mirrors-prettier
    rev: v3.1.0
    hooks:
      - id: prettier
        additional_dependencies:
          - [email protected] # npm
          - [email protected] # npm
          - [email protected] # npm

Spellcheck

Check the bash files and Docker files

  - repo: https://github.com/shellcheck-py/shellcheck-py
    rev: v0.9.0.6
    hooks:
      - id: shellcheck

Git

Check that the git-attribute are respected

  - repo: https://github.com/jumanjihouse/pre-commit-hooks
    rev: 3.0.0
    hooks:
      - id: git-check

Python

  - repo: https://github.com/asottile/pyupgrade
    rev: v3.15.0
    hooks:
      - id: pyupgrade
        args:
          - --py310-plus
  - repo: https://github.com/PyCQA/autoflake
    rev: v2.2.1
    hooks:
      - id: autoflake
  - repo: https://github.com/PyCQA/isort
    rev: 5.13.2
    hooks:
      - id: isort
  - repo: https://github.com/psf/black
    rev: 24.1.1
    hooks:
      - id: black

JSON schema

Validate against the present in $schema os in # yaml-language-server: $schema=

  - repo: https://github.com/sbrunner/jsonschema-validator
    rev: 0.1.0
    hooks:
      - id: jsonschema-validator

Spell check

  - repo: https://github.com/mheap/json-schema-spell-checker
    rev: main
    hooks:
      - id: json-schema-spell-checker
        files: ...
        args:
          - --fields=description
          - --ignore-numbers
          - --ignore-acronyms
          - --en-us
          - --spelling=.github/spell-ignore-words.txt

Generate documentation

  - repo: https://github.com/sbrunner/jsonschema2md2
    rev: 1.1.0
    hooks:
      - id: jsonschema2md
        files: <src>
        args:
          - --pre-commit
          - <src>
          - <dst>

Generate Python types

  - repo: https://github.com/camptocamp/jsonschema-gentypes
    rev: 2.4.0
    hooks:
      - id: jsonschema-gentypes
        files: |-
          (?x)^(
            jsonschema-gentypes\.yaml
            |<src>
          )$

Requires a jsonschema-gentypes.yaml file with

headers: |
  """
  Automatically generated file from a JSON schema.
  """

pre_commit:
  enabled: true
  hooks_skip:
    - jsonschema-gentypes
  arguments:
    - --color=never

generate:
  - source: <src>
    destination: <dst>
    root_name: Configuration
    api_arguments:
      additional_properties: Only explicit