Merge branch 'main' into renovate/pydantic-core-2.x

.github/pull_request_template.md

@@ -17,6 +17,6 @@ Applicable spec: <link>
 - [ ] The [contributing guide](https://github.com/canonical/is-charms-contributing-guide) was applied
 - [ ] The documentation is generated using `src-docs`
 - [ ] The PR is tagged with appropriate label (`urgent`, `trivial`, `complex`)
-- [ ] Version has been incremented on `pyproject.toml`
+- [ ] Version has been incremented on `pyproject.toml` and `rockcraft.yaml`
 
 <!-- Explanation for any unchecked items above -->

.github/workflows/comment_contributing.yaml

@@ -0,0 +1,13 @@
+name: Comment on the pull request
+
+on:
+  pull_request:
+    types:
+      - opened
+    branches:
+      - 'track/**'
+
+jobs:
+  comment-on-pr:
+    uses: canonical/operator-workflows/.github/workflows/comment_contributing.yaml@main
+    secrets: inherit

.github/workflows/integration_test.yaml

@@ -13,7 +13,7 @@ jobs:
     uses: canonical/operator-workflows/.github/workflows/integration_test.yaml@main
     secrets: inherit
     with:
-      juju-channel: 3.1/stable
+      juju-channel: 3.4/stable
       channel: 1.28-strict/stable
       self-hosted-runner: true
       self-hosted-runner-label: "edge"

.github/workflows/publish_charm.yaml

@@ -1,16 +1,15 @@
 name: Publish to edge
 
 on:
-  workflow_dispatch:
   push:
     branches:
       - main
+      - track/*
 
 jobs:
   publish-to-edge:
     uses: canonical/operator-workflows/.github/workflows/publish_charm.yaml@main
     secrets: inherit
     with:
-      channel: latest/edge
       charmcraft-channel: "latest/edge"
       resource-mapping: '{"repo-policy-compliance": "flask-app-image"}'

.github/workflows/tests.yaml

@@ -18,7 +18,10 @@ jobs:
       - name: Run tests (unit + integration)
         id: run-tests
         env:
-          GITHUB_TOKEN: ${{ secrets.PERSONAL_GITHUB_TOKEN }}
+          AUTH_GITHUB_TOKEN: ${{ secrets.PERSONAL_GITHUB_TOKEN }}
+          AUTH_GITHUB_APP_ID : ${{ secrets.TEST_GITHUB_APP_ID }}
+          AUTH_GITHUB_APP_INSTALLATION_ID : ${{ secrets.TEST_GITHUB_APP_INSTALLATION_ID }}
+          AUTH_GITHUB_APP_PRIVATE_KEY : ${{ secrets.TEST_GITHUB_APP_PRIVATE_KEY }}
         run: |
           # Ensure that stdout appears as normal and redirect to file and exit depends on exit code of first command
           STDOUT_LOG=$(mktemp --suffix=stdout.log)

README.md

@@ -46,15 +46,21 @@ failing check to be used for testing purposes.
 There are two types of test: the application test and the charm test.
 
 ### Application tests
-To run the application tests, the `GITHUB_TOKEN` environment variable must be set. This
+To run the application tests, the `AUTH_GITHUB_TOKEN` environment variable must be set. This
 should be a token of a user with full repo permissions for the test repository. 
+You can also pass in `AUTH_APP_ID`, `AUTH_INSTALLATION_ID`, and `AUTH_PRIVATE_KEY` 
+to test the authentication using GitHub App Auth. In that case, the tests will additionally
+be executed using GitHub app auth. Note that the GitHub app should be installed
+in the test repository organisation/user namespace, with access granted to the test repository.
+
 The command `tox -e test` can be used to run all tests, which are primarily integration tests.
 You can also select the repository against which to run the tests by setting
 the `--repository` flag. The tests will fork the repository and create PRs against it.
 Note that the tests are currently designed to work for specific Canonical repositories, 
-and may need to be for other repositories 
+and may need to be adapted for other repositories 
 (e.g. `tests.app.integration.test_target_branch_protection.test_fail` 
-assumes that certain collaborators are in the `users_bypass_pull_request_allowances` list).
+assumes that certain collaborators are in the `users_bypass_pull_request_allowances` list). 
+The test repository must also have a branch protection defined for the main branch.
 Also note that the forks are created in the personal space of the user whose token is being used, 
 and that the forks are not deleted after the run. 
 The reason for this is that it is only possible to create one fork of a repository, 
@@ -66,6 +72,8 @@ bot to test things like comments from a user with no write permissions or above.
 GitHub actions should have access to the GitHub token via a secret
 called `PERSONAL_GITHUB_TOKEN`. It is recommended to use either a fine-grained PAT or a 
 token that is short-lived, e.g. 7 days. When it expires, a new token must be set.
+For the GitHub App Auth, the `TEST_GITHUB_APP_ID`, `TEST_GIHUB_APP_INSTALLATION_ID`, and
+`TEST_GITHUB_APP_PRIVATE_KEY` should be set as secrets.
 
 ### Charm tests
 

charm/charmcraft.yaml

@@ -48,10 +48,35 @@ config:
         write and higher permissions for the repository to run jobs from forks.
       type: boolean
       default: false
-    github_token:
+    github_app_id:
       description: >-
-        The token to use for comms with GitHub. This can be a PAT or a fine-grained token
-        with permissions to read collaborators (and collaborators' permissions) and branches 
-        for all repositories that need to be checked.
+        The app or client ID of the GitHub App to use for communication with GitHub.
+        If provided, the other github_app_* options must also be provided.
+        The Github App needs to have read permission for Administration. If private repositories
+        are checked, the Github App does also need read permission for Contents and Pull request.
+        Either this or the github_token must be provided.
       type: string
-      required: true
+    github_app_installation_id:
+      description: >-
+        The installation ID of the GitHub App to use for communication with GitHub.
+        If provided, the other github_app_* options must also be provided.
+        The Github App needs to have read permission for Administration. If private repositories
+        are checked, the Github App does also need read permission for Contents and Pull request.
+        Either this or the github_token must be provided.
+      type: string
+    github_app_private_key:
+      # this will become a juju user secret once paas-app-charmer supports it
+      description: >-
+        The private key of the GitHub App to use for communication with GitHub.
+        If provided, the other github_app_* options must also be provided.
+        The Github App needs to have read permission for Administration. If private repositories
+        are checked, the Github App does also need read permission for Contents and Pull request.
+        Either this or the github_token must be provided.
+      type: string
+    github_token:
+      description: >-
+        The token to use for communication with GitHub. This can be a PAT (with repo scope)
+        or a fine-grained token with read permission for Administration. If private repositories 
+        are checked, the fine-grained token does also need read permission for Contents and 
+        Pull request.
+        Either this or the GitHub App configuration must be provided.

charm/docs/reference/github-auth.md

@@ -0,0 +1,30 @@
+# GitHub Authentication
+
+This section describes the GitHub authentication options available for the charm.
+
+You can either choose to use
+
+- classic personal access tokens
+- fine-grained personal access tokens
+- a GitHub app
+
+for authentication. The latter two options are recommended for better security and access control.
+They require the fine-grained permissions as mentioned below.
+
+**Note**:  If you are using a personal access tokens rather than a GitHub app,
+the user who owns the token must have administrative access to the organisation or repository,
+in addition to having a token with the necessary permissions.
+
+
+## Classic personal access token scopes
+
+If you want to use classic personal access tokens, you will need to select the `repo` 
+scope when generating them.
+
+## Fine grained permissions
+
+For fine-grained access control, the following repository permissions are required:
+
+- Administration: read
+- Contents: read (if you want to check private repositories)
+- Pull requests: read (if you want to check private repositories)

charm/requirements.txt

@@ -1 +1 @@
-paas-app-charmer==1.0.3
+paas-app-charmer==1.0.4

pyproject.toml

@@ -3,7 +3,7 @@
 
 [tool.poetry]
 name = "repo-policy-compliance"
-version = "1.9.0"
+version = "1.10.0"
 description = "Checks GitHub repository settings for compliance with policy"
 authors = ["Canonical IS DevOps <launchpad.net/~canonical-is-devops>"]
 license = "Apache 2.0"

repo_policy_compliance/check.py

@@ -8,14 +8,15 @@
 from enum import Enum
 from typing import Callable, NamedTuple, ParamSpec, TypeVar
 
-from github import Github
+from github import Github, GithubException
 from github.Branch import Branch
 from github.Repository import Repository
 
 from repo_policy_compliance import log
 from repo_policy_compliance.comment import remove_quote_lines
 from repo_policy_compliance.exceptions import (
     ConfigurationError,
+    GithubApiNotFoundError,
     GithubClientError,
     RetryableGithubClientError,
 )
@@ -108,6 +109,13 @@ def wrapper(*args: P.args, **kwargs: P.kwargs) -> R | Report:
                 reason="Checking repository compliance policy failed due to Github rate limit "
                 "exceeded. Please wait before retrying",
             )
+        except GithubApiNotFoundError as exc:
+            return Report(
+                result=Result.FAIL,
+                reason=f"The repository is not set up correctly. "
+                f"A particular GitHub resource could not be found. "
+                f"{('The API returned:' + exc.api_message) if exc.api_message else ''} ",
+            )
         except GithubClientError:
             return Report(
                 result=Result.ERROR,
@@ -142,9 +150,9 @@ def branch_protected(branch: Branch) -> Report:
     return Report(result=Result.PASS, reason=None)
 
 
+@log.call
 @github_exceptions_to_fail_report
 @inject_github_client
-@log.call
 def target_branch_protection(
     github_client: Github, repository_name: str, branch_name: str, source_repository_name: str
 ) -> Report:
@@ -158,6 +166,9 @@ def target_branch_protection(
 
     Returns:
         Whether the branch has appropriate protections.
+
+    Raises:
+        GithubException: If there is a non-404 error on getting the branch protection.
     """
     branch = get_branch(
         github_client=github_client, repository_name=repository_name, branch_name=branch_name
@@ -170,7 +181,22 @@ def target_branch_protection(
     # the default branch
     repository = github_client.get_repo(repository_name)
     if branch_name == repository.default_branch or repository_name != source_repository_name:
-        protection = branch.get_protection()
+        try:
+            # There can be the case that the branch is protected via rulesets and not
+            # the branch protection API in which case the branch protection API will return
+            # a 404 error
+            protection = branch.get_protection()
+        except GithubException as exc:
+            if exc.status == 404:
+                return Report(
+                    result=Result.FAIL,
+                    reason=(
+                        f"{FAILURE_MESSAGE}branch protection not enabled "
+                        "(maybe rulesets have been defined instead of branch protection),"
+                        f" {branch_name=!r}"
+                    ),
+                )
+            raise
         pull_request_reviews = protection.required_pull_request_reviews
         if pull_request_reviews is None:
             return Report(
@@ -192,9 +218,9 @@ def target_branch_protection(
     return Report(result=Result.PASS, reason=None)
 
 
+@log.call
 @github_exceptions_to_fail_report
 @inject_github_client
-@log.call
 def collaborators(github_client: Github, repository_name: str) -> Report:
     """Check that no outside contributors have higher access than read.
 
@@ -246,9 +272,9 @@ class JobMetadata:
     fork_or_branch_repository_name: str
 
 
+@log.call
 @github_exceptions_to_fail_report
 @inject_github_client
-@log.call
 def execute_job(github_client: Github, job_metadata: JobMetadata) -> Report:
     """Check that the execution of the workflow for a SHA has been granted for a PR from a fork.
 
@@ -368,9 +394,9 @@ def _check_authorization_comment(
     return Report(result=Result.PASS, reason=None)
 
 
+@log.call
 @github_exceptions_to_fail_report
 @inject_github_client
-@log.call
 def pull_request_disallow_fork(github_client: Github, job_metadata: JobMetadata) -> Report:
     """Check that the pull request from 3rd party is disallowed.
 

repo_policy_compliance/exceptions.py

@@ -20,5 +20,17 @@ class GithubClientError(BaseError):
     """Error occurred on Github API."""
 
 
+class GithubApiNotFoundError(GithubClientError):
+    """Error occurred on Github API that the resource is not found."""
+
+    def __init__(self, api_message: str | None = None):
+        """Initialize the exception.
+
+        Args:
+            api_message: The message from the Github API. Should be something with "not found".
+        """
+        self.api_message = api_message
+
+
 class RetryableGithubClientError(GithubClientError):
     """Error occurred on Github API that can be retried on user's end."""