Celestia updates (#8)

* first commit * .github: add actions, workflows, and issue template directories * .github: added composite actions and CI Release workflow Brought in a few helpful github actions, for Yamllint, dockerfile linting, release, and versioning. * fix(.github): switch changes to only run on schedule * .github: include workflow dispatch in if condition for changes * fix(.github): fix permission issue with add write-all permission for release * .github: add markdown-lint action and remove scheduled run (#2) * Create CONTRIBUTING.md * .github: add team leads as codeowners (#9) * .github: allow for custom failure threshold for dockerfile linting (#8) * .github: update markdown-lint action to include link checker and add stale workflow (#3) * Create PR template (#12) Co-authored-by: Rootul P <[email protected]> Co-authored-by: Tomasz Zdybał <[email protected]> * chore: remove redundant title (#14) * fix: typo apprioprate -> appropriate (#18) * .github: create reuseable housekeeping workflow (#16) * .github: update housekeeping workflow to use pull_request_target (#20) * Create SECURITY.md (#21) * Add dependabot and bump python setup action (#22) * SECURITY: update from celestia program to security program owner (#23) * feat: Increase the yaml linter length line to 120 (#28) * Bump actions/stale from 6 to 7 (#27) Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump actions/add-to-project from 0.3.0 to 0.4.0 (#26) Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump hadolint/hadolint-action from 2.1.0 to 3.1.0 (#25) Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump necojackarc/auto-request-review from 0.8.0 to 0.10.0 (#24) Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * feat: Common Docker CI pipeline (#29) * Bump docker/build-push-action from 3 to 4 (#30) Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * chore: update codeowners (#31) * fix: repositories naming (#33) * feat: Add master to publish (#34) * fix: Specify Dockerfile path in docker security (#35) * feat: Admit new input: packageName (#36) Co-authored-by: Nguyen Nhu Viet <[email protected]> * chore(deps): bump actions/add-to-project from 0.4.0 to 0.4.1 (#38) Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * .github: add further distinction for codeowners to improve code review (#41) * feat: build container with matrix, it might speed up the process (#43) * fix: fix name of matrics variable (#45) Signed-off-by: Smuu <[email protected]> * fix: undo matrix (#47) * copy over adr template (#40) * chore(deps): bump actions/stale from 7 to 8 (#42) Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * chore(deps): bump actions/add-to-project from 0.4.1 to 0.5.0 (#55) Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Add repository dispatch reusable workflow (#46) Co-authored-by: Nguyen Nhu Viet <[email protected]> * feat: TestGround GA (#57) * SECURITY: minor update to scope language (#48) * feat: speed up amd64 availability & push amd64 on each commit (#60) Signed-off-by: Smuu <[email protected]> * chore(deps): bump necojackarc/auto-request-review from 0.10.0 to 0.12.0 (#61) Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Remove failing docker publish line (#62) * feat: use context - seems like it fixes the ldflags issue (#65) Signed-off-by: Jose Ramon Mañes <[email protected]> * Revert "Remove failing docker publish line" (#64) * remove unneeded workflow and update celestia reference --------- Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: Smuu <[email protected]> Signed-off-by: Jose Ramon Mañes <[email protected]> Co-authored-by: Rootul P <[email protected]> Co-authored-by: Tomasz Zdybał <[email protected]> Co-authored-by: Evan Forbes <[email protected]> Co-authored-by: CHAMI Rachid <[email protected]> Co-authored-by: Jose Ramon Mañes <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Nguyen Nhu Viet <[email protected]> Co-authored-by: Samuel Enderwitz <[email protected]>
rollkit · Jul 18, 2023 · b761dcf · b761dcf
1 parent 4405cc6
commit b761dcf
Show file tree

Hide file tree

Showing 6 changed files with 410 additions and 13 deletions.
diff --git a/.github/workflows/housekeeping.yml b/.github/workflows/housekeeping.yml
@@ -0,0 +1,72 @@
+name: Housekeeping
+
+on:
+  # triage new issues
+  issues:
+    types: [opened, edited]
+  pull_request_target:
+
+jobs:
+  # calls housekeeping for issues
+  # In this example, we are adding `needs: triage` label to all newly created
+  # and unlabeled issues. Additionally, the section that is commented out would
+  # be adding all issues to the defined project
+  issues:
+    name: Issue Management
+    if: ${{ github.event_name == 'issues' }}
+    uses: ./.github/workflows/reusable_housekeeping.yml
+    secrets: inherit
+    # write access for issues and pull requests is needed because the called
+    # workflow requires write access to issues and pull requests and the
+    # permissions must match
+    permissions:
+      issues: write
+      pull-requests: write
+    with:
+      # Labels
+      run-labels: true
+      labels-to-add: "needs: triage"
+      ignore-if-labeled: true
+      # Projects
+      # run-projects: true
+      # project-url: ${{ env.PROJECT_URL }}
+
+  # This illustrates how to only trigger on issues with `EPIC` in the title and
+  # then add the corresponding EPIC issue to multiple projects
+  epic_issues:
+    name: Manage epic_issues
+    if: contains(github.event.issue.title, 'EPIC')
+    strategy:
+      matrix:
+        project_url:
+          - "https://github.com/orgs/rollkit/projects/3"
+          - "https://github.com/orgs/rollkit/projects/7"
+    uses: ./.github/workflows/reusable_housekeeping.yml
+    secrets: inherit
+    permissions:
+      issues: write
+      pull-requests: write
+    with:
+      # Projects
+      run-projects: true
+      project-url: ${{ matrix.project_url }}
+
+  # calls housekeeping for PRs
+  # In this example, we are adding PRs with the `project` label to the defined
+  # project board. Additionally we are using the custom PR assignment rules
+  # defined in `.github/auto_request_review.yml`.
+  pull_request:
+    name: PR Management
+    if: ${{ github.event_name == 'pull_request' }}
+    uses: ./.github/workflows/reusable_housekeeping.yml
+    secrets: inherit
+    permissions:
+      issues: write
+      pull-requests: write
+    with:
+      # Projects
+      run-projects: true
+      project-url: "https://github.com/orgs/rollkit/projects/7"
+      project-labels: "project"
+      # Reviewers
+      run-auto-request-review: true
diff --git a/.github/workflows/reusable_dockerfile_pipeline.yml b/.github/workflows/reusable_dockerfile_pipeline.yml
@@ -66,6 +66,7 @@ jobs:
           OUTPUT_SHORT_SHA: ${{ needs.prepare-env.outputs.output_short_sha }}
           OUTPUT_IMAGE_NAME: ${{ needs.prepare-env.outputs.output_image_name }}
         with:
+          context: .
           push: false
           platforms: linux/amd64
           # we're building the container before the scan, use the short sha tag
@@ -82,12 +83,12 @@ jobs:
           OUTPUT_IMAGE_NAME: ${{ needs.prepare-env.outputs.output_image_name }}
         with:
           # here we use the local tag that we've built before
-          image-ref: '${{ env.OUTPUT_IMAGE_NAME }}:${{ env.OUTPUT_SHORT_SHA }}'
-          format: 'table'
+          image-ref: "${{ env.OUTPUT_IMAGE_NAME }}:${{ env.OUTPUT_SHORT_SHA }}"
+          format: "table"
           #exit-code: '1' # uncomment to stop the CI if the scanner fails
           ignore-unfixed: true
-          vuln-type: 'os,library'
-          severity: 'CRITICAL,HIGH'
+          vuln-type: "os,library"
+          severity: "CRITICAL,HIGH"
 
   docker-build:
     runs-on: "ubuntu-latest"
@@ -125,7 +126,7 @@ jobs:
           tags: |
             # output minimal (short sha)
             type=raw,value={{sha}}
-            # output v0.2.1/v*-*
+            # output v0.2.1/v*-* (or sha of no tag)
             type=semver,pattern={{raw}}
             # pull request event
             type=ref,enable=true,prefix=pr-,suffix=,event=pr
@@ -134,20 +135,50 @@ jobs:
       - name: Set up Docker Buildx
         uses: docker/setup-buildx-action@v2
 
-      # We always build the image but we only push if we are on the `main`,
-      # `master` branch or a versioned `v*` branch
-      - name: Build and PushDocker Image
+      # Build amd64 images always, and publish when it is not a fork. The Github
+      # security model prevents forks from pushing to the registry so we can
+      # only push if the branch/PR is not generated from a fork. Even though
+      # forks can't push, we still want to try and build the image to catch
+      # bugs. For testing purposes we only need an amd64 image.
+      - name: Build and Push Docker Image amd64
         uses: docker/build-push-action@v4
         env:
           OUTPUT_SHORT_SHA: ${{ needs.prepare-env.outputs.output_short_sha }}
           OUTPUT_IMAGE_NAME: ${{ needs.prepare-env.outputs.output_image_name }}
         with:
-          platforms: linux/amd64,linux/arm64
+          context: .
+          platforms: linux/amd64
+          # Only push if the head and base repos match, meaning it is not a fork
           # yamllint disable
-          # The following line, is execute as an if statement, only push when
-          # the branch is main, master or starts with v*
-          push: ${{ github.ref == 'refs/heads/main' || github.ref == 'refs/heads/master' || startsWith(github.ref, 'refs/tags/v') }}
+          push: ${{ github.event.pull_request.head.repo.full_name == github.event.pull_request.base.repo.full_name }}
           # yamllint enable
           tags: ${{ steps.meta.outputs.tags }}
           labels: ${{ steps.meta.outputs.labels }}
           file: ${{ inputs.dockerfile }}
+
+      # Build and Publish images on main, master, and versioned branches.
+      #
+      # NOTES:
+      # This step overrides the tag from the previous step. It will re-use
+      # the cached image that was built and only build the remaining images.
+      #
+      # The reason we split out these steps into 2 is for better handling of
+      # forks when building amd64 images and to enable faster availability of
+      # the amd64 image since building the arm64 image takes significantly
+      # longer.
+      - name: Build and Push Docker Images
+        uses: docker/build-push-action@v4
+        # yamllint disable
+        # only run when the branch is main, master or starts with v*
+        if: ${{ github.ref == 'refs/heads/main' || github.ref == 'refs/heads/master' || startsWith(github.ref, 'refs/tags/v') }}
+        # yamllint enable
+        env:
+          OUTPUT_SHORT_SHA: ${{ needs.prepare-env.outputs.output_short_sha }}
+          OUTPUT_IMAGE_NAME: ${{ needs.prepare-env.outputs.output_image_name }}
+        with:
+          context: .
+          platforms: linux/amd64,linux/arm64
+          push: true
+          tags: ${{ steps.meta.outputs.tags }}
+          labels: ${{ steps.meta.outputs.labels }}
+          file: ${{ inputs.dockerfile }}
diff --git a/.github/workflows/reusable_housekeeping.yml b/.github/workflows/reusable_housekeeping.yml
@@ -99,7 +99,7 @@ jobs:
     permissions:
       pull-requests: write
     steps:
-      - uses: necojackarc/auto-request-review@v0.10.0
+      - uses: necojackarc/auto-request-review@v0.12.0
         with:
           # There is a rollkit level PAT_FOR_AUTO_REQUEST_REVIEW secret that
           # belongs to MSevey who should have sufficient access for all public

diff --git a/.github/workflows/reusable_repository_dispatch.yml b/.github/workflows/reusable_repository_dispatch.yml
@@ -0,0 +1,111 @@
+name: Repository Dispatch
+
+# This workflow is used to call workflows in external repositories and bring the
+# result back into the calling repository.
+#
+# This is ideal for E2E testing when a repository can trigger a dependent
+# repository's integration tests in order to check for breaking changes.
+#
+# The called repository should have a workflow that triggers on `workflow_dispatch`
+#
+# Example:
+#
+# name: Repository Dispatch
+# on:
+#   workflow_dispatch:
+#     inputs:
+#       distinct_id:
+#       key:
+#
+# jobs:
+#   test:
+#     runs-on: ubuntu-latest
+#     steps:
+#       - name: echo distinct ID ${{ github.event.inputs.distinct_id }}
+#         run: |
+#           echo ${{ github.event.inputs.distinct_id }}
+#           echo 'my input key ${{ inputs.key }}'
+#
+#
+# At a minimum a `distinct_id` input is required in the called workflow so that
+# this workflow can find the workflow run in the API since the `distinct_id` is
+# then printed in the run name. This is just needed in one step, so as a
+# template, the echo statement can be used for debugging purposes.
+#
+# This example also shows how you can access addition inputs via the
+# `workflow_inputs` variable. These `workflow_inputs` should correspond with
+# `workflow_dispatch` inputs. In this example, the `workflow_inputs` would have
+# been '{"key": "my_value"}'
+
+on:
+  workflow_call:
+    inputs:
+      owner:
+        description: "Repository owner for the target repository"
+        type: string
+        required: true
+      repo:
+        description: "Repository being targeted"
+        type: string
+        required: true
+      ref:
+        description: "The branch of the target repository that should be targeted, i.e. main or refs/heads/main"
+        type: string
+        required: false
+        default: main
+      workflow:
+        description: "The workflow in the target repository that should be triggered"
+        type: string
+        required: true
+      workflow_inputs:
+        description: "A key value map of custom inputs, i.e. `{'my_key':'my_value'}`"
+        type: string
+        required: false
+        default: ""
+      workflow_timeout_seconds:
+        description: "Timeout for called workflow"
+        type: number
+        required: false
+        default: 300
+
+jobs:
+  triggerMyEvent:
+    runs-on: ubuntu-latest
+    steps:
+      - name: Dispatch an action and get the run ID
+        uses: codex-/return-dispatch@v1
+        id: return_dispatch
+        with:
+          token: ${{ secrets.PAT_REPO_DISPATCH }} # this is an org level secret
+          ref: ${{inputs.repo}}
+          repo: ${{inputs.repo}}
+          owner: ${{inputs.owner}}
+          workflow: ${{inputs.workflow}}
+          workflow_inputs: ${{ inputs.workflow_inputs }} # Optional
+          workflow_timeout_seconds: ${{inputs.workflow_timeout_seconds}} # Default: 300
+
+      # I added this as I observed the API response sometimes being empty as the
+      # API route used to get the conclusion is different than the API route
+      # used in the previous action
+      - name: Delay for api to update
+        run: sleep 2
+
+      - name: Get Conclusion
+        uses: octokit/[email protected]
+        id: get_run_conclusion
+        with:
+          route: GET /repos/{owner}/{repo}/actions/runs/{run_id}
+          owner: ${{inputs.owner}}
+          repo: ${{inputs.repo}}
+          run_id: ${{steps.return_dispatch.outputs.run_id}}
+        env:
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+
+      - name: Fail if not successful
+        if: ${{ fromJSON(steps.get_run_conclusion.outputs.data).conclusion != 'success' }}
+        run: exit 1
+        # Alternative if descriptive exit code is helpful
+        # uses: actions/github-script@v3
+        # with:
+        #   script: |
+        #       core.setFailed('My detailed error response')
diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
@@ -0,0 +1,108 @@
+# Contributing to Rollkit
+
+Welcome and thank you for contributing to building Rollkit.
+
+In this guide, you will find information about how the Rollkit team manages the
+`rollkit` Github as well as expectations around our engineering standards.
+
+## Github Management
+
+The `rollkit` Github uses teams to manage access to the organization and its
+repositories. Currently there is just one `celestia` team which is the team from
+Celestia that is working on Rollkit. As more teams contritube to Rollkit, more
+teams will be added.
+
+### Permissions
+
+By default, all teams have `Triage` access to all repositories in the
+`rollkit` Github. This allows for anyone to help manage issues and pull
+requests (i.e. adding labels). Teams are given write access to the repositories
+that they are responsible for working on. 
+
+### Codeowners
+
+All production repos must use Codeowners or the Auto Review github workflow.
+Codeowners are typically the team leads and/or engineering leadership members.
+
+### .github
+
+The Rollkit team utilizes the organization's `.github` repository. This
+repository is used to store common organization level content like Github
+actions, issue templates, PR templates, etc.
+
+### Repository Settings
+
+The following is a list of key settings that should be enabled on all production
+repositories:
+
+**Enabled:**
+
+- Issues
+- Projects
+- Perserve this repository
+- Allow merge commits **Forked Repos Only**
+- Allow squash merging
+  - Default to pull request title
+- Always suggest updating pull request branches
+- Allow auto-merge
+- Automatically delete head branches
+
+**Disabled:**
+
+- Allow merge commits **Except Forked Repos**
+- Allow rebase merging
+
+## Development
+
+### What the Fork
+
+The default development flow is to fork the repository that you are working on
+in order to submit a PR. If you have write access to a repository, because you
+are a member of that sub team, then you can push your development branches
+directly to the repository.
+
+### ADRs
+
+For architectural changes or improvements, Rollkit uses Architecture Decision
+Record (ADRs) to flush out the design scope. These live in the code under
+`docs/adr`. PRs are used to open new ADRs for approval.
+
+### Issues
+
+When proposing new work, an issue should be created. Issues can be created for
+bugs, feature requests, improvements based on ADRs, etc. Issue templates should
+be used whenever possible, but especially for bug reports, feature requests, and
+ADRs to ensure all the necessary information is captured.
+
+### Pull Requests
+
+Before opening a PR, make sure that the scope of work was previously
+communicated, either via an ADR or an issue. Submitting code that has no
+background context is likely to be rejected because the implication and design
+has not been properly discussed.
+
+The Rollkit team has a culture of prioritizing the review of PRs. This
+prioritization focuses on unblocking others and finishing existing work before
+starting new work.
+
+As a developer, you are responsible for ensuring your code gets merged. This
+means you should be verifying that the appropriate reviewers are assigned and
+that you are responding to review comments promptly. When given the choice to
+start a new PR or work on closing out an existing PR, you should usually choose
+closing out the existing PR.
+
+As a reviewer, it is your responsibility to be providing prompt, action oriented
+reviews. Clearing out your requests reviews should be a daily activity. Action
+oriented reviews mean that there is a clear action step for the developer of
+the PR to take in order to get the PR approved and merged. Open ended questions
+and statements should be avoided. Being clear when a comment is a blocking change,
+okay to be a follow up, or just a personal preference enables developers to
+effectively implement the feedback on a PR.  
+
+All production repos have the following branch protections requirements:
+
+- 2 approvals
+- Codeowner approval
+- New commits dismiss approvals
+- Status checks must be passing
+- Conversations must be resolved