Add GHA workflow to lint with pantsbuild

[cognifloyd]

Background

This is another part of introducing pants, as discussed in the TSC Meetings on 12 July 2022, 02 Aug 2022, 06 Sept 2022, and 04 Oct 2022. Pants has fine-grained per-file caching of results for lint, fmt (like black), test, etc. It also has lockfiles that work well for monorepos that have multiple python packages. With these lockfiles CI should not break when any of our dependencies or our transitive dependencies release new versions, because CI will continue to use the locked version until we explicitly relock with updates.

To keep PRs as manageable/reviewable as possible, introducing pants will take a series of PRs. I do not know yet how many PRs; I will break this up into logical steps with these goals:

• introduce pants to the st2 repo, and
• teach some (but not all) TSC members about pants step-by-step.

Other pants PRs include:

• Initialize pantsbuild #5713
• Configure pantsbuild source roots #5724
• Enable pantsbuild language backends #5725
• Adjust .gitignore to work better with pantsbuild #5726
• Add GHA workflow to validate pants BUILD files #5732
• Configure pants_ignore and bump pants to v2.14.0rc1 #5733
• Explicitly disable pantsbuild opt-in anonymous telemetry #5737
• Create pantsbuild BUILD files with ./pants tailor :: #5738
• Use shellcheck (via pantsbuild) #5751

Overview of this PR

We begin to add linters in #5751. This adds a GHA workflow to automatically run the linters. This workflow copies a few pants tasks from the pants validation workflow added in #5732 and the apt tasks from our current CI workflow.

Until one of the linter backends is registered, ./pants lint :: will just return with success because there is nothing to do.

When we add python linters (black, flake8, etc) we will need to expand this workflow to handle a matrix of the various python versions. For now, this is sufficient for running shellcheck (in #5751) and a few other linters.

Relevant Pants documentation

• ./pants lint goal

Things you can do with pantsbuild

You can run ./pants lint :: but it doesn't do anything yet. So, output looks like this:

$ ./pants lint ::
13:00:22.28 [INFO] Initializing scheduler...
13:00:22.57 [INFO] Scheduler initialized.

$ echo $?
0
$ ./pants lint ::

$ echo $?
0

[Eric-Arellano]

Why this?

[cognifloyd]

I copied that from our other workflows which we run every day. We did that in part to catch issues with transitive deps, which will not be a significant issue once we've got the lockfiles in. So, I suppose it is unlikely for linters to fail from one day to another, so maybe we don't have to be quite so aggressive here. 🤔

What do others think? Drop this? Comment it out for now?

[Eric-Arellano]

which will not be a significant issue once we've got the lockfiles in.

Note that you already are using "tool lockfiles". Pants ships them by default. So your installation of tools like Flake8 and Pylint will be deterministic. It is only installs of your own user code that need to manually set up lockfiles, since we can't ship them without knowing before what your code is.

So running this is cron is likely a waste of compute resources (and trees)

[cognifloyd]

I commented it out.

[rush-skills]

Lgtm overall, added some comments below

[rush-skills]

can't we point to master or latest release or something here

[cognifloyd]

This repo does not yet have tags we can point to. If there are changes, we might need to adjust how we use the action, app pinning to a particular version seemed the best way to go.

When we choose another approach (eg when there are tags available), then we'll need to change the pants BUILD validation workflow add well:

st2/.github/workflows/pants.yaml

Line 33 in 989e4ec

uses: pantsbuild/actions/init-pants@c0ce05ee4ba288bb2a729a2b77294e9cb6ab66f7

[rush-skills]

is the root level requirements.txt enough in this scenario?

[cognifloyd]

Not entirely. I plan to change this to use the lockfiles instead once the lockfiles are available.

[amanda11]

LGTM - once the other queries on the PR have been addressed.