Skip to content

Latest commit

 

History

History
158 lines (114 loc) · 19.7 KB

README.md

File metadata and controls

158 lines (114 loc) · 19.7 KB
Raw

sbt-github-actions

sbt-github-actions is an sbt plugin for assisting in building sbt projects using GitHub Actions.

Usage

Add the following to your plugins.sbt:

addSbtPlugin("com.github.sbt" % "sbt-github-actions" % <latest>)

To use the generative functionality, run sbt githubWorkflowGenerate and commit the results. If your sbt build is ever changed such that the generated workflow is no longer in sync, the workflow run in GitHub Actions will begin failing and you will need to re-run this task (and commit the results).

General Plugin

The GitHubActionsPlugin provides general functionality, giving builds the ability to introspect on their host workflow and whether or not they are running in GitHub Actions. This latter functionality, exposed by the githubIsWorkflowBuild global setting, is the most commonly used functionality of this plugin. If you need behavior within your build which is conditional on whether or not the build is running in CI, this is the setting you should branch on.

githubWorkflowName and githubWorkflowDefinition are designed to allow introspection on the exact definition of the workflow which is running the current build, if any. This kind of introspection is not common, but seems like it could be useful.

Generative Plugin

sbt-github-actions provides a mechanism for generating GitHub Actions workflows from the sbt build definition. With sbt-github-actions, the scala: entry in the job matrix: is populated from the ThisBuild / crossScalaVersions key in your build.sbt.

The GenerativePlugin is designed to make it easier to maintain GitHub Actions builds for sbt projects by generating ci.yml and clean.yml workflow definition files, and then forcibly failing the build if these files ever fall out of step with the build itself. The ci.yml workflow, by default, contains both build and publish jobs, though you will likely need to add extra steps to the githubWorkflowPublishPreamble and/or githubWorkflowEnv (e.g. decrypting and importing a GPG signing key) in order for publication to actually work.

If a publish job is not desired, simply set githubWorkflowPublishTargetBranches to Seq(). By default, publish is restricted to run on main, and additional restrictions may be configured within the build.

sbt and Coursier caching are all handled by the generated ci.yml by default, as well as standard things like Git checkout, Scala setup (using excellent setup-java action), and more. The matrix for the build job will be generated from crossScalaVersions and has additional support for multiple JVMs and OSes. Additionally, compiled artifacts are properly uploaded so that jobs which are dependent on build can avoid redundant work (most notably, publish). Thus, publication is guaranteed to be based on binary files that were generated and tested by the build job, rather than re-generated by publish. (NB: due to what appear to be issues in Zinc, this isn't quite working yet; expect it to be fixed in a coming release of sbt-github-actions)

clean.yml is generated based on a static description because it should just be the default in all GitHub Actions projects. This is basically a hack to work around the fact that artifacts produced by GitHub Actions workflows count against personal and organization storage limits, but those artifacts also are retained indefinitely up until 2 GB. This is entirely unnecessary and egregious, since artifacts are transient and only useful for passing state between jobs within the same workflow. To make matters more complicated, artifacts from a given workflow are invisible to the GitHub API until that workflow is finished, which is why clean.yml has to be a separate workflow rather than part of ci.yml. It runs on every push to the repository.

This plugin is quite prescriptive in that it forcibly manages the contents of the ci.yml and clean.yml files. By default, ci.yml will contain a step which verifies that its contents (and the contents of clean.yml) correspond precisely to the most up-to-date generated version of themselves. If this is not the case, then the build is failed. However, there is no restriction in adding other workflows not named ci.yml or clean.yml. These will be ignored entirely by the plugin.

JDK settings

We recommend you set the following setting in build.sbt:

// sbt-github-actions defaults to using JDK 8 for testing and publishing.
// The following adds JDK 17 for testing.
ThisBuild / githubWorkflowJavaVersions += JavaSpec.temurin("17")

Integration with sbt-ci-release

Integrating with sbt-ci-release is a relatively straightforward process, and the plugins are quite complementary. First, follow all of the setup instructions in sbt-ci-release's readme. Once this is complete, add the following to your build.sbt:

ThisBuild / githubWorkflowTargetTags ++= Seq("v*")
ThisBuild / githubWorkflowPublishTargetBranches :=
  Seq(RefPredicate.StartsWith(Ref.Tag("v")))

ThisBuild / githubWorkflowPublish := Seq(
  WorkflowStep.Sbt(
    commands = List("ci-release"),
    name = Some("Publish project"),
  )
)

This is assuming that you only wish to publish tags. If you also wish to publish snapshots upon successful main builds, use the following githubWorkflowPublishTargetBranches declaration:

ThisBuild / githubWorkflowPublishTargetBranches :=
  Seq(
    RefPredicate.StartsWith(Ref.Tag("v")),
    RefPredicate.Equals(Ref.Branch("main"))
  )

Note the use of += rather than :=.

ThisBuild / githubWorkflowPublish := Seq(
  WorkflowStep.Sbt(
    commands = List("ci-release"),
    name = Some("Publish project"),
    env = Map(
      "PGP_PASSPHRASE" -> "${{ secrets.PGP_PASSPHRASE }}",
      "PGP_SECRET" -> "${{ secrets.PGP_SECRET }}",
      "SONATYPE_PASSWORD" -> "${{ secrets.SONATYPE_PASSWORD }}",
      "SONATYPE_USERNAME" -> "${{ secrets.SONATYPE_USERNAME }}"
    )
  )
)

Tasks

Generative

  • githubWorkflowGenerate – Generates (and overwrites if extant) ci.yml and clean.yml workflows according to configuration within sbt. The clean.yml workflow is something that GitHub Actions should just do by default: it removes old build artifacts to prevent them from running up your storage usage (it has no effect on currently running builds). This workflow is unconfigurable and is simply drawn from the static contents of the clean.yml resource file within this repository.
  • githubWorkflowCheck – Checks to see if the ci.yml and clean.yml files are equivalent to what would be generated and errors if otherwise. This task is run from within the generated ci.yml to ensure that the build and the workflow are kept in sync. As a general rule, any time you change the workflow configuration within sbt, you should regenerate the ci.yml and commit the results, but inevitably people forget. This check fails the build if that happens. Note that if you need to manually fiddle with the ci.yml contents, for whatever reason, you will need to remove the call to this check from within the workflow, otherwise your build will simply fail.

Settings

General

  • githubWorkflowDir : File - Where to place the workflow directory which contains the generated ci.yml and clean.yml files. (default: baseDirectory.value / ".github")
  • githubIsWorkflowBuild : Boolean – Indicates whether or not the build is currently running within a GitHub Actions Workflow
  • githubWorkflowName : String – The name of the currently-running workflow. Will be undefined if not running in GitHub Actions.
  • githubWorkflowDefinition : Map[String, Any] – The raw (parsed) contents of the workflow YAML definition. Will be undefined if not running in GitHub Actions, or if (for some reason) the workflow could not be identified. Workflows are located by taking the githubWorkflowName and finding the YAML definition which has the corresponding name: key/value pair.

Generative

Any and all settings which affect the behavior of the generative plugin should be set in the ThisBuild scope (for example, ThisBuild / crossScalaVersions := rather than just crossScalaVersions := ). This is important because GitHub Actions workflows are global across the entire build, regardless of how individual projects are configured. A corollary of this is that it is not possible (yet) to have specific subprojects which build with different Scala versions, Java versions, or OSes. This is theoretically possible but it's very complicated. For now, I'm going to be lazy and wait for someone to say "pretty please" before implementing it.

General

  • githubWorkflowGeneratedCI : Seq[WorkflowJob] — Contains a description of the ci.yml jobs that will drive the generation if used. This setting can be overridden to customize the jobs (e.g. by adding additional jobs to the workflow).
  • githubWorkflowGeneratedUploadSteps : Seq[WorkflowStep] – Contains a list of steps which are used to upload generated intermediate artifacts from the build job. This is mostly for reference and introspection purposes; one would not be expected to change this setting.
  • githubWorkflowGeneratedDownloadSteps : Seq[WorkflowStep] – Contains a list of steps which are used to download generated intermediate artifacts from the build job. This is mostly for reference and introspection purposes; one would not be expected to change this setting. This setting is particularly useful in conjunction with githubWorkflowAddedJobs: if you're adding a job which needs access to intermediate artifacts, you should make sure these steps are part of the process.
  • githubWorkflowGeneratedCacheSteps : Seq[WorkflowStep] – Contains a list of steps which are used to set up caching for ivy, sbt, and coursier artifacts (respecting changes to files which contain versioning information). This is mostly for reference and introspection purposes; one would not be expected to change this setting. This setting is particularly useful in conjunction with githubWorkflowAddedJobs: if you're adding a job which needs to use sbt, you should probably ensure that these steps are part of the job.
  • githubWorkflowSbtCommand : String – Any use of sbt within the generated workflow will compile to an invocation of this bash command. This defaults to "sbt", but can be overridden to anything that is valid in bash syntax (e.g. "csbt", or "$SBT").
  • githubWorkflowUseSbtThinClient : Boolean – Controls whether or not the --client option will be added to sbt command invocations, accelerating build times (default: true for sbt ≥ 1.4, false otherwise)
  • githubWorkflowIncludeClean : Boolean – Controls whether to include the clean.yml file (default: true)
  • githubWorkflowDependencyPatterns : Seq[String] – A list of file globs which dictate where dependency information is stored. This is conventionally just **/*.sbt and project/build.properties. If you store dependency information in some other file (for example, project/Versions.scala), then you should add a glob which matches that file in this setting. This is used for determining the appropriate cache keys for the Ivy and Coursier caches.
  • githubWorkflowTargetBranches : Seq[String] – A list of globs which will match branches and tags for push and pull-request event types to trigger the ci.yml workflow. Defaults to [*].
  • githubWorkflowTargetTags : Seq[String] – A list of globs which will match tags and tags for push event types to trigger the ci.yml workflow. Defaults to [].
  • githubWorkflowTargetPaths : Paths – Paths which will match modified files for push and pull_request event types to trigger the ci.yml workflow. May be Paths.None, Paths.Include(patterns), or Paths.Ignore(patterns). Paths.Include may include negative patterns. Defaults to Paths.None.
  • githubWorkflowPREventTypes : Seq[PREventType] – A list of event types which will be used to determine which Pull Request events trigger the ci.yml workflow. This follows GitHub's defaults: [opened, synchronize, reopened].
  • githubWorkflowArtifactUpload : Boolean – Controls whether or not to upload target directories in the event that multiple jobs are running sequentially. Can be set on a per-project basis. Defaults to true.
  • githubWorkflowJobSetup : Seq[WorkflowStep] – The automatically-generated checkout, setup, and cache steps which are common to all jobs which touch the build (default: autogenerated)
  • githubWorkflowEnv : Map[String, String] – An environment which is global to the entire ci.yml workflow. Defaults to Map("GITHUB_TOKEN" -> "${{ secrets.GITHUB_TOKEN }}") since it's so commonly needed.
  • githubWorkflowAddedJobs : Seq[WorkflowJob] – A convenience mechanism for adding extra custom jobs to the ci.yml workflow (though you can also do this by modifying githubWorkflowGeneratedCI). Defaults to empty.

build Job

  • githubWorkflowBuildMatrixFailFast : Option[Boolean] – Whether or not to enable the fail-fast strategy for the build job. By default, no strategy option is written to the build configuration (None), which means that the fail-fast strategy is used.
  • githubWorkflowBuildMatrixAdditions : Map[String, List[String]] – Contains a map of additional matrix: dimensions which will be added to the build job (on top of the auto-generated ones for Scala/Java/JVM version). As an example, this can be used to manually achieve additional matrix expansion for ScalaJS compilation. Matrix variables can be referenced in the conventional way within steps by using the ${{ matrix.keynamehere }} syntax. Defaults to empty.
  • githubWorkflowBuildMatrixInclusions : Seq[MatrixInclude] – A list of matrix inclusions. This is useful for when you have a specific matrix job which needs to do extra work, or wish to add an individual matrix job to the configuration set. The matching keys and values are verified against the known matrix configuration. Defaults to empty.
  • githubWorkflowBuildMatrixExclusions : Seq[MatrixExclude] – A list of matrix exclusions. This is useful for when there is a matrix expansion (or set of expansions) which you wish to filter out of the set. Note that exclusions are applied before inclusions, allowing you to subtract jobs before re-adding them. Also – and the documentation isn't clear on this point – it is possible that the matching must cover the full set of matrix keys and cannot contain partial values. Defaults to empty.
  • githubWorkflowBuildPreamble : Seq[WorkflowStep] – Contains a list of steps which will be inserted into the build job in the ci.yml workflow after setup but before the sbt test invocation. Defaults to empty.
  • githubWorkflowBuildPostamble : Seq[WorkflowStep] – Similar to the Preamble variant, this contains a list of steps which will be added to the build job after the sbt test invocation but before cleanup. Defaults to empty.
  • githubWorkflowBuildSbtStepPreamble : Seq[String] - See below.
  • githubWorkflowBuild : Seq[WorkflowStep] – The steps which invoke sbt (or whatever else you want) to build and test your project. This defaults to just [sbt test], but can be overridden to anything. For example, sbt plugin projects probably want to redefine this to be Seq(WorkflowStep.Sbt(List("test", "scripted"))), which would run the test and scripted sbt tasks, in order. Note that all uses of WorkflowStep.Sbt are compiled using the configured githubWorkflowSbtCommand invocation, and prepended with githubWorkflowBuildSbtStepPreamble (default: [++{matrix.scala}]).
  • githubWorkflowJavaVersions : Seq[JavaSpec] – A list of Java versions to be used for the build job. The publish job will use the first of these versions. Defaults to JavaSpec.temurin("8")).
  • githubWorkflowScalaVersions : Seq[String] – A list of Scala versions which will be used to build your project. Defaults to crossScalaVersions in build, and simply scalaVersion in publish.
  • githubWorkflowOSes : Seq[String] – A list of operating systems, which will be ultimately passed to the runs-on: directive, on which to build your project. Defaults to ubuntu-latest. Note that, regardless of the value of this setting, only ubuntu-latest will be used for the publish job. The first value of this setting will be used for the clean job.
  • githubWorkflowBuildRunsOnExtraLabels : Seq[String] - A list of additional runs-on labels, which will be combined with the matrix.os from githubWorkflowOSes above allowing for singling out more specific runners.
  • githubWorkflowBuildTimeout : Option[FiniteDuration] - The maximum duration to let the build job run before GitHub automatically cancels it. Defaults to None.

publish Job

  • githubWorkflowPublishPreamble : Seq[WorkflowStep] – Similar to githubWorkflowBuildPreamble, this contains a series of steps which will be inserted into the publish job after setup but before the publication step. Defaults to empty.
  • githubWorkflowPublishPostamble : Seq[WorkflowStep] – Similar to the Preamble variant, this contains a series of steps which will be inserted into the publish job after publication has completed, but before cleanup. Defaults to empty.
  • githubWorkflowPublishSbtStepPreamble : Seq[String] - Defaults to Nil. This opts out of ++{matrix.scala} preamble during publishing since publishing step typically includes cross building.
  • githubWorkflowPublish : Seq[WorkflowStep] – The steps which will be invoked to publish your project. This defaults to [sbt +publish].
  • githubWorkflowPublishTargetBranches : Seq[RefPredicate] – A list of branch predicates which will be applied to determine whether the publish job will run. Defaults to just == main. The supports all of the predicate types currently allowed by GitHub Actions. This exists because, while you usually want to run the build job on every branch, publish is obviously much more limited in applicability. If this list is empty, then the publish job will be omitted entirely from the workflow.
  • githubWorkflowPublishCond : Option[String] – This is an optional added conditional check on the publish branch, which must be defined using GitHub Actions expression syntax, which will be conjoined to determine the if: predicate on the publish job. Defaults to None.
  • githubWorkflowPublishTimeout : Option[FiniteDuration] - The maximum duration to let the publish job run before GitHub automatically cancels it. Defaults to None.

Windows related

  • githubWorkflowWindowsPagefileFix : Option[windows.PagefileFix] - Due to the fact that Windows is more strict in how it treats pagefile size compared to *unix systems, certain windows related workflows (typically scala-native) can run out of memory without this fix. Defaults to Some(windows.PagefileFix(2GB,8GB)).