Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use setup-java/setup-graalvm instead of setup-scala #91

Merged
merged 4 commits into from
Dec 1, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 12 additions & 8 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ jobs:
matrix:
os: [ubuntu-latest, macos-latest, windows-latest]
scala: [2.12.15]
java: [[email protected]]
java: [temurin@11]
runs-on: ${{ matrix.os }}
steps:
- name: Ignore line ending differences in git
Expand All @@ -36,10 +36,12 @@ jobs:
with:
fetch-depth: 0

- name: Setup Java and Scala
uses: olafurpg/setup-scala@v13
- name: Setup Java (temurin@11)
if: matrix.java == 'temurin@11'
uses: actions/setup-java@v2
with:
java-version: ${{ matrix.java }}
distribution: temurin
java-version: 11

- name: Cache sbt
uses: actions/cache@v2
Expand Down Expand Up @@ -78,7 +80,7 @@ jobs:
matrix:
os: [ubuntu-latest]
scala: [2.12.15]
java: [[email protected]]
java: [temurin@11]
runs-on: ${{ matrix.os }}
steps:
- name: Ignore line ending differences in git
Expand All @@ -90,10 +92,12 @@ jobs:
with:
fetch-depth: 0

- name: Setup Java and Scala
uses: olafurpg/setup-scala@v13
- name: Setup Java (temurin@11)
if: matrix.java == 'temurin@11'
uses: actions/setup-java@v2
with:
java-version: ${{ matrix.java }}
distribution: temurin
java-version: 11

- name: Cache sbt
uses: actions/cache@v2
Expand Down
2 changes: 1 addition & 1 deletion build.sbt
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@

name := "sbt-github-actions"

ThisBuild / baseVersion := "0.13"
ThisBuild / baseVersion := "0.14"

ThisBuild / organization := "com.codecommit"
ThisBuild / publishGithubUser := "djspiewak"
Expand Down
Binary file added dumps/1638295906794/graal_diagnostics_72509.zip
Binary file not shown.
2 changes: 1 addition & 1 deletion src/main/scala/sbtghactions/GenerativeKeys.scala
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ trait GenerativeKeys {
lazy val githubWorkflowPublishTargetBranches = settingKey[Seq[RefPredicate]]("A set of branch predicates which will be applied to determine whether the current branch gets a publication stage; if empty, publish will be skipped entirely (default: [== main])")
lazy val githubWorkflowPublishCond = settingKey[Option[String]]("A set of conditionals to apply to the publish job to further restrict its run (default: [])")

lazy val githubWorkflowJavaVersions = settingKey[Seq[String]]("A list of Java versions to be used for the build job. The publish job will use the *first* of these versions. (default: [[email protected]])")
lazy val githubWorkflowJavaVersions = settingKey[Seq[JavaSpec]]("A list of Java versions to be used for the build job. The publish job will use the *first* of these versions. (default: [temurin@11])")
lazy val githubWorkflowScalaVersions = settingKey[Seq[String]]("A list of Scala versions on which to build the project (default: crossScalaVersions.value)")
lazy val githubWorkflowOSes = settingKey[Seq[String]]("A list of OS names (default: [ubuntu-latest])")

Expand Down
15 changes: 9 additions & 6 deletions src/main/scala/sbtghactions/GenerativePlugin.scala
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,9 @@ object GenerativePlugin extends AutoPlugin {

type Paths = sbtghactions.Paths
val Paths = sbtghactions.Paths

type JavaSpec = sbtghactions.JavaSpec
val JavaSpec = sbtghactions.JavaSpec
}

import autoImport._
Expand Down Expand Up @@ -387,7 +390,7 @@ strategy:${renderedFailFast}
matrix:
os:${compileList(job.oses, 3)}
scala:${compileList(job.scalas, 3)}
java:${compileList(job.javas, 3)}${renderedMatrices}
java:${compileList(job.javas.map(_.render), 3)}${renderedMatrices}
runs-on: ${runsOn}${renderedEnvironment}${renderedContainer}${renderedEnv}
steps:
${indent(job.steps.map(compileStep(_, sbt, declareShell = declareShell)).mkString("\n\n"), 1)}"""
Expand Down Expand Up @@ -427,7 +430,7 @@ s"""
val renderedPaths = paths match {
case Paths.None =>
""
case Paths.Include(paths) =>
case Paths.Include(paths) =>
"\n" + indent(s"""paths: [${paths.map(wrap).mkString(", ")}]""", 2)
case Paths.Ignore(paths) =>
"\n" + indent(s"""paths-ignore: [${paths.map(wrap).mkString(", ")}]""", 2)
Expand Down Expand Up @@ -476,7 +479,7 @@ ${indent(jobs.map(compileJob(_, sbt)).mkString("\n\n"), 1)}
githubWorkflowPublishTargetBranches := Seq(RefPredicate.Equals(Ref.Branch("main"))),
githubWorkflowPublishCond := None,

githubWorkflowJavaVersions := Seq("[email protected]"),
githubWorkflowJavaVersions := Seq(JavaSpec.temurin("11")),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't that a bit dangerous - publishing by default for Java 11?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's basically two possible dangers. One is that the classfiles have the wrong version header and are thus incompatible with <11, and the other is standard library stuff. In the former case, that only happens if you have Java sources mixed into your project and you aren't using the -target option. You're likely to be aware of the pitfalls if you're in that situation.

The standard library stuff is more complicated, but also much rarer. These are things like ByteBuffer#swap. They're hard to figure out in advance, but most people who are around these types of issues have some idea of what's going on.

At the very least, it's an easy fix downstream: you can just opt-in to Java 8. This change just makes Java 8 runtime an opt-in rather than an opt-out. I think it's important to apply some gentle pressure to the larger ecosystem to move forward. Java 8 is ancient at this point.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But does this plugin set the -target option? 🤔

I agree it's time to move on but the reality is that production systems are slow in that regard.
I'm not sure what would be the trigger for the entire ecosystem to move forward, but we need something.
I'm just afraid that this isn't gonna provide sufficient motivation.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel like the motivation has to be slow and subtle. Nothing else really makes sense or will work. What this change does is effectively lean into the idea that bending over backwards to support a deprecated JVM is the "unusual" thing to do, still possible but unusual in a small way.

For what it's worth, most production systems I see these days are on 11 or higher. It's very rare that I see something other than a library on 8.

Copy link
Contributor

@joroKr21 joroKr21 Dec 2, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is nothing subtle about publishing a library though - it's either on Java 8 or 11 and if it's on 11 you can't use 8 and that's it. If it's on 8 nobody that is using it would notice. I.e. it's more or less a binary choice for the ecosystem and at some point we will have to say it's time to bootstrap on Java 11. But it would be good to reach a widely supported decision to do so.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Only if they have Java sources though. How wide of a cohort is that?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Only if they have Java sources though.

Why? Doesn't the same apply to Scala sources as well? They also need to produce bytecode.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The Scala compiler always targets Java 8 for bytecode output. If you do a quick publishLocal on anything that is pure Scala from Java 11 (or higher) and then grab it in ammonite on Java 8, it'll work perfectly.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The Scala compiler always targets Java 8 for bytecode output. If you do a quick publishLocal on anything that is pure Scala from Java 11 (or higher) and then grab it in ammonite on Java 8, it'll work perfectly.

Ok nice 👍 Somehow I missed that, I guess I'm still confused about this stuff.

Copy link
Member

@SethTisue SethTisue Dec 6, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The standard library stuff is more complicated, but also much rarer. These are things like ByteBuffer#swap. They're hard to figure out in advance, but most people who are around these types of issues have some idea of what's going on.

I'm not so sure about that.

Once something has already gone wrong (once a call to an 11+ or 17+ Java stdlib method has been used, and CI isn't set up to catch it) and the problem is noticed and reported, "most people" who do Scala OSS will know right away what happened. But by that point it's already too late; a library version has already accidentally been published that doesn't work on 8.

Perhaps it's reasonable — though bold — to make publish-on-11 the default as a way of encouraging people to drop 8, but I don't think "problems will be rare" is a good supporting argument.

githubWorkflowScalaVersions := crossScalaVersions.value,
githubWorkflowOSes := Seq("ubuntu-latest"),
githubWorkflowDependencyPatterns := Seq("**/*.sbt", "project/build.properties"),
Expand Down Expand Up @@ -610,9 +613,9 @@ ${indent(jobs.map(compileJob(_, sbt)).mkString("\n\n"), 1)}
Nil
}

autoCrlfOpt ::: List(
WorkflowStep.CheckoutFull,
WorkflowStep.SetupScala) :::
autoCrlfOpt :::
List(WorkflowStep.CheckoutFull) :::
WorkflowStep.SetupJava(githubWorkflowJavaVersions.value.toList) :::
githubWorkflowGeneratedCacheSteps.value.toList
},

Expand Down
41 changes: 41 additions & 0 deletions src/main/scala/sbtghactions/JavaSpec.scala
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
/*
* Copyright 2020-2021 Daniel Spiewak
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

package sbtghactions

final case class JavaSpec(dist: JavaSpec.Distribution, version: String) {
def render: String = dist match {
case JavaSpec.Distribution.GraalVM(gversion) => s"graal:$gversion@$version"
case dist => s"${dist.rendering}@$version"
}
}

object JavaSpec {

def temurin(version: String): JavaSpec = JavaSpec(Distribution.Temurin, version)
def graalvm(graal: String, version: String): JavaSpec = JavaSpec(Distribution.GraalVM(graal), version)

sealed abstract class Distribution(val rendering: String) extends Product with Serializable

object Distribution {
case object Temurin extends Distribution("temurin")
case object Zulu extends Distribution("zulu")
case object Adopt extends Distribution("adopt-hotspot")
case object OpenJ9 extends Distribution("adopt-openj9")
case object Liberica extends Distribution("liberica")
final case class GraalVM(version: String) extends Distribution(version)
}
}
2 changes: 1 addition & 1 deletion src/main/scala/sbtghactions/Paths.scala
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
/*
* Copyright 2021 Daniel Spiewak
* Copyright 2020-2021 Daniel Spiewak
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
Expand Down
2 changes: 1 addition & 1 deletion src/main/scala/sbtghactions/WorkflowJob.scala
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ final case class WorkflowJob(
env: Map[String, String] = Map(),
oses: List[String] = List("ubuntu-latest"),
scalas: List[String] = List("2.13.6"),
javas: List[String] = List("[email protected]"),
javas: List[JavaSpec] = List(JavaSpec.temurin("11")),
needs: List[String] = List(),
matrixFailFast: Option[Boolean] = None,
matrixAdds: Map[String, List[String]] = Map(),
Expand Down
21 changes: 20 additions & 1 deletion src/main/scala/sbtghactions/WorkflowStep.scala
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,26 @@ object WorkflowStep {

val Checkout: WorkflowStep = Use(UseRef.Public("actions", "checkout", "v2"), name = Some("Checkout current branch (fast)"))

val SetupScala: WorkflowStep = Use(UseRef.Public("olafurpg", "setup-scala", "v13"), name = Some("Setup Java and Scala"), params = Map("java-version" -> s"$${{ matrix.java }}"))
def SetupJava(versions: List[JavaSpec]): List[WorkflowStep] =
versions map {
case jv @ JavaSpec(JavaSpec.Distribution.GraalVM(graalVersion), version) =>
WorkflowStep.Use(
UseRef.Public("DeLaGuardo", "setup-graalvm", "5.0"),
name = Some(s"Setup GraalVM (${jv.render})"),
cond = Some(s"matrix.java == '${jv.render}'"),
params = Map(
"graalvm" -> graalVersion,
"java" -> version))

case jv @ JavaSpec(dist, version) =>
WorkflowStep.Use(
UseRef.Public("actions", "setup-java", "v2"),
name = Some(s"Setup Java (${jv.render})"),
cond = Some(s"matrix.java == '${jv.render}'"),
params = Map(
"distribution" -> dist.rendering,
"java-version" -> version))
}

val Tmate: WorkflowStep = Use(UseRef.Public("mxschmitt", "action-tmate", "v2"), name = Some("Setup tmate session"))

Expand Down
2 changes: 1 addition & 1 deletion src/sbt-test/sbtghactions/check-and-regenerate/build.sbt
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ ThisBuild / scalaVersion := crossScalaVersions.value.head

ThisBuild / githubWorkflowTargetTags += "v*"

ThisBuild / githubWorkflowJavaVersions += "graalvm@21.1.0"
ThisBuild / githubWorkflowJavaVersions += JavaSpec.graalvm("21.1.0", "8")
ThisBuild / githubWorkflowPublishTargetBranches += RefPredicate.Equals(Ref.Tag("test"))

ThisBuild / githubWorkflowBuildMatrixAdditions += "test" -> List("this", "is")
Expand Down
34 changes: 26 additions & 8 deletions src/sbt-test/sbtghactions/check-and-regenerate/expected-ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ jobs:
matrix:
os: [ubuntu-latest]
scala: [2.13.6, 2.12.15]
java: [[email protected], graalvm@21.1.0]
java: [temurin@11, 'graal:21.1.0@8']
test: [this, is]
include:
- test: this
Expand All @@ -39,10 +39,19 @@ jobs:
with:
fetch-depth: 0

- name: Setup Java and Scala
uses: olafurpg/setup-scala@v13
- name: Setup Java (temurin@11)
if: matrix.java == 'temurin@11'
uses: actions/setup-java@v2
with:
java-version: ${{ matrix.java }}
distribution: temurin
java-version: 11

- name: 'Setup GraalVM (graal:21.1.0@8)'
if: 'matrix.java == ''graal:21.1.0@8'''
uses: DeLaGuardo/[email protected]
with:
graalvm: 21.1.0
java: 8

- name: Cache sbt
uses: actions/cache@v2
Expand Down Expand Up @@ -81,18 +90,27 @@ jobs:
matrix:
os: [ubuntu-latest]
scala: [2.13.6]
java: [[email protected]]
java: [temurin@11]
runs-on: ${{ matrix.os }}
steps:
- name: Checkout current branch (full)
uses: actions/checkout@v2
with:
fetch-depth: 0

- name: Setup Java and Scala
uses: olafurpg/setup-scala@v13
- name: Setup Java (temurin@11)
if: matrix.java == 'temurin@11'
uses: actions/setup-java@v2
with:
distribution: temurin
java-version: 11

- name: 'Setup GraalVM (graal:21.1.0@8)'
if: 'matrix.java == ''graal:21.1.0@8'''
uses: DeLaGuardo/[email protected]
with:
java-version: ${{ matrix.java }}
graalvm: 21.1.0
java: 8

- name: Cache sbt
uses: actions/cache@v2
Expand Down
20 changes: 12 additions & 8 deletions src/sbt-test/sbtghactions/no-clean/.github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -23,18 +23,20 @@ jobs:
matrix:
os: [ubuntu-latest]
scala: [2.13.6, 2.12.15]
java: [[email protected]]
java: [temurin@11]
runs-on: ${{ matrix.os }}
steps:
- name: Checkout current branch (full)
uses: actions/checkout@v2
with:
fetch-depth: 0

- name: Setup Java and Scala
uses: olafurpg/setup-scala@v13
- name: Setup Java (temurin@11)
if: matrix.java == 'temurin@11'
uses: actions/setup-java@v2
with:
java-version: ${{ matrix.java }}
distribution: temurin
java-version: 11

- name: Cache sbt
uses: actions/cache@v2
Expand Down Expand Up @@ -71,18 +73,20 @@ jobs:
matrix:
os: [ubuntu-latest]
scala: [2.13.6]
java: [[email protected]]
java: [temurin@11]
runs-on: ${{ matrix.os }}
steps:
- name: Checkout current branch (full)
uses: actions/checkout@v2
with:
fetch-depth: 0

- name: Setup Java and Scala
uses: olafurpg/setup-scala@v13
- name: Setup Java (temurin@11)
if: matrix.java == 'temurin@11'
uses: actions/setup-java@v2
with:
java-version: ${{ matrix.java }}
distribution: temurin
java-version: 11

- name: Cache sbt
uses: actions/cache@v2
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -23,18 +23,20 @@ jobs:
matrix:
os: [ubuntu-latest]
scala: [2.13.6]
java: [[email protected]]
java: [temurin@11]
runs-on: ${{ matrix.os }}
steps:
- name: Checkout current branch (full)
uses: actions/checkout@v2
with:
fetch-depth: 0

- name: Setup Java and Scala
uses: olafurpg/setup-scala@v13
- name: Setup Java (temurin@11)
if: matrix.java == 'temurin@11'
uses: actions/setup-java@v2
with:
java-version: ${{ matrix.java }}
distribution: temurin
java-version: 11

- name: Cache sbt
uses: actions/cache@v2
Expand Down Expand Up @@ -70,18 +72,20 @@ jobs:
matrix:
os: [ubuntu-latest]
scala: [2.13.6]
java: [[email protected]]
java: [temurin@11]
runs-on: ${{ matrix.os }}
steps:
- name: Checkout current branch (full)
uses: actions/checkout@v2
with:
fetch-depth: 0

- name: Setup Java and Scala
uses: olafurpg/setup-scala@v13
- name: Setup Java (temurin@11)
if: matrix.java == 'temurin@11'
uses: actions/setup-java@v2
with:
java-version: ${{ matrix.java }}
distribution: temurin
java-version: 11

- name: Cache sbt
uses: actions/cache@v2
Expand Down
Loading