Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Favor -SNAPSHOT version when publishing Mill locally #3615

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

alexarchambault
Copy link
Contributor

@alexarchambault alexarchambault commented Sep 27, 2024

This makes the millVersion task return a more stable value during local development (the version on CI doesn't change on the other hand). That value ends up in BuildInfo files, so having it not change prevents it trigerring re-compilations.

Without this change, adding files to the staging area, committing things, or sometimes just creating new files, was changing the millVersion, and as a consequence trigerring a re-compilation of many things.

This makes the millVersion task return a more stable value. That value
ends up in BuildInfo files, so having it not change prevents it
trigerring re-compilations.

Without this change, adding files to the staging area, committing
things, or sometimes just creating new files, was changing the
millVersion, and as a consequence trigerring a re-compilation of many
things.
@alexarchambault
Copy link
Contributor Author

Don't know if you rely locally on the more detailed versions. They're really a loss of time for me.

@lihaoyi lihaoyi requested a review from lefou September 27, 2024 15:03
@lihaoyi
Copy link
Member

lihaoyi commented Sep 27, 2024

@lefou could you help take a look at this? All this code is interacting with the vcs version and you probably have thought about it more than I have

@lefou
Copy link
Member

lefou commented Sep 28, 2024

Sure, we can change the configuration. mill-vcs-version should support many use-cases. The question is, what we want? Should local modifations to the git hash result in a changed version at all? Currently, we let the detailed change reflect in the version via a hashed value, resulting in ever changing versions. @lihaoyi already worked around this by don't requiring jars but classes-paths as dependencies.

In a work project, we instead use a static -DIRTY suffix. That way, we always detect that a build might not reflect the exact git hash, but once we are in that state, the version is stable. Here is the config for that:

VcsVersion.vcsState().format(dirtyHashDigits = 0)

In the same project, we also have a local override.

def staticVersion: T[Option[String]] = T {
    val file = staticVersionFile().path
    if (os.exists(file)) {
      val version = Option(os.read(file).trim()).filter(_.nonEmpty)
      T.log.info(s"Using static version `${version}` from file: ${file}")
      version
    } else None
  }

  def version: T[String] = T {
    val git = VcsVersion.vcsState().format(dirtyHashDigits = 0, commitCountPad = 4, countSep = "-")
    staticVersion().getOrElse(git)
  }

Comment on lines +222 to +247
def millVersion: T[String] = T {
val vcsVersion = VcsVersion.vcsState().format()
val isCI = System.getenv("CI") != null
if (!isCI && (vcsVersion.contains("DIRTY") || vcsVersion.count(_ == '-') >= 2)) {
// vcsVersion looks like "0.11.11-6-49d084-DIRTYa0693949", we only keep
// the "0.11.11" part here
val baseVersion = vcsVersion.takeWhile(_ != '-')
val lastDotIndex = baseVersion.lastIndexOf('.')
if (lastDotIndex < 0)
// unexpected shape, keeping vcsVersion
vcsVersion
else {
val patchVersionOpt = baseVersion.substring(lastDotIndex + 1).toIntOption
patchVersionOpt match {
case None =>
// unexpected shape, keeping vcsVersion
vcsVersion
case Some(patchVersion) =>
val base = baseVersion.substring(0, lastDotIndex)
s"$base.${patchVersion + 1}-SNAPSHOT"
}
}
}
else
vcsVersion
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I might misreading this longish blob, but I guess what you want is:

Suggested change
def millVersion: T[String] = T {
val vcsVersion = VcsVersion.vcsState().format()
val isCI = System.getenv("CI") != null
if (!isCI && (vcsVersion.contains("DIRTY") || vcsVersion.count(_ == '-') >= 2)) {
// vcsVersion looks like "0.11.11-6-49d084-DIRTYa0693949", we only keep
// the "0.11.11" part here
val baseVersion = vcsVersion.takeWhile(_ != '-')
val lastDotIndex = baseVersion.lastIndexOf('.')
if (lastDotIndex < 0)
// unexpected shape, keeping vcsVersion
vcsVersion
else {
val patchVersionOpt = baseVersion.substring(lastDotIndex + 1).toIntOption
patchVersionOpt match {
case None =>
// unexpected shape, keeping vcsVersion
vcsVersion
case Some(patchVersion) =>
val base = baseVersion.substring(0, lastDotIndex)
s"$base.${patchVersion + 1}-SNAPSHOT"
}
}
}
else
vcsVersion
}
def millVersion: T[String] = T {
VcsVersion.vcsState().format(dirtySep = "-SNAPSHOT", dirtyHashDigits = 0)
}

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@lihaoyi
Copy link
Member

lihaoyi commented Sep 28, 2024

I'd be ok with publishLocal using stable {stable-tag}-SNAPSHOT versions. The only thing you lose is to have multiple local publishes that you can swap between, which isn't something I've ever done: I always only have one local publish that I'm using at any point in time.

I do use installLocalCache quite a lot for testing things out in scenarios where I don't want dist.run, e.g. I don't want to have a wrapper Mill process running along with the child Mill process to confuse me. dist.launcher is also an alternative that provides that, but currently there's a bit of boilerplate around using it (see readme.md). Maybe if we smoothened out the workflow for using dist.launcher, we could rely on that more heavily for more scenarios except when we really-truly need to exercise the assembly-publish-download-jar code paths.

But For this PR that seems like it will speed up installLocalCache, I'm fine with proceeding

@lefou
Copy link
Member

lefou commented Oct 1, 2024

I'd be ok with publishLocal using stable {stable-tag}-SNAPSHOT versions. The only thing you lose is to have multiple local publishes that you can swap between, which isn't something I've ever done: I always only have one local publish that I'm using at any point in time.

The pattern I was proposing is {stable-tag}-{commit-count-since}-{commit-hash}-SNAPSHOT. It will invalidate once you commit somthing, but as long as you just publish local edits, you won't see unnecessary rebuilds.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants