Skip to content

Commit

Permalink
Add detection of MC launcher installations and reuse libraries stored…
Browse files Browse the repository at this point in the history
… there (#27)
  • Loading branch information
shartte authored Jul 13, 2024
1 parent 0dbead6 commit 289c990
Show file tree
Hide file tree
Showing 8 changed files with 326 additions and 60 deletions.
63 changes: 43 additions & 20 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,8 @@ Since it is used as part of the NeoForge toolchain, it extends NeoForm by adding
apply [NeoForge](https://github.com/neoforged/NeoForge) patches and produces the necessary artifacts to compile against
the NeoForge APIs.

You'll find the [latest releases](https://projects.neoforged.net/neoforged/neoformruntime) on the NeoForged Project Listing.
You'll find the [latest releases](https://projects.neoforged.net/neoforged/neoformruntime) on the NeoForged Project
Listing.

## Usage

Expand Down Expand Up @@ -85,38 +86,60 @@ assets_root=...path to assets...
While it may seem odd that NFRT supports passing NeoForm or NeoForge versions to this command, this is in service
of potential Gradle plugins never having to actually read and parse the NeoForm configuration file.

## Caches
## Common Options

NFRT has to store various files to speed up later runs. It does this in several cache
These options affect all NFRT subcommands.

### Set NFRT Home Directory

The `--home-dir` option changes where NFRT stores its caches, intermediate working directories, assets, etc.

It defaults to `.neoformruntime` in your user directory on Windows and Mac OS X, and `.cache/neoformruntime` on Linux,
where it also respects `XDG_CACHE_DIR`.

### Change Temporary Working Directories

The `--work-dir` option changes where NFRT creates temporary working directories.
Defaults to the given home directory, otherwise.

### Adding Custom Launcher Directories

NFRT will try to reuse files found in Minecraft launcher directories. It will scan known locations to find installation
directories.
If you have moved your launcher, you can supply additional launcher directories using the `--launcher-dir` option.

### Cache Directories
### Artifact Resolution

On Linux, NFRT will store its caches by default at `$XDG_CACHE_HOME/neoformruntime`. If that variable is not set or not an
absolute path, it falls back to `~/.cache/neoformruntime`.
NFRT comes with built-in Maven repositories for downloading required files.
To override these repositories, use the `--repository` option one or more times.
If you'd only like to add repositories instead of overriding them completely, you can use the `--add-repository` option.

For other operating systems (Windows, Mac), it defaults to `.neoformruntime` in your home directory.
When running NFRT through a tool like Gradle, it might be desired to externally inject all needed dependencies,
by redirecting them to the local Gradle artifact cache.
NFRT supports this use case by supporting an artifact manifest, which can be supplied using the `--artifact-manifest`
option.

Please note that Gradle plugins using this runtime may set different cache directories.
This manifest is a Java properties file in ISO-8859-1 encoding, which uses a Maven coordinate as the key and the full
path for that artifact as the value. For example:

### Reusing Gradle Artifacts
`com.google.guava\:guava\:32.1.2-jre=C\:\\path\\to\\file.jar`

To prevent NFRT from re-downloading all the libraries and artifacts **again** when it is being used through Gradle,
it supports passing an "artifact manifest". This property file maps from Maven coordinates to the full path of
those files on disk.
To aid with detecting missing entries to fully cover all required artifacts, the `--warn-on-artifact-manifest-miss`
option
enables warnings when an artifact is being looked up, but not found in the manifest.
NFRT will continue to download the artifact remotely in this case.

The path to this manifest is passed to NFRT via the `--artifact-manifest` command-line option.
### Mojang Launcher Manifest

Example:
The full URL to the [Launcher version manifest](https://launchermeta.mojang.com/mc/game/version_manifest_v2.json) can be overridden using `--launcher-meta-uri`.

```properties
net.neoforged.fancymodloader\:loader\:3.0.53-pr-54-junit=C\:\\Gradle Home\\caches\\modules-2\\files-2.1\\net.neoforged.fancymodloader\\loader\\3.0.53-pr-54-junit\\eacd6fc41449ff1dc84b1a4593c7e6c96599374f\\loader-3.0.53-pr-54-junit.jar
[...]
```
### Output Settings

For more verbose output, pass `--verbose`.

The Gradle plugin can prepare such a file to make NFRT use a local build of certain artifacts too in case includeBuild is used on the
containing project.
To force the use of ANSI color on the console, pass `--color`, or `--no-color` to disable it. The `NO_COLOR` environment variable is also respected.

The use of Emojis in console output can be toggled with `--emojis` and `--no-emojis`.

## Example Execution Graphs

Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
package net.neoforged.neoform.runtime.artifacts;

import net.neoforged.neoform.runtime.cache.CacheManager;
import net.neoforged.neoform.runtime.cache.LauncherInstallations;
import net.neoforged.neoform.runtime.cli.LockManager;
import net.neoforged.neoform.runtime.downloads.DownloadManager;
import net.neoforged.neoform.runtime.downloads.DownloadSpec;
Expand All @@ -10,6 +11,7 @@
import net.neoforged.neoform.runtime.manifests.MinecraftVersionManifest;
import net.neoforged.neoform.runtime.utils.AnsiColor;
import net.neoforged.neoform.runtime.utils.FilenameUtil;
import net.neoforged.neoform.runtime.utils.HashingUtil;
import net.neoforged.neoform.runtime.utils.Logger;
import net.neoforged.neoform.runtime.utils.MavenCoordinate;

Expand All @@ -28,6 +30,7 @@
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.Objects;
import java.util.Properties;

public class ArtifactManager {
Expand All @@ -41,17 +44,20 @@ public class ArtifactManager {
private final Path artifactsCache;
private final Map<MavenCoordinate, Artifact> externallyProvided = new HashMap<>();
private boolean warnOnArtifactManifestMiss;
private final LauncherInstallations launcherInstallations;

public ArtifactManager(List<URI> repositoryBaseUrls,
CacheManager cacheManager,
DownloadManager downloadManager,
LockManager lockManager,
URI launcherManifestUrl) {
URI launcherManifestUrl,
LauncherInstallations launcherInstallations) {
this.repositoryBaseUrls = repositoryBaseUrls;
this.downloadManager = downloadManager;
this.lockManager = lockManager;
this.launcherManifestUrl = launcherManifestUrl;
this.artifactsCache = cacheManager.getArtifactCacheDir();
this.launcherInstallations = launcherInstallations;
}

public Artifact get(MinecraftLibrary library) throws IOException {
Expand All @@ -60,15 +66,30 @@ public Artifact get(MinecraftLibrary library) throws IOException {
throw new IllegalArgumentException("Cannot download a library that has no artifact defined: " + library);
}

// TODO: if we identify where the Minecraft installation is, we could try to copy the library from there

var artifactCoordinate = MavenCoordinate.parse(library.artifactId());
var externalArtifact = getFromExternalManifest(artifactCoordinate);
if (externalArtifact != null) {
return externalArtifact;
}

var finalLocation = artifactsCache.resolve(artifactCoordinate.toRelativeRepositoryPath());
var relativePath = artifactCoordinate.toRelativeRepositoryPath();

// Try reusing it from a local Minecraft installation, which ultimately is structured like a Maven repo
var localMinecraftLibraries = new ArrayList<>(launcherInstallations.getInstallationRoots());
for (var localRepo : localMinecraftLibraries) {
var localPath = localRepo.resolve("libraries").resolve(relativePath);
try {
// Ensure the file matches before using it
var fileHash = HashingUtil.hashFile(localPath, artifact.checksumAlgorithm());
if (Objects.equals(fileHash, artifact.checksum())) {
return getArtifactFromPath(localPath);
}
} catch (IOException ignored) {
// Ignore if it doesn't exist or is otherwise fails to be read
}
}

var finalLocation = artifactsCache.resolve(relativePath);

return download(finalLocation, artifact);
}
Expand Down Expand Up @@ -150,6 +171,14 @@ public List<Path> resolveClasspath(Collection<ClasspathItem> classpathItems) thr
* Special purpose method to get the version manifest for a specific Minecraft version.
*/
public Artifact getVersionManifest(String minecraftVersion) throws IOException {
// Check local Minecraft launchers for a copy of it
for (var root : launcherInstallations.getInstallationRoots()) {
var localPath = root.resolve("versions").resolve(minecraftVersion).resolve(minecraftVersion + ".json");
if (Files.isReadable(localPath)) {
return getArtifactFromPath(localPath);
}
}

var finalLocation = artifactsCache.resolve("minecraft_" + minecraftVersion + "_version_manifest.json");
return download(finalLocation, () -> {
var launcherManifestArtifact = getLauncherManifest();
Expand All @@ -168,6 +197,10 @@ public Artifact getVersionManifest(String minecraftVersion) throws IOException {
* Gets the v2 Launcher Manifest.
*/
public Artifact getLauncherManifest() throws IOException {

// Note that we're not reusing launcher manifests from known launcher installations,
// since we don't know how old they are

var finalLocation = artifactsCache.resolve("minecraft_launcher_manifest.json");

downloadManager.download(DownloadSpec.of(launcherManifestUrl), finalLocation);
Expand Down Expand Up @@ -228,10 +261,6 @@ private Artifact getArtifactFromPath(Path path) throws IOException {
return new Artifact(path, attrView.lastModifiedTime().toMillis(), attrView.size());
}

public DownloadManager getDownloadManager() {
return downloadManager;
}

@FunctionalInterface
public interface DownloadAction {
void run() throws IOException;
Expand Down Expand Up @@ -285,10 +314,6 @@ private Artifact download(Path finalLocation, DownloadSpec spec) throws IOExcept
return download(finalLocation, () -> downloadManager.download(spec, finalLocation));
}

public boolean isWarnOnArtifactManifestMiss() {
return warnOnArtifactManifestMiss;
}

public void setWarnOnArtifactManifestMiss(boolean warnOnArtifactManifestMiss) {
this.warnOnArtifactManifestMiss = warnOnArtifactManifestMiss;
}
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,162 @@
package net.neoforged.neoform.runtime.cache;

import net.neoforged.neoform.runtime.utils.AnsiColor;
import net.neoforged.neoform.runtime.utils.Logger;
import org.jetbrains.annotations.Nullable;

import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.util.ArrayList;
import java.util.List;
import java.util.regex.Matcher;
import java.util.regex.Pattern;

/**
* Searches for installation locations of the Minecraft launcher.
*/
public class LauncherInstallations {
private static final Logger LOG = Logger.create();

// Some are sourced from:
// https://github.com/SpongePowered/VanillaGradle/blob/ccc45765d9881747b2c922be7a13c453c32ce9ed/subprojects/gradle-plugin/src/main/java/org/spongepowered/gradle/vanilla/internal/Constants.java#L61-L71
// Placeholders of the form ${variable} are interpreted as system properties.
// If the variable starts with "env.", it is sourced from environment variables.
private static final String[] CANDIDATES = {
"${env.APPDATA}/.minecraft/", // Windows, default launcher
"${user.home}/.minecraft/", // linux, default launcher
"${user.home}/Library/Application Support/minecraft/", // macOS, default launcher
"${user.home}/curseforge/minecraft/Install/", // Windows, Curseforge Client
"${env.APPDATA}/com.modrinth.theseus/meta/", // Windows, Modrinth App
"${env.LOCALAPPDATA}/.ftba/bin/", // Windows, FTB App
"${user.home}/.local/share/PrismLauncher/", // linux, PrismLauncher
"${user.home}/.local/share/multimc/", // linux, MultiMC
"${user.home}/Library/Application Support/PrismLauncher/", // macOS, PrismLauncher
"${env.APPDATA}/PrismLauncher/", // Windows, PrismLauncher
"${user.home}/scoop/persist/multimc/", // Windows, MultiMC via Scoop
};
private static final Pattern PLACEHOLDER_PATTERN = Pattern.compile("\\$\\{([^}]+)}");

private final List<LauncherDirectory> launcherDirectories = new ArrayList<>();

/** If true, the scan was already performed and launcherDirectories is up-to-date */
private boolean scanned;

private boolean verbose;

public LauncherInstallations(List<Path> additionalLauncherDirs) {
for (var dir : additionalLauncherDirs) {
var launcherDir = analyzeLauncherDirectory(dir);
if (launcherDir != null) {
launcherDirectories.add(launcherDir);
}
}
}

public void setVerbose(boolean verbose) {
this.verbose = verbose;
}

/**
* Returns all installation roots that have been found.
* Please note that none of the directories or files you might
* expect might actually exist or be readable.
*/
public List<Path> getInstallationRoots() {
scanIfNecessary();

return launcherDirectories.stream().map(LauncherDirectory::directory).toList();
}

private void scanIfNecessary() {
if (scanned) {
return;
}
scanned = true;

// In CI, we can assume that the Minecraft launcher is not going to be installed.
// Skip scanning for it there.
// See: https://docs.gitlab.com/ee/ci/variables/predefined_variables.html
// See: https://docs.github.com/en/actions/learn-github-actions/variables#default-environment-variables
if ("true".equals(System.getenv("CI"))) {
if (verbose) {
LOG.println("Not scanning for Minecraft Launcher installations in CI");
}
return;
}

if (verbose) {
LOG.println("Scanning for Minecraft Launcher installations");
}

for (var candidate : CANDIDATES) {
var resolvedPath = resolvePlaceholders(candidate);
if (resolvedPath == null) {
continue;
}

try {
var result = analyzeLauncherDirectory(Paths.get(candidate));
if (result != null) {
launcherDirectories.add(result);
}
} catch (Exception e) {
if (verbose) {
LOG.println(" Failed to scan launcher directory " + candidate + ": " + e);
}
}
}

if (verbose) {
LOG.println("Launcher directories found:");
for (var launcherDirectory : launcherDirectories) {
LOG.println(AnsiColor.MUTED + " " + launcherDirectory.directory
+ AnsiColor.RESET);
}
}
}

@Nullable
private String resolvePlaceholders(String candidate) {
var matcher = PLACEHOLDER_PATTERN.matcher(candidate);

var unmatchedVariables = new ArrayList<String>();
candidate = matcher.replaceAll(match -> {
var variable = match.group(1);
String value;
if (variable.startsWith("env.")) {
value = System.getenv(variable.substring("env.".length()));
} else {
value = System.getProperty(variable);
}

if (value == null) {
unmatchedVariables.add(variable);
return "";
}
return Matcher.quoteReplacement(value);
});
if (!unmatchedVariables.isEmpty()) {
if (verbose) {
LOG.println(" Skipping candidate " + candidate + " due to undefined references: " + unmatchedVariables);
}
return null; // Ignoring due to unmatched variables
}
return candidate;
}

@Nullable
private LauncherDirectory analyzeLauncherDirectory(Path installDir) {
if (!Files.isDirectory(installDir)) {
if (verbose) {
LOG.println(AnsiColor.MUTED + " Not found: " + installDir + AnsiColor.RESET);
}
return null;
}

return new LauncherDirectory(installDir);
}

private record LauncherDirectory(Path directory) {
}
}
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@
package net.neoforged.neoform.runtime.cli;

import net.neoforged.neoform.runtime.cache.CacheManager;
import picocli.CommandLine;

import java.util.concurrent.Callable;
Expand All @@ -12,9 +11,7 @@ public class CacheMaintenance implements Callable<Integer> {

@Override
public Integer call() throws Exception {
try (var cacheManager = new CacheManager(commonOptions.homeDir, commonOptions.getWorkDir())) {
cacheManager.setVerbose(commonOptions.verbose);

try (var cacheManager = commonOptions.createCacheManager()) {
cacheManager.performMaintenance();
}

Expand Down
Loading

0 comments on commit 289c990

Please sign in to comment.