Skip to content

Commit

Permalink
Authenticate with the build cache using an access token
Browse files Browse the repository at this point in the history
Closes gh-73
  • Loading branch information
wilkinsona committed Apr 25, 2024
1 parent 709fc65 commit 66623ed
Show file tree
Hide file tree
Showing 6 changed files with 113 additions and 84 deletions.
58 changes: 20 additions & 38 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,39 +7,18 @@ Conventions for Gradle projects that use the Gradle Enterprise instance hosted a
When applied, the conventions will configure the build cache to:

- Enable local caching.
- Use https://ge.spring.io/cache/ as the remote cache.
- Use https://ge.spring.io as the remote cache server.
- Enable pulling from the remote cache.
- Enable pushing to the remote cache if the required credentials are available.
- Enable pushing to the remote cache when a CI environment is detected and the required access token is available.

### Remote cache

#### URL

By default, https://ge.spring.io/cache/ will be used as the remote cache.
The URL can be configured using the `GRADLE_ENTERPRISE_CACHE_URL` environment variable.

#### Credentials

:rotating_light: **Credentials must not be configured in environments where pull requests are built.** :rotating_light:

Pushing to the remote cache requires authentication.
The necessary credentials can be provided using the `GRADLE_ENTERPRISE_CACHE_USERNAME` and `GRADLE_ENTERPRISE_CACHE_PASSWORD` environment variables.

#### Bamboo

The username and password environment variables should be set using `${bamboo.gradle_enterprise_cache_user}` and `${bamboo.gradle_enterprise_cache_password}` respectively.

#### Concourse

The username and password environment variables should be set using `((gradle_enterprise_cache_user.username))` and `((gradle_enterprise_cache_user.password))` from Vault respectively.

#### GitHub Actions

The username and password environment variables should be set using the `GRADLE_ENTERPRISE_CACHE_USER` and `GRADLE_ENTERPRISE_CACHE_PASSWORD` organization secrets respectively.

#### Jenkins

The username and password environment variables should be set using the `gradle_enterprise_cache_user` username with password credential.
By default, https://ge.spring.io will be used as the remote cache server.
The server can be configured using the `GRADLE_ENTERPRISE_CACHE_SERVER` environment variable.
For backwards compatibility, `GRADLE_ENTERPRISE_CACHE_URL` is also supported for a limited time.
`/cache/` is removed from the end of the URL and the remainder is used to configure the remote cache server.

## Build scan conventions

Expand All @@ -62,15 +41,24 @@ The build scans will be customized to:
- Enable capturing of file fingerprints
- Upload build scans in the foreground when running on CI

### Build scan publishing credentials
### Git branch names

:rotating_light: **Credentials must not be configured in environments where pull requests are built.** :rotating_light:
`git rev-parse --abbrev-ref HEAD` is used to determine the name of the current branch.
This does not work on Concourse as its git resource places the repository in a detached head state.
To work around this, an environment variable named `BRANCH` can be set on the task to provide the name of the branch.

Publishing to [ge.spring.io](https://ge.spring.io) requires authentication via an access key.
When running on CI, the access key should be made available via the `GRADLE_ENTERPRISE_ACCESS_KEY` environment variable.
### Anonymous publication

When using Gradle, build scans can be published anonymously to scans.gradle.com by running the build with `--scan`.

## Authentication

:rotating_light: **Credentials must not be configured in environments where pull requests are built.** :rotating_light:

Publishing build scans and pushing to the remote cache requires authentication via an access key.
Additionally, pushing to the remote cache also requires that a CI environment be detected.
When running on CI, the access key should be made available via the `GRADLE_ENTERPRISE_ACCESS_KEY` environment variable.

#### Bamboo

The environment variable should be set to `${bamboo.gradle_enterprise_secret_access_key}`.
Expand All @@ -91,13 +79,7 @@ The environment variable should be set using the `gradle_enterprise_secret_acces

An access key can be provisioned by running `./gradlew provisionGradleEnterpriseAccessKey` once the project has been configured to use this plugin.

### Git branch names

`git rev-parse --abbrev-ref HEAD` is used to determine the name of the current branch.
This does not work on Concourse as its git resource places the repository in a detached head state.
To work around this, an environment variable named `BRANCH` can be set on the task to provide the name of the branch.

### Detecting CI
## Detecting CI

Bamboo is detected by looking for an environment variable named `bamboo_resultsUrl`.

Expand Down
2 changes: 1 addition & 1 deletion gradle.properties
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
version=0.0.17-SNAPSHOT

gradleEnterprisePluginVersion=3.17
gradleEnterprisePluginVersion=3.17.2
javaFormatVersion=0.0.39
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
/*
* Copyright 2020-2022 the original author or authors.
* Copyright 2020-2024 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
Expand All @@ -18,24 +18,27 @@

import java.util.Map;

import com.gradle.develocity.agent.gradle.buildcache.DevelocityBuildCache;
import org.gradle.caching.configuration.BuildCacheConfiguration;
import org.gradle.caching.http.HttpBuildCache;

/**
* Conventions that are applied to the build cache for Maven and Gradle builds.
* Conventions that are applied to the build cache.
*
* @author Andy Wilkinson
*/
public class BuildCacheConventions {

private final Map<String, String> env;

public BuildCacheConventions() {
this(System.getenv());
private final Class<? extends DevelocityBuildCache> buildCacheType;

public BuildCacheConventions(Class<? extends DevelocityBuildCache> buildCache) {
this(buildCache, System.getenv());
}

BuildCacheConventions(Map<String, String> env) {
BuildCacheConventions(Class<? extends DevelocityBuildCache> buildCacheType, Map<String, String> env) {
this.env = env;
this.buildCacheType = buildCacheType;
}

/**
Expand All @@ -44,21 +47,35 @@ public BuildCacheConventions() {
*/
public void execute(BuildCacheConfiguration buildCache) {
buildCache.local((local) -> local.setEnabled(true));
buildCache.remote(HttpBuildCache.class, (remote) -> {
buildCache.remote(this.buildCacheType, (remote) -> {
remote.setEnabled(true);
remote.setUrl(this.env.getOrDefault("GRADLE_ENTERPRISE_CACHE_URL", "https://ge.spring.io/cache/"));
String username = this.env.get("GRADLE_ENTERPRISE_CACHE_USERNAME");
String password = this.env.get("GRADLE_ENTERPRISE_CACHE_PASSWORD");
if (hasText(username) && hasText(password)) {
String cacheServer = this.env.get("GRADLE_ENTERPRISE_CACHE_SERVER");
if (cacheServer == null) {
cacheServer = serverOfCacheUrl(this.env.get("GRADLE_ENTERPRISE_CACHE_URL"));
if (cacheServer == null) {
cacheServer = "https://ge.spring.io";
}
}
remote.setServer(cacheServer);
String accessKey = this.env.get("GRADLE_ENTERPRISE_ACCESS_KEY");
if (hasText(accessKey) && ContinuousIntegration.detect(this.env) != null) {
remote.setPush(true);
remote.credentials((credentials) -> {
credentials.setUsername(username);
credentials.setPassword(password);
});
}
});
}

private String serverOfCacheUrl(String cacheUrl) {
if (cacheUrl != null) {
if (cacheUrl.endsWith("/cache/")) {
return cacheUrl.substring(0, cacheUrl.length() - 7);
}
if (cacheUrl.endsWith("/cache")) {
return cacheUrl.substring(0, cacheUrl.length() - 6);
}
}
return null;
}

private boolean hasText(String string) {
return string != null && string.length() > 0;
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -45,14 +45,13 @@ public GradleEnterpriseConventionsPlugin(ProcessOperations processOperations) {

@Override
public void apply(Settings settings) {
settings.getPlugins().withType(DevelocityPlugin.class, (plugin) -> {
DevelocityConfiguration extension = settings.getExtensions().getByType(DevelocityConfiguration.class);
configureBuildScanConventions(extension, extension.getBuildScan(), settings.getStartParameter(),
settings.getRootDir());
});
DevelocityConfiguration extension = settings.getExtensions().getByType(DevelocityConfiguration.class);
settings.getPlugins()
.withType(DevelocityPlugin.class, (plugin) -> configureBuildScanConventions(extension,
extension.getBuildScan(), settings.getStartParameter(), settings.getRootDir()));
if (settings.getStartParameter().isBuildCacheEnabled()) {
settings
.buildCache((buildCacheConfiguration) -> new BuildCacheConventions().execute(buildCacheConfiguration));
settings.buildCache((buildCacheConfiguration) -> new BuildCacheConventions(extension.getBuildCache())
.execute(buildCacheConfiguration));
}
}

Expand Down
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
/*
* Copyright 2020-2022 the original author or authors.
* Copyright 2020-2024 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
Expand All @@ -16,17 +16,19 @@

package io.spring.ge.conventions.gradle;

import java.net.URI;
import java.util.Collections;
import java.util.HashMap;
import java.util.Map;

import com.gradle.develocity.agent.gradle.buildcache.DevelocityBuildCache;
import org.gradle.api.Action;
import org.gradle.caching.BuildCacheServiceFactory;
import org.gradle.caching.configuration.BuildCache;
import org.gradle.caching.configuration.BuildCacheConfiguration;
import org.gradle.caching.http.HttpBuildCache;
import org.gradle.caching.local.DirectoryBuildCache;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.params.ParameterizedTest;
import org.junit.jupiter.params.provider.ValueSource;

import static org.assertj.core.api.Assertions.assertThat;

Expand All @@ -41,44 +43,73 @@ class BuildCacheConventionsTests {

@Test
void localCacheIsEnabled() {
new BuildCacheConventions().execute(this.buildCache);
new BuildCacheConventions(DevelocityBuildCache.class).execute(this.buildCache);
assertThat(this.buildCache.local.isEnabled()).isTrue();
}

@Test
void remoteCacheIsEnabled() {
new BuildCacheConventions().execute(this.buildCache);
new BuildCacheConventions(DevelocityBuildCache.class).execute(this.buildCache);
assertThat(this.buildCache.remote.isEnabled()).isTrue();
assertThat(this.buildCache.remote.getUrl()).isEqualTo(URI.create("https://ge.spring.io/cache/"));
assertThat(this.buildCache.remote.getServer()).isEqualTo("https://ge.spring.io");
assertThat(this.buildCache.remote.isPush()).isFalse();
}

@ParameterizedTest
@ValueSource(strings = { "https://ge.example.com/cache/", "https://ge.example.com/cache" })
void remoteCacheUrlCanBeConfigured(String cacheUrl) {
Map<String, String> env = new HashMap<>();
env.put("GRADLE_ENTERPRISE_CACHE_URL", cacheUrl);
new BuildCacheConventions(DevelocityBuildCache.class, env).execute(this.buildCache);
assertThat(this.buildCache.remote.isEnabled()).isTrue();
assertThat(this.buildCache.remote.getServer()).isEqualTo("https://ge.example.com");
assertThat(this.buildCache.remote.isPush()).isFalse();
}

@Test
void remoteCacheUrlCanBeConfigured() {
void remoteCacheServerCanBeConfigured() {
Map<String, String> env = new HashMap<>();
env.put("GRADLE_ENTERPRISE_CACHE_URL", "https://ge.example.com/cache/");
new BuildCacheConventions(env).execute(this.buildCache);
env.put("GRADLE_ENTERPRISE_CACHE_SERVER", "https://ge.example.com");
new BuildCacheConventions(DevelocityBuildCache.class, env).execute(this.buildCache);
assertThat(this.buildCache.remote.isEnabled()).isTrue();
assertThat(this.buildCache.remote.getUrl()).isEqualTo(URI.create("https://ge.example.com/cache/"));
assertThat(this.buildCache.remote.getServer()).isEqualTo("https://ge.example.com");
assertThat(this.buildCache.remote.isPush()).isFalse();
}

@Test
void remoteCacheServerHasPrecedenceOverRemoteCacheUrl() {
Map<String, String> env = new HashMap<>();
env.put("GRADLE_ENTERPRISE_CACHE_URL", "https://ge-cache.example.com/cache/");
env.put("GRADLE_ENTERPRISE_CACHE_SERVER", "https://ge.example.com");
new BuildCacheConventions(DevelocityBuildCache.class, env).execute(this.buildCache);
assertThat(this.buildCache.remote.isEnabled()).isTrue();
assertThat(this.buildCache.remote.getServer()).isEqualTo("https://ge.example.com");
assertThat(this.buildCache.remote.isPush()).isFalse();
}

@Test
void whenAccessTokenIsProvidedInALocalEnvironmentThenPushingToTheRemoteCacheIsNotEnabled() {
new BuildCacheConventions(DevelocityBuildCache.class,
Collections.singletonMap("GRADLE_ENTERPRISE_ACCESS_KEY", "ge.example.com=a1b2c3d4"))
.execute(this.buildCache);
assertThat(this.buildCache.remote.isPush()).isFalse();
}

@Test
void whenCredentialsAreProvidedThenPushingToTheRemoteCacheIsEnabled() {
void whenAccessTokenIsProvidedInACiEnvironmentThenPushingToTheRemoteCacheIsNotEnabled() {
Map<String, String> env = new HashMap<>();
env.put("GRADLE_ENTERPRISE_CACHE_USERNAME", "user");
env.put("GRADLE_ENTERPRISE_CACHE_PASSWORD", "secret");
new BuildCacheConventions(env).execute(this.buildCache);
env.put("GRADLE_ENTERPRISE_ACCESS_KEY", "ge.example.com=a1b2c3d4");
env.put("CI", "true");
new BuildCacheConventions(DevelocityBuildCache.class, env).execute(this.buildCache);
assertThat(this.buildCache.remote.isPush()).isTrue();
assertThat(this.buildCache.remote.getCredentials().getUsername()).isEqualTo("user");
assertThat(this.buildCache.remote.getCredentials().getPassword()).isEqualTo("secret");
}

private static final class TestBuildCacheConfiguration implements BuildCacheConfiguration {

private final DirectoryBuildCache local = new DirectoryBuildCache();

private final HttpBuildCache remote = new HttpBuildCache();
private final DevelocityBuildCache remote = new DevelocityBuildCache() {
};

@Override
public DirectoryBuildCache getLocal() {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@ void whenThePluginIsAppliedThenBuildScanConventionsAreApplied(@TempDir File proj
void whenThePluginIsAppliedThenBuildCacheConventionsAreApplied(@TempDir File projectDir) {
prepareProject(projectDir);
BuildResult result = build(projectDir, "6.0.1", "verifyBuildCacheConfig");
assertThat(result.getOutput()).contains("Build cache remote: https://ge.spring.io/cache/");
assertThat(result.getOutput()).contains("Build cache server: https://ge.spring.io");
}

@Test
Expand Down Expand Up @@ -95,7 +95,7 @@ void whenThePluginIsAppliedAndScanIsSpecifiedThenServerIsNotCustomized(@TempDir
void whenThePluginIsAppliedAndBuildCacheIsDisabledThenBuildCacheConventionsAreNotApplied(@TempDir File projectDir) {
prepareProject(projectDir);
BuildResult result = build(projectDir, "6.0.1", "verifyBuildCacheConfig", "--no-build-cache");
assertThat(result.getOutput()).contains("Build cache remote: null");
assertThat(result.getOutput()).contains("Build cache server: null");
}

private void prepareProject(File projectDir) {
Expand All @@ -116,8 +116,8 @@ private void prepareProject(File projectDir) {
writer.println("}");
writer.println("task verifyBuildCacheConfig {");
writer.println(" doFirst {");
writer
.println(" println \"Build cache remote: ${project.ext['settings'].buildCache?.remote?.url}\"");
writer.println(
" println \"Build cache server: ${project.ext['settings'].buildCache?.remote?.server}\"");
writer.println(" }");
writer.println("}");
});
Expand All @@ -142,8 +142,8 @@ private void prepareMultiModuleProject(File projectDir) {
writer.println("}");
writer.println("task verifyBuildCacheConfig {");
writer.println(" doFirst {");
writer
.println(" println \"Build cache remote: ${project.ext['settings'].buildCache?.remote?.url}\"");
writer.println(
" println \"Build cache server: ${project.ext['settings'].buildCache?.remote?.server}\"");
writer.println(" }");
writer.println("}");
});
Expand Down

0 comments on commit 66623ed

Please sign in to comment.