Use native gummy bears type descriptors

When using the Android platform shorthand, use type descriptors in the native Expediter protobuf format (published by https://github.com/open-toast/gummy-bears). Animal Sniffer signatures are still supported via explicit configuration. Allow using Gradle dependencies (configurations) to define the platform APIs. This can be used to emulate the typical Animal Sniffer setup where only the project source code is validated against platform APIs. See the `android compat source only` integration test for an example.
open-toast · Nov 13, 2023 · 2d76403 · 2d76403
1 parent 1f16499
commit 2d76403
Show file tree

Hide file tree

Showing 12 changed files with 241 additions and 44 deletions.
diff --git a/README.md b/README.md
@@ -35,11 +35,10 @@ Conceptually, the inputs for running the tool are _application classes_ and _pla
 classes are the classes that need to be validated, and the platform APIs are the APIs provided by the runtime
 environment of the application, e.g. the JVM or the Android SDK.
 
-Typically, the application classes are the classes compiled directly by the project and its runtime dependencies,
-and the platform APIs are the classes provided by the JVM's platform classloader. However, platform APIs can
-also be specified as a set of AnimalSniffer signatures.
+Typically, the application classes are the classes compiled directly by the project and its runtime dependencies.
+The platform APIs can be specified by the JVM, a set of dependencies, or serialized type descriptors.
 
-## Setup
+## Basic setup
 
 The most convenient way to run this tool is via the provided Gradle plugin.
 
@@ -49,7 +48,7 @@ plugins {
 }
 ```
 
-### Application classes
+## Application classes
 
 By default, the application classes are the classes from the main source set and the runtime dependencies of the project.
 
@@ -58,17 +57,24 @@ You can customize this behavior, e.g. change the Gradle configuration that descr
 ```kotlin
 expediter {
     application {
+        sourceSet("<< something other than main >>")
         configuration("<< something other than runtimeClasspath >>")
     }
 }
 ```
 
-For example, in an Android project, you may want to use the `productionReleaseRuntime` configuration.
+For example, in an Android project, you will want to use a different configuration, such as `productionReleaseRuntime`.
+
+## Platform APIs
+
+Platform APIs can be provided by the JVM or specified as a set of dependencies or published type descriptors in the
+native [Expediter](proto/src/main/proto/toasttab/expediter/v1) or [Animal Sniffer](https://www.mojohaus.org/animal-sniffer/)
+format.
 
 ### JVM platform APIs
 
-By default, the platform APIs are defined by the JVM running the build. You can configure a specific version of the
-JVM instead. Then only the APIs provided by that specific version of the JVM will be included. 
+To use platform APIs provided by a specific version of the JVM, use the `jvmVersion` property. You need to run the build
+on a JVM version that is at least the specified `jvmVersion`.
 
 ```
 expediter {
@@ -78,13 +84,13 @@ expediter {
 }
 ```
 
-Setting an explicit `jvmVersion` is recommended unless checking Android compatibility or using custom signatures, see below.
+### Serialized type descriptors
 
-### Animal Sniffer and Android support
+You can use published type descriptors instead of, or in addition to, the JVM to describe platform APIs. 
 
-You can use AnimalSniffer signatures instead of, or in addition to, the current JVM to describe platform APIs. This is useful, for example, when checking compatibility with a specific Android SDK.
+This is useful, for example, when checking compatibility with a specific Android SDK.
 
-For Android compatibility, you may use a shorthand, which will set up the [Gummy Bears](https://github.com/open-toast/gummy-bears)-powered Android signatures.
+For Android compatibility, you may use a shorthand, which will set up the [Gummy Bears](https://github.com/open-toast/gummy-bears)-powered Android type descriptors.
 
 ```kotlin
 expediter {
@@ -94,23 +100,74 @@ expediter {
 }
 ```
 
-Alternatively, configure the AnimalSniffer signatures explicitly.
+Alternatively, configure the type descriptors explicitly.
 
 ```kotlin
 expediter {
     platform {
-        animalSnifferConfiguration("animalSniffer")
+        expediterConfiguration("_descriptors_")
     }
 }
 
-configurations.create("animalSniffer")
+configurations.create("_descriptors_")
 
 dependencies {
-    add("animalSniffer", "com.toasttab.android:gummy-bears-api-21:0.5.1@signature")
+    add("_descriptors_", "com.toasttab.android:gummy-bears-api-21:0.6.1@expediter")
 }
 ```
 
-### Run the task
+Expediter can also consume Animal Sniffer signatures
+
+```kotlin
+expediter {
+    platform {
+        animalSnifferConfiguration("_animal_sniffer_descriptors_")
+    }
+}
+
+configurations.create("_animal_sniffer_descriptors_")
+
+dependencies {
+    add("_animal_sniffer_descriptors_", "com.toasttab.android:gummy-bears-api-21:0.6.1@signature")
+}
+```
+
+### Dependencies as platform APIs
+
+In addition to type descriptors and the JVM, plain dependencies can be used to describe platform APIs.
+
+```kotlin
+expediter {
+    platform {
+        configuration("runtimeClasspath")
+    }
+}
+```
+
+This can be used to emulate the default behavior of Animal Sniffer, where only the project classes are validated
+against platform APIs, but dependencies are not.
+
+For example, to validate that a library's source code is compatible with a specific version of Android without
+validating the library's dependencies, use the setup below.
+
+```kotlin
+expediter {
+    application {
+        sourceSet("main")
+    }
+    platform {
+        androidSdk = 21
+        configuration("runtimeClasspath")
+    }
+}
+```
+
+### Fallback platform APIs
+
+If no platform APIs are explicitly specified, the Gradle plugin will fall back to using the platform classloader
+of the JVM running the build. This setup is rarely useful and will emit a warning.
+
+## Run the task
 
 Now you can run
 
@@ -120,7 +177,7 @@ Now you can run
 
 which will generate a report of binary incompatibilities under `build/expediter.json`.
 
-### Ignores
+## Ignores
 
 Typically, there is a lot of noise in the report from unused parts of libraries and missing optional dependencies.
 
@@ -135,7 +192,7 @@ expediter {
 }
 ```
 
-### Failing the build
+## Failing the build
 
 Once you tune the ignores, you can make the plugin fail the build if any issues are still found.
 

diff --git a/gradle/libs.versions.toml b/gradle/libs.versions.toml
@@ -7,6 +7,7 @@ nexus = "1.3.0"
 ktlint = "0.50.0"
 spotless = "6.21.0"
 protokt = "1.0.0-alpha.10"
+protobuf = "3.24.0"
 
 # test
 junit = "5.10.0"
@@ -23,6 +24,7 @@ kotlin-gradle = { module = "org.jetbrains.kotlin:kotlin-gradle-plugin", version.
 spotless = { module = "com.diffplug.spotless:spotless-plugin-gradle", version.ref = "spotless" }
 gradle-publish = { module = "com.gradle.publish:plugin-publish-plugin", version = "1.2.0" }
 protokt-extensions = { module = "com.toasttab.protokt:protokt-extensions", version.ref = "protokt" }
+protobuf-java = { module = "com.google.protobuf:protobuf-java", version.ref = "protobuf" }
 
 # test
 junit = { module = "org.junit.jupiter:junit-jupiter", version.ref = "junit" }

diff --git a/plugin/build.gradle.kts b/plugin/build.gradle.kts
@@ -8,6 +8,7 @@ plugins {
 dependencies {
     implementation(projects.core)
     implementation(projects.animalSnifferFormat)
+    implementation(libs.protobuf.java)
 
    testImplementation(libs.testkit.junit5)
 }

diff --git a/plugin/src/main/kotlin/com/toasttab/expediter/gradle/ExpediterExtension.kt b/plugin/src/main/kotlin/com/toasttab/expediter/gradle/ExpediterExtension.kt
@@ -19,7 +19,7 @@ import org.gradle.api.Action
 
 abstract class ExpediterExtension {
     var application: ApplicationClassSelector = ApplicationClassSelector(configuration = "runtimeClasspath", sourceSet = "main")
-    var platform: PlatformClassSelector = PlatformClassSelector(platformClassloader = true)
+    var platform: PlatformClassSelector = PlatformClassSelector()
 
     val ignoreSpec = IgnoreSpec()
 
@@ -34,7 +34,6 @@ abstract class ExpediterExtension {
     }
 
     fun platform(configure: Action<PlatformClassSelector>) {
-        platform = PlatformClassSelector(false)
         configure.execute(platform)
     }
 

diff --git a/plugin/src/main/kotlin/com/toasttab/expediter/gradle/ExpediterPlugin.kt b/plugin/src/main/kotlin/com/toasttab/expediter/gradle/ExpediterPlugin.kt
@@ -44,23 +44,31 @@ class ExpediterPlugin : Plugin<Project> {
 
             jvmVersion = extension.platform.jvmVersion
 
-            val animalSnifferConfigurations = extension.platform.animalSnifferConfigurations.toMutableList()
+            val expediterConfigurations = extension.platform.expediterConfigurations.toMutableList()
 
             if (extension.platform.androidSdk != null) {
-                val config = project.configurations.create("_expediter_animal_sniffer_")
+                val config = project.configurations.create("_expediter_type_descriptors_")
                 project.dependencies.add(
                     config.name,
-                    "com.toasttab.android:gummy-bears-api-${extension.platform.androidSdk}:0.5.1@signature"
+                    "com.toasttab.android:gummy-bears-api-${extension.platform.androidSdk}:0.6.0@expediter"
                 )
-                animalSnifferConfigurations.add(config.name)
+                expediterConfigurations.add(config.name)
+            }
+
+            for (conf in expediterConfigurations) {
+                typeDescriptors.from(project.configurations.getByName(conf))
+            }
+
+            for (conf in extension.platform.animalSnifferConfigurations) {
+                animalSnifferSignatures.from(project.configurations.getByName(conf))
             }
 
             if (extension.platform.jvmVersion != null && extension.platform.androidSdk != null) {
                 logger.warn("Both jvmVersion and androidSdk are set.")
             }
 
-            for (conf in animalSnifferConfigurations) {
-                animalSnifferSignatures.from(project.configurations.getByName(conf))
+            for (conf in extension.platform.configurations) {
+                platformArtifactCollection(selector.artifacts(conf))
             }
 
             ignore = extension.ignoreSpec.buildIgnore()

diff --git a/plugin/src/main/kotlin/com/toasttab/expediter/gradle/ExpediterTask.kt b/plugin/src/main/kotlin/com/toasttab/expediter/gradle/ExpediterTask.kt
@@ -16,7 +16,9 @@
 package com.toasttab.expediter.gradle
 
 import com.toasttab.expediter.ClasspathApplicationTypesProvider
+import com.toasttab.expediter.ClasspathScanner
 import com.toasttab.expediter.Expediter
+import com.toasttab.expediter.TypeParsers
 import com.toasttab.expediter.ignore.Ignore
 import com.toasttab.expediter.issue.IssueReport
 import com.toasttab.expediter.sniffer.AnimalSnifferParser
@@ -39,30 +41,39 @@ import org.gradle.api.tasks.OutputFile
 import org.gradle.api.tasks.PathSensitive
 import org.gradle.api.tasks.PathSensitivity
 import org.gradle.api.tasks.TaskAction
+import protokt.v1.toasttab.expediter.v1.TypeDescriptors
 import java.io.File
+import java.util.zip.GZIPInputStream
 
 @CacheableTask
 abstract class ExpediterTask : DefaultTask() {
-    private val configurationArtifacts = mutableListOf<ArtifactCollection>()
+    private val applicationConfigurationArtifacts = mutableListOf<ArtifactCollection>()
+    private val platformConfigurationArtifacts = mutableListOf<ArtifactCollection>()
 
     @get:InputFiles
     @get:PathSensitive(PathSensitivity.ABSOLUTE)
-    val artifacts: FileCollection get() {
-        return if (configurationArtifacts.isEmpty()) {
-            project.objects.fileCollection()
-        } else {
-            configurationArtifacts.map {
-                it.artifactFiles
-            }.reduce(FileCollection::plus)
-        }
+    val applicationArtifacts get() = applicationConfigurationArtifacts.asFileCollection()
+
+    @get:InputFiles
+    @get:PathSensitive(PathSensitivity.ABSOLUTE)
+    val platformArtifacts get() = platformConfigurationArtifacts.asFileCollection()
+
+    private fun Collection<ArtifactCollection>.asFileCollection() = if (isEmpty()) {
+        project.objects.fileCollection()
+    } else {
+        map { it.artifactFiles }.reduce(FileCollection::plus)
     }
 
     @get:InputFiles
     @get:PathSensitive(PathSensitivity.ABSOLUTE)
     abstract val files: ConfigurableFileCollection
 
     fun artifactCollection(artifactCollection: ArtifactCollection) {
-        configurationArtifacts.add(artifactCollection)
+        applicationConfigurationArtifacts.add(artifactCollection)
+    }
+
+    fun platformArtifactCollection(artifactCollection: ArtifactCollection) {
+        platformConfigurationArtifacts.add(artifactCollection)
     }
 
     @OutputFile
@@ -79,6 +90,10 @@ abstract class ExpediterTask : DefaultTask() {
     @get:PathSensitive(PathSensitivity.RELATIVE)
     abstract val animalSnifferSignatures: ConfigurableFileCollection
 
+    @get:InputFiles
+    @get:PathSensitive(PathSensitivity.RELATIVE)
+    abstract val typeDescriptors: ConfigurableFileCollection
+
     @InputFile
     @PathSensitive(PathSensitivity.RELATIVE)
     @Optional
@@ -95,13 +110,27 @@ abstract class ExpediterTask : DefaultTask() {
             providers.add(JvmTypeProvider.forTarget(it))
         }
 
+        if (!platformArtifacts.isEmpty) {
+            providers.add(
+                InMemoryPlatformTypeProvider(
+                    ClasspathScanner(platformArtifacts).scan { i, _ -> TypeParsers.typeDescriptor(i) }
+                )
+            )
+        }
+
         for (signaturesFile in animalSnifferSignatures) {
             signaturesFile.inputStream().buffered().use {
                 providers.add(InMemoryPlatformTypeProvider(AnimalSnifferParser.parse(it)))
             }
         }
 
-        if (jvmVersion == null && animalSnifferSignatures.isEmpty) {
+        for (descriptorFile in typeDescriptors) {
+            GZIPInputStream(descriptorFile.inputStream().buffered()).use {
+                providers.add(InMemoryPlatformTypeProvider(TypeDescriptors.deserialize(it).types))
+            }
+        }
+
+        if (jvmVersion == null && animalSnifferSignatures.isEmpty && typeDescriptors.isEmpty && platformArtifacts.isEmpty) {
             logger.warn("No platform APIs specified, falling back to the platform classloader of the current JVM.")
 
             providers.add(PlatformClassloaderTypeProvider)
@@ -115,7 +144,7 @@ abstract class ExpediterTask : DefaultTask() {
 
         val issues = Expediter(
             ignore,
-            ClasspathApplicationTypesProvider(artifacts + files),
+            ClasspathApplicationTypesProvider(applicationArtifacts + files),
             PlatformTypeProviderChain(
                 providers
             )

diff --git a/plugin/src/main/kotlin/com/toasttab/expediter/gradle/PlatformClassSelector.kt b/plugin/src/main/kotlin/com/toasttab/expediter/gradle/PlatformClassSelector.kt
@@ -16,12 +16,21 @@
 package com.toasttab.expediter.gradle
 
 class PlatformClassSelector(
-    var platformClassloader: Boolean,
     val animalSnifferConfigurations: MutableList<String> = mutableListOf(),
+    val expediterConfigurations: MutableList<String> = mutableListOf(),
+    val configurations: MutableList<String> = mutableListOf(),
     var androidSdk: Int? = null,
     var jvmVersion: Int? = null
 ) {
     fun animalSnifferConfiguration(configuration: String) {
         animalSnifferConfigurations.add(configuration)
     }
+
+    fun expediterConfiguration(configuration: String) {
+        expediterConfigurations.add(configuration)
+    }
+
+    fun configuration(configuration: String) {
+        configurations.add(configuration)
+    }
 }