Support incremental annotation processing

[stephanenicolas]

Gradle 4.7 provides support for incremental annotation processing.
https://docs.gradle.org/nightly/userguide/java_plugin.html#sec:incremental_annotation_processing

It would be nice to see auto (at least auto-value) supporting it. The change is not that big as auto value is an isolating AP, and even simpler as it doesn't handle inheritance.

[eamonnmcmanus]

We'd be happy to receive a PR from someone who has tested that this works. Just a couple of cautionary notes:

• The cited document says that processors "can't read resources". AutoValueProcessor does read resources, namely the template files it uses to generate code. I suspect that the document means that a processor can't read arbitrary resources, rather than these fixed ones.

• AutoValue does handle inheritance. For example the @AutoValue class can inherit abstract methods from a superclass or superinterface and those will be implemented. But I think we are still within the documented constraint "Must make all decisions about an element based on information reachable from its AST." The only exception I can think of is that we use javax.annotation.Generated or javax.annotation.processing.Generated depending on what is available. That's very unlikely to matter in practice.

[stephanenicolas]

Hi @eamonnmcmanus ,

you're right

1. this resource doesn't matter. All is about reproducibility of the processing. So a fixed resource is fine. It would be more about reading stuff that can change (updated data, time, stuff from internet..)
2. I didn't know that, cool then. Does auto value support inheritance across different modules already ? I mean if a subclass can inherit a super class in a different module, does all work well (and efficiently? )

They are different approaches to resolve the second point. A naive approach is simply to crawl up the hierarchy again and re process all annotations in super classes. That would work wonderfully here.
Some different approoaches try to optimize this process (for instance the generated code for a subclass can call the generated code from a super class) and they are harder to make compatible with modularization or incremental annotation processing.

[eamonnmcmanus]

The supertype processing is just used to determine what methods are inherited by the @AutoValue class. We don't allow one @AutoValue class to inherit from another, and even if we did you still couldn't inherit from (or otherwise reference) a generated AutoValue_Foo class in another module, so I don't think there are any concerns about modularity. As far as I can tell, everything should Just Work if we put the right thing in META-INF/gradle/incremental.annotation.processors.

[stephanenicolas]

I created a PR ^. It needs a little more testing before merging. Is there a sample somewhere using gradle that we can use to test this? Do you have any edge cases you would like to test?

I don't think it's possible to unit / integration (well, gradle uses spock for integration testing of this kind of features. But I am not sure we want to introduce this in auto...) test the incremental compilation due to the current state of incremental compilation in compile testing and also because it would require the gradle wrapper around compilation tasks to be used. But we can test that we are passing the right stuff to the filer when auto value generates a class. I used this technique in the second commit of this PR for butter knife: JakeWharton/butterknife#1230

[eamonnmcmanus]

I think manual testing would be fine. If you have a project using AutoValue where Gradle always recompiles it before the incremental.annotation.processors change, and does not recompile it after the change unless needed, that should be enough. I don't see any edge cases, and at worst we will sometimes recompile when we didn't have to.

It should be straightforward to make a simple project using the instructions here.

[stephanenicolas]

I will make one to make sure, but I would be much more confident after that to also add a unit test, just to make sure we still pass the right information to gradle.

[eamonnmcmanus]

I have released AutoValue 1.6.3rc1 which includes the code from @tbroyer for incremental support. I would appreciate it if someone with more Gradle experience than me could test it. I would also be very happy if someone could add a unit test, as @stephanenicolas suggests.

[anuraaga]

Hello - found this issue when looking for incremental support for AutoFactory. I notice value and service were updated with support, but is there a reason to leave out factory? It's my favorite processor of the three :)

[friedroman]

I've tested 1.6.3rc1 and gradle 4.10.2 reports
Full recompilation is required because com.google.auto.value.processor.AutoValueProcessor is not incremental.
I'm not using any extensions and reproduced it with minimal project setup.
Looking at the code and gradle documentation it seems supported option text in net.ltgt.gradle.incap.IncrementalAnnotationProcessorType is incorrect:
It is org.gradle.annotation.compile., but it should be org.gradle.annotation.processing. according to documentation

[ronshapiro]

@tbroyer ^

[ronshapiro]

That actually seems to be a separate bug - though I'm not even sure what those are for.

But inspecting the file, I see:

com.google.auto.value.extension.memoized.processor.MemoizedValidator,ISOLATING     
com.google.auto.value.processor.AutoAnnotationProcessor,ISOLATING                  
com.google.auto.value.processor.AutoOneOfProcessor,ISOLATING                       
com.google.auto.value.processor.AutoValueBuilderProcessor,ISOLATING                
com.google.auto.value.processor.AutoValueProcessor,DYNAMIC                         

Looks like we need a toLowerCase() in there. I've opened tbroyer/gradle-incap-helper#1 for that.

[tbroyer]

Looks like we need a toLowerCase() in there.

While it might be better to use lowercase, this wouldn't change Gradle behavior, as it toUpperCases the value anyway: https://github.com/gradle/gradle/blob/v4.10.2/subprojects/language-java/src/main/java/org/gradle/api/internal/tasks/compile/processing/AnnotationProcessorDetector.java#L188

But yes, the dynamic value (for AutoValueProcessor) returned in getSupportedOptions() is indeed wrong: https://github.com/gradle/gradle/blob/v4.10.2/subprojects/language-java/src/main/java/org/gradle/api/internal/tasks/compile/processing/IncrementalAnnotationProcessorType.java#L32 vs. https://github.com/tbroyer/gradle-incap-helper/blob/v0.1/lib/src/main/java/net/ltgt/gradle/incap/IncrementalAnnotationProcessorType.java#L49

Will fix and release v0.2 asap (and will try to add an integration test)

[tbroyer]

Version 0.2 published to a staging Maven repository: https://oss.sonatype.org/content/repositories/netltgt-1037, will add the test later.

@friedroman if you could test it to make sure it fixes the issue, then I'll release it to Maven Central:

1. Add the repository:

   repositories {
     // …
     maven { url "https://oss.sonatype.org/content/repositories/netltgt-1037" }
   }

2. Add the dependency before auto-value:

   dependencies {
      annotationProcessor "net.ltgt.incap:incap:0.2"
      annotationProcessor "com.google.auto.value:auto-value:0.6.3rc1"
   }

BTW @ronshapiro / @eamonnmcmanus, the dependency is bundled into the auto-value JAR but not shaded:

$ jar tf auto-value-1.6.3rc1.jar |grep incap
net/ltgt/gradle/incap/
net/ltgt/gradle/incap/IncrementalAnnotationProcessorType.class

[friedroman]

I have tested it and incremental compilation does indeed work correctly. Only I had to change
"annotationProcessor "net.ltgt.incap:incap:0.2" to
"annotationProcessor "net.ltgt.gradle.incap:incap:0.2"
Thank you and looking forward to using it once it's on Maven Central

[tbroyer]

I released 0.2 to Central (please note that it's different from the previous 0.2 in staging) and created #678 to update the dependency and correctly shade it.

[bardyamomeni]

Hi everybody.

I am using auto-value in Android with Gradle 4.7. After editing an auto-value class and compiling the sources, the incremental build does not fire up I get this message in the info log:

:app:compileDebugJavaWithJavac - is not incremental. The following annotation processors don't support incremental compilation:
        com.google.auto.value.processor.AutoValueProcessor (type: UNKNOWN)

my app module auto dependencies are as follows:

dependencies {
    annotationProcessor group: 'net.ltgt.gradle.incap', name: 'incap-processor', version: '0.2'
    annotationProcessor "net.ltgt.gradle.incap:incap:0.2"

    annotationProcessor group: 'com.google.auto.value', name: 'auto-value', version: '1.6.3rc1'

    api group: 'com.google.auto.value', name: 'auto-value-annotations', version: '1.6.3rc1'

   //other dependencies
}

I also use gson auto-value adapter and glide too.

[tbroyer]

@bardyamomeni "Dynamic" incremental processors was added in Gradle 4.8: https://docs.gradle.org/4.8/release-notes.html#improved-incremental-annotation-processing

Also, auto-value-gson is ready for incremental processing but this hasn't been released yet. And finally, glide is not incremental yet (bumptech/glide#2983), so it wouldn't really help your build to have auto-value be incremental.

[ReginFell]

Any updates on this?
Glide already supports incremental https://github.com/bumptech/glide/releases/tag/v4.9.0

[tbroyer]

@ReginFell What kind of update are you looking for?

AutoValue 1.6.4 (but you'll probably want to use 1.6.5) supports incremental annotation processing (1.6.3 should have, but had a bug).
(note: you'll need at least Gradle 4.8)

AutoService 1.0-rc5 is said to have support, but actually has a bug (see comment below)

AutoFactory will soon have support too (see #708 (comment))

[timurstrekalov]

@tbroyer I feel like I'm going crazy, but I can't see the incremental.annotation.processors file in the AutoService 1.0-rc5 jar, which correlates with the fact that Gradle reports it as not incremental.

[tbroyer]

Oh you're right, AutoService 1.0-rc5 has a bug, already fixed on master but not released yet: cf61cff

@ronshapiro Could you maybe do a new release of AutoService? (also possibly update the release notes for AutoService 1.0-rc5 to remove Gradle's incremental processing)

[ronshapiro]

Done @tbroyer!

[technoir42]

@ronshapiro Could you please make a new AutoFactory release too?

[emartynov]

Guys, what about auto factory?

[ZacSweers]

AutoFactory 1.0-beta7 was released 24 days ago, presumably with incremental support: https://github.com/google/auto/releases/tag/auto-factory-1.0-beta7. Have you tried that?

[carrino]

I just tried auto service rc6 and got the following error

Full recompilation is required because '@AutoService' has source retention. Aggregating annotation processors require class or runtime retention.

[carrino]

I also found this comment on the gradle project that seems related: gradle/gradle#9946 (comment)

[ZacSweers]

Can this issue be closed as complete now?

[carrino]

verify failed: I just tried auto service rc6 and got the following error

Full recompilation is required because '@autoservice' has source retention. Aggregating annotation processors require class or runtime retention.