Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Gradle bufBuild hang on newer version of Protobuf #172

Closed
jeffawx opened this issue Dec 25, 2023 · 5 comments
Closed

Gradle bufBuild hang on newer version of Protobuf #172

jeffawx opened this issue Dec 25, 2023 · 5 comments

Comments

@jeffawx
Copy link

jeffawx commented Dec 25, 2023

The Gradle command hangs infinitely without any response: ./gradlew clean bufBuild --no-build-cache

This only happens after I upgraded protobuf-java from version 3.24.4 to 3.25.1

My project info:

  • uses protobuf-gradle-plugin but no explicit workspace
  • Only has buf.yaml with content:
version: v1
breaking:
  use:
    - WIRE
  ignore:
    - google
lint:
  use:
    - DEFAULT
  except:
    - RPC_REQUEST_RESPONSE_UNIQUE
    - RPC_REQUEST_STANDARD_NAME
    - RPC_RESPONSE_STANDARD_NAME
  • In gradle, buf is configured as:
buf {
    publishSchema = true
}

Tried both plugin version 0.8.3 and newest 0.8.6, as long as I upgraded project Protobuf version it started to hang.
@andrewparmet
Copy link
Collaborator

Can you provide a reproducer? I can upgrade the version of protobuf-java in the plugin's integration tests and they continue to run without issue.

@jeffawx
Copy link
Author

jeffawx commented Dec 26, 2023

Can you provide a reproducer? I can upgrade the version of protobuf-java in the plugin's integration tests and they continue to run without issue.

hi Andrew, please check this simple setup for reproducing the issue:

https://github.com/jeffawx/buf-proto-issue

I have put details in the README, one thing to correct: this is only a problem for protobuf-kotlin which has transitive dependency of protobuf-java

@gavvvr
Copy link

gavvvr commented Feb 15, 2024

Hi @andrewparmet

I can confirm the issue. The :bufBuild hangs forever if 3 conditions are met:

  • there is a protobuf-kotlin in dependencies { ... }
  • protobuf-java and protobuf-kotlin dependencies are 3.25.0+
  • there are is a Google's protobuf Gradle plugin in this build

Here is the minimal build.gradle.kts to reproduce the problem:

plugins {
    id("java")
    id("com.google.protobuf") version "0.9.4"
    id("build.buf") version "0.9.0"
}

repositories {
    mavenCentral()
}

val protobufJvm = "3.25.0"
dependencies {
    implementation("com.google.protobuf:protobuf-java:$protobufJvm")
    implementation("com.google.protobuf:protobuf-kotlin:$protobufJvm")
}

buf {
    toolVersion = "1.29.0"
}

Running Gradle with --debug reveals the following repeated again and again:

Click me 
2024-02-15T18:55:08.037+0300 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Waiting to acquire shared lock on daemon addresses registry.
2024-02-15T18:55:08.037+0300 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Lock acquired on daemon addresses registry.
2024-02-15T18:55:08.037+0300 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Releasing lock on daemon addresses registry.
2024-02-15T18:55:14.333+0300 [DEBUG] [org.gradle.internal.resources.AbstractTrackedResourceLock] Daemon worker Thread 3: acquired lock on worker lease
2024-02-15T18:55:14.336+0300 [DEBUG] [org.gradle.internal.resources.AbstractTrackedResourceLock] Daemon worker Thread 3: released lock on worker lease
2024-02-15T18:55:14.336+0300 [DEBUG] [org.gradle.internal.resources.AbstractTrackedResourceLock] Execution worker Thread 9: acquired lock on worker lease
2024-02-15T18:55:14.336+0300 [DEBUG] [org.gradle.internal.resources.AbstractTrackedResourceLock] Execution worker Thread 9: released lock on worker lease
2024-02-15T18:55:14.336+0300 [DEBUG] [org.gradle.internal.resources.AbstractTrackedResourceLock] Execution worker: acquired lock on worker lease
2024-02-15T18:55:14.337+0300 [DEBUG] [org.gradle.internal.resources.AbstractTrackedResourceLock] Execution worker: released lock on worker lease
2024-02-15T18:55:14.337+0300 [DEBUG] [org.gradle.internal.resources.AbstractTrackedResourceLock] Execution worker Thread 7: acquired lock on worker lease
2024-02-15T18:55:14.337+0300 [DEBUG] [org.gradle.internal.resources.AbstractTrackedResourceLock] Execution worker Thread 7: released lock on worker lease
2024-02-15T18:55:14.337+0300 [DEBUG] [org.gradle.internal.resources.AbstractTrackedResourceLock] Execution worker Thread 8: acquired lock on worker lease
2024-02-15T18:55:14.337+0300 [DEBUG] [org.gradle.internal.resources.AbstractTrackedResourceLock] Execution worker Thread 8: released lock on worker lease
2024-02-15T18:55:14.337+0300 [DEBUG] [org.gradle.internal.resources.AbstractTrackedResourceLock] included builds: acquired lock on worker lease
2024-02-15T18:55:14.337+0300 [DEBUG] [org.gradle.internal.resources.AbstractTrackedResourceLock] included builds: released lock on worker lease
2024-02-15T18:55:14.337+0300 [DEBUG] [org.gradle.internal.resources.AbstractTrackedResourceLock] Execution worker Thread 3: acquired lock on worker lease
2024-02-15T18:55:14.337+0300 [DEBUG] [org.gradle.internal.resources.AbstractTrackedResourceLock] Execution worker Thread 3: released lock on worker lease
2024-02-15T18:55:14.337+0300 [DEBUG] [org.gradle.internal.resources.AbstractTrackedResourceLock] Execution worker Thread 2: acquired lock on worker lease
2024-02-15T18:55:14.338+0300 [DEBUG] [org.gradle.internal.resources.AbstractTrackedResourceLock] Execution worker Thread 2: released lock on worker lease
2024-02-15T18:55:14.338+0300 [DEBUG] [org.gradle.internal.resources.AbstractTrackedResourceLock] Execution worker Thread 6: acquired lock on worker lease
2024-02-15T18:55:14.338+0300 [DEBUG] [org.gradle.internal.resources.AbstractTrackedResourceLock] Execution worker Thread 6: released lock on worker lease
2024-02-15T18:55:14.338+0300 [DEBUG] [org.gradle.internal.resources.AbstractTrackedResourceLock] Execution worker Thread 5: acquired lock on worker lease
2024-02-15T18:55:14.338+0300 [DEBUG] [org.gradle.internal.resources.AbstractTrackedResourceLock] Execution worker Thread 5: released lock on worker lease
2024-02-15T18:55:18.034+0300 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Waiting to acquire shared lock on daemon addresses registry.
2024-02-15T18:55:18.035+0300 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Lock acquired on daemon addresses registry.
2024-02-15T18:55:18.035+0300 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Releasing lock on daemon addresses registry.
2024-02-15T18:55:18.036+0300 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Waiting to acquire shared lock on daemon addresses registry.
2024-02-15T18:55:18.036+0300 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Lock acquired on daemon addresses registry.
2024-02-15T18:55:18.036+0300 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Releasing lock on daemon addresses registry.
2024-02-15T18:55:28.036+0300 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Waiting to acquire shared lock on daemon addresses registry.
2024-02-15T18:55:28.037+0300 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Lock acquired on daemon addresses registry.
2024-02-15T18:55:28.038+0300 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Releasing lock on daemon addresses registry.
2024-02-15T18:55:28.039+0300 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Waiting to acquire shared lock on daemon addresses registry.
2024-02-15T18:55:28.039+0300 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Lock acquired on daemon addresses registry.
2024-02-15T18:55:28.039+0300 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Releasing lock on daemon addresses registry.
<===========--> 88% EXECUTING [1m 48s]

@berksean
Copy link
Contributor

berksean commented Feb 21, 2024 •
edited
Loading

Note: this seems to be a problem on the protobuf-kotlin side. For some reason they included the wellknown types in the artifact under the src/google/protobuf directory (which seems wrong). This causes duplicate definitions for the wellknown types (with those imported by protobuf-java) and causes bufBuild to hang.

Running the command manually fails immediately showing logs indicating duplicate definitions. Buf could also probably handle this more nicely by not failing.

For what it's worth -- I've handled this locally by adding a task to manually strip the superfluous files. This issue is open on the protobuf-kotlin side.

@srikrsna-buf
Copy link
Member

Closing this based on @berksean's comment

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@gavvvr @andrewparmet @jeffawx @srikrsna-buf @berksean