Over-triggering with rules_go and Gazelle under certain circumstances

[mikberg]

This issue is basically a better documented version of #70

Example repository: https://github.com/mikberg/bazel-diff-problem

This repository sets up rules_go with Gazelle, a go binary with an external dependency and some rule which depends on the WORKSPACE file. I think all three are important.

The second commit adds a comment to the WORKSPACE file, demonstrating that any change to WORKSPACE will now result in bazel-diff regarding all external Go dependencies as impacted. This presumably also is the reason why the go_binary is also regarded as impacted.

./bazel-diff-example.sh <path-to>/bazel-diff-test /usr/local/bin/bazelisk d075588217ec39bac4064dc7feb7026cd899a1b0 d6e1e901dcd5dd9e9b32c2ce575938946839055f
Generating Hashes for Revision 'd075588217ec39bac4064dc7feb7026cd899a1b0'
Generating Hashes for Revision 'd6e1e901dcd5dd9e9b32c2ce575938946839055f'
Determining Impacted Targets
Impacted Targets between d075588217ec39bac4064dc7feb7026cd899a1b0 and d6e1e901dcd5dd9e9b32c2ce575938946839055f:
//:project_lib //external:com_github_kr_pretty //external:com_github_bazelbuild_buildtools //external:in_gopkg_check_v1 //external:com_github_pelletier_go_toml //external:org_golang_x_mod //external:bazel_gazelle_go_repository_config //external:com_github_fsnotify_fsnotify //external:com_github_rs_zerolog //:some-script //external:in_gopkg_yaml_v2 //external:com_github_bazelbuild_rules_go //external:com_github_kr_text //:project //external:com_github_davecgh_go_spew //external:com_github_bmatcuk_doublestar //external:org_golang_x_net //external:org_golang_x_text //:WORKSPACE //external:com_github_kr_pty //external:com_github_pmezard_go_difflib //external:org_golang_x_sync //external:com_github_burntsushi_toml //external:com_github_google_go_cmp //external:org_golang_x_crypto

I haven't been able to nail this down further.

As far as I can tell, this problem was introduced between 2.3.0 and 2.4.0.

Thank you for a fantastic tool!

[tinder-maxwellelliott]

Thank you for this great example repo. I will put together an integration test now in a draft PR

[tinder-maxwellelliott]

Alright my PR above captures the issue

[tinder-maxwellelliott]

Okay so here is the result of doing the diff manually, somehow all the external targets have changed

diff /tmp/starting.json /tmp/final.json              
2,3c2,3
<   "//external:com_github_kr_pretty": "c4e4c3bc011fa6d15dfd99fa463bd11765e8c654518313e428aa08bd64d82b0a",
<   "//external:com_github_bazelbuild_buildtools": "b8b5d58a377e5e4318f83a0faf8681a9d3620c676008a661e908613fe24a6ecc",
---
>   "//external:com_github_kr_pretty": "f3d19f7fee981d26d6859ff30e10c715d37f78dc823d6d2783df63461347b181",
>   "//external:com_github_bazelbuild_buildtools": "e3f000b9c665df29a67ab9f47e8b8fa7e644b14fd244f9f3a596e6a32d1c8856",
6c6
<   "//external:com_github_pelletier_go_toml": "1d9442e54b744300238e0590d7216c2ce34ec1c51516e50f422946a60810f71f",
---
>   "//external:com_github_pelletier_go_toml": "2131bbfc71e19722cbcfcd6262f349de139bd198dc1478e1078f77468aef5ac8",
11,12c11,12
<   "//external:com_github_fsnotify_fsnotify": "705f7f1800547b80625ed9550543f276b3cc1124f58a97a86e5d2c4fe0e10be3",
<   "//external:com_github_rs_zerolog": "1ae495c13ad68f81f8a12d15e4ca23b9e359e7e52dd51bf29b8e6abd3e45cb9e",
---
>   "//external:com_github_fsnotify_fsnotify": "78fadeb3f4f535b0d74b674b93769e6819fa89b68ab2f528a34148e95806ad5d",
>   "//external:com_github_rs_zerolog": "1028e56c28478575f8d3a64ffec25dc85d2469ef89b75ec425e1525823a99488",
15c15
<   "//external:in_gopkg_yaml_v2": "2b4d8cfde049e875c93e38528bdb7a249bff551b7ff4caf0547f07953b4409bb",
---
>   "//external:in_gopkg_yaml_v2": "152fc1a853ba006b778634f18bfd3c086e519a259bce6941ec731deb9ab835ec",
18c18
<   "//external:com_github_bazelbuild_rules_go": "3a0865bfddf75458c31485e4c305904c04cdb2df52c93986053ad4b24c30b2aa",
---
>   "//external:com_github_bazelbuild_rules_go": "edf5a21b54d40aeda6585964afb7a7fbcbe2c860ed67c726038d3bd6e23046f4",
22c22
<   "//external:com_github_kr_text": "e8d75b53525ddb2ec3ba29b3a4aaa827c204dac696a6762f0a23659939ec9220",
---
>   "//external:com_github_kr_text": "998574c1d7bdcdecd2c3dd8cb3afef39ab5a63427b5aff9fd24ee0970b3cdbe4",
27,28c27,28
<   "//external:com_github_davecgh_go_spew": "e3212ef7ab80aab70c7250597d5fee5ebe8e34e69369a804f55f4dbc07a3bea6",
<   "//external:org_golang_x_net": "31c6917695194619df8f9a9fdec4c17fa1f3c5a691e6ede28e97640dd708eed5",
---
>   "//external:com_github_davecgh_go_spew": "d86252a55c6032a749ecacc7c900719a3d14ab718e4325516b62b7529fdbc920",
>   "//external:org_golang_x_net": "764f0843cdc8367c7e789f724817fe441692a5fd53cc2a257dd109affa6bd09e",
32c32
<   "//external:org_golang_x_text": "40fa854ffc099e0c8d89b3ae50aea9c9bf40fc588b081dfa0923243e6867d010",
---
>   "//external:org_golang_x_text": "d5ce531a28c54e3682e2fc64d685cba107d7891dd15a6e7b65a6fcd416643870",
38,40c38,40
<   "//:WORKSPACE": "946f540400d0819f2a9560a5e1415b1a09b95a046106b8f23588b7a457d7de5b",
<   "//external:com_github_kr_pty": "09277840b3f11e949947b07314020a7579e7ead4fc0abd87729d778d7b273b71",
<   "//external:com_github_pmezard_go_difflib": "a334ff2164b1018f256343f6729efd976d1659777fea888dcf4a8820b0d8fd56",
---
>   "//:WORKSPACE": "ec1cc9b48b848154725a374f8fddb8948ed740618d8fd02c9cf3bf8b02faf718",
>   "//external:com_github_kr_pty": "3485edec5acd4d701fde18c7f897daeb1e8a86be2a384eb9990d3298f04f3da9",
>   "//external:com_github_pmezard_go_difflib": "b1300b1184c695f203e22e51ce6f9b1d5994e7a59c68ca93695c585f700381b8",
42c42
<   "//external:org_golang_x_sync": "e39921babe450e121cfad126dc06988762a65c0b4825168212099aa92435e269",
---
>   "//external:org_golang_x_sync": "ae56798306b072b1c3bde8df25915800b4c5964546bd0919624d59221b7ef2ae",
47,48c47,48
<   "//external:org_golang_x_crypto": "c4e38b3ab791ab1643a17495962211eb5c2278ef0241c44e6104d4844b4ad4c2",
<   "//:project_lib": "b10df648bbb657679c6ab4344992e50c56ccf1d1d45d75364a67fad0c737a742",
---
>   "//external:org_golang_x_crypto": "90c56911e12a985da508b84ebfb6e887bea61e12da102a0ebce44221509f5676",
>   "//:project_lib": "1fefe55c301fe583b399dd6b1b930ef8d0a4f3962ba4f1b1435e2770232cecd8",
50c50
<   "//external:in_gopkg_check_v1": "b8e2106e05b8f536647c5d7e827f12a7b711e1a0b53e4010821cdebbe3846d07",
---
>   "//external:in_gopkg_check_v1": "fbce66b424002962111fbd882f6d09ca8b1039f3fb9dffdc2d715a010e31e29e",
54c54
<   "//external:org_golang_x_mod": "25083e56a5d2fd373223f9752e32114c999959046f0db85b0a5841ae77d16532",
---
>   "//external:org_golang_x_mod": "8716352ae28861ccdc1f4bb1ae9a0839d7aaf224d76bd0d2782a37d84a19b0f7",
56c56
<   "//external:bazel_gazelle_go_repository_config": "4dbf6519f3eecdc7aafad3f4ccf4a8db35c90d5a5686b9625e943250ca993c42",
---
>   "//external:bazel_gazelle_go_repository_config": "81bb955050b1682e632e51f1c12a741e853df7d361a1c489f5fd581e3dad1bd4",
65c65
<   "//:some-script": "60564e9161cd8c233f139c1c337eabe54a6ec9f01348576e12a805abda6d1993",
---
>   "//:some-script": "d63f3f179cf609cc35ac5fb514c69ff2498a2aa1d55f56d9f5bb86abe76f0523",
72c72
<   "//:project": "bff5ea58ba16b1a8046fc7fadc4fa5aa4938f469650b2783c8901b9cd12704ae",
---
>   "//:project": "90dddabbd0ddc6609defa9161b796a401c3596fc5ec23c558a8e7489b15206c2",
81c81
<   "//external:com_github_bmatcuk_doublestar": "a7d16a635ff53fafc705aae4532399724acd8f385236b6e4e3b5e2ce1743a2a9",
---
>   "//external:com_github_bmatcuk_doublestar": "bf52aea7749804d168dfdc9a7259eb5c0b16a34f5835bf7065027c4e7ca4b504",
93,94c93,94
<   "//external:com_github_burntsushi_toml": "c04a2cccf68f0a6bffbe27264d859db5ff17e53db8ed9da68ae8c87b967f0426",
<   "//external:com_github_google_go_cmp": "0b67469f337ad2e7bf5e488aa2a1fda52faf50616d4d6b29c4daeaf2df4803ca",
---
>   "//external:com_github_burntsushi_toml": "5334099e2e590d474aae6b19d9c9e4fc370ea8b70dc686db9a8d587c2a43a13a",
>   "//external:com_github_google_go_cmp": "2f11107a60838e66150a634f619061902870db30d7d89cdae32c2b2e5592f77e",

[tinder-maxwellelliott]

@mikberg is there any way to debug the BUILD files that are automatically created for go deps in a Gazelle project? Trying to understand how we are getting differences

[mikberg]

@mikberg is there any way to debug the BUILD files that are automatically created for go deps in a Gazelle project? Trying to understand how we are getting differences

Hmm, you can at least bazel query @com_github_rs_zerolog//... --output=location to get the filesystem locations of the BUILD files.

[com6056]

This seems to also be impacting us, any debug info I can provide to help figure this out?

[cristifalcas]

I believe we are impacted as well because of this.
Between practically no changes between main and a test branch, we get a diff with hundred of targets

[com6056]

Curious if #174 will solve this, might give it a try and see 👀

[aw185176]

We are also running into this while trying to adopt bazel-diff. I looked into #174 and I don't think its quite right, since rules_go creates individual external repositories for each third party dependency. IIUC, #174 helps with situations where all external packages get put under one external repo namespace (eg @npm//..., etc).

I can confirm that a simple diff only touching WORKSPACE file produces a list of impacted targets that includes all external dependencies