Skip to content

Commit 758ce61

Browse files
tinder-maxwellelliotttinder-maxwellelliott
committed
v1.0
0 parents  commit 758ce61

23 files changed

+1435
-0
lines changed

‎.bazelrc

+1
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
run --show_loading_progress=false --show_progress=false --ui_event_filters='ERROR'

‎.github/workflows/ci.yaml

+28
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,28 @@
1+
name: CI
2+
3+
on:
4+
push:
5+
branches: [ master ]
6+
pull_request:
7+
branches: [ master ]
8+
9+
jobs:
10+
build:
11+
runs-on: ubuntu-latest
12+
13+
steps:
14+
- name: Setup Java JDK
15+
uses: actions/[email protected]
16+
with:
17+
java-version: 1.8
18+
id: java
19+
- name: Setup Go environment
20+
uses: actions/[email protected]
21+
with:
22+
go-version: ^1.14
23+
id: go
24+
- name: Setup Bazelisk
25+
run: go get github.com/bazelbuild/bazelisk
26+
- uses: actions/checkout@v2
27+
- name: Run bazel-diff tests
28+
run: ~/go/bin/bazelisk test //test/...

‎.gitignore

+28
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,28 @@
1+
# Ignore backup files.
2+
*~
3+
# Ignore Vim swap files.
4+
.*.swp
5+
# Ignore files generated by IDEs.
6+
/.classpath
7+
/.factorypath
8+
/.idea/
9+
/.ijwb/
10+
/.project
11+
/.settings
12+
/.vscode/
13+
/bazel.iml
14+
*.iml
15+
# Ignore all bazel-* symlinks. There is no full list since this can change
16+
# based on the name of the directory bazel is cloned into.
17+
/bazel-*
18+
/out
19+
bazel-*
20+
out
21+
# Ignore outputs generated during Bazel bootstrapping.
22+
/output/
23+
# Bazelisk version file
24+
.bazelversion
25+
# User-specific .bazelrc
26+
user.bazelrc
27+
!bazel-diff-example.sh
28+
.DS_Store

‎BUILD

+4
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
alias(
2+
name = "bazel-diff",
3+
actual = "//src/main/java/com/bazel-diff:bazel-diff"
4+
)

‎LICENSE

+24
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
Copyright (c) 2020, Match Group, LLC
2+
All rights reserved.
3+
4+
Redistribution and use in source and binary forms, with or without
5+
modification, are permitted provided that the following conditions are met:
6+
* Redistributions of source code must retain the above copyright
7+
notice, this list of conditions and the following disclaimer.
8+
* Redistributions in binary form must reproduce the above copyright
9+
notice, this list of conditions and the following disclaimer in the
10+
documentation and/or other materials provided with the distribution.
11+
* Neither the name of Match Group, LLC nor the names of its contributors
12+
may be used to endorse or promote products derived from this software
13+
without specific prior written permission.
14+
15+
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
16+
ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
17+
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
18+
DISCLAIMED. IN NO EVENT SHALL MATCH GROUP, LLC BE LIABLE FOR ANY
19+
DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
20+
(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
21+
LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
22+
ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
23+
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
24+
SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

‎README.md

+257
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,257 @@
1+
# bazel-diff
2+
3+
`bazel-diff` is a command line tool for Bazel projects that allows users to determine the exact affected set of impacted targets between two Git revisions. Using this set, users can test or build the exact modified set of targets.
4+
5+
`bazel-diff` offers several key advantages over rolling your own target diffing solution
6+
7+
1. `bazel-diff` is designed for very large Bazel projects. We use Java Protobuf's `parseDelimitedFrom` method alongside Bazel Query's `streamed_proto` output option. These two together allow you to parse Gigabyte or larger protobuf messages. We have tested it with projects containing tens of thousands of targets.
8+
2. We avoid usage of large command line query lists when interacting with Bazel, [issue here](https://github.com/bazelbuild/bazel/issues/8609). When you interact with Bazel with thousands of query parameters you can reach an upper maximum limit, seeing this error:
9+
10+
bash: /usr/local/bin/bazel: Argument list too long
11+
12+
`bazel-diff` is smart enough to prevent these errors
13+
3. `bazel-diff` has been tested with file renames, deletions, and modifications. Works on `bzl` files, `WORKSPACE` files, `BUILD` files and regular files
14+
15+
Track the feature request for target diffing in Bazel [here](https://github.com/bazelbuild/bazel/issues/7962)
16+
17+
This approach was inspired by the [following BazelConf talk](https://www.youtube.com/watch?v=9Dk7mtIm7_A) by Benjamin Peterson.
18+
19+
### Prerequisites
20+
21+
* Git
22+
* Bazel 3.3.0 or higher
23+
* `bazel_skylib` version 1.0.2 or higher
24+
* Java 8 JDK or higher (Bazel requires this)
25+
26+
## Getting Started
27+
28+
To start using `bazel-diff` immediately, simply clone down the repo and then run the example shell script:
29+
30+
git clone https://github.com/Tinder/bazel-diff.git
31+
cd bazel-diff
32+
./bazel-diff-example.sh WORKSPACE_PATH BAZEL_PATH START_GIT_REVISION END_GIT_REVISION
33+
34+
Here is a breakdown of those arguments:
35+
* `WORKSPACE_PATH`: Path to directory containing your `WORKSPACE` file in your Bazel project.
36+
* Note: Your project must use Git for `bazel-diff` to work!
37+
* `BAZEL_PATH`: Path to your Bazel executable
38+
* `START_GIT_REVISION`: Starting Git Branch or SHA for your desired commit range
39+
* `END_GIT_REVISION`: Final Git Branch or SHA for your desired commit range
40+
41+
You can see the example shell script in action below:
42+
43+
![Demo](demo.gif)
44+
45+
Open `bazel-diff-example.sh` to see how this is implemented. This is purely an example use-case, but it is a great starting point to using `bazel-diff`.
46+
47+
## How it works
48+
49+
`bazel-diff` works as follows
50+
* The `modified-filepaths` command is used to fetch the modified filepaths between two Git revisions, we write this list of modified filepaths to a file.
51+
52+
* The previous revision is checked out, then we run `generate-hashes`. This gives us the hashmap representation for the entire Bazel graph, then we write this JSON to a file. We do not pass `modified-filepaths` here since it is the starting revision.
53+
54+
* Next we checkout the initial revision, then we run `generate-hashes` with the file output of `modified-filepaths` and write that JSON to a file. Now we have our final hashmap representation for the Bazel graph.
55+
56+
* We run `bazel-diff` on the starting and final JSON hash filepaths to get our affected set of targets. This impacted set of targets is written to a file
57+
58+
* Finally we run `impacted-tests` with the filepath to the list of impacted targets. This returns us the impacted test targets.
59+
60+
## CLI Interface
61+
62+
`bazel-diff` Command
63+
~~~
64+
Usage: bazel-diff [-hV] -b=<bazelPath> [-fh=<finalHashesJSONPath>]
65+
[-o=<outputPath>] [-sh=<startingHashesJSONPath>]
66+
-w=<workspacePath> [COMMAND]
67+
Writes to a file the impacted targets between two Bazel graph JSON files
68+
-b, --bazelPath=<bazelPath>
69+
Path to Bazel binary
70+
-fh, --finalHashes=<finalHashesJSONPath>
71+
The path to the JSON file of target hashes for the final
72+
revision. Run 'generate-hashes' to get this value.
73+
-h, --help Show this help message and exit.
74+
-o, --output=<outputPath>
75+
Filepath to write the impacted Bazel targets to, newline
76+
separated
77+
-sh, --startingHashes=<startingHashesJSONPath>
78+
The path to the JSON file of target hashes for the initial
79+
revision. Run 'generate-hashes' to get this value.
80+
-V, --version Print version information and exit.
81+
-w, --workspacePath=<workspacePath>
82+
Path to Bazel workspace directory.
83+
~~~
84+
85+
`impacted-tests` Command
86+
~~~
87+
Usage: bazel-diff impacted-tests [-hV] -b=<bazelPath> -w=<workspacePath>
88+
<impactedBazelTargetsPath> <outputPath>
89+
Write to a file the impacted test targets for the list of Bazel targets in the
90+
provided file
91+
<impactedBazelTargetsPath>
92+
The filepath to a newline separated list of Bazel targets
93+
<outputPath> The filepath to write the impacted test targets to
94+
-b, --bazelPath=<bazelPath>
95+
Path to Bazel binary
96+
-h, --help Show this help message and exit.
97+
-V, --version Print version information and exit.
98+
-w, --workspacePath=<workspacePath>
99+
Path to Bazel workspace directory.
100+
~~~
101+
102+
`modified-filepaths` Command
103+
~~~
104+
Usage: bazel-diff modified-filepaths [-hV] -b=<bazelPath> -w=<workspacePath>
105+
<startingGitRevision> <endingGitRevision>
106+
<outputPath>
107+
Writes to the file the modified filepaths between two revisions.
108+
<startingGitRevision> The starting Git revision, e.g. "HEAD^"
109+
<endingGitRevision> The final Git revision, e.g. "HEAD"
110+
<outputPath> Path that the list of modified files will be
111+
written to
112+
-b, --bazelPath=<bazelPath>
113+
Path to Bazel binary
114+
-h, --help Show this help message and exit.
115+
-V, --version Print version information and exit.
116+
-w, --workspacePath=<workspacePath>
117+
Path to Bazel workspace directory.
118+
~~~
119+
120+
`generate-hashes` Command
121+
~~~
122+
Usage: bazel-diff generate-hashes [-hV] -b=<bazelPath> [-m=<modifiedFilepaths>]
123+
-w=<workspacePath> <outputPath>
124+
Writes to a file the SHA256 hashes for each Bazel Target in the provided
125+
workspace.
126+
<outputPath> The filepath to write the resulting JSON of dictionary
127+
target => SHA-256 values
128+
-b, --bazelPath=<bazelPath>
129+
Path to Bazel binary
130+
-h, --help Show this help message and exit.
131+
-m, --modifiedFilepaths=<modifiedFilepaths>
132+
The path to a file containing the list of modified
133+
filepaths in the workspace, you can use the
134+
'modified-filepaths' command to get this list
135+
-V, --version Print version information and exit.
136+
-w, --workspacePath=<workspacePath>
137+
Path to Bazel workspace directory.
138+
~~~
139+
140+
## Installing
141+
142+
### Run from Source
143+
144+
After cloning down the repo, you are good to go, Bazel will handle the rest
145+
146+
To run the project
147+
148+
```
149+
bazel run :bazel-diff -- bazel-diff -h
150+
```
151+
152+
### Run Via JAR Release
153+
154+
curl -LO bazel-diff.jar GITHUB_RELEASE_JAR_URL
155+
java -jar bazel-diff.jar -h
156+
157+
### Build your own deployable JAR
158+
159+
```
160+
bazel build //src/main/java/com/bazel-diff:bazel-diff_deploy.jar
161+
java -jar bazel-bin/src/main/java/com/bazel-diff/bazel-diff_deploy.jar # This JAR can be run anywhere
162+
```
163+
164+
### Integrate directly into your Bazel Project
165+
166+
Add the following to your `WORKSPACE` file to add the external repositories, replacing the `RELEASE_ARCHIVE_URL` with the archive url of the bazel-diff release you wish to depend on:
167+
168+
load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive", "http_file")
169+
170+
http_archive(
171+
name = "bazel_diff",
172+
urls = [
173+
"RELEASE_ARCHIVE_URL",
174+
],
175+
sha256 = "UPDATE_ME",
176+
)
177+
178+
load("@bazel_diff//:repositories.bzl", "bazel_diff_dependencies")
179+
load("@bazel_diff//:constants.bzl", "bazel_diff_dependencies")
180+
181+
http_archive(
182+
name = "bazel_skylib",
183+
urls = [
184+
"ARCHIVE_URL_VERSION_ABOVE_1.0.2",
185+
],
186+
sha256 = "UPDATE_ME",
187+
)
188+
189+
load("@bazel_skylib//:workspace.bzl", "bazel_skylib_workspace")
190+
191+
bazel_skylib_workspace()
192+
193+
load("@bazel_skylib//lib:versions.bzl", "versions")
194+
195+
load("@bazel_diff//:repositories.bzl", "bazel_diff_dependencies")
196+
197+
# Allows bazel-diff to run based on your projects' version of Bazel
198+
bazel_diff_dependencies(versions.get())
199+
200+
load("@rules_jvm_external//:defs.bzl", "maven_install")
201+
load("@bazel_diff//:constants.bzl", "BAZEL_DIFF_MAVEN_ARTIFACTS")
202+
203+
maven_install(
204+
name = "bazel_diff_maven",
205+
artifacts = BAZEL_DIFF_MAVEN_ARTIFACTS,
206+
repositories = [
207+
"http://uk.maven.org/maven2",
208+
"https://jcenter.bintray.com/",
209+
],
210+
)
211+
212+
Now you can simply run `bazel-diff` from your project:
213+
214+
```
215+
bazel run @bazel_diff//:bazel-diff -- bazel-diff -h
216+
```
217+
218+
## Running the tests
219+
220+
To run the tests simply run
221+
222+
```
223+
bazel test //test/...
224+
```
225+
226+
## Versioning
227+
228+
We use [SemVer](http://semver.org/) for versioning. For the versions available,
229+
see the [tags on this repository](https://github.com/Tinder/bazel-diff/tags).
230+
231+
License
232+
---
233+
~~~
234+
Copyright (c) 2020, Match Group, LLC
235+
All rights reserved.
236+
237+
Redistribution and use in source and binary forms, with or without
238+
modification, are permitted provided that the following conditions are met:
239+
* Redistributions of source code must retain the above copyright
240+
notice, this list of conditions and the following disclaimer.
241+
* Redistributions in binary form must reproduce the above copyright
242+
notice, this list of conditions and the following disclaimer in the
243+
documentation and/or other materials provided with the distribution.
244+
* Neither the name of Match Group, LLC nor the names of its contributors
245+
may be used to endorse or promote products derived from this software
246+
without specific prior written permission.
247+
248+
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
249+
ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
250+
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
251+
DISCLAIMED. IN NO EVENT SHALL MATCH GROUP, LLC BE LIABLE FOR ANY
252+
DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
253+
(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
254+
LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
255+
ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
256+
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
257+
SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

‎WORKSPACE

+38
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,38 @@
1+
workspace(name = "bazel_diff")
2+
3+
load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")
4+
load("//:repositories.bzl", "bazel_diff_dependencies")
5+
load("//:constants.bzl", "BAZEL_DIFF_MAVEN_ARTIFACTS")
6+
7+
http_archive(
8+
name = "bazel_skylib",
9+
urls = [
10+
"https://mirror.bazel.build/github.com/bazelbuild/bazel-skylib/releases/download/1.0.2/bazel-skylib-1.0.2.tar.gz",
11+
"https://github.com/bazelbuild/bazel-skylib/releases/download/1.0.2/bazel-skylib-1.0.2.tar.gz",
12+
],
13+
sha256 = "97e70364e9249702246c0e9444bccdc4b847bed1eb03c5a3ece4f83dfe6abc44",
14+
)
15+
16+
load("@bazel_skylib//:workspace.bzl", "bazel_skylib_workspace")
17+
18+
bazel_skylib_workspace()
19+
20+
load("@bazel_skylib//lib:versions.bzl", "versions")
21+
22+
bazel_diff_dependencies(versions.get())
23+
24+
load("@rules_proto//proto:repositories.bzl", "rules_proto_dependencies", "rules_proto_toolchains")
25+
26+
rules_proto_dependencies()
27+
rules_proto_toolchains()
28+
29+
load("@rules_jvm_external//:defs.bzl", "maven_install")
30+
31+
maven_install(
32+
name = "bazel_diff_maven",
33+
artifacts = BAZEL_DIFF_MAVEN_ARTIFACTS,
34+
repositories = [
35+
"http://uk.maven.org/maven2",
36+
"https://jcenter.bintray.com/",
37+
]
38+
)

0 commit comments

Comments
 (0)