Add bazel query --output_file option, which writes query results directly to a file
#24297
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
When a `PathFragment` is created on Windows and one of its segments looks like a Windows short path, additional file I/O is performed to resolve it to its long form. Certain Bzlmod-managed repository names could look like such paths, which potentially slows down analysis. Instead, prepend a `v` to the version of multi-version modules to avoid this. Requires bumping the lockfile version as canonical label literals may be embedded in lockfiles. Closes bazelbuild#22219. PiperOrigin-RevId: 630468932 Change-Id: I54d58ac23e779cda50301301f16109db27d11306 --------- Co-authored-by: Fabian Meumertzheim <[email protected]>
…azelbuild#22328) This is required for `exports_filter` to match any external repositories with Bzlmod enabled. `native.package_relative_label` is used to convert the user-specified filter strings, which are always valid labels, to canonical label literals that are then passed to the actual `cc_shared_library` rule. Fixes bazelbuild#21872 Closes bazelbuild#21622. PiperOrigin-RevId: 632624776 Change-Id: I2f80563d8c434b2726f4facb0551b316b2cd2e1c Co-authored-by: Fabian Meumertzheim <[email protected]>
This allows BES consumers to access this file without guessing its path and existence, as well as when the actual coverage generation action runs locally. Also removes an obsolete `CoverageReport` event. RELNOTES: The combined coverage report produced via `--combined_report=lcov` is now announced on the BES via the new `CoverageReport` event. Closes bazelbuild#22171. PiperOrigin-RevId: 632641499 Change-Id: I0c323a371ec2fdd085bea23a772a85b72c52093f Commit bazelbuild@bb1fb53 Co-authored-by: Fabian Meumertzheim <[email protected]>
RELNOTES: The compact and full execution logs now contain start times for spawns (if available). Closes bazelbuild#22281. PiperOrigin-RevId: 633108842 Change-Id: I3b7fbf8559b7583dcfe5ed0ff5eaf736de4f1f8d Commit bazelbuild@73b0faf Co-authored-by: Fabian Meumertzheim <[email protected]>
* Add more logging to exception (b9741ed) * Sandboxing: Fix crash triggered by precondition (978c50b) --------- Co-authored-by: Googler <[email protected]>
…azelbuild#22348) ... the rollback of ee11240. Closes bazelbuild#20816.
Original commits: * 40ec2cd * b9a0578 --------- Co-authored-by: Fabian Meumertzheim <[email protected]>
This test was disabled on presubmit in bazelbuild#22351 but we forgot to make the corresponding change for postsubmit.
Removes unnecessary lockfile entries. Closes bazelbuild#22335. PiperOrigin-RevId: 633311588 Change-Id: Ia0ccb2fd14b28123115109db9fa921380319451e Co-authored-by: Fabian Meumertzheim <[email protected]>
…d#22368) If the cli-update-thread is crashing, it may attempt to interrupt and join on itself. Hopefully no updateThread could be in stopUpdateThread without going through handleCrash() -> Event.FATAL sequence through BlazeRuntime. Fixes bazelbuild#22051 Closes bazelbuild#22122. PiperOrigin-RevId: 633653817 Change-Id: Iaef5df56358d74bd7210ad8cb3562b452de9eb6a Commit bazelbuild@6306240 Co-authored-by: George Gensure <[email protected]>
RELNOTES: The new `refresh` value for `--lockfile_mode` behaves like the `update` mode, but additionally forces a refresh of mutable registry content (yanked versions and missing module versions) when switched to or from time to time while enabled. Closes bazelbuild#22333. PiperOrigin-RevId: 633737372 Change-Id: I2cae552f6adfcd07a5b3263b5e4997e21653f106 Commit bazelbuild@88a230f --------- Co-authored-by: Fabian Meumertzheim <[email protected]> Co-authored-by: Xùdōng Yáng <[email protected]>
…zelbuild#22369) Also remove Ubuntu 16.04 workarounds from layering_check tests. RELNOTES: The default Unix C++ toolchain now supports the `parse_headers` feature to validate header files with `--process_headers_in_dependencies`. Closes bazelbuild#21560. PiperOrigin-RevId: 633657012 Change-Id: Iaaa2623bcc86b219b02567eca1c7bf9e1a77ae7d Fixes bazelbuild#22361 Commit bazelbuild@231dfc2
… throwing. (bazelbuild#22381) This adds a bit of convenience when executing pragmas that echo back the setting. PiperOrigin-RevId: 633913983 Change-Id: If2a9f747a1fe73393dfe5608ae17d366d271892e
…ubdirectory. (bazelbuild#22383) This will make it easier for garbage collection to identify them. PiperOrigin-RevId: 634007262 Change-Id: I78b836f4fbae448c2ef408e7c2f8313447cedc22
…emote_build_event_upload=MINIMAL. (bazelbuild#22384) In particular, this ensures that the compact execution log is uploaded (for which a test has been added). ByteStreamBuildEventArtifactUploader would have benefitted from a refactor, but I'm deliberately making a minimal change that can be cherry-picked into 7.2.0 without risking a regression. PiperOrigin-RevId: 634033159 Change-Id: I4d3df757f612337b5d2b222278d1171cc8e1aed4
This will be necessary to implement retry loops when the database is locked by another process (SQLITE_BUSY). However, I'm exporting all of the known error codes for future convenience. They're a stable API, so hardcoding the values should be fine. PiperOrigin-RevId: 633948990 Change-Id: Ic089d2792ddab28bcba20d583fbdedbaea421c4c
…2391) PiperOrigin-RevId: 621452493 Change-Id: I812062db638f171e9e2dcb570f1797cf5118984e Commit bazelbuild@31b522f Co-authored-by: Googler <[email protected]>
…ld#22407) The bind mounting scheme used with the Linux sandbox' hermetic `/tmp` feature is modified to preserve all paths as they are outside the sandbox, which removes the need to rewrite paths when staging inputs into and, crucially, moving outputs out of the sandbox. Source roots and output base paths under `/tmp` are now treated just like any user-specified bind mount under `/tmp`: They are mounted under the hermetic tmp directory with their path relativized against `/tmp` before the hermetic tmp directory is mounted as `/tmp` as the final step. There is one caveat compared to user-specified mounts: Source roots, which may themselves not lie under `/tmp`, can be symlinks to directories under `/tmp` (e.g., when they arise from a `local_repository`). To handle this situation in the common case, all parent directories of package path entries (up to direct children of `/tmp`) are mounted into the sandbox. If users use `local_repository`s with fixed target paths under `/tmp`, they will need to specify `--sandbox_add_mount_pair`. Overlayfs has been considered as an alternative to this approach, but ultimately doesn't seem to work for this use case since its `lowerpath`, which would be `/tmp`, is not allowed to have child mounts from a different user namespace (see https://unix.stackexchange.com/questions/776030/mounting-overlayfs-in-a-user-namespace-with-child-mounts). However, this is exactly the situation created by a Bazel-in-Bazel test and can also arise if the user has existing mounts under `/tmp` when using Bazel (e.g. the JetBrains toolbox on Linux uses such mounts). This replaces and mostly reverts the following commits, but keeps their tests: * bazelbuild@bf6ebe9 * bazelbuild@fb6658c * bazelbuild@bc1d9d3 * bazelbuild@1829883 * bazelbuild@70691f2 * bazelbuild@a556969 * bazelbuild@8e32f44 (had its test lost in an incorrect merge conflict resolution, this PR adds it back) Fixes bazelbuild#20533 Work towards bazelbuild#20753 Fixes bazelbuild#21215 Fixes bazelbuild#22117 Fixes bazelbuild#22226 Fixes bazelbuild#22290 RELNOTES: Paths in the Linux sandbox are now again identical to those outside the sandbox, even with `--incompatible_sandbox_hermetic_tmp`. Closes bazelbuild#22001. PiperOrigin-RevId: 634381503 Change-Id: I9f7f3948c705be120c55c9b0c51204e5bea45f61 Fixes bazelbuild#22291
Necessary to remove references to `@bazel_tools//tools/android`, which is necessary to remove bind()s from Starlark android_sdk_repository. PiperOrigin-RevId: 633693304 Change-Id: Ib49fd219847e63135a4b3e771bf67013e12830b4 Commit bazelbuild@df9f76a Co-authored-by: Googler <[email protected]>
…azelbuild#22413) The external_include_paths feature enables specifying include paths locally with -I and with -isystem when the project is included as an external repo. This makes it possible to set -Werror against your own headers for development without propagating this to consumers of your libraries. For some reason this was only previously enabled on linux. I do not see a reason for not also enabling it on macos. Closes bazelbuild#21949. PiperOrigin-RevId: 634394452 Change-Id: I0df91b2a2c9b4a5bd52b22e64f99ea0745cb9759 Commit bazelbuild@d831214 Co-authored-by: Isaac Torres <[email protected]>
Previously the code failed when the repository was in detached HEAD state (e.g. on CI). Closes bazelbuild#22405. PiperOrigin-RevId: 634564660 Change-Id: I60d87f1f24a1dceec6b74709a571c9630cf6bcc5 Commit bazelbuild@2028c47 Co-authored-by: Florian Weikert <[email protected]>
…bazelbuild#22450) The logic for sandboxing hermetic tmp needs to take into account all paths under `/tmp` used during the build. A user may also pass a `sandbox_base` under `/tmp` even when the `output_base` is not. This change adds `sandbox_base` to the list. Fix suggested by fmeum Closes bazelbuild#22435. PiperOrigin-RevId: 635382607 Change-Id: I32497a514ff16e64a0e0d84c307f280d3d37544b Commit bazelbuild@eebf5f9 Co-authored-by: oquenchil <[email protected]>
The reverts in 3fddc7f accidentally replaced `sandboxExecRoot` with `execRoot` in the function that computes writable directories for sandbox spawn runners. This issue was observed in MobileNativeFoundation/rules_xcodeproj#3029. Closes bazelbuild#22443. PiperOrigin-RevId: 635383877 Change-Id: I3d74bf78cf97ecd267ba90b68cf1b715850aba27 Commit bazelbuild@b0ed4ca Co-authored-by: Fabian Meumertzheim <[email protected]>
…zelbuild#22460) This is a reland of 30b95e3 with a different approach to emitting display form labels that avoids adding a new `to_display_form()` method to `Label`: * In action command lines, which are the most frequent use of labels in rule implementation functions, labels are automatically emitted in display form since 9d3a8b0. * In module extensions and repository rules, if labels can be turned into display form, the inverse of the repository mapping would need to be tracked in lockfiles and marker files for correct incrementality. Furthermore, allowing implementation functions to access apparent names would allow them to "discriminate" against them, thus possibly breaking the user's capability to map repos at will via `use_repo` and `repo_name`. Similar to how providers on a target can't be enumerated, it is thus safer to not provide this information to the implementation functions directly. This change thus implements `StarlarkValue#debugPrint` for `Label` to allow ruleset authors to emit labels in display form in warnings and error messages while ensuring that Starlark logic doesn't have access to this information. `print("My message", label)` degrades gracefully with older Bazel versions (it just prints a canonical label literal) and can thus be adopted immediately without a need for feature detection. This requires changing the signature of `StarlarkValue#debugPrint` to receive the `StarlarkThread` instead of just the `StarlarkSemantics`. Since `debugPrint` is meant for emitting potentially non-deterministic information, it is safe to give it access to `StarlarkThread`. Also improves the Bzlmod cycle reporter so that it prints helpful information on a cycle encountered in a previous iteration of this PR. Fixes bazelbuild#20486 RELNOTES: `Label` instances passed to `print` or `fail` as positional arguments are now formatted with apparent repository names (optimized for human readability). Closes bazelbuild#21963. PiperOrigin-RevId: 635589078 Change-Id: If60fdc632a59f19dad0cb02312690c15a0540c8e Closes bazelbuild#22136
Closes bazelbuild#22409. PiperOrigin-RevId: 635869476 Change-Id: I8992950723b112d8bb8072500840e7a377e5957f Co-authored-by: Fabian Meumertzheim <[email protected]>
JSON-based merge conflict resolution for `MODULE.bazel.lock` can end up producing entries with, e.g., both a `general` and an `os:Linux` factor result when the OS/arch dependence of the extension changes. `BazelLockFileModule` now invalidates each factor result individually. Also sort the extension factors. Closes bazelbuild#22378. PiperOrigin-RevId: 635913837 Change-Id: I0064098806c856f16e8f4c0270f609f06cebc945 Commit bazelbuild@c4092e9 Co-authored-by: Fabian Meumertzheim <[email protected]>
Fixes bazelbuild#14418 Closes bazelbuild#22420. PiperOrigin-RevId: 635740423 Change-Id: Iffd4d172c4175be2e1b6cfad04ddad2759adb987 Commit bazelbuild@01a90d6 Co-authored-by: Keith Smiley <[email protected]>
…gs (bazelbuild#22467) Targets tagged with `local` or `no-sandbox` and `no-remote` can't successfully use path mapping and thus have it disabled implicitly. Closes bazelbuild#21921. PiperOrigin-RevId: 635832339 Change-Id: Ib5cac0b202cbcd1704410f06fa3cda645581b849 Commit bazelbuild@46dce83 Co-authored-by: Fabian Meumertzheim <[email protected]>
Co-authored-by: hvd <[email protected]> Co-authored-by: Googler <[email protected]>
…4123) Since `WorkRequest` encodes strings as UTF-8, Bazel's internal string encoding of file paths needs to be undone before passing them to the protobuf builder. Closes bazelbuild#23924. PiperOrigin-RevId: 690737819 Change-Id: I3eb341c05e045c75ac004da57e1f46f3587a6810 Commit bazelbuild@89f798e Co-authored-by: Fabian Meumertzheim <[email protected]>
…uild#24132) Looks like f6687ad didn't fully fix things, based on looking at https://bazel.build/external/module today: <img width="907" alt="Screenshot 2024-10-15 at 1 13 37 PM" src="https://github.com/user-attachments/assets/800487d4-0845-418d-8042-bd2c4bad017c"> Closes bazelbuild#23985. PiperOrigin-RevId: 691071540 Change-Id: Ic474af8308843314aaeaaa85a40dfc487cf7efca Commit bazelbuild@4e4b88f Co-authored-by: Matt Brown <[email protected]>
…uild#23406)" (bazelbuild#24211) This reverts commit cb1bc28. Closes bazelbuild#24158
…zelbuild#24228) This PR removes the DownloadManager from the registry factory implementation. As the registry created by the factory is cached by the SkyFunction mechanism, the DownloadManager instance was living too long - it is supposed to be re-created for every command instantiation to respect changes in command line options, but for the registry, it ignored those changes. Instead, the DownloadManager is set directly into the affected SkyFunctions that require access to it. This way, the per-command DownloadManager instance is correctly used. This fixes bazelbuild#24166. Note for reviewers: This is my first time touching code with SkyFunctions, so I don't really know what I'm doing. Closes bazelbuild#24212. PiperOrigin-RevId: 693644409 Change-Id: I7b16684e52673043615290d114f078ab7ab99fcf Co-authored-by: Cornelius Riemenschneider <[email protected]>
Release Notes:
github-actions
bot
added
the
awaiting-review
keithl-stripe
force-pushed
the
query-output-to-file
branch
from
6979c80
to
5521241
Compare
…rectly to a file
keithl-stripe
force-pushed
the
query-output-to-file
branch
from
5521241
to
c08500f
Compare
keithl-stripe
requested review from
a team,
lberki,
brandjon,
tetromino,
comius,
pzembrod,
trybka,
gregestren,
fweikert,
Wyverald,
meteorcloudy,
ahumesky,
ted-xie and
kevin1e100
as code owners
github-actions
bot
removed
the
awaiting-review
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
closed in favor of #24298