Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Setup bazel single compilation thread in release deb/rpm #1016

Merged
merged 2 commits into from
Mar 26, 2024

Conversation

j-rivero
Copy link
Contributor

@j-rivero j-rivero commented Jan 4, 2024

For preparing the buildfarm to host Bazel builds, one of the requirements is to have a single compilation thread during the release buildings. The is usually accomplish in deb creation by relying in the default behaviour of dpkg-buildpackage of using a single thread builds when using make.

In Bazel, we can setup the jobs limits by having a user configuration user.bazelrc in HOME. The PR adds these files to the deb and rpm release Dockerfiles.

An alternative considered was to use environment variables to setup this preference (ala MAKEJOBS/MAKEFLAGS) but I was able to find any documentation in Bazel that supports this use case.

@j-rivero j-rivero force-pushed the jrivero/bazel_release_limit_1 branch from 6d2baa6 to 2c1ddb4 Compare February 6, 2024 18:00
Copy link
Member

@tfoote tfoote left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good to me. I don't have a quick way to test it.

@j-rivero
Copy link
Contributor Author

j-rivero commented Feb 8, 2024

Tested in the test buildfarm for Drake. Build Status

Expected results were good:

  • The build was able to finish without killing the agent completely
  • The build was long taking more than 2:30 hours

I can see in the logs that bazel is reading the configuration:

[32mINFO: �[0mReading 'startup' options from /tmp/binarydeb/ros-rolling-drake-vendor-1.25.5/.obj-x86_64-linux-gnu/drake-prefix/src/drake-build/drake_build_cwd/.bazelrc: --output_base=/tmp/binarydeb/ros-rolling-drake-vendor-1.25.5/.obj-x86_64-linux-gnu/drake-prefix/src/drake-build/_bazel_/54c7e0b241e3ddac25283e0b95a56108
�[32mINFO: �[0mOptions provided by the client:
  Inherited 'common' options: --isatty=0 --terminal_columns=80
�[32mINFO: �[0mReading rc options for 'info' from /tmp/binarydeb/ros-rolling-drake-vendor-1.25.5/.obj-x86_64-linux-gnu/drake-prefix/src/drake/tools/bazel.rc:
  Inherited 'common' options: --enable_bzlmod=false
�[32mINFO: �[0mReading rc options for 'info' from /tmp/binarydeb/ros-rolling-drake-vendor-1.25.5/.obj-x86_64-linux-gnu/drake-prefix/src/drake-build/drake_build_cwd/.bazelrc:
  Inherited 'common' options: --announce_rc=yes --repo_env=CC=/usr/bin/cc --repo_env=CXX=/usr/bin/c++
�[32mINFO: �[0mReading rc options for 'info' from /etc/bazel.bazelrc:
  Inherited 'build' options: --jobs=1
�[32mINFO: �[0mReading rc options for 'info' from /tmp/binarydeb/ros-rolling-drake-vendor-1.25.5/.obj-x86_64-linux-gnu/drake-prefix/src/drake/tools/bazel.rc:
  Inherited 'build' options: -c opt --strip=never --strict_system_includes --test_output=errors --test_summary=terse --test_tag_filters=-gurobi,-mosek,-snopt --test_env=DISPLAY --test_env=GRB_LICENSE_FILE --test_env=MOSEKLM_LICENSE_FILE --test_env=LCM_DEFAULT_URL=memq:// --test_env=MPLBACKEND=Template
�[32mINFO: �[0mReading rc options for 'info' from /tmp/binarydeb/ros-rolling-drake-vendor-1.25.5/.obj-x86_64-linux-gnu/drake-prefix/src/drake/tools/ubuntu.bazelrc:
  Inherited 'build' options: --force_pic --fission=dbg --features=per_object_debug_info --action_env=PATH=/usr/bin:/bin --action_env=PYTHONNOUSERSITE=1 --test_env=PYTHONNOUSERSITE=1
�[32mINFO: �[0mReading rc options for 'info' from /tmp/binarydeb/ros-rolling-drake-vendor-1.25.5/.obj-x86_64-linux-gnu/drake-prefix/src/drake/tools/ubuntu-jammy.bazelrc:
  Inherited 'build' options: --cxxopt=-std=c++20 --host_cxxopt=-std=c++20
�[32mINFO: �[0mReading rc options for 'info' from /tmp/binarydeb/ros-rolling-drake-vendor-1.25.5/.obj-x86_64-linux-gnu/drake-prefix/src/drake-build/drake_build_cwd/.bazelrc:
  Inherited 'build' options: --symlink_prefix=/ --color=yes --subcommands=yes --package_path=%workspace%:/tmp/binarydeb/ros-rolling-drake-vendor-1.25.5/.obj-x86_64-linux-gnu/drake-prefix/src/drake --notrim_test_configuration --action_env=CCACHE_DISABLE=1 --config=Release

@j-rivero j-rivero marked this pull request as ready for review February 8, 2024 17:32
@j-rivero j-rivero merged commit da33299 into master Mar 26, 2024
31 checks passed
@j-rivero j-rivero deleted the jrivero/bazel_release_limit_1 branch April 15, 2024 16:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants