Skip to content

Commit

Permalink
getdeps: add xxhash ubuntu and homebrew packages, fix actions generat…
Browse files Browse the repository at this point in the history
…ion and regenerate

Summary:
EdenFS build on github CI was failing with missing xxhash include: https://github.com/facebook/sapling/actions/runs/11311478649/job/31457761727#step:108:3838

Add package mappings to the xxhash manifest

When I went to regenerate the github actions found I'd previously introduced a bug in the actions generation with a mis-merged run_tests check.  Fix it and regenerate

Test Plan:

Build locally on ubuntu-22.04 toolbox to repro issue, its broken,
```
./build/fbcode_builder/getdeps.py --allow-system-packages build --src-dir=. eden
```

Install the packages:
```
./build/fbcode_builder/getdeps.py install-system-deps --recursive eden
```

Run again, it builds:
```
./build/fbcode_builder/getdeps.py --allow-system-packages build --src-dir=. eden
```

Regenerate the github workflow:
```
 ./build/fbcode_builder/getdeps.py --allow-system-packages generate-github-actions --no-tests --free-up-disk --os-type=linux --src-dir=. --output-dir=.github/workflows --job-name "EdenFS " --job-file-prefix=edenfs_ eden --num-jobs=8  --project-install-prefix sapling:/usr/local
```
  • Loading branch information
ahornby committed Oct 13, 2024
1 parent dce7886 commit edcb20e
Show file tree
Hide file tree
Showing 3 changed files with 41 additions and 30 deletions.
5 changes: 5 additions & 0 deletions .github/workflows/edenfs_linux.yml
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,8 @@ jobs:
run: python3 build/fbcode_builder/getdeps.py --allow-system-packages fetch --no-tests glog
- name: Fetch googletest
run: python3 build/fbcode_builder/getdeps.py --allow-system-packages fetch --no-tests googletest
- name: Fetch xxhash
run: python3 build/fbcode_builder/getdeps.py --allow-system-packages fetch --no-tests xxhash
- name: Fetch zstd
run: python3 build/fbcode_builder/getdeps.py --allow-system-packages fetch --no-tests zstd
- name: Fetch boost
Expand Down Expand Up @@ -148,6 +150,8 @@ jobs:
run: python3 build/fbcode_builder/getdeps.py --allow-system-packages build --free-up-disk --no-tests glog
- name: Build googletest
run: python3 build/fbcode_builder/getdeps.py --allow-system-packages build --free-up-disk --no-tests googletest
- name: Build xxhash
run: python3 build/fbcode_builder/getdeps.py --allow-system-packages build --free-up-disk --no-tests xxhash
- name: Build zstd
run: python3 build/fbcode_builder/getdeps.py --allow-system-packages build --free-up-disk --no-tests zstd
- name: Build boost
Expand Down Expand Up @@ -237,4 +241,5 @@ jobs:
name: eden
path: _artifacts
- name: Show disk space at end
if: always()
run: df -h
59 changes: 29 additions & 30 deletions build/fbcode_builder/getdeps.py
Original file line number Diff line number Diff line change
Expand Up @@ -1081,38 +1081,37 @@ def write_job_for_platform(self, platform, args): # noqa: C901
free_up_disk = ""

allow_sys_arg = ""
if run_tests:
sudo_arg = "sudo "
allow_sys_arg = " --allow-system-packages"
if build_opts.host_type.get_package_manager() == "deb":
out.write(" - name: Update system package info\n")
out.write(f" run: {sudo_arg}apt-get update\n")

out.write(" - name: Install system deps\n")
if build_opts.is_darwin():
# brew is installed as regular user
sudo_arg = ""
sudo_arg = "sudo "
allow_sys_arg = " --allow-system-packages"
if build_opts.host_type.get_package_manager() == "deb":
out.write(" - name: Update system package info\n")
out.write(f" run: {sudo_arg}apt-get update\n")

out.write(" - name: Install system deps\n")
if build_opts.is_darwin():
# brew is installed as regular user
sudo_arg = ""
out.write(
f" run: {sudo_arg}python3 build/fbcode_builder/getdeps.py --allow-system-packages install-system-deps --recursive {manifest.name}\n"
)
if build_opts.is_linux() or build_opts.is_freebsd():
out.write(" - name: Install packaging system deps\n")
out.write(
f" run: {sudo_arg}python3 build/fbcode_builder/getdeps.py --allow-system-packages install-system-deps --recursive {manifest.name}\n"
)
if build_opts.is_linux() or build_opts.is_freebsd():
out.write(" - name: Install packaging system deps\n")
out.write(
f" run: {sudo_arg}python3 build/fbcode_builder/getdeps.py --allow-system-packages install-system-deps --recursive patchelf\n"
)
required_locales = manifest.get(
"github.actions", "required_locales", ctx=manifest_ctx
f" run: {sudo_arg}python3 build/fbcode_builder/getdeps.py --allow-system-packages install-system-deps --recursive patchelf\n"
)
if (
build_opts.host_type.get_package_manager() == "deb"
and required_locales
):
# ubuntu doesn't include this by default
out.write(" - name: Install locale-gen\n")
out.write(f" run: {sudo_arg}apt-get install locales\n")
for loc in required_locales.split():
out.write(f" - name: Ensure {loc} locale present\n")
out.write(f" run: {sudo_arg}locale-gen {loc}\n")
required_locales = manifest.get(
"github.actions", "required_locales", ctx=manifest_ctx
)
if (
build_opts.host_type.get_package_manager() == "deb"
and required_locales
):
# ubuntu doesn't include this by default
out.write(" - name: Install locale-gen\n")
out.write(f" run: {sudo_arg}apt-get install locales\n")
for loc in required_locales.split():
out.write(f" - name: Ensure {loc} locale present\n")
out.write(f" run: {sudo_arg}locale-gen {loc}\n")

projects = loader.manifests_in_dependency_order()

Expand Down
7 changes: 7 additions & 0 deletions build/fbcode_builder/manifests/xxhash
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,13 @@ sha256 = baee0c6afd4f03165de7a4e67988d16f0f2b257b51d0e3cb91909302a26a79c4
[rpms]
xxhash-devel

[debs]
libxxhash-dev
xxhash

[homebrew]
xxhash

[build.not(os=windows)]
builder = make
subdir = xxHash-0.8.2
Expand Down

0 comments on commit edcb20e

Please sign in to comment.