failures to build (and needed dependencies) #11

samth · 2022-11-22T19:12:55Z

Describe the bug

As of 56f1390 the repository fails to build with cargo build. I get the error:

[samth@huor:~/tmp/hermit (main) plt] cargo build
warning: skipping duplicate package `main` found at `/home/samth/.cargo/git/checkouts/reverie-9a587e40a0d7d3be/6f03658/experimental/riptrace`
   Compiling reverie-ptrace v0.1.0 (https://github.com/facebookexperimental/reverie.git?branch=main#6f036584)
   Compiling hermetic_infra_hermit_tests v0.0.0 (/home/samth/tmp/hermit/tests)
   Compiling hermit-verify v0.0.0 (/home/samth/tmp/hermit/hermit-verify)
error[E0432]: unresolved import `detcore::preemptions`
  --> tests/standalone/stacktrace_events.rs:13:14
   |
13 | use detcore::preemptions::PreemptionReader;
   |              ^^^^^^^^^^^ could not find `preemptions` in `detcore`

error[E0432]: unresolved import `detcore::preemptions`
  --> tests/standalone/stacktrace_events.rs:14:14
   |
14 | use detcore::preemptions::PreemptionRecord;
   |              ^^^^^^^^^^^ could not find `preemptions` in `detcore`

For more information about this error, try `rustc --explain E0432`.
error: could not compile `hermetic_infra_hermit_tests` due to 2 previous errors
warning: build failed, waiting for other jobs to finish...
error[E0433]: failed to resolve: use of undeclared crate or module `detcore`
  --> hermit-verify/src/internal/schedule_trace.rs:13:5
   |
13 | use detcore::preemptions::read_trace;
   |     ^^^^^^^ use of undeclared crate or module `detcore`

error[E0433]: failed to resolve: use of undeclared crate or module `detcore`
  --> hermit-verify/src/internal/schedule_trace.rs:14:5
   |
14 | use detcore::preemptions::PreemptionRecord;
   |     ^^^^^^^ use of undeclared crate or module `detcore`

error[E0432]: unresolved import `edit_distance`
  --> hermit-verify/src/internal/schedule_trace.rs:15:5
   |
15 | use edit_distance::iterable_bubble_sort;
   |     ^^^^^^^^^^^^^ use of undeclared crate or module `edit_distance`

Some errors have detailed explanations: E0432, E0433.
For more information about an error, try `rustc --explain E0432`.
error: could not compile `hermit-verify` due to 3 previous errors

Additionally, I needed to install libunwind-dev in order to build, otherwise it failed (with a panic) when compiling unwind-sys.

Also, I notice that the GitHub actions CI doesn't do anything.

The text was updated successfully, but these errors were encountered:

rnbguy · 2022-11-22T19:48:22Z

adding the following lines in [dependencies] of hermit-verify/Cargo.toml resolves the use of undeclared crate or module `detcore` errors

detcore = { version = "0.0.0", path = "../detcore" }
edit-distance = { version = "0.0.0", path = "../common/edit-distance" }

but I still get could not find `preemptions` in `detcore` errors - which is weird. rust-analyzer can find the references all fine, but not cargo build.

error[E0432]: unresolved import `detcore::preemptions`
  --> tests/standalone/stacktrace_events.rs:13:14
   |
13 | use detcore::preemptions::PreemptionReader;
   |              ^^^^^^^^^^^ could not find `preemptions` in `detcore`

error[E0432]: unresolved import `detcore::preemptions`
  --> tests/standalone/stacktrace_events.rs:14:14
   |
14 | use detcore::preemptions::PreemptionRecord;
   |              ^^^^^^^^^^^ could not find `preemptions` in `detcore`

error[E0432]: unresolved import `detcore::Config`
  --> detcore/tests/testutils/src/lib.rs:18:5
   |
18 | use detcore::Config;
   |     ^^^^^^^^^^^^^^^ no `Config` in the root

error[E0432]: unresolved import `detcore::Detcore`
  --> detcore/tests/testutils/src/lib.rs:19:5
   |
19 | use detcore::Detcore;
   |     ^^^^^^^^^^^^^^^^ no `Detcore` in the root

error[E0432]: unresolved import `detcore::SchedHeuristic`
  --> detcore/tests/testutils/src/lib.rs:20:5
   |
20 | use detcore::SchedHeuristic;
   |     ^^^^^^^^^^^^^^^^^^^^^^^ no `SchedHeuristic` in the root

error[E0433]: failed to resolve: could not find `logdiff` in `detcore`
   --> detcore/tests/testutils/src/lib.rs:582:38
    |
582 |                 let str_a = detcore::logdiff::strip_log_entry(&x[ix]);
    |                                      ^^^^^^^ could not find `logdiff` in `detcore`

error[E0433]: failed to resolve: could not find `logdiff` in `detcore`
   --> detcore/tests/testutils/src/lib.rs:583:38
    |
583 |                 let str_b = detcore::logdiff::strip_log_entry(&filtered[ix]);

androm3da · 2022-11-22T20:04:17Z

adding the following lines in [dependencies] of hermit-verify/Cargo.toml resolves the use of undeclared crate or module `detcore` errors
detcore = { version = "0.0.0", path = "../detcore" }
edit-distance = { version = "0.0.0", path = "../common/edit-distance" }

With those changes I get a smaller set of errors:

error[E0432]: unresolved import `detcore::preemptions`
  --> hermit-verify/src/internal/schedule_trace.rs:13:14
   |
13 | use detcore::preemptions::read_trace;
   |              ^^^^^^^^^^^ could not find `preemptions` in `detcore`

error[E0432]: unresolved import `detcore::preemptions`
  --> hermit-verify/src/internal/schedule_trace.rs:14:14
   |
14 | use detcore::preemptions::PreemptionRecord;
   |              ^^^^^^^^^^^ could not find `preemptions` in `detcore`

androm3da · 2022-11-22T20:05:56Z

adding the following lines in [dependencies] of hermit-verify/Cargo.toml resolves the use of undeclared crate or module `detcore` errors
detcore = { version = "0.0.0", path = "../detcore" }
edit-distance = { version = "0.0.0", path = "../common/edit-distance" }

With those changes I get a smaller set of errors:

error[E0432]: unresolved import `detcore::preemptions`
  --> hermit-verify/src/internal/schedule_trace.rs:13:14
   |
13 | use detcore::preemptions::read_trace;
   |              ^^^^^^^^^^^ could not find `preemptions` in `detcore`

error[E0432]: unresolved import `detcore::preemptions`
  --> hermit-verify/src/internal/schedule_trace.rs:14:14
   |
14 | use detcore::preemptions::PreemptionRecord;
   |              ^^^^^^^^^^^ could not find `preemptions` in `detcore`

Hmm, oddly - on subsequent build attempts, several more build errors are reported.

jasonwhite · 2022-11-22T20:15:36Z

We're working on it! We don't use Cargo internally and keeping two different build systems in sync is a little challenging.

rrnewton · 2022-11-22T20:18:13Z

Erk, sorry, looks like this JUST got messed up by a recent push. Bad timing.

@samth @rnbguy - does the prior commit, cacc346, work for you?

We'll stick a Dockerfile in there momentarily to address the libunwind dependence.

androm3da · 2022-11-22T20:32:14Z

I get the same or substantially similar results when building cacc346

Both with or without this patch:

diff --git a/hermit-verify/Cargo.toml b/hermit-verify/Cargo.toml
index 36524c5..9ba41df 100644
--- a/hermit-verify/Cargo.toml
+++ b/hermit-verify/Cargo.toml
@@ -7,6 +7,8 @@ edition = "2021"
 
 [dependencies]
 anyhow = "1.0.65"
+detcore = { version = "0.0.0", path = "../detcore" }
+edit-distance = { version = "0.0.0", path = "../common/edit-distance" }
 clap = { version = "3.2.17", features = ["derive", "env", "regex", "unicode", "wrap_help"] }
 colored = "1.9"
 fbinit = { version = "0.1.2", git = "https://github.com/facebookexperimental/rust-shed.git", branch = "main" }

androm3da · 2022-11-22T20:34:01Z

FYI my nightly happens to be brand new, I suppose that could contribute.

nightly-x86_64-unknown-linux-gnu (overridden by '.../hermit/rust-toolchain.toml')
rustc 1.67.0-nightly (b7bc90fea 2022-11-21)

rnbguy · 2022-11-22T20:37:09Z

the detcore error persists on cacc346 at my side too. I tried local build and on an empty docker - both gave same results.

my nightly is also the latest.

rrnewton · 2022-11-22T20:52:42Z

Jeez, sorry about this. At the current main branch, 6cb5575, it cargo builds for me fine on a Centos Stream 8 server and Fedora 36 laptop (with libunwind). I'm trying this from the rust/ Debian base image:

git clone [email protected]:facebookexperimental/hermit.git
cd hermit
docker run --volume `pwd`:/build -it --rm --platform linux/amd64 rust:latest
apt-get update && apt-get install -y libunwind-dev
cd /build
cargo update # make sure there is enough memory
cargo build

(and having some trouble with cargo update getting OOM killed.. will update this in a moment)

androm3da · 2022-11-22T20:59:08Z

Interesting. Does your cargo and rustc come from rustup? Or does debian have its own installation?

I had not tried cargo update before but it did not change the results.

$ lsb_release -a
No LSB modules are available.
Distributor ID:	Ubuntu
Description:	Ubuntu 18.04.6 LTS
Release:	18.04
Codename:	bionic

$ free -m
              total        used        free      shared  buff/cache   available
Mem:          64318        6052        2420        1771       55845       55820
Swap:         32767        2928       29839

$ cargo build | tail
warning: skipping duplicate package `main` found at `/local/mnt/workspace/install/rust/git/checkouts/reverie-9a587e40a0d7d3be/6f03658/experimental/traceviz`
   Compiling hermetic_infra_hermit_tests v0.0.0 (/local/mnt/workspace/vp/hermit/tests)
   Compiling hermit v0.0.0 (/local/mnt/workspace/vp/hermit/hermit-cli)
   Compiling detcore-testutils v0.0.0 (/local/mnt/workspace/vp/hermit/detcore/tests/testutils)
error[E0432]: unresolved import `detcore::preemptions`
  --> tests/standalone/stacktrace_events.rs:13:14
   |
13 | use detcore::preemptions::PreemptionReader;
   |              ^^^^^^^^^^^ could not find `preemptions` in `detcore`

$ git show|head
commit 6cb5575ffd287289769144ec82e2900cbf6cd1ad
Author: Ryan Newton <[email protected]>
Date:   Tue Nov 22 15:20:46 2022 -0500

    Revert "tooling to compare schedules and run midpoints on CLI" (#12)
    
    This reverts commit 56f1390a474d3b123d8fc8d3e734410b593e04f9.

diff --git a/common/edit-distance/src/bubbles.rs b/common/edit-distance/src/bubbles.rs
index fb1ef9b..e3486c5 100644

rnbguy · 2022-11-22T21:05:18Z

I am using archlinux, and rustup for rust toolchains.

Anyway, using git bisect I found the earliest commit with a failed build - 66a93ab

Successful build on d37f732 🙂

rnbguy · 2022-11-22T21:11:15Z

Ah this is the culprit

hermit/detcore/src/lib.rs

Line 16 in 6cb5575

#![cfg(not(version("1.66")))]

Removing this line builds everything all fine on current main 🎉

Maybe turn on the CI runs using cargo-action 🙂 Atleast the cargo build one.

hermit/.github/workflows/ci.yml

Lines 19 to 24 in 6cb5575

    
           - name: Build 
        
             working-directory: detcore-model 
        
             run: echo cargo build --verbose 
        
           - name: Run tests 
        
             working-directory: detcore-model 
        
             run: echo cargo test --verbose

rrnewton · 2022-11-22T21:11:25Z

Thanks for the investigation!! So ceee5bd probably builds for you on that arch setup then then? Reverting that business about the map_first_last feature.

@rnbguy, thanks again for your patience. Our external build was half baked here. (It builds fine and runs 750 integration tests, with buck, over in monorepo land ;-). ) We'll invest here to get this in good shape, buildable with Docker & Nix, and with images on dockerhub.

Also, of course, reenabling the barest level of build in CI (#13). Porting real integration tests will take a bit more work as described in the README.

samth · 2022-11-22T21:13:12Z

That being the issue makes me think it's about the difference between what Debian installs for rust:latest and what rustup installs for nightly (I also used rustup).

androm3da · 2022-11-22T21:27:09Z

I am using archlinux, and rustup for rust toolchains.

Anyway, using git bisect I found the earliest commit with a failed build - 66a93ab

Successful build on d37f732 🙂

This commit makes my build succeed but might be worth mentioning that there's a test failure. Not sure how significant it is.

test scheduler::test::test_my_thread_group1 ... FAILED
...
failures:

---- scheduler::test::test_my_thread_group1 stdout ----
thread 'scheduler::test::test_my_thread_group1' panicked at 'Unable to install global subscriber: SetGlobalDefaultError("a global default trace dispatcher has already been set")', /local/mnt/workspace/install/rust/registry/src/github.com-1ecc6299db9ec823/tracing-subscriber-0.3.16/src/fmt/mod.rs:1232:16

jasonwhite · 2022-11-22T22:08:10Z

@androm3da Does it work with cargo test -- --test-threads=1?

Edit: Never mind. I can reproduce the failure as well. Looking into it!

rnbguy · 2022-11-22T22:09:29Z

main build is successful now.

$ cargo +nightly install --git https://github.com/facebookexperimental/hermit hermit
$ hermit run date
Sat Jan  1 12:59:59 AM CET 2000

🎉

androm3da · 2022-11-22T22:10:13Z

@androm3da Does it work with cargo test -- --test-threads=1?

It does. However a different test fails when I use that command line.

$ cargo test -- --test-threads=1
...
test scheduler::replay_cursor::tests::test_peek_nth ... ok
test scheduler::replay_cursor::tests::test_peek_pair ... ok
test scheduler::replay_cursor::tests::test_peek_pair_empty ... ok
test scheduler::replay_cursor::tests::test_peek_pair_one ... ok
test scheduler::test::test_my_thread_group1 ... ok
test scheduler::test::test_my_thread_group2 ... FAILED
test syscalls::files::test::linux_flags_assumptions ... ok

failures:

---- scheduler::test::test_my_thread_group2 stdout ----
thread 'scheduler::test::test_my_thread_group2' panicked at 'Unable to install global subscriber: SetGlobalDefaultError("a global default trace dispatcher has already been set")', /local/mnt/workspace/install/rust/registry/src/github.com-1ecc6299db9ec823/tracing-subscriber-0.3.16/src/fmt/mod.rs:1232:16


failures:
    scheduler::test::test_my_thread_group2

test result: FAILED. 41 passed; 1 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.10s

error: test failed, to rerun pass `-p detcore --lib`

VladimirMakaev · 2022-11-23T12:50:00Z

@androm3da Does it work with cargo test -- --test-threads=1?

It does. However a different test fails when I use that command line.

$ cargo test -- --test-threads=1
...
test scheduler::replay_cursor::tests::test_peek_nth ... ok
test scheduler::replay_cursor::tests::test_peek_pair ... ok
test scheduler::replay_cursor::tests::test_peek_pair_empty ... ok
test scheduler::replay_cursor::tests::test_peek_pair_one ... ok
test scheduler::test::test_my_thread_group1 ... ok
test scheduler::test::test_my_thread_group2 ... FAILED
test syscalls::files::test::linux_flags_assumptions ... ok

failures:

---- scheduler::test::test_my_thread_group2 stdout ----
thread 'scheduler::test::test_my_thread_group2' panicked at 'Unable to install global subscriber: SetGlobalDefaultError("a global default trace dispatcher has already been set")', /local/mnt/workspace/install/rust/registry/src/github.com-1ecc6299db9ec823/tracing-subscriber-0.3.16/src/fmt/mod.rs:1232:16


failures:
    scheduler::test::test_my_thread_group2

test result: FAILED. 41 passed; 1 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.10s

error: test failed, to rerun pass `-p detcore --lib`

We'll fix this particular issue soon. Currently we have a bit of desync problem with our internal repo but at least you can build and try hermit.

samth · 2022-11-23T14:40:20Z

This now builds correctly for me (although I then experience #16) so I'm closing this.

sanmai-NL · 2023-02-26T20:33:32Z

@samth The official instructions are incomplete still, like you I had to install the libunwind-dev OS package under Debian Bullseye. Can you reopen this?

samth added the bug Something isn't working label Nov 22, 2022

rrnewton mentioned this issue Nov 22, 2022

Revert "remove feature flag which is now stabilized" #14

Merged

samth closed this as completed Nov 23, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

failures to build (and needed dependencies) #11

failures to build (and needed dependencies) #11

samth commented Nov 22, 2022

rnbguy commented Nov 22, 2022 •

edited

Loading

androm3da commented Nov 22, 2022

androm3da commented Nov 22, 2022

jasonwhite commented Nov 22, 2022

rrnewton commented Nov 22, 2022 •

edited

Loading

androm3da commented Nov 22, 2022

androm3da commented Nov 22, 2022

rnbguy commented Nov 22, 2022 •

edited

Loading

rrnewton commented Nov 22, 2022 •

edited

Loading

androm3da commented Nov 22, 2022

rnbguy commented Nov 22, 2022 •

edited

Loading

rnbguy commented Nov 22, 2022 •

edited

Loading

rrnewton commented Nov 22, 2022 •

edited

Loading

samth commented Nov 22, 2022

androm3da commented Nov 22, 2022

jasonwhite commented Nov 22, 2022 •

edited

Loading

rnbguy commented Nov 22, 2022

androm3da commented Nov 22, 2022

VladimirMakaev commented Nov 23, 2022

samth commented Nov 23, 2022

sanmai-NL commented Feb 26, 2023

failures to build (and needed dependencies) #11

failures to build (and needed dependencies) #11

Comments

samth commented Nov 22, 2022

rnbguy commented Nov 22, 2022 • edited Loading

androm3da commented Nov 22, 2022

androm3da commented Nov 22, 2022

jasonwhite commented Nov 22, 2022

rrnewton commented Nov 22, 2022 • edited Loading

androm3da commented Nov 22, 2022

androm3da commented Nov 22, 2022

rnbguy commented Nov 22, 2022 • edited Loading

rrnewton commented Nov 22, 2022 • edited Loading

androm3da commented Nov 22, 2022

rnbguy commented Nov 22, 2022 • edited Loading

rnbguy commented Nov 22, 2022 • edited Loading

rrnewton commented Nov 22, 2022 • edited Loading

samth commented Nov 22, 2022

androm3da commented Nov 22, 2022

jasonwhite commented Nov 22, 2022 • edited Loading

rnbguy commented Nov 22, 2022

androm3da commented Nov 22, 2022

VladimirMakaev commented Nov 23, 2022

samth commented Nov 23, 2022

sanmai-NL commented Feb 26, 2023

rnbguy commented Nov 22, 2022 •

edited

Loading

rrnewton commented Nov 22, 2022 •

edited

Loading

rnbguy commented Nov 22, 2022 •

edited

Loading

rrnewton commented Nov 22, 2022 •

edited

Loading

rnbguy commented Nov 22, 2022 •

edited

Loading

rnbguy commented Nov 22, 2022 •

edited

Loading

rrnewton commented Nov 22, 2022 •

edited

Loading

jasonwhite commented Nov 22, 2022 •

edited

Loading