Support -Z build-std

[Stupremee]

See

• https://github.com/rust-lang/wg-cargo-std-aware
• https://doc.rust-lang.org/cargo/reference/unstable.html#build-std
• https://internals.rust-lang.org/t/dogfooding-z-build-std-in-rustbuild/13775/13

Currently, if you try to build a project that uses build-std, you'll get errors about dependencies that are not found, but definitely
exist.

Example project

.cargo/config.toml

[build]
target = "riscv64gc-unknown-none-elf"

[unstable]
build-std = ["core"]

src/lib.rs

#![no_std]

default.nix

let
  sources = import ./nix/sources.nix;
  nixpkgs-mozilla = import sources.nixpkgs-mozilla;
  rust = pkgs.latest.rustChannels.nightly.rust.override {
    targets = [ "riscv64gc-unknown-none-elf" ];
    extensions = [ "rust-src" ];
  };
  pkgs = import sources.nixpkgs {
    overlays = [
      nixpkgs-mozilla
      (self: super: {
        rustc = rust;
        cargo = rust;
      })
    ];
  };
  naersk = pkgs.callPackage sources.naersk { };
in naersk.buildPackage ./.

If I try to build this project, I get the following error:

Error

@nix { "action": "setPhase", "phase": "unpackPhase" }
unpacking sources
unpacking source archive /nix/store/8aaja1qq93a0bvv48bij1n5gcwsblidy-foo
source root is foo
@nix { "action": "setPhase", "phase": "patchPhase" }
patching sources
@nix { "action": "setPhase", "phase": "configurePhase" }
configuring
[naersk] cargo_version (read): 1.51.0-nightly (34170fcd6 2021-02-04)
[naersk] cargo_message_format (set): json-diagnostic-rendered-ansi
[naersk] cargo_release: --release
[naersk] cargo_options: -Z unstable-options
[naersk] cargo_build_options: $cargo_release -j "$NIX_BUILD_CORES" --out-dir out --message-format=$cargo_message_format
[naersk] cargo_test_options: $cargo_release -j "$NIX_BUILD_CORES"
[naersk] RUST_TEST_THREADS: 16
[naersk] cargo_bins_jq_filter: select(.reason == "compiler-artifact" and .executable != null and .profile.test == false)
[naersk] cargo_build_output_json (created): /build/tmp.6WWPHk4QeB
[naersk] crate_sources: /nix/store/yslb0dsplxhn0lyzziy7acpy3gfvl1sy-crates-io
[naersk] RUSTFLAGS: 
[naersk] CARGO_BUILD_RUSTFLAGS: 
[naersk] CARGO_BUILD_RUSTFLAGS (updated): --remap-path-prefix /nix/store/yslb0dsplxhn0lyzziy7acpy3gfvl1sy-crates-io=/sources
[naersk] pre-installing dep /nix/store/a1dz3j1gpk470f1pnh7vd8znxfgm0254-foo-deps-0.1.0
@nix { "action": "setPhase", "phase": "buildPhase" }
building
cargo -Z unstable-options build $cargo_release -j "$NIX_BUILD_CORES" --out-dir out --message-format=$cargo_message_format
error: no matching package named `cfg-if` found
location searched: registry `https://github.com/rust-lang/crates.io-index`
required by package `test v0.0.0 (/nix/store/ll0i3ynrs532lnc7a3hahlsbvj794lcd-rust-1.52.0-nightly-2021-02-07-9778068cb/lib/rustlib/src/rust/library/test)`
[naersk] cargo returned with exit code 101, exiting

[rpearce]

I came across this because I got this warning and tried using the latest stable:

trace: Rust 1.53.0-nightly-2021-04-06:
Pre-aggregated package `rust` is not encouraged for stable channel since it contains almost all and uncertain components.
Consider use `default` profile like `rust-bin.stable.latest.default` and override it with extensions you need.

error: builder for '/nix/store/vvs55mhz40hlkv4vkcrg62vdpi63wpw0-hull-deps-0.1.0.drv' failed with exit code 101;
       last 10 log lines:
       > [naersk] cargo_build_output_json (created): /private/tmp/nix-build-hull-deps-0.1.0.drv-0/tmp.uDXoj1I7Oy
       > [naersk] crate_sources: /nix/store/yz4z6dsblj4i1pawqshn7nmda8zf8m4m-crates-io
       > [naersk] RUSTFLAGS:
       > [naersk] CARGO_BUILD_RUSTFLAGS:
       > [naersk] CARGO_BUILD_RUSTFLAGS (updated): --remap-path-prefix /nix/store/yz4z6dsblj4i1pawqshn7nmda8zf8m4m-crates-io=/sources
       > building
       > cargo -Z unstable-options build $cargo_release -j "$NIX_BUILD_CORES" --out-dir out --message-format=$cargo_message_format
       > error: the `-Z` flag is only accepted on the nightly channel of Cargo, but this is the `stable` channel
       > See https://doc.rust-lang.org/book/appendix-07-nightly-rust.html for more information about Rust release channels.
       > [naersk] cargo returned with exit code 101, exiting

I'm also using https://github.com/oxalica/rust-overlay/ over nixpkgs-mozilla, though it does support the latter's API.

[Shados]

I've ran into the same error as @Stupremee while using build-std to build the stdlib with various options intended to save on binary size; this works fine in a Nix-based development environment, but naersk barfs:

building
cargo -Z unstable-options build $cargo_release -j "$NIX_BUILD_CORES" --out-dir out --message-format=$cargo_message_format
error: no matching package named `cfg-if` found
location searched: registry `https://github.com/rust-lang/crates.io-index`
required by package `std v0.0.0 (/nix/store/mb7rijsslpzffvqcx5jgpiwq7zc64vlv-rust-1.53.0-nightly-2021-04-24-42816d61e/lib/rustlib/src/rust/library/std)`
[naersk] cargo returned with exit code 101, exiting

[blitz]

There is also a relevant discussion on the Nix Discourse: https://discourse.nixos.org/t/build-rust-app-using-cargos-build-std-feature-with-naersk-fails-because-rust-src-is-missing/13161/5

The current workaround is to list all extra dependencies in the cargo file of the project you want to build. This is pretty unwieldy unfortunately.

[bouk]

I've created a PR that aims to make this workflow easier: #328

[bouk]

You can use additionalCargoLock as follows, if you're using rust-overlay and trying to compile a wasm package for multithreading:

let
  rust-toolchain = pkgs.rust-bin.fromRustupToolchainFile ./rust-toolchain.toml;
in
  naersk.buildPackage {
	...

    cargoBuildOptions = x: x ++ [ "-Z" "build-std=core,std,panic_abort,alloc,test" ];
	additionalCargoLock = "${rust-toolchain.availableComponents.rust-src}/lib/rustlib/src/rust/Cargo.lock";
	CARGO_BUILD_TARGET = "wasm32-unknown-unknown";
    RUSTFLAGS = "-C target-feature=+atomics,+bulk-memory,+mutable-globals -C link-arg=--max-memory=4294967296";
  }

And then you don't even need to copy any Cargo.lock but directly use the one from the Rust toolchain you're using

[nmattia]

I haven't tried it out but it looks like the issue should be resolved. Thank you very much @bouk ! Do you think it makes sense to add those instructions to the README?