From 9b395f1a2f652bbc359849b10b864cc55e219065 Mon Sep 17 00:00:00 2001 From: Michal Nazarewicz Date: Fri, 3 Jun 2022 04:27:10 +0200 Subject: [PATCH 1/2] contribute-nearcore: add info on ROCKSDB_LIB_DIR and other minor updates MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Most notably, add a section about compiling while using rocksdb library provided by the system. Based on my observation, this helps with OOMs. I think (though I haven’t tested it) that rust-rocksdb builds rocksdb library with LTO enabled which consumes bunch of memory. While at it, make some minor fixes to the document. Change ‘bash’ preformatted blocks to ‘console’ where the latter is more appropriate and remove mention of `start_localnet.py` script which no longer exists. --- docs/community/contribute/nearcore.md | 104 ++++++++++++++++++-------- 1 file changed, 71 insertions(+), 33 deletions(-) diff --git a/docs/community/contribute/nearcore.md b/docs/community/contribute/nearcore.md index dd1d0ecabe0..2903271d994 100644 --- a/docs/community/contribute/nearcore.md +++ b/docs/community/contribute/nearcore.md @@ -38,8 +38,8 @@ cd nearcore Navigate to the root of the repository, and run: ```bash -cargo run --package neard --bin neard -- init -cargo run --package neard --bin neard -- run +cargo run -p neard -- init +cargo run -p neard -- run ``` This will setup a local chain with `init` and will run the node. @@ -63,10 +63,11 @@ If you are using CLion IDE you can configure it to run rustfmt automatically eve ## Testing {#testing} -To run NEARCore node in the testing mode, for example to test it or for development of `near-api-js` or `near-cli` you can use scripts that sets up special tests-only local testnet: +To run NEARCore node in the testing mode, for example to test it or for development of `near-api-js` or `near-cli` you can use commands that sets up special tests-only local testnet: ```bash -./scripts/start_localnet.py +cargo run -p neard -- localnet -v1 +cargo run -p neard -- run ``` This sets up a new single node testnet, with predetermined private key of the validator and turns on "fast" mode of block production to make development and testing easy. @@ -76,7 +77,7 @@ This sets up a new single node testnet, with predetermined private key of the va Many times in development of the node it's useful to see detailed logs about what is happening. `neard` binary has `--verbose` mode to show more details about what is happening: ```bash -cargo run --package neard --bin neard -- --verbose=true run +cargo run -p neard -- --verbose= run ``` You can also use the `RUST_LOG` environment variable, with `env_logger` [semantics](https://docs.rs/env_logger/0.6.0/env_logger/#enabling-logging) to override the log level for specific targets. `RUST_LOG` can also be used in integration tests which spawn runnable apps. @@ -115,7 +116,7 @@ Official image is published at `nearprotocol/nearcore` > > Ask it on StackOverflow! -# How to speed up rust compilation time +## How to speed up rust compilation time This describes on how to improve nearcore compilation time without having to modify Cargo.toml. It's possible to override Cargo.toml setting by setting @@ -124,14 +125,14 @@ modify Cargo.toml. It's possible to override Cargo.toml setting by setting ### Default build {#default-build} -```bash -# cargo clean -# time RUSTFLAGS= RUSTC_WRAPPER= cargo build -p neard --release +```console +$ cargo clean +$ time RUSTFLAGS= RUSTC_WRAPPER= cargo build -p neard --release real 2m13.773s user 44m49.204s sys 1m22.583s -# touch */*/*/*.rs -# time RUSTFLAGS= RUSTC_WRAPPER= cargo build -p neard --release +$ touch */*/*/*.rs +$ time RUSTFLAGS= RUSTC_WRAPPER= cargo build -p neard --release real 0m38.021s user 9m37.045s sys 0m5.302s @@ -149,14 +150,14 @@ lto=off`. Requires installing lld linker. -```bash -# cargo clean -# time RUSTFLAGS='-C lto=off -C link-arg=-fuse-ld=lld' RUSTC_WRAPPER= cargo build -p neard --release +```console +$ cargo clean +$ time RUSTFLAGS='-C lto=off -C link-arg=-fuse-ld=lld' RUSTC_WRAPPER= cargo build -p neard --release real 1m50.802s user 36m50.251s sys 1m19.069s -# touch */*/*/*.rs -# time RUSTFLAGS='-C lto=off -C link-arg=-fuse-ld=lld' RUSTC_WRAPPER= cargo build -p neard --release +$ touch */*/*/*.rs +$ time RUSTFLAGS='-C lto=off -C link-arg=-fuse-ld=lld' RUSTC_WRAPPER= cargo build -p neard --release real 0m28.951s user 6m56.670s sys 0m4.307s @@ -166,32 +167,69 @@ sys 0m4.307s Works only with nightly compiler. -```bash -# cargo clean -# time RUSTFLAGS='-C lto=off -C link-arg=-fuse-ld=lld -Zshare-generics=y' RUSTC_WRAPPER= cargo build -p neard --release +```console +$ cargo clean +$ time RUSTFLAGS='-C lto=off -C link-arg=-fuse-ld=lld -Zshare-generics=y' RUSTC_WRAPPER= cargo build -p neard --release real 1m42.999s user 33m31.341s sys 1m25.773s -# touch */*/*/*.rs -# time RUSTFLAGS='-C lto=off -C link-arg=-fuse-ld=lld -Zshare-generics=y' RUSTC_WRAPPER= cargo build -p neard --release +$ touch */*/*/*.rs +$ time RUSTFLAGS='-C lto=off -C link-arg=-fuse-ld=lld -Zshare-generics=y' RUSTC_WRAPPER= cargo build -p neard --release real 0m23.501s user 4m30.597s sys 0m3.804s ``` +### Use system-provided rocksdb + +By default, building `neard` will result in compiling `librocksdb` +which takes up some non-negligible amount of memory especially when +building on a system with many cores. It’s possible to configure +build process to provide system-provided library instead. This is +done by setting `ROCKSDB_LIB_DIR` environment variable to location +where `librocksdb.a` file is installed: + +```console +$ cargo clean +$ time ROCKSDB_LIB_DIR=/usr/lib/x86_64-linux-gnu cargo build -p neard --release +real 1m31.014s +user 30m46.544s +sys 1m26.932s +$ touch */*/*/*.rs +$ time ROCKSDB_LIB_DIR=/usr/lib/x86_64-linux-gnu cargo build -p neard --release +real 0m35.061s +user 9m7.968s +sys 0m10.486s +``` + +Note that the system must provide a recent version of the library +which, depending on operating system you’re using, may require +installing packages from testing branches. For example, on Debian it +requires installing `librocksdb-dev` from `experimental` version: + +```bash +echo 'deb http://ftp.debian.org/debian experimental main contrib non-free' | + sudo tee -a /etc/apt/sources.list +sudo apt update +sudo apt -t experimental install librocksdb-dev + +ROCKSDB_LIB_DIR=/usr/lib/x86_64-linux-gnu +export ROCKSDB_LIB_DIR +``` + ### Cache results from previous compilations using sccache {#cache-results-from-previous-compilations-using-sccache} Requires installing sscache. If you want to compile sccache with `cargo install sccache` make sure you use stable version of Rust. -```bash -# cargo clean -# rm -rf ~/.cache/sccache/ -# time RUSTFLAGS='-C lto=off -C link-arg=-fuse-ld=lld -Zshare-generics=y' RUSTC_WRAPPER=sccache cargo build -p neard --release +```console +$ cargo clean +$ rm -rf ~/.cache/sccache/ +$ time RUSTFLAGS='-C lto=off -C link-arg=-fuse-ld=lld -Zshare-generics=y' RUSTC_WRAPPER=sccache cargo build -p neard --release real 2m6.452s user 3m24.797s sys 0m30.919s -# cargo clean -# time RUSTFLAGS='-C lto=off -C link-arg=-fuse-ld=lld -Zshare-generics=y' RUSTC_WRAPPER=sccache cargo build -p neard --release +$ cargo clean +$ time RUSTFLAGS='-C lto=off -C link-arg=-fuse-ld=lld -Zshare-generics=y' RUSTC_WRAPPER=sccache cargo build -p neard --release real 0m24.292s user 3m3.627s sys 0m27.619s @@ -201,9 +239,9 @@ sys 0m27.619s ### Setting for building release binary with debug symbols and reduced inlining {#setting-for-building-release-binary-with-debug-symbols-and-reduced-inlining} -```bash -# cargo clean -# time RUSTFLAGS='-C lto=off -C link-arg=-fuse-ld=lld -Zshare-generics=y -C inline-threshold=25 -C debuginfo=2' RUSTC_WRAPPER=sccache cargo build -p neard --release +```console +$ cargo clean +$ time RUSTFLAGS='-C lto=off -C link-arg=-fuse-ld=lld -Zshare-generics=y -C inline-threshold=25 -C debuginfo=2' RUSTC_WRAPPER=sccache cargo build -p neard --release real 1m50.521s user 3m39.398s sys 0m32.069s @@ -214,9 +252,9 @@ sys 0m32.069s Building this way cuts down build time, but the node will likely be too slow to run. This is useful in case you need to build package to run tests or do build within CLion/Intellij. -```bash -# cargo clean -# time RUSTFLAGS='-C lto=off -C link-arg=-fuse-ld=lld -Zshare-generics=y' RUSTC_WRAPPER=sccache cargo build -p neard +```console +$ cargo clean +$ time RUSTFLAGS='-C lto=off -C link-arg=-fuse-ld=lld -Zshare-generics=y' RUSTC_WRAPPER=sccache cargo build -p neard real 1m18.198s user 4m35.409s sys 0m32.220s From 6a15f205a1296d3f82f88bd7c3505bb02eb39fae Mon Sep 17 00:00:00 2001 From: Michal Nazarewicz Date: Fri, 3 Jun 2022 15:47:40 +0200 Subject: [PATCH 2/2] time env --- docs/community/contribute/nearcore.md | 24 ++++++++++++------------ 1 file changed, 12 insertions(+), 12 deletions(-) diff --git a/docs/community/contribute/nearcore.md b/docs/community/contribute/nearcore.md index 2903271d994..4b1c28e30b5 100644 --- a/docs/community/contribute/nearcore.md +++ b/docs/community/contribute/nearcore.md @@ -127,12 +127,12 @@ modify Cargo.toml. It's possible to override Cargo.toml setting by setting ```console $ cargo clean -$ time RUSTFLAGS= RUSTC_WRAPPER= cargo build -p neard --release +$ time env RUSTFLAGS= RUSTC_WRAPPER= cargo build -p neard --release real 2m13.773s user 44m49.204s sys 1m22.583s $ touch */*/*/*.rs -$ time RUSTFLAGS= RUSTC_WRAPPER= cargo build -p neard --release +$ time env RUSTFLAGS= RUSTC_WRAPPER= cargo build -p neard --release real 0m38.021s user 9m37.045s sys 0m5.302s @@ -152,12 +152,12 @@ Requires installing lld linker. ```console $ cargo clean -$ time RUSTFLAGS='-C lto=off -C link-arg=-fuse-ld=lld' RUSTC_WRAPPER= cargo build -p neard --release +$ time env RUSTFLAGS='-C lto=off -C link-arg=-fuse-ld=lld' RUSTC_WRAPPER= cargo build -p neard --release real 1m50.802s user 36m50.251s sys 1m19.069s $ touch */*/*/*.rs -$ time RUSTFLAGS='-C lto=off -C link-arg=-fuse-ld=lld' RUSTC_WRAPPER= cargo build -p neard --release +$ time env RUSTFLAGS='-C lto=off -C link-arg=-fuse-ld=lld' RUSTC_WRAPPER= cargo build -p neard --release real 0m28.951s user 6m56.670s sys 0m4.307s @@ -169,12 +169,12 @@ Works only with nightly compiler. ```console $ cargo clean -$ time RUSTFLAGS='-C lto=off -C link-arg=-fuse-ld=lld -Zshare-generics=y' RUSTC_WRAPPER= cargo build -p neard --release +$ time env RUSTFLAGS='-C lto=off -C link-arg=-fuse-ld=lld -Zshare-generics=y' RUSTC_WRAPPER= cargo build -p neard --release real 1m42.999s user 33m31.341s sys 1m25.773s $ touch */*/*/*.rs -$ time RUSTFLAGS='-C lto=off -C link-arg=-fuse-ld=lld -Zshare-generics=y' RUSTC_WRAPPER= cargo build -p neard --release +$ time env RUSTFLAGS='-C lto=off -C link-arg=-fuse-ld=lld -Zshare-generics=y' RUSTC_WRAPPER= cargo build -p neard --release real 0m23.501s user 4m30.597s sys 0m3.804s @@ -191,12 +191,12 @@ where `librocksdb.a` file is installed: ```console $ cargo clean -$ time ROCKSDB_LIB_DIR=/usr/lib/x86_64-linux-gnu cargo build -p neard --release +$ time env ROCKSDB_LIB_DIR=/usr/lib/x86_64-linux-gnu cargo build -p neard --release real 1m31.014s user 30m46.544s sys 1m26.932s $ touch */*/*/*.rs -$ time ROCKSDB_LIB_DIR=/usr/lib/x86_64-linux-gnu cargo build -p neard --release +$ time env ROCKSDB_LIB_DIR=/usr/lib/x86_64-linux-gnu cargo build -p neard --release real 0m35.061s user 9m7.968s sys 0m10.486s @@ -224,12 +224,12 @@ Requires installing sscache. If you want to compile sccache with `cargo install ```console $ cargo clean $ rm -rf ~/.cache/sccache/ -$ time RUSTFLAGS='-C lto=off -C link-arg=-fuse-ld=lld -Zshare-generics=y' RUSTC_WRAPPER=sccache cargo build -p neard --release +$ time env RUSTFLAGS='-C lto=off -C link-arg=-fuse-ld=lld -Zshare-generics=y' RUSTC_WRAPPER=sccache cargo build -p neard --release real 2m6.452s user 3m24.797s sys 0m30.919s $ cargo clean -$ time RUSTFLAGS='-C lto=off -C link-arg=-fuse-ld=lld -Zshare-generics=y' RUSTC_WRAPPER=sccache cargo build -p neard --release +$ time env RUSTFLAGS='-C lto=off -C link-arg=-fuse-ld=lld -Zshare-generics=y' RUSTC_WRAPPER=sccache cargo build -p neard --release real 0m24.292s user 3m3.627s sys 0m27.619s @@ -241,7 +241,7 @@ sys 0m27.619s ```console $ cargo clean -$ time RUSTFLAGS='-C lto=off -C link-arg=-fuse-ld=lld -Zshare-generics=y -C inline-threshold=25 -C debuginfo=2' RUSTC_WRAPPER=sccache cargo build -p neard --release +$ time env RUSTFLAGS='-C lto=off -C link-arg=-fuse-ld=lld -Zshare-generics=y -C inline-threshold=25 -C debuginfo=2' RUSTC_WRAPPER=sccache cargo build -p neard --release real 1m50.521s user 3m39.398s sys 0m32.069s @@ -254,7 +254,7 @@ to build package to run tests or do build within CLion/Intellij. ```console $ cargo clean -$ time RUSTFLAGS='-C lto=off -C link-arg=-fuse-ld=lld -Zshare-generics=y' RUSTC_WRAPPER=sccache cargo build -p neard +$ time env RUSTFLAGS='-C lto=off -C link-arg=-fuse-ld=lld -Zshare-generics=y' RUSTC_WRAPPER=sccache cargo build -p neard real 1m18.198s user 4m35.409s sys 0m32.220s