Buildnet WIP (massalabs#4527)

* Buildnet WIP

* Test dir + base case test

* Mock Final State/Improve tests execution (massalabs#4512)

* Use a controller for final state to be able to mock it

* Fix all formatting issues.

* Use more mocks in execution tests

* Fix test nested call usage

* improve some tests execution

* Fix tests async message and compact the test of executed

* Fix more execution tests

* Fix more tests.

* Update more tests

* Finish fixing all executions tests.

* Improve naming

* add more comment

* Fix comment

* Minor addition to test_standalone_selection

* Add test_invalid_roll_distribution

* Minor test_invalid_roll_distribution change

* fix clippy

* Fix double cursor advance (massalabs#4531)

Co-authored-by: sydhds <[email protected]>

* Fix massa db change history handling (massalabs#4533)

* Initial fix and test

To check: if < or <= to allow changes

* fmt + clippy

* fix comments

* Remove debug prints

* Cache miss error rename + u8::MAX

* Added comments about Included and CacheMiss

* Comment + Add StreamingStep::Ongoing to test

* Fix previous tests

* Bump self_cell from 1.0.1 to 1.0.2 (massalabs#4538)

Bumps [self_cell](https://github.com/Voultapher/self_cell) from 1.0.1 to 1.0.2.
- [Release notes](https://github.com/Voultapher/self_cell/releases)
- [Commits](Voultapher/self_cell@v1.0.1...v1.0.2)

---
updated-dependencies:
- dependency-name: self_cell
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Add test code coverage + checksums (massalabs#4470)

* ci: add test code coverage

* ci: add checksums to artifacts

* fix: cargo clean

* fix: trigger ci

* ci: last chance

* ci: skip macOS ...

* ci: silver bullet cache

* ci: add codecov

* ci: clean coverage utility

* ci: reset to main version

* Keep only event count in ExecutionContextSnapshot

Signed-off-by: Jean-François <[email protected]>

* updates

* update packages

* make test_standalone_selection generic (massalabs#4539)

* Create universes for bootstrap (massalabs#4528)

* Use a controller for final state to be able to mock it

* Fix all formatting issues.

* Use more mocks in execution tests

* Fix test nested call usage

* improve some tests execution

* Fix tests async message and compact the test of executed

* Fix more execution tests

* Fix more tests.

* Update more tests

* Finish fixing all executions tests.

* Improve naming

* Create universes for bootstrap

* add more comment

* Add bootstrap whitelist test correctly

* Update to make a correct bootstrap flow

* Change big bootstrap test and cleanup some unused variables

* Version of bootstrap tests with mock well organized

* Update bootstrap check database

* fix all tests bootstrap

* Fix tests massa db

* Update better tests

* modify condition

* Fix readonly (massalabs#4543)

* fix readonly

* update proto dep

* Fix bootstrap deser (massalabs#4537)

* Add length_value check in deserializer

* Update the logic

TODO:
- Fix UT fails
- Remove debug logs
- Refactor the serde of KV in a separate parser

* fmt

* Fix ser

* Remove prints

* Add duplicate key test in deserializer

* Add faulty message tests

* Fix & prints

* Update to take serialized len into account for streamBatch fill

* Rename + add comments

* Use Enum for faulty case instead of int

* Fix clippy

* Adjust node APIs settings (massalabs#4536)

* refactor: adjust APIs settings

* refactor: revert bind public address

* Improve ReadOnly estimations (massalabs#4545)

* Fix read only call & execution estimations

* Update comment

* Set async pool size to 1000 and add async msg fixed costs (massalabs#4546)

Signed-off-by: Jean-François <[email protected]>

* Update genesis

---------

Signed-off-by: dependabot[bot] <[email protected]>
Signed-off-by: Jean-François <[email protected]>
Co-authored-by: Thomas Plisson <[email protected]>
Co-authored-by: AurelienFT <[email protected]>
Co-authored-by: AurelienFT <[email protected]>
Co-authored-by: Sydhds <[email protected]>
Co-authored-by: sydhds <[email protected]>
Co-authored-by: Eitu33 <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Moncef AOUDIA <[email protected]>
Co-authored-by: Jean-François <[email protected]>
Co-authored-by: Damir Vodenicarevic <[email protected]>
Co-authored-by: Moncef AOUDIA <[email protected]>

.config/nextest.toml

@@ -0,0 +1,10 @@
+[profile.ci]
+# Print out output for failing tests as soon as they fail, and also at the end
+# of the run (for easy scrollability).
+failure-output = "immediate-final"
+# Do not cancel the test run on the first failure.
+fail-fast = false
+# retry policy for flaky tests
+retries = { backoff = "fixed", count = 2, delay = "1s" }
+# timeout policy for slow tests
+slow-timeout = { period = "60s", terminate-after = 4, grace-period = "30s" }

.github/workflows/cd.yml

@@ -17,14 +17,14 @@ jobs:
   release:
     strategy:
       matrix:
-        os: [ubuntu-20.04, macOS-latest, windows-latest]
+        os: [ubuntu-latest, macOS-latest, windows-latest]
         platform: [x86_64, arm64]
         include:
-          - os: ubuntu-20.04
+          - os: ubuntu-latest
             platform: x86_64
             target: x86_64-unknown-linux-gnu
             name: release_linux.tar.gz
-          - os: ubuntu-20.04
+          - os: ubuntu-latest
             platform: arm64
             target: aarch64-unknown-linux-gnu
             name: release_linux_arm64.tar.gz
@@ -92,7 +92,23 @@ jobs:
             tar czvf massa_${GITHUB_REF/refs\/tags\//}_${{ matrix.name }} massa
           fi
           cd -
+        working-directory: ${{ github.workspace }}
+      - name: Create Checksums
+        id: checksums
+        run: |
+          checksums=""
+          for file in massa/*.{gz,zip}; do
+            if [ -f "$file" ]; then
+              checksum=$(sha256sum "$file" | cut -d ' ' -f 1)
+              checksums="$checksums\n$checksum $(basename $file)"
+            fi
+          done
+          echo -e "$checksums" > checksums.txt
+        working-directory: ${{ github.workspace }}
       - name: Publish
         uses: softprops/action-gh-release@v1
         with:
-          files: 'massa_*'
+          files: |
+            massa_*.zip
+            massa_*.gz
+            checksums.txt

.github/workflows/ci.yml

@@ -4,7 +4,7 @@ on:
   push:
     branches: [main, staging, trying]
   pull_request:
-    branches: [main, 'testnet_*']
+    branches: [main, 'testnet_*', 'buildnet_wip']
     types:
     - opened
     - reopened
@@ -37,7 +37,7 @@ jobs:
       - uses: Swatinem/rust-cache@v2
         with:
           shared-key: "sanity"
-          save-if: ${{ github.ref_name == 'main' }}
+          save-if: ${{ github.ref_name == 'buildnet_wip' }}
       - uses: actions-rs/cargo@v1
         with:
           command: fmt
@@ -59,7 +59,7 @@ jobs:
       - uses: Swatinem/rust-cache@v2
         with:
           shared-key: "check"
-          save-if: ${{ github.ref_name == 'main' }}
+          save-if: ${{ github.ref_name == 'buildnet_wip' }}
       - uses: arduino/setup-protoc@v1
         with:
           version: '3.x'
@@ -86,7 +86,7 @@ jobs:
       - uses: Swatinem/rust-cache@v2
         with:
           shared-key: "clippy"
-          save-if: ${{ github.ref_name == 'main' }}
+          save-if: ${{ github.ref_name == 'buildnet_wip' }}
       - uses: arduino/setup-protoc@v1
         with:
           version: '3.x'
@@ -194,7 +194,7 @@ jobs:
       - uses: Swatinem/rust-cache@v2
         with:
           shared-key: "massa"
-          save-if: ${{ github.ref_name == 'main' }}
+          save-if: ${{ github.ref_name == 'buildnet_wip' }}
       - uses: arduino/setup-protoc@v1
         with:
           version: '3.x'
@@ -214,6 +214,10 @@ jobs:
         with:
           command: test
           args: --doc
+      - uses: codecov/codecov-action@v3
+        with:
+          files: lcov.info
+          fail_ci_if_error: false
 
   build: # quick hack because bors wrongly detect matrix jobs status
     needs: full
@@ -273,7 +277,7 @@ jobs:
       - uses: Swatinem/rust-cache@v2
         with:
           shared-key: "doc"
-          save-if: ${{ github.ref_name == 'main' }}
+          save-if: ${{ github.ref_name == 'buildnet_wip' }}
       - uses: arduino/setup-protoc@v1
         with:
           version: '3.x'

.gitignore

@@ -8,7 +8,6 @@
 *.swp*
 **/Cargo.lock
 **/target
-lcov.info
 massa-client/config/*
 massa-client/wallet.dat
 massa-node/config/*
@@ -30,4 +29,4 @@ extract_massa_unit_tests_src_*
 massa-client/wallets/*
 # coverage
 *.profraw
-coverage
+coverage