Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CI tests: CairoPie deserialization #39

Closed
wants to merge 25 commits into from
Closed

Conversation

odesenfans
Copy link

TITLE

Description

Description of the pull request changes and motivation.

Checklist

  • Linked to Github Issue
  • Unit tests added
  • Integration tests added.
  • This change requires new documentation.
    • Documentation has been added/updated.
    • CHANGELOG has been updated.

Copy link

github-actions bot commented Jan 18, 2024

Benchmark Results for unmodified programs 🚀

Command Mean [s] Min [s] Max [s] Relative
base big_factorial 2.256 ± 0.009 2.242 2.268 1.00
head big_factorial 2.265 ± 0.018 2.245 2.304 1.00 ± 0.01
Command Mean [s] Min [s] Max [s] Relative
base big_fibonacci 2.242 ± 0.024 2.223 2.294 1.00
head big_fibonacci 2.254 ± 0.033 2.210 2.321 1.01 ± 0.02
Command Mean [s] Min [s] Max [s] Relative
base blake2s_integration_benchmark 8.244 ± 0.059 8.187 8.334 1.00
head blake2s_integration_benchmark 8.541 ± 0.214 8.420 9.141 1.04 ± 0.03
Command Mean [s] Min [s] Max [s] Relative
base compare_arrays_200000 2.287 ± 0.011 2.265 2.299 1.00
head compare_arrays_200000 2.318 ± 0.012 2.300 2.339 1.01 ± 0.01
Command Mean [s] Min [s] Max [s] Relative
base dict_integration_benchmark 1.445 ± 0.006 1.439 1.453 1.00
head dict_integration_benchmark 1.446 ± 0.009 1.433 1.462 1.00 ± 0.01
Command Mean [s] Min [s] Max [s] Relative
base field_arithmetic_get_square_benchmark 1.287 ± 0.007 1.276 1.295 1.00
head field_arithmetic_get_square_benchmark 1.392 ± 0.011 1.379 1.412 1.08 ± 0.01
Command Mean [s] Min [s] Max [s] Relative
base integration_builtins 8.250 ± 0.050 8.160 8.322 1.00
head integration_builtins 8.465 ± 0.096 8.385 8.722 1.03 ± 0.01
Command Mean [s] Min [s] Max [s] Relative
base keccak_integration_benchmark 8.489 ± 0.073 8.417 8.626 1.00
head keccak_integration_benchmark 8.705 ± 0.055 8.635 8.835 1.03 ± 0.01
Command Mean [s] Min [s] Max [s] Relative
base linear_search 2.308 ± 0.021 2.293 2.359 1.00
head linear_search 2.329 ± 0.017 2.311 2.365 1.01 ± 0.01
Command Mean [s] Min [s] Max [s] Relative
base math_cmp_and_pow_integration_benchmark 1.580 ± 0.006 1.571 1.589 1.00
head math_cmp_and_pow_integration_benchmark 1.821 ± 0.021 1.801 1.870 1.15 ± 0.01
Command Mean [s] Min [s] Max [s] Relative
base math_integration_benchmark 1.467 ± 0.027 1.447 1.538 1.00
head math_integration_benchmark 1.636 ± 0.019 1.606 1.659 1.12 ± 0.02
Command Mean [s] Min [s] Max [s] Relative
base memory_integration_benchmark 1.279 ± 0.012 1.265 1.308 1.00
head memory_integration_benchmark 1.292 ± 0.011 1.274 1.312 1.01 ± 0.01
Command Mean [s] Min [s] Max [s] Relative
base operations_with_data_structures_benchmarks 1.603 ± 0.004 1.597 1.610 1.00
head operations_with_data_structures_benchmarks 1.929 ± 0.033 1.898 2.005 1.20 ± 0.02
Command Mean [ms] Min [ms] Max [ms] Relative
base pedersen 607.4 ± 5.6 603.8 622.9 1.02 ± 0.01
head pedersen 593.6 ± 5.7 588.0 607.2 1.00
Command Mean [s] Min [s] Max [s] Relative
base poseidon_integration_benchmark 1.075 ± 0.019 1.063 1.127 1.09 ± 0.02
head poseidon_integration_benchmark 0.985 ± 0.012 0.967 1.001 1.00
Command Mean [s] Min [s] Max [s] Relative
base secp_integration_benchmark 1.929 ± 0.014 1.913 1.964 1.00
head secp_integration_benchmark 1.959 ± 0.014 1.940 1.982 1.02 ± 0.01
Command Mean [ms] Min [ms] Max [ms] Relative
base set_integration_benchmark 753.7 ± 5.3 749.8 768.0 1.04 ± 0.01
head set_integration_benchmark 727.9 ± 2.7 723.2 731.7 1.00
Command Mean [s] Min [s] Max [s] Relative
base uint256_integration_benchmark 4.572 ± 0.074 4.520 4.773 1.00
head uint256_integration_benchmark 4.681 ± 0.065 4.631 4.806 1.02 ± 0.02

@odesenfans odesenfans force-pushed the ml/deserialize-cairo-pie branch 3 times, most recently from 0402e81 to 1db8a29 Compare January 25, 2024 10:23
fmoletta and others added 25 commits February 6, 2024 13:20
…roof_mode (lambdaclass#1597)

* Force an output builtin to be present when running in proof_mode

* Vm changes

* Minor progress + notes

* Append last return value to the output segment in the proof_mode header

* remove handling that is confusing/not needed

* Extend solution to more than one return value

* Remove block of comment

* Update comments

* Add test that was removed when merged

* Skip output builtin in special final stack method

* Manually set the output builtin's segment size

* Move previous solution to where it is needed

* fmt + Improve comments

* Restore comment

* Improve spelling

* Add changelog entry

* Restore fuzzer/Cargo.lock
* Add allow_missing_builtins flag and handle it

* Add + Update tests

* changelog

* Add doc comment for initialize_builtins

* Extend CLI documentation

* Update changelog

* Update code behind feature flags

* Update tests

* Update tests

* Fix

* Fix

* Fix

* fmt
* Create a cairo_run for cairo1-run crate + Refactor cairo1 execution into different functions

* Clippy

* Merge imports + Impl Default for Cairo1RunConfig

* Remove allow unused imports

* Add changelog entry

* Use FuncArg slice instead of FuncArgs wrapper in public api

* Update changelog
…urn values when running in proof-mode (lambdaclass#1602)

* Create a cairo_run for cairo1-run crate + Refactor cairo1 execution into different functions

* Clippy

* Test first draft

* Merge imports + Impl Default for Cairo1RunConfig

* Remove allow unused imports

* Fix test

* Add a case for each test file

* Improve test

* Clippy

* Add changelog entry

* Use FuncArg slice instead of FuncArgs wrapper in public api

* Update changelog

* remove test case for ghost file

* Fix misleading comment
…bdaclass#1614)

* Use locked dependencies to install cargo-nextest

* Also lock wasm-pack
* Update rust version to 1.74.1

* cargo fmt -all

* Update Makefile

* cargo clippy --fix

* fix cargo clippy

* restore fuzzer/Cargo.lock

---------

Co-authored-by: Pedro Fontana <[email protected]>
* Update cairo-lang version to 2.5.3

* Fix cairo programs

* Update rust changes

* update CHANGELOG.md

* typo

* Fix array_append.cairo program

* Update rust version to 1.74.1

* cargo fmt -all

* Update Makefile

* cargo clippy --fix

* fix cargo clippy

* restore fuzzer/Cargo.lock

* restore cairo1-run

* restore cairo programs

* Restore Makefile

* Restore cairo1-run/Makefile

* restore vm/src/tests/cairo_1_run_from_entrypoint_tests.rs

* Revert "restore cairo programs"

This reverts commit d3c9836.

* Update cairo1-run crate

* Revert "restore vm/src/tests/cairo_1_run_from_entrypoint_tests.rs"

This reverts commit 4212f1e.

* fix uint256_div_mod entrypoint

---------

Co-authored-by: Pedro Fontana <[email protected]>
* change felt for maybe relocatable

* add changelog and improve code

* Add unit test

* cippy

---------

Co-authored-by: Pedro Fontana <[email protected]>
Updates the README file to reflect the migration of Cairo VM to the latest implementation in Rust.
…ss#1619)

* Add docs label to changelog workflow skiplist

* Update changelog.yml

---------

Co-authored-by: Pedro Fontana <[email protected]>
* Update hint_processor docs

* Remove unecessary comma

* Update README.md
* Replace symlinks with script

* Add target to deps

* Create symlimks in workflow

* Remove script

* Simplify

* Debug

* Try

* Reorder targets

* Try fix

* Use gitkeep to push proof_programs directory

* restore command

* Change target name

* Restore

* Update workflow

* Always build symlinks

* Create symlinks before fetching from cache

* Create symlinks before fetching from cache before computing traces

* Create symlinks before fetching from cache before running comparisons

* Remove new symlinks

* Create symlinks before fetching from cache clippy

* Use artifacts to ensure ther eis no hash mismatch when retrieving files

* Fix
* Impl Display for MemorySegmentManager

* Remove old code + Add changelog entry

* Fix

* fmt

* Update test

---------

Co-authored-by: Pedro Fontana <[email protected]>
Problem: the Python VM generates Cairo PIEs as ZIP archives containing
several JSON files and the memory as a binary file. We do not have a
solution yet to deserialize these files into CairoPie objects.

Solution: add a new `CairoPie::from_file(path)` method that reads the
ZIP file and extracts its contents.
Added a `from_bytes` class method to build a `CairoPie` method in
addition to the existing `from_file` method.
Problem: Deserializing the PIE additional data as a hashmap of
`BuiltinAdditionalData` enums because of an issue with deserializing
untagged unions in `serde`
(see serde-rs/json#1103).

Solution: add a new `AdditionalData` struct with explicit fields for each
builtin, circumventing the untagged union issue. This solution has the
advantage of always associating the correct data type for each builtin
(it's not possible anymore to associate a builtin with a different data
type), but requires modifications if a new builtin is added.
Problem: the ECDSA/signature builtin additional data is stored
internally as a hashmap, but the Python VM stores it as a vector of
tuples.

Solution: Add a `SignatureBuiltinAdditionalData` struct and implement a
custom deserializer for it that can take either a hashmap or a vector.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants