Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add BorshSerializeAsync/BorshDeserializeAsync (WIP) #337

Draft
wants to merge 74 commits into
base: master
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from 69 commits
Commits
Show all changes
74 commits
Select commit Hold shift + click to select a range
d4d92f7
WIP: implement support for async I/O. FIXME: resolve issue with type …
DanikVitek Jan 21, 2025
23a292f
resolved issue with type parameters needing `Send` bound
DanikVitek Jan 21, 2025
5b7f19d
remove redundant `cfg_attr`s
DanikVitek Jan 21, 2025
cf2807c
Implement async I/O for all standard types. TODO: derive, etc.
DanikVitek Jan 21, 2025
447d7bb
remove `async-generic` version
DanikVitek Jan 21, 2025
984b16d
Merge remote-tracking branch 'upstream/master'
DanikVitek Jan 22, 2025
31a4c24
Remove dependency on sync counterparts
DanikVitek Jan 23, 2025
d484aa4
use `impl Future` for async traits
DanikVitek Jan 27, 2025
a2526e2
Implement async traits, using `impl Future`
DanikVitek Jan 27, 2025
ecdbd3c
inline inner `as_bytes` fns; use `write_u32` for `len`
DanikVitek Jan 27, 2025
037bc31
Update workflow Rust version
DanikVitek Jan 27, 2025
e5216c5
Formatting & remove redundant imports
DanikVitek Jan 27, 2025
bb3d210
Rollback remove unused imports. Add `#[allow(unused_imports)]`
DanikVitek Jan 27, 2025
7dacd4c
Add support for mutex/rwlock (de-)serialization (including async coun…
DanikVitek Jan 28, 2025
e322cb5
Fix docs
DanikVitek Jan 28, 2025
6e250a5
set patch `syn` version
DanikVitek Jan 28, 2025
0d0d550
Manually declare async trait variants. Update `async_generic` usage
DanikVitek Jan 28, 2025
a677a17
Fix formatting in macros
DanikVitek Jan 28, 2025
48f7d09
Fixate `async-generic` commit version
DanikVitek Jan 28, 2025
d2ea430
Fixate `async-generic` commit version
DanikVitek Jan 28, 2025
0c85a18
Box errors instead of strings
DanikVitek Jan 28, 2025
09735b2
Remove support locks and unnecessary trait bounds
DanikVitek Jan 28, 2025
f9aed6b
Remove unnecessary trait bounds
DanikVitek Jan 28, 2025
ae05c33
Remove wrong `#[cfg]`
DanikVitek Jan 28, 2025
c57e5e6
use `cfg!` for io error creation
DanikVitek Jan 29, 2025
5da74f3
move private `fn _new` into `pub fn new`
DanikVitek Jan 29, 2025
22bb75b
Fix `async-generic` revision specification
DanikVitek Jan 29, 2025
dc317ad
Partial revert
DanikVitek Jan 29, 2025
8ff7681
replace `cfg!` with `#[cfg]`
DanikVitek Jan 29, 2025
97febbe
move `ToString` import
DanikVitek Jan 29, 2025
a34c3e5
Fix macros formatting issues
DanikVitek Jan 29, 2025
b7f7fec
use `crate::io::Result`
DanikVitek Jan 29, 2025
43bec58
WIP: implement derive macro for async traits
DanikVitek Jan 29, 2025
d64ff6b
Rename async-related features
DanikVitek Jan 29, 2025
22b2bde
use `Cow<'static, str>` in place of `String` to reduce cloning
DanikVitek Jan 29, 2025
c142a0f
Merge pull request #2 from DanikVitek/master
DanikVitek Jan 29, 2025
b98e23d
formatting
DanikVitek Jan 29, 2025
7358d3a
Merge pull request #3 from DanikVitek/master
DanikVitek Jan 29, 2025
a48e9fa
add `pub use` for derive macros for async traits
DanikVitek Jan 29, 2025
2891864
Code cleanup. Reduce `quote!` usage where unnecessary. WIP: update de…
DanikVitek Jan 30, 2025
42fd2b2
Add compilation tests
DanikVitek Jan 30, 2025
8001084
Add default `read_exact` impl. Fix doc test
DanikVitek Jan 30, 2025
2d1c059
Merge pull request #4 from DanikVitek/master
DanikVitek Jan 30, 2025
4e04e09
Fix doc tests
DanikVitek Jan 30, 2025
e13e6ef
Add tests. Use `write_u8` for `variant_idx` in `BorshSerializeAsync` …
DanikVitek Jan 31, 2025
6c78116
fmt
DanikVitek Jan 31, 2025
a13066c
Merge pull request #1 from DanikVitek/async/derive
DanikVitek Jan 31, 2025
475a6ce
Revert rustfmt.toml
DanikVitek Feb 22, 2025
8419d81
Parametrize `needs_bounds_derive` and `collect_bounds` with `IS_ASYNC`
DanikVitek Feb 22, 2025
92316bd
Fix doc tests
DanikVitek Feb 22, 2025
1b663e2
Add description for the `unstable__async`, `unstable__tokio` and `uns…
DanikVitek Feb 22, 2025
fb1e01d
guard async-related code under the `async` feature flag
DanikVitek Feb 22, 2025
f842c46
add docs on IO
DanikVitek Feb 22, 2025
a729695
Fix tests in CI
DanikVitek Feb 22, 2025
49cf096
Fix benchmarks features
DanikVitek Feb 22, 2025
b669baf
Fix clippy warnings
DanikVitek Feb 22, 2025
c5b99e5
fmt
DanikVitek Feb 22, 2025
9f4c5e3
Fix doc links
DanikVitek Feb 22, 2025
f23ed33
Use monospace for types and features
DanikVitek Feb 22, 2025
57a0a40
Fix tests
DanikVitek Feb 22, 2025
584f36f
Revert changes
DanikVitek Feb 22, 2025
bc70f92
Add docs to the `async_io` declaration
DanikVitek Feb 22, 2025
a5e027e
Fix lifetimes
DanikVitek Feb 22, 2025
018c356
fmt
DanikVitek Feb 22, 2025
19e1571
Fix clippy & fmt warnings
DanikVitek Feb 22, 2025
66447cb
Fix docs and doc tests
DanikVitek Feb 22, 2025
7c3eb77
Fix docs and checks in CI. Fix clippy warnings about mixed attributes…
DanikVitek Feb 22, 2025
0a1aab4
Readability fixes. Add test for async array deserialization
DanikVitek Feb 22, 2025
d624435
Readability fix
DanikVitek Feb 22, 2025
6c52ef9
Fix lifetimes. Add `ascii` feature to docs metadata
DanikVitek Feb 28, 2025
da103d3
clippy
DanikVitek Feb 28, 2025
253b868
fmt
DanikVitek Feb 28, 2025
e58f2b9
make `captures` module public
DanikVitek Feb 28, 2025
611e8b7
revert remove `inline`
DanikVitek Mar 4, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .github/test.sh
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,9 @@ pushd borsh
cargo test --no-run
cargo test
cargo test --features derive
cargo test --features derive,unstable__tokio
cargo test --features derive,unstable__async-std
cargo test --features derive,unstable__async 'compile_derives::async_derives'
cargo test --features unstable__schema
########## features = ["ascii"] group
cargo test --features ascii 'roundtrip::test_ascii_strings'
Expand Down
12 changes: 9 additions & 3 deletions .github/workflows/rust.yml
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ jobs:
tests:
strategy:
matrix:
rust_version: [1.67, stable]
rust_version: [1.75.0, stable]
runs-on: ubuntu-20.04

steps:
Expand Down Expand Up @@ -63,7 +63,13 @@ jobs:
steps:
- uses: actions/checkout@v4
- name: Run clippy
run: cargo clippy --features unstable__schema --benches -- -D clippy::all
run: cargo clippy --features rc,ascii,unstable__schema --benches -- -D clippy::all
- name: Run clippy (async)
run: cargo clippy --features rc,ascii,unstable__schema,unstable__async --benches -- -D clippy::all
- name: Run clippy (tokio)
run: cargo clippy --features rc,ascii,unstable__schema,unstable__tokio --benches -- -D clippy::all
- name: Run clippy (async-std)
run: cargo clippy --features rc,ascii,unstable__schema,unstable__async-std --benches -- -D clippy::all

cargo-fmt:
runs-on: ubuntu-20.04
Expand All @@ -82,7 +88,7 @@ jobs:
steps:
- uses: actions/checkout@v4
- name: run cargo doc
run: RUSTDOCFLAGS="-D warnings" cargo doc --features derive,unstable__schema
run: RUSTDOCFLAGS="-D warnings" cargo doc --features derive,rc,ascii,unstable__schema,unstable__async

release-plz:
runs-on: ubuntu-latest
Expand Down
4 changes: 3 additions & 1 deletion Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,7 +1,9 @@
[workspace]
resolver = "2"
members = ["borsh", "borsh-derive", "fuzz/fuzz-run", "benchmarks"]

[workspace.package]
# shared version of all public crates in the workspace
version = "1.5.5"
rust-version = "1.67.0"
edition = "2021"
rust-version = "1.75.0"
7 changes: 2 additions & 5 deletions benchmarks/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ name = "benchmarks"
version.workspace = true
rust-version.workspace = true
authors = ["Near Inc <[email protected]>"]
edition = "2018"
edition.workspace = true
publish = false

# This is somehow needed for command line arguments to work: https://github.com/bheisler/criterion.rs/issues/193#issuecomment-415740713
Expand All @@ -13,7 +13,7 @@ bench = false
[dependencies]
rand_xorshift = "0.2.0"
rand = "0.7.0"
borsh = { path = "../borsh", default-features = false }
borsh = { path = "../borsh", default-features = false, features = ["std", "derive"] }
serde = { version = "1.0", features = ["derive"] }
speedy-derive = "0.5"
speedy = "0.5"
Expand All @@ -39,6 +39,3 @@ harness = false
[[bench]]
name = "object_length"
harness = false

[features]
default = ["borsh/std", "borsh/derive"]
8 changes: 4 additions & 4 deletions benchmarks/benches/bench.rs
Original file line number Diff line number Diff line change
Expand Up @@ -14,8 +14,8 @@ where
+ BorshDeserialize
+ SerdeSerialize
+ SerdeDeserialize<'a>
+ Readable<'a, speedy::Endianness>
+ Writable<speedy::Endianness>
+ Readable<'a, Endianness>
+ Writable<Endianness>
+ 'static,
{
let mut rng = rand_xorshift::XorShiftRng::from_seed([0u8; 16]);
Expand Down Expand Up @@ -71,8 +71,8 @@ where
+ BorshDeserialize
+ SerdeSerialize
+ SerdeDeserialize<'a>
+ Readable<'a, speedy::Endianness>
+ Writable<speedy::Endianness>
+ Readable<'a, Endianness>
+ Writable<Endianness>
+ 'static,
{
let mut rng = rand_xorshift::XorShiftRng::from_seed([0u8; 16]);
Expand Down
4 changes: 2 additions & 2 deletions benchmarks/benches/object_length.rs
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
use benchmarks::{Generate, ValidatorStake};
use borsh::{to_vec, BorshSerialize};
use borsh::BorshSerialize;
use criterion::{criterion_group, criterion_main, BenchmarkId, Criterion, Throughput};
use rand::SeedableRng;

Expand All @@ -11,7 +11,7 @@ where
let mut group = c.benchmark_group(group_name);

let objects: Vec<_> = (0..num_samples).map(|_| T::generate(&mut rng)).collect();
let borsh_datas: Vec<Vec<u8>> = objects.iter().map(|t| to_vec(t).unwrap()).collect();
let borsh_datas: Vec<Vec<u8>> = objects.iter().map(|t| borsh::to_vec(t).unwrap()).collect();
let borsh_sizes: Vec<_> = borsh_datas.iter().map(|d| d.len()).collect();

for i in 0..objects.len() {
Expand Down
Loading
Loading