-
Notifications
You must be signed in to change notification settings - Fork 972
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
WIP: Shwap prototype #3184
Closed
Closed
WIP: Shwap prototype #3184
Changes from 1 commit
Commits
Show all changes
141 commits
Select commit
Hold shift + click to select a range
d953117
chore(moddas): speed up sampling for LN
Wondertan f122a71
prototype
Wondertan 7237ad7
feat(modp2p): listen on WebTransport by default
Wondertan ab6bec8
lint
Wondertan 2f08bdd
now test verifies all the share proofs
Wondertan b33515e
refactor sampling protocol and use proto for serialization
Wondertan 59aa730
docs and tests for ipldv2
Wondertan 1b3d881
add support for col proofs sampling
Wondertan b53769b
blockstore impl and various cleanups and improvements
Wondertan ce31854
initial support for ODS Mode
Wondertan 3ab6b37
implement axis sampling
Wondertan 041ed3d
introduce File interface and decouple ipldv2 tests from on disk file
Wondertan 1601460
use height as block id
Wondertan 600d186
chore: extract proto helper
Wondertan 6673564
successful experiment with request size shortening for axis sampling
Wondertan 830860d
docs fix
Wondertan 9ffb284
request size optimization for share sample
Wondertan 52f3ab9
refactor AxisID away and many more improvements
Wondertan 21bd2fc
remove serialization ambigiouty and ensure there is only one serializ…
Wondertan cdbd694
cleanup proto field names
Wondertan c3e8450
namespace mh
Wondertan 8a66fd5
namespace mh but finished and tested
Wondertan bbcd956
lol
Wondertan 583481b
pass by value and cid must constructors
Wondertan a28cfef
fix data id test
Wondertan f6db8f9
blockservice constructor
Wondertan 9a8b5ed
implement Getter and tests for it
Wondertan 0d4dd27
rename to shwap
Wondertan c96579a
ensure only shares a cached in blockstore
Wondertan bb034b3
add sessions
Wondertan e6d39dc
create store v2 file interface (#2989)
walldiss 82150f2
feat(store/mem_file): add in-memory eds file implementation (#2992)
walldiss 4feea07
add ods eds files
walldiss e34394b
single read of ods for recomputed axes
walldiss 698ae46
reduce parity allocations
walldiss 90672ec
reuse allocated memory
walldiss 0a9daec
minor rafactoring
walldiss fd82475
allow Store implementation to choose proof axis for Share
walldiss 17e9821
move shareWithProof outside to share pkg
walldiss 8b29af3
allow Store implementation to choose proof axis for Share
walldiss eae5359
remove option to select proof axis from store interface
walldiss daf4bcc
fix shadow subslicing by data copy
walldiss 4d71dad
allow reconstructSome using direct reedsolomon
walldiss fc082f4
protocol updates:
Wondertan 0f61c50
Merge branch 'update-store-interface' into vlad/shwamp-prototype
walldiss 730f05e
Merge branch 'ods-file' into vlad/shwamp-prototype
walldiss 6655b1d
Share with proof
walldiss c8e1926
Switch implementation to reconstructSome
walldiss ec71e12
add codec benchmark
walldiss 74ab0b9
add different sizes to codec benchmark
walldiss 7183820
Merge branch 'ods-file' into vlad/shwamp-prototype
walldiss ed9b593
update EDSFile implementations to support new Share interface
walldiss f5c1183
- update file interface
walldiss 1d21a22
- add store
walldiss 1272917
add cache
walldiss 4190362
add shwamp multiplexer
walldiss 5c6aa8d
add blockstore
walldiss e332f0a
add support for store in shrex
walldiss 4ab0fbf
add support for store in store getter
walldiss 2d5299a
add support for store in availability
walldiss 85ba38d
update mem file
walldiss 8f0315b
lots of changes
walldiss 2f5563e
add store tests and benchmarks
walldiss efd829a
fix hashing for in-mem proofs cache
walldiss da5533b
fix shrex tests
walldiss c004096
add non-inclusion tests to file
walldiss fb93edb
fix shwap tests
walldiss 6ad1791
handle empty sqaure in store and getters
walldiss 7d85083
refactor shrexNd client to use rowIdx
walldiss 8bde751
limit reader by known header instead of the one send over the wire
walldiss 85e069b
add reader tests to all types of files
walldiss 70dddf6
add proper prealloc for shwamp data request builder
walldiss 0c8bc88
fix offset calc for file streaming
walldiss f3186f1
add streaming for mem file
walldiss f1556a4
add support for empty root in getters and store
walldiss 05e5a5e
sort imports
walldiss ad117d7
fix shrexnd tests
walldiss 4052a00
various tests fixes and improvements
walldiss 134c554
store empty heights in availability
walldiss 9a6d1a0
remove proofs collection into proofAdder
walldiss cf2d29e
close file after read everywhere
walldiss d79338e
put heights inside blocks folder
walldiss 6b7080e
add concurrency safety for cached file
walldiss f8bbaca
improve logging for shrex eds
walldiss eb6e377
log amount of written bytes by server when stream interrupted
walldiss 02b2e80
log amount of bytes read from stream
walldiss dd8a225
remove extra alloc in ods reader
walldiss 2f7cb65
remove height method from edsdile
walldiss db0bb88
handle blocks duplicates in store
walldiss aeafcad
fix core tests
walldiss ae6a0af
add test blockstore for shwap and fix shwap roundtrip tests
walldiss 9a0cc1d
close files inside blockstore instead of shwap handlers
walldiss 72c4d5e
sort imports
walldiss 3113f73
remove old store benchmarks
walldiss 1a2256c
fix nodebuilder tests
walldiss c7dd8b3
some todos to rework availability tests
walldiss d148bad
fix shrex-eds tests
walldiss 2886665
refactor shwap blockstore test
walldiss 035a272
add validating file
walldiss 9e29a18
wrap opened files in store with validation file and cache
walldiss db97ae6
store size and datahash in close_once_file
walldiss f2d4704
minor cleanup and renames
walldiss dfe79fd
add comment how server side shwap requests validation should work
walldiss e6c64df
iterate shrex-sub version
walldiss 25250c8
version bitswap
walldiss 6e654df
add share size validation to readShares
walldiss 3e91d02
bump discovery dht tag version
walldiss 777564d
fix validation file test
walldiss 743fed2
aligh shwap with the spec
Wondertan 1b1c9a1
Merge remote-tracking branch 'celestia/shwap-prototype' into shwap-pr…
walldiss 7021639
use AxisHalf type in file interface
walldiss 3e71d57
use AxisHalf in file implementations
walldiss 74ed5c0
add q1q4 file
walldiss ab92e82
rename wrapping files to middleware pattern
walldiss 28fe967
use Q1Q4 file in store
walldiss 3b46993
remove redundant files from eds store
walldiss ba2f512
add edsid
Wondertan 960ea10
extract ErrOperationNotSupported to getter interface
walldiss a0c885b
aggregate testing utils in the same folder
walldiss 683bb93
- register verifier on message creation
walldiss 83beea0
refactor shwap getter
walldiss 772ec00
Merge remote-tracking branch 'celestia/shwap-prototype' into shwap-pr…
walldiss 3ccf47b
aggregate testing utils in the same folder
walldiss 6f7fdd2
rework retriever and add reconstruction getter
walldiss 214392f
Merge remote-tracking branch 'celestia/shwap-prototype' into shwap-pr…
walldiss 02030bf
feat(pruner): Implement `full` and `bridge` node pruning (#3150)
renaynay 2ba8ec6
use rlock in retriever
walldiss 04ad041
use reconstruction getter in full and light nodes
walldiss 84241a4
Merge branch 'shwap-prototype' into prototype-v2
walldiss eef869d
address Hlibs comments
walldiss 6394d37
move retriever test to proper pkg
walldiss 09e93d7
integrate with pruner
walldiss 39c7276
remove redundant panic on marshaling error
walldiss 948013c
remove generics from shwap verification code
walldiss 6f5db44
sort imports
walldiss 7ea43f8
improve locking in store
walldiss b4b5a03
store roots instead of verifier
walldiss 623a3d3
remove unnecessary key methods
Wondertan a02f31d
wrap file on get in store
walldiss 94f1f02
refactor file wrapping in store
walldiss 7580cdf
add validation conditions to shrex-nd server
walldiss File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
little reminder to godoc this