Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(shwap): Efficient blob retrieval by shwap protocol #4098

Open
walldiss opened this issue Feb 7, 2025 · 0 comments
Open

feat(shwap): Efficient blob retrieval by shwap protocol #4098

walldiss opened this issue Feb 7, 2025 · 0 comments
Assignees
Labels
area:blob enhancement New feature or request

Comments

@walldiss
Copy link
Member

walldiss commented Feb 7, 2025

Description:
Currently, the blob module requires requesting the entire namespace data to retrieve a single blob. Because the namespace may contain multiple blobs, the client node must parse all of that data locally to find the requested blob. And also it needs to recalculate nmt proof of single blob from whole namespace proof.

  • This approach is inefficient and consumes unnecessary bandwidth. Instead, the blob should be transmitted directly.
  • NMT does not support proof recalculation. Current API returns namespace row proof instead of blob proof.

Efficient blob retrieval can be implemented using shwap, which will solve both issues. Server has capabilities to read specified blob as well as calculate blob proof, because it has access to full ODS data.

Implementation Plan:

  1. Shwap protocol changes

    • Define new message types for BlobRequest and BlobResponse to enable more efficient blob transmission.
    • The server node should serve the blob as-is.
    type BlobID struct {
        NamespaceDataID // embedded
        Commitment      []byte
    }
    message BlobData {
        blob NamespaceData // NamespaceData type can be reused
    }
  2. Accessor interface

    • Add a GetBlob() method to the Accessor interface and all corresponding implementations.
  3. Full node adjustments

    • Update the full node code (e.g., ODS, ODSQ4 files) to provide an efficient way to read a single blob.
    • Calculate blob proof(s) on server using nmt tree builder
  • These changes will significantly reduce bandwidth usage and improve overall performance for single blob retrieval.
  • Returned proof will be for blob only.
@walldiss walldiss added area:blob enhancement New feature or request labels Feb 7, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:blob enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

3 participants