Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(shwap):Add eds streaming #3531

Merged
merged 10 commits into from
Jul 2, 2024
Merged

Conversation

walldiss
Copy link
Member

Add streaming: read/write for files. Read is implemented only for Rsmt2d file by ReadFrom method. There is no usecase so farm to directly read for ods file, so ReadFrom is not added to eds.Accessor interface

@walldiss walldiss added kind:feat Attached to feature PRs shwap labels Jun 26, 2024
@walldiss walldiss self-assigned this Jun 26, 2024
@walldiss walldiss marked this pull request as draft June 27, 2024 10:38
@walldiss walldiss force-pushed the add-eds-streaming branch from ed532f8 to b11b360 Compare July 1, 2024 15:38
@walldiss walldiss marked this pull request as ready for review July 1, 2024 15:39
}

func WithCloser(a Accessor, c io.Closer) AccessorCloser {
return &accessorCloser{a, c}
func AccessorAndStreamer(a Accessor, s Streamer) AccessorStreamer {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When we discussed the idiomatic approach, I think you wanted to find iotest pkg, which has things like TimeoutReader

Copy link
Member Author

@walldiss walldiss Jul 2, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

// TimeoutReader returns ErrTimeout on the second read with no data. Subsequent calls to read succeed.

Not sure how it is relevant. It does not seem to combine 2 interfaces. Or you reffered different topic?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I mean, the pattern is similar to what here

Comment on lines -220 to -228
if len(ax.half.Shares) == 0 {
half, err := c.AxisHalf(ctx, axisType, axisIdx)
if err != nil {
return nil, err
}
ax.half = half
half, err := c.AxisHalf(ctx, axisType, axisIdx)
if err != nil {
return nil, err
}

shares, err := ax.half.Extended()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the reason for this change? I don't follow.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It caches half axis independently inside AxisHalf call

@walldiss walldiss merged commit ba26102 into celestiaorg:shwap Jul 2, 2024
27 checks passed
walldiss added a commit to walldiss/celestia-node that referenced this pull request Jul 6, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind:feat Attached to feature PRs shwap
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants