Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Range 0:0 for the whole payload #313

Closed
roman-khimov opened this issue Dec 10, 2024 · 5 comments · Fixed by #315
Closed

Range 0:0 for the whole payload #313

roman-khimov opened this issue Dec 10, 2024 · 5 comments · Fixed by #315
Labels
enhancement Improving existing functionality I3 Minimal impact S3 Minimally significant U4 Nothing urgent
Milestone

Comments

@roman-khimov
Copy link
Member

Is your feature request related to a problem? Please describe.

I'm always frustrated when I think of how S3 gateway processes GET requests. Most of the time it goes https://github.com/nspcc-dev/neofs-s3-gw/blob/bf5a07fa5c4d6b4bc21a489b600b7548c78db6ab/internal/neofs/neofs.go#L547-L560 route, fetching the whole object from storage (regular GET operation). While in fact the gateway doesn't need and doesn't use the header in any way (see https://github.com/nspcc-dev/neofs-s3-gw/blob/bf5a07fa5c4d6b4bc21a489b600b7548c78db6ab/api/layer/object.go#L107-L123 also).

Describe the solution you'd like

We're missing easy ways to get payload of the object only. Take HTTP range requests for example (https://www.rfc-editor.org/rfc/rfc7233#section-2), much more rich semantics is provided there. We can't do all of it immediately, our range parameters are unsigned, but we can give an additional meaning to otherwise useless 0:0 and return whole payload in this case. S3 will get the data it needs to, no more and no less, we'll send less messages and will be able to benefit from other node-side optimizations like nspcc-dev/neofs-node#2925 (omitting header processing at all).

Describe alternatives you've considered

Keep things as is or extend range requests with additional parameters.

@roman-khimov roman-khimov added enhancement Improving existing functionality U4 Nothing urgent S3 Minimally significant I3 Minimal impact labels Dec 10, 2024
@roman-khimov
Copy link
Member Author

Or 0:Uint64Max maybe.

@cthulhu-rider
Copy link
Contributor

i like treating zero range as whole payload request. It is unlikely that it can be interpreted other way

Or 0:Uint64Max maybe.

maxuint64 is 16EB which is enormous nowadays, but still a range of a specific size. So zero wins to me

@AnnaShaleva
Copy link
Member

Will be useful for NeoFS BlockFetcher as far, +1 from NeoGo side.

@roman-khimov
Copy link
Member Author

Related to #276.

@roman-khimov
Copy link
Member Author

It's not critical, but we have #276 and in general MaxUint64 is the maximum that our protocol supports, so if that's exactly the size of the object --- we're not breaking anything with 0:MaxUint64. But 0:0 is less bytes transmitted over the wire (if anyone cares about them).

🚀 for 0:0
🎉 for 0:MaxUint64

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement Improving existing functionality I3 Minimal impact S3 Minimally significant U4 Nothing urgent
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants