Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Limitation on the amount of tuple allowed to be uploaded #45

Open
sbckr opened this issue Jan 30, 2023 · 0 comments
Open

Limitation on the amount of tuple allowed to be uploaded #45

sbckr opened this issue Jan 30, 2023 · 0 comments
Labels
good first issue Denotes an issue ready for a new contributor. kind/cleanup Categorizes issue or PR as related to cleaning up code, process, or technical debt. priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release. triage/accepted Indicates an issue or PR is ready to be actively worked on.

Comments

@sbckr
Copy link
Member

sbckr commented Jan 30, 2023

The maximum number of tuples in a so-called TupleChunk is limited to 100000 (hardcoded in DefaultCastorUploadClient) which highly affects usability and eventually performance when uploading tuples.

Also the Websocket message size is capped at 10MB, which further restricts upload options. (See Castor-Service application.yaml)
@sbckr sbckr added the good first issue Denotes an issue ready for a new contributor. label Jan 30, 2023
@github-actions github-actions bot added the needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. label Jan 30, 2023
@strieflin strieflin added needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. and removed needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. labels Jan 30, 2023
@strieflin strieflin changed the title limitation on the amount of tuple allowed to be uploaded Limitation on the amount of tuple allowed to be uploaded Feb 13, 2023
@strieflin strieflin added kind/cleanup Categorizes issue or PR as related to cleaning up code, process, or technical debt. priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release. triage/accepted Indicates an issue or PR is ready to be actively worked on. and removed needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. labels Feb 13, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
good first issue Denotes an issue ready for a new contributor. kind/cleanup Categorizes issue or PR as related to cleaning up code, process, or technical debt. priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release. triage/accepted Indicates an issue or PR is ready to be actively worked on.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@strieflin @sbckr