-
Notifications
You must be signed in to change notification settings - Fork 0
Feat: Added tiered_storage add/list/remove #47
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: mdai_cli
Are you sure you want to change the base?
Conversation
…ttle table for reference
…sn't work yet to add anything, visually works sort of
"clickhouse", | ||
"druid", | ||
"pinot"}, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
These are not formats and shouldn't be included as options that you would be choosing from in the context of choosing s3 or gcs storage.
This applies to the cold storage option as well.
I believe parquet
fits here while store should include options like my-clickhouse
.
}, | ||
Formats: []string{"7zip", | ||
"arc", | ||
"br", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
"br", | |
"brotli", |
Formats: []string{"7zip", | ||
"arc", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Formats: []string{"7zip", | |
"arc", | |
Formats: []string{"7zip", |
"br", | ||
"rar", | ||
"tar", | ||
"GZIP", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
"GZIP", | |
"gzip", |
"rar", | ||
"tar", | ||
"GZIP", | ||
"zpaq", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I had to look this one up. Should it really be included?
"tar", | ||
"GZIP", | ||
"zpaq", | ||
"zst"}, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
"zst"}, | |
"zstd"}, |
boldStyle.Render("Performance") + ": Very slow access (hours or days to retrieve).", | ||
boldStyle.Render("Cost") + ": Extremely low cost per GB, ideal for long-term retention.", | ||
}, | ||
Formats: []string{"7zip", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't think I ever see 7zip mentioned these days.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hm, doing a quick google search for best text based compression and maybe i'm wrong here, it gets mentioned.
If we want anymore, assuming the ones I think should be removed are, there is also snappy, lzo and lz4.
"arc", | ||
"br", | ||
"rar", | ||
"tar", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
"tar", |
tar
isn't a compression format.
Formats: []string{"7zip", | ||
"arc", | ||
"br", | ||
"rar", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
"rar", |
Really like this interface but had some nitpicks. A general question is if the like cold storage is meant to assume the use of the same format as previous selections of hot or something. Because you should need to choose both json/csv/parquet and compression protocol (note, list of compression protocols supported by parquet https://parquet.apache.org/docs/file-format/data-pages/compression/). Really any of the steps hot or cold should give the option to select a compression type. Maybe in hot they'd choose a faster but less efficient compression, or none at all, but makes sense to not be just a format on cold. |
Unable to get the add form to add to configMap. I can see tiered storage on kubectl, but nothing in it.