Swagger: scylla-manager, move towards just restored bytes #4214
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
In overall/keyspace/table progress we were reporting both downloaded and restored bytes. There are two problems with it: 1. Users does not really care for the distinction between downloaded and restored bytes - they just want to know the amount of bytes that have been successfully restored. 2. With new native Scylla API restore, we no longer separate restoration into download/load&stream stages. That's why we should move towards reporting just the amount of successfully restored bytes, and not keep on adding another fields to our API for each new type of restore. This commit: - Adds restored_bytes/duration to host progress, since it was lacking there. It does not deprecate the downloaded/streamed fields because they were just added in the previous minor release, so that would be confusing. Instead, for now we allow for host progress to contain detailed operation bandwidth information. - Deprecates downloaded bytes in overall/keyspace/table progress, as we want to move towards observing just the restored bytes. A general note is that we should use metrics for monitoring performance, not the sctool progress cmd, but since we already started going in this direction, we need to support it.
There was a problem hiding this comment.
Why do you want to deprecate some of the fields ?
Why not to introduce new fields (you actually did it) handling progress of the restore with Scylla API only ?
downloaded properties cannot be deprecated yet. (even though the deprecation here is just a description).
Would be worth to extend the description of "restored_bytes" and "restore_duration" with an information that it's part of the restore handled by with ScyllaAPI.
karol-kokoszka
approved these changes
Jan 21, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
In overall/keyspace/table progress we were reporting both downloaded and restored bytes. There are two problems with it:
Users does not really care for the distinction between downloaded and restored bytes - they just want to know the amount of bytes that have been successfully restored.
With new native Scylla API restore, we no longer separate restoration into download/load&stream stages.
That's why we should move towards reporting just the amount of successfully restored bytes, and not keep on adding another fields to our API for each new type of restore.
This commit:
A general note is that we should use metrics for monitoring performance, not the sctool progress cmd, but since we already started going in this direction, we need to support it.