-
Notifications
You must be signed in to change notification settings - Fork 54
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
doc: added deletion overview with very high level chart. #288 #295
base: main
Are you sure you want to change the base?
Conversation
All the comments has been resolved !! |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just one more change :-)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the changes! 👍
@bari12 Can you verify your requested changes are completed? |
It's good. Thanks @panta-123 and @voetberg |
|
||
- Rule based deletion: | ||
|
||
Each rule has a corresponding lifetime associated with it (default is None). The attribute `expires at` of the rule is the time when the lifetime is set plus the lifetime associated with rule. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Each rule has a corresponding lifetime associated with it (default is None). The attribute `expires at` of the rule is the time when the lifetime is set plus the lifetime associated with rule. | |
Each rule has a corresponding lifetime associated with it (default is None). The attribute `expires_at` of the rule is the time when the lifetime is set plus the lifetime associated with the rule. |
title: Deletion Overview | ||
--- | ||
|
||
Deletion in rucio can be performed in broadly two ways. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Deletion in rucio can be performed in broadly two ways. | |
Deletion in Rucio can be performed in broadly two ways. |
- DID based deletion: | ||
|
||
Additionally, each did also has a lifetime associated with it (default is None). The metadata `expires_at` of the did is calculated similarly to a rule. | ||
The `Undertaker` daemon is responsible for handling expired dids. The daemon checks if none of the associated rules to the did are locked, if not it removes all rules of the did (see section above). The did itself is then removed from the catalog. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The `Undertaker` daemon is responsible for handling expired dids. The daemon checks if none of the associated rules to the did are locked, if not it removes all rules of the did (see section above). The did itself is then removed from the catalog. | |
The `Undertaker` daemon is responsible for handling expired DIDs. The daemon checks if none of the associated rules to the DID are locked, if not it removes all rules of the DID (see section above). The DID itself is then removed from the catalog. |
Additionally, each did also has a lifetime associated with it (default is None). The metadata `expires_at` of the did is calculated similarly to a rule. | ||
The `Undertaker` daemon is responsible for handling expired dids. The daemon checks if none of the associated rules to the did are locked, if not it removes all rules of the did (see section above). The did itself is then removed from the catalog. | ||
|
||
**The did expiration overrules the rule expiration. But the locked rules are protected.** |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
**The did expiration overrules the rule expiration. But the locked rules are protected.** | |
**The DID expiration overrules the rule expiration. But the locked rules are protected.** |
|
||
- Greedy | ||
|
||
The reaper daemon gets all the replicas with tombstone in the RSE and immediately delete all replicas. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The reaper daemon gets all the replicas with tombstone in the RSE and immediately delete all replicas. | |
The reaper daemon gets all the replicas with tombstone in the RSE and immediately deletes all replicas. |
|
||
- Non-greedy | ||
|
||
The reaper daemon first check if the free space is needed in the RSE. The needed free space is the difference of minimum free space (attrribute set for RSE) and actual free space in RSE. Deletion only occurs once free space is needed. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The reaper daemon first check if the free space is needed in the RSE. The needed free space is the difference of minimum free space (attrribute set for RSE) and actual free space in RSE. Deletion only occurs once free space is needed. | |
The reaper daemon first checks if the free space is needed in the RSE. The needed free space is the difference of minimum free space (attrribute set for RSE) and actual free space in RSE. Deletion only occurs once free space is needed. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: all other filenames in this asset folder are lowercase; for consistency, this could be renamed to judge-cleaner.png
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In case future changes need to be done, it would be great to have the source code for the flowcharts as part of the documentation repo. Maybe we could use Mermaid.js to handle this?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
"Are rule locked?" should be "Is rule locked?"
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
also - "child(s)" should likely be "child(ren)"
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
"get list of all replica with tombstone" -> "get list of all replicas with tombstone"
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There are two identical "get list of all replicas with tombstone" blocks; the chart could be simplified so that there is only one block, to which you can either get via "Is greedy? YES" or "needed_free_space >= 0"
Added the deletion overveiw in different chapter.
Few things I am not clear.
When setting TOMBSTONE by cleaner and undertaker , what is the value it sets. I look through the code and not clear to me.
The chart are very crude and may need some imporvement suggestion needed here.