Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

doc: added deletion overview with very high level chart. #288 #295

Open
wants to merge 4 commits into
base: main
Choose a base branch
from

Conversation

panta-123
Copy link

Added the deletion overveiw in different chapter.

Few things I am not clear.

  1. When setting TOMBSTONE by cleaner and undertaker , what is the value it sets. I look through the code and not clear to me.

  2. The chart are very crude and may need some imporvement suggestion needed here.

docs/started/concepts/deletion_overview.md Outdated Show resolved Hide resolved
docs/started/concepts/deletion_overview.md Outdated Show resolved Hide resolved
docs/started/concepts/deletion_overview.md Outdated Show resolved Hide resolved
docs/started/concepts/deletion_overview.md Outdated Show resolved Hide resolved
docs/started/concepts/deletion_overview.md Outdated Show resolved Hide resolved
docs/started/concepts/deletion_overview.md Outdated Show resolved Hide resolved
docs/started/concepts/deletion_overview.md Outdated Show resolved Hide resolved
docs/started/concepts/replica_management.md Outdated Show resolved Hide resolved
website/static/img/Repear-repear.png Outdated Show resolved Hide resolved
website/static/img/Undertaker-undertaker.png Outdated Show resolved Hide resolved
@panta-123
Copy link
Author

All the comments has been resolved !!

Copy link
Member

@bari12 bari12 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just one more change :-)

docs/started/concepts/deletion_overview.md Outdated Show resolved Hide resolved
Copy link
Contributor

@voetberg voetberg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the changes! 👍

@voetberg
Copy link
Contributor

@bari12 Can you verify your requested changes are completed?

@voetberg voetberg requested a review from bari12 November 14, 2024 14:35
@bari12
Copy link
Member

bari12 commented Nov 20, 2024

It's good. Thanks @panta-123 and @voetberg
You need to add the file to the sidebars.json though, otherwise it won't show up in the doc


- Rule based deletion:

Each rule has a corresponding lifetime associated with it (default is None). The attribute `expires at` of the rule is the time when the lifetime is set plus the lifetime associated with rule.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Each rule has a corresponding lifetime associated with it (default is None). The attribute `expires at` of the rule is the time when the lifetime is set plus the lifetime associated with rule.
Each rule has a corresponding lifetime associated with it (default is None). The attribute `expires_at` of the rule is the time when the lifetime is set plus the lifetime associated with the rule.

title: Deletion Overview
---

Deletion in rucio can be performed in broadly two ways.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Deletion in rucio can be performed in broadly two ways.
Deletion in Rucio can be performed in broadly two ways.

- DID based deletion:

Additionally, each did also has a lifetime associated with it (default is None). The metadata `expires_at` of the did is calculated similarly to a rule.
The `Undertaker` daemon is responsible for handling expired dids. The daemon checks if none of the associated rules to the did are locked, if not it removes all rules of the did (see section above). The did itself is then removed from the catalog.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
The `Undertaker` daemon is responsible for handling expired dids. The daemon checks if none of the associated rules to the did are locked, if not it removes all rules of the did (see section above). The did itself is then removed from the catalog.
The `Undertaker` daemon is responsible for handling expired DIDs. The daemon checks if none of the associated rules to the DID are locked, if not it removes all rules of the DID (see section above). The DID itself is then removed from the catalog.

Additionally, each did also has a lifetime associated with it (default is None). The metadata `expires_at` of the did is calculated similarly to a rule.
The `Undertaker` daemon is responsible for handling expired dids. The daemon checks if none of the associated rules to the did are locked, if not it removes all rules of the did (see section above). The did itself is then removed from the catalog.

**The did expiration overrules the rule expiration. But the locked rules are protected.**
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
**The did expiration overrules the rule expiration. But the locked rules are protected.**
**The DID expiration overrules the rule expiration. But the locked rules are protected.**


- Greedy

The reaper daemon gets all the replicas with tombstone in the RSE and immediately delete all replicas.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
The reaper daemon gets all the replicas with tombstone in the RSE and immediately delete all replicas.
The reaper daemon gets all the replicas with tombstone in the RSE and immediately deletes all replicas.


- Non-greedy

The reaper daemon first check if the free space is needed in the RSE. The needed free space is the difference of minimum free space (attrribute set for RSE) and actual free space in RSE. Deletion only occurs once free space is needed.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
The reaper daemon first check if the free space is needed in the RSE. The needed free space is the difference of minimum free space (attrribute set for RSE) and actual free space in RSE. Deletion only occurs once free space is needed.
The reaper daemon first checks if the free space is needed in the RSE. The needed free space is the difference of minimum free space (attrribute set for RSE) and actual free space in RSE. Deletion only occurs once free space is needed.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: all other filenames in this asset folder are lowercase; for consistency, this could be renamed to judge-cleaner.png

Copy link
Contributor

@rdimaio rdimaio left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In case future changes need to be done, it would be great to have the source code for the flowcharts as part of the documentation repo. Maybe we could use Mermaid.js to handle this?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"Are rule locked?" should be "Is rule locked?"

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also - "child(s)" should likely be "child(ren)"

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"get list of all replica with tombstone" -> "get list of all replicas with tombstone"

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are two identical "get list of all replicas with tombstone" blocks; the chart could be simplified so that there is only one block, to which you can either get via "Is greedy? YES" or "needed_free_space >= 0"

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants