-
Notifications
You must be signed in to change notification settings - Fork 4
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
(docs): Add lakeFS-spec and lakeFS-SDK transaction differences #252
Conversation
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #252 +/- ##
==========================================
- Coverage 94.10% 94.08% -0.02%
==========================================
Files 5 5
Lines 407 406 -1
Branches 72 72
==========================================
- Hits 383 382 -1
Misses 15 15
Partials 9 9 ☔ View full report in Codecov by Sentry. |
I cannot directly comment on code not (yet) affected by this PR. In the |
I guess it's not necessary, but it does not hurt, either, so I'm unsure. If you can verify that this does not break any tests, I'm happy to see it removed. |
docs/guides/transactions.md
Outdated
@@ -6,6 +6,11 @@ A transaction is basically a context manager that collects all file uploads, def | |||
They are an "all or nothing" proposition: If an error occurs during the transaction, none of the queued files are uploaded. | |||
For more information on fsspec transactions, see the official [documentation](https://filesystem-spec.readthedocs.io/en/latest/features.html#transactions). | |||
|
|||
!!! info | |||
The transactions in `lakeFS-spec` are different from the transactions in the [lakeFS-SDK](https://docs.lakefs.io/integrations/python.html#transactions), which were added in `v0.2.0`. | |||
- `lakeFS-SDK` transactions create an ephemeral branch, perform the operations in the context block on this branch and, upon exiting the context manager, merge it back into the source branch. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
- `lakeFS-SDK` transactions create an ephemeral branch, perform the operations in the context block on this branch and, upon exiting the context manager, merge it back into the source branch. | |
- High-level `lakefs` SDK transactions create an ephemeral branch, perform the operations in the context block on that ephemeral branch and, upon exiting the context, merge it back into the source branch. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I would leave out the backticks (as mentioned above).
docs/guides/transactions.md
Outdated
!!! info | ||
The transactions in `lakeFS-spec` are different from the transactions in the [lakeFS-SDK](https://docs.lakefs.io/integrations/python.html#transactions), which were added in `v0.2.0`. | ||
- `lakeFS-SDK` transactions create an ephemeral branch, perform the operations in the context block on this branch and, upon exiting the context manager, merge it back into the source branch. | ||
- `lakeFS-spec` transactions collect the actions and perform them one by one directly on the source branch once the context manager is exited. This allows, for example, multiple commits per transaction. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
- `lakeFS-spec` transactions collect the actions and perform them one by one directly on the source branch once the context manager is exited. This allows, for example, multiple commits per transaction. | |
- `lakeFS-spec` transactions collect the actions and perform them one by one directly on the source branch once the context manager is exited. This allows more flexible operations, for example, creating multiple commits per transaction. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As above, no backticks.
@@ -6,6 +6,11 @@ A transaction is basically a context manager that collects all file uploads, def | |||
They are an "all or nothing" proposition: If an error occurs during the transaction, none of the queued files are uploaded. | |||
For more information on fsspec transactions, see the official [documentation](https://filesystem-spec.readthedocs.io/en/latest/features.html#transactions). | |||
|
|||
!!! info |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The placement of this callout is a bit (visually) unfortunate, IMO, since it separates the last sentence of the introduction from the rest, creating a sort-of orphan.
Instead of a list inside the callout, I would simply have two sentences (maybe with * *
emphasis on the package names).
In case you want to keep the list, please include a blank link before the first list item, otherwise the list does not render correctly in Mkdocs.
docs/guides/transactions.md
Outdated
@@ -6,6 +6,11 @@ A transaction is basically a context manager that collects all file uploads, def | |||
They are an "all or nothing" proposition: If an error occurs during the transaction, none of the queued files are uploaded. | |||
For more information on fsspec transactions, see the official [documentation](https://filesystem-spec.readthedocs.io/en/latest/features.html#transactions). | |||
|
|||
!!! info | |||
The transactions in `lakeFS-spec` are different from the transactions in the [lakeFS-SDK](https://docs.lakefs.io/integrations/python.html#transactions), which were added in `v0.2.0`. | |||
- `lakeFS-SDK` transactions create an ephemeral branch, perform the operations in the context block on this branch and, upon exiting the context manager, merge it back into the source branch. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I would leave out the backticks (as mentioned above).
docs/guides/transactions.md
Outdated
!!! info | ||
The transactions in `lakeFS-spec` are different from the transactions in the [lakeFS-SDK](https://docs.lakefs.io/integrations/python.html#transactions), which were added in `v0.2.0`. | ||
- `lakeFS-SDK` transactions create an ephemeral branch, perform the operations in the context block on this branch and, upon exiting the context manager, merge it back into the source branch. | ||
- `lakeFS-spec` transactions collect the actions and perform them one by one directly on the source branch once the context manager is exited. This allows, for example, multiple commits per transaction. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As above, no backticks.
docs/guides/transactions.md
Outdated
!!! info | ||
The transactions in `lakeFS-spec` are different from the transactions in the [lakeFS-SDK](https://docs.lakefs.io/integrations/python.html#transactions), which were added in `v0.2.0`. | ||
- `lakeFS-SDK` transactions create an ephemeral branch, perform the operations in the context block on this branch and, upon exiting the context manager, merge it back into the source branch. | ||
- `lakeFS-spec` transactions collect the actions and perform them one by one directly on the source branch once the context manager is exited. This allows, for example, multiple commits per transaction. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
- `lakeFS-spec` transactions collect the actions and perform them one by one directly on the source branch once the context manager is exited. This allows, for example, multiple commits per transaction. | |
- `lakeFS-spec` transactions collect the actions and perform them one by one directly on the source branch once the context manager is exited. This allows for a more fine-grained control over the versioning operations performed. For example, multiple commits can be performed as part of a single transaction. |
since that is already taken care of in the __exit__ method of the parent class
Co-authored-by: Nicholas Junge <[email protected]>
Closes https://github.com/aai-institute/MLOps-Engineering/issues/29