-
Notifications
You must be signed in to change notification settings - Fork 54
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: make ExpressionHandler::get_evaluator
fallible
#577
base: main
Are you sure you want to change the base?
feat: make ExpressionHandler::get_evaluator
fallible
#577
Conversation
Make get_evaluator() return DeltaResult<Arc<dyn ExpressionEvaluator>> to properly handle potential errors. Update all call sites to handle the Result type and simplify error handling using the ok()? operator where appropriate.
…tions - Modified scan_action_iter to return DeltaResult<impl Iterator> - Fixed code formatting in log_replay.rs files - Reordered imports in mod.rs for better readability - Improved error propagation in transaction.rs Co-Authored-By: Calvin Giroud <[email protected]>
Added warning logs when evaluator creation fails in DataSkippingFilter: - Log stats selector evaluator failures - Log skipping evaluator failures - Log filter evaluator failures This improves debuggability when data skipping does not occur as expected. Co-Authored-By: Calvin Giroud <[email protected]>
…olving conflicts Co-Authored-By: Calvin Giroud <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please update the PR description to correctly follow the breaking change template -- our release management scrapes PR descriptions to build the changelog. It looks like the commented-out template was deleted from the PR description, so you'll need to dig it up from some other PR.
kernel/src/scan/data_skipping.rs
Outdated
.map_err(|e| { | ||
warn!("Failed to create stats selector evaluator: {}", e); | ||
e | ||
}) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
.map_err(|e| { | |
warn!("Failed to create stats selector evaluator: {}", e); | |
e | |
}) | |
.inspect_err(|e| warn!("Failed to create stats selector evaluator: {e}")) |
(more below)
kernel/src/scan/log_replay.rs
Outdated
action_iter | ||
)?; | ||
|
||
Ok(action_iter |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: it's very subjective, but in general I find this sort of multi-line, multi-operation Ok
(or Some
, etc) to be unnecessarily hard to read because of all the nesting. Factoring it out as a value is easier to read (and change later):
let actions = action_iter
...
.filter(...);
Ok(actions)
Wrapping a single function call or struct creation isn't so bad, because conceptually only one thing is happening, e.g. this code above is not particularly difficult to read:
Ok(Arc::new(DefaultExpressionEvaluator {
...
}))
Clippy seems to take a similar stance on monadic chains -- it will almost always force newlines between chained function calls, if more than one of the functions takes args or even if the line gets very long (long before the 100-char line limit).
Updated the PR description, not quite sure if I did it right! Let me know after you review. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM except one new type annotation that seems unnecessary?
…devin-open-source/delta-kernel-rs into devin/1733609186-fallible-get-evaluator
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #577 +/- ##
==========================================
- Coverage 83.46% 83.36% -0.11%
==========================================
Files 74 74
Lines 16858 16854 -4
Branches 16858 16854 -4
==========================================
- Hits 14071 14050 -21
- Misses 2129 2130 +1
- Partials 658 674 +16 ☔ View full report in Codecov by Sentry. |
Fixes #566 . Makes get_evaluator() return DeltaResult<Arc> to properly handle potential errors. This change is intended to support
get_evaluator()
doing eager validation checking, but doesn't do the validation itself.What changes are proposed in this pull request?
Here's a comprehensive summary of all changes made:
How was this change tested?
Tested with
cargo test --all-features --all-targets -- --skip read_table_version_hdfs
. Had an issue with getting Java setup but that test doesn't seem relevant to the PR. No new tests need to be written since it's a no-op refactor.This PR affects the following public APIs
Allows get_evaluator to return a Result type, which opens up the possibility to do eager schema validation.
Additional Context
This PR was entirely written by Devin with a little bit of review from me. Happy to address any feedback and get this over the finish line.
Original Run: https://preview.devin.ai/sessions/e839a4e9bcb444b69b99205babdcf1af