-
Notifications
You must be signed in to change notification settings - Fork 453
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[4189] Fix colliding filters by updating match method for MultiCharSequenceFilter to iterate through all patterns for the most complete match and not return on the first pattern found #4188
Merged
Merged
Changes from 6 commits
Commits
Show all changes
9 commits
Select commit
Hold shift + click to select a range
caeb6f6
Fix colliding filters by updating match method for MultiCharSequenceF…
saad-zaman 6086f86
Remove empty newline
saad-zaman 1b9ea72
Update new unit test to test length of returned byte array
saad-zaman 4b36d5f
Update location of new test added to fold into an existing test
saad-zaman 8f0402a
Simplify code and add test for multiCharSequenceFilter with multiple …
saad-zaman 2222592
Add benchmarks tests for new logic in multicharsequence matches method
saad-zaman 604df47
Add benchmark tests and optimizations to short circuit iterating thro…
saad-zaman dceb8da
Fix linting error
saad-zaman 64fe147
Merge branch 'master' into szaman/M3-2186/fix-colliding-filters
saad-zaman File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -349,6 +349,16 @@ func TestMultiCharSequenceFilter(t *testing.T) { | |
validateLookup(t, f, "12test", true, "12") | ||
validateLookup(t, f, "12test2", true, "12") | ||
validateLookup(t, f, "123book", true, "123") | ||
|
||
f, err = newMultiCharSequenceFilter([]byte("arachne_failures,arachne_failures_by_rack"), false) | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Add a similar test case where There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Added |
||
require.NoError(t, err) | ||
validateLookup(t, f, "arachne_failures", true, "") | ||
validateLookup(t, f, "arachne_failures_by_rack", true, "") | ||
|
||
f, err = newMultiCharSequenceFilter([]byte("arachne_failures,test_arachne_failures"), true) | ||
require.NoError(t, err) | ||
validateLookup(t, f, "arachne_failures", true, "") | ||
validateLookup(t, f, "test_arachne_failures", true, "") | ||
} | ||
|
||
func validateLookup(t *testing.T, f chainFilter, val string, expectedMatch bool, expectedRemainder string) { | ||
|
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@prateek PTAL
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please put the results of the benchmark before and after the change.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
To Justin's point - please add a comment to the code with "After change results"
And in the PR, update description with both before/after results
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Updated the PR description to add benchmark results and a table to help with comparison. Also updated the code to add a small optimization to stop iterating through patterns if an optimal match was found and updated tests to ensure it was working
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Updated table with a more legible benchmark test comparison
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can you add
b.ReportAllocs()
to these benchmarks?Having a >50% impact seems pretty steep for this change, but I'm not seeing any obvious ways to make this function faster. Perhaps this impact is just very high because we're purposely checking the scenario where these "colliding filters" are present. Can we see the before/after numbers for the vanilla case where there aren't any colliding filters?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I ran it with
b.ReportAllocs()
and there are 0 allocs with the old and new logic.Updated the tests to show the change for all benchmark tests related to my code. There is a small difference (~0-15%) in test cases where the prefix/suffix collision does not occur likely due to the additional conditionals present after a prefix or suffix detection is found.