Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Don't add measure-level filters to post-agg filters #1452

Merged
merged 8 commits into from
Oct 15, 2024

Conversation

WilliamDee
Copy link
Contributor

@WilliamDee WilliamDee commented Oct 9, 2024

Context

We should respect filter hierarchy and have,

  • measure-level filters should be applied only pre-aggregation
  • metric-level filters and query-level filters should be applied pre AND post-aggregation

Meaning, measure-level filters should not be applied post-aggregation.

Refactor

Changed filter_specs from being a tuple of filter specs all mashed together to a WhereFilterSpecSet where it stores which levels each of the filters were defined at. This was changed inside MetricSpec and MetricInputMeasureSpec.

Resolves SL-2956

@cla-bot cla-bot bot added the cla:yes label Oct 9, 2024
@WilliamDee WilliamDee changed the base branch from main to court/ts-filter-fix October 9, 2024 23:01
Copy link

linear bot commented Oct 9, 2024

ON
subq_15.ds = subq_13.metric_time__day
) subq_16
WHERE metric_time__day > '2020-01-01'
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

booking__is_instant measure-level filter is not applied here

@courtneyholcomb courtneyholcomb force-pushed the court/ts-filter-fix branch 3 times, most recently from a3e07bf to 2affb6d Compare October 10, 2024 00:29
Base automatically changed from court/ts-filter-fix to main October 10, 2024 00:31
@WilliamDee WilliamDee force-pushed the will/filter-hierarchy branch 2 times, most recently from de0a385 to b21dc21 Compare October 10, 2024 22:29
Copy link
Contributor

@courtneyholcomb courtneyholcomb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great!! Should be good to go once you add those check query tests!

@courtneyholcomb
Copy link
Contributor

After rebasing and adding the check query tests, can you run the SQL engine tests on this PR too?

@WilliamDee WilliamDee added the Run Tests With Other SQL Engines Runs the test suite against the SQL engines in our target environment label Oct 12, 2024
@github-actions github-actions bot removed the Run Tests With Other SQL Engines Runs the test suite against the SQL engines in our target environment label Oct 12, 2024
@WilliamDee WilliamDee merged commit 022c88b into main Oct 15, 2024
25 checks passed
@WilliamDee WilliamDee deleted the will/filter-hierarchy branch October 15, 2024 16:49
WilliamDee added a commit that referenced this pull request Oct 15, 2024
## Context
After merging #1452, seems
like snapshots in MF-semantics were missing (CI still passed - odd). But
updated those and also updated the script to update snapshots in MF
semantics as that seems to be missing.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants