Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add custom granularities to YAML spec #10664

Merged
merged 4 commits into from
Sep 17, 2024
Merged

Add custom granularities to YAML spec #10664

merged 4 commits into from
Sep 17, 2024

Conversation

courtneyholcomb
Copy link
Contributor

@courtneyholcomb courtneyholcomb commented Sep 5, 2024

Resolves #9265

Description

The semantic layer is in the process of adding a custom calendar feature that will allow users to query using custom granularities like retail_month. This requires an addition to the YAML spec for models that looks like this:

models:
 - name: my_time_spine
   description: "my favorite time spine"
   time_spine:
      standard_granularity_column: date_day
      custom_granularities:
        - name: fiscal_year
          column_name: fiscal_year_column

The only new piece here is custom_granularities and everything nested under that key. This spec has been approved by the core product team.

The corresponding change to schemas.getdbt.com is here.

Checklist

  • I have read the contributing guide and understand what's expected of me.
  • I have run this code in development, and it appears to resolve the stated issue.
  • This PR includes tests, or tests are not required or relevant for this PR.
  • This PR has no interface changes (e.g., macros, CLI, logs, JSON artifacts, config files, adapter interface, etc.) or this PR has already received feedback and approval from Product or DX.
  • This PR includes type annotations for new and modified functions.

@courtneyholcomb courtneyholcomb added semantic Issues related to the semantic layer artifact_minor_upgrade To bypass the CI check by confirming that the change is not breaking community This PR is from a community member labels Sep 5, 2024
@cla-bot cla-bot bot added the cla:yes label Sep 5, 2024
@dbt-labs dbt-labs deleted a comment from github-actions bot Sep 5, 2024
Copy link

codecov bot commented Sep 5, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 88.90%. Comparing base (9b7f4ff) to head (bf5646f).
Report is 14 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main   #10664      +/-   ##
==========================================
- Coverage   88.94%   88.90%   -0.05%     
==========================================
  Files         180      180              
  Lines       22778    22856      +78     
==========================================
+ Hits        20260    20319      +59     
- Misses       2518     2537      +19     
Flag Coverage Δ
integration 86.10% <100.00%> (-0.18%) ⬇️
unit 62.42% <47.05%> (+0.07%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components Coverage Δ
Unit Tests 62.42% <47.05%> (+0.07%) ⬆️
Integration Tests 86.10% <100.00%> (-0.18%) ⬇️

@courtneyholcomb courtneyholcomb marked this pull request as ready for review September 5, 2024 01:31
@courtneyholcomb courtneyholcomb requested review from a team as code owners September 5, 2024 01:31
@courtneyholcomb courtneyholcomb requested review from wpowers-dbt and removed request for a team September 5, 2024 01:31
@courtneyholcomb courtneyholcomb marked this pull request as draft September 5, 2024 01:31
@courtneyholcomb courtneyholcomb removed the request for review from wpowers-dbt September 5, 2024 01:31
@dbt-labs dbt-labs deleted a comment from github-actions bot Sep 5, 2024
Copy link
Contributor

@QMalcolm QMalcolm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks really great! Thank you for doing this work ❤️ My only request is that I believe we're doing some unnecessary class duplication across artifiacts/resources/v1/model.py and contracts/graph/unparsed.py. Tests look good and everything else looks good.

@dataclass
class TimeSpine(dbtClassMixin):
standard_granularity_column: str
custom_granularities: List[CustomGranularity] = field(default_factory=list)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perfect, thank you for the defaulting behavior

Comment on lines +109 to +114
custom_granularities=[
PydanticTimeSpineCustomGranularityColumn(
name=custom_granularity.name, column_name=custom_granularity.column_name
)
for custom_granularity in time_spine.custom_granularities
],
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The more we do this, the more I wish we had a better way to automatically translate them. Alas unless we're literally using the same object (or subclassing) we'll need the translation logic somewhere

core/dbt/contracts/graph/unparsed.py Outdated Show resolved Hide resolved
Copy link
Contributor

@QMalcolm QMalcolm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me 🚀

@QMalcolm QMalcolm merged commit c6b8f7e into main Sep 17, 2024
60 of 62 checks passed
@QMalcolm QMalcolm deleted the court/custom-gran branch September 17, 2024 18:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
artifact_minor_upgrade To bypass the CI check by confirming that the change is not breaking cla:yes community This PR is from a community member semantic Issues related to the semantic layer
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[CT-3485] Semantic Layer custom calendar config support
2 participants