Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow setting access in config in addition to properties #8635

Merged
merged 4 commits into from
Sep 14, 2023

Conversation

gshank
Copy link
Contributor

@gshank gshank commented Sep 12, 2023

resolves #8383

Problem

Setting "access" had been limited to setting as a node property, but users want to be able to set it in dbt_project.yml

Solution

Create a ModelConfig config with an "access" field. Wire up all of the pieces to allow it to be properly parsed and promoted to a node property.

Checklist

  • I have read the contributing guide and understand what's expected of me
  • I have run this code in development and it appears to resolve the stated issue
  • This PR includes tests, or tests are not required/relevant for this PR
  • This PR has no interface changes (e.g. macros, cli, logs, json artifacts, config files, adapter interface, etc) or this PR has already received feedback and approval from Product or DX

@gshank gshank requested review from a team as code owners September 12, 2023 19:34
@gshank gshank requested review from jzhu13 and MichelleArk and removed request for a team September 12, 2023 19:34
@cla-bot cla-bot bot added the cla:yes label Sep 12, 2023
@gshank gshank marked this pull request as draft September 12, 2023 19:34
@codecov
Copy link

codecov bot commented Sep 12, 2023

Codecov Report

Patch coverage: 100.00% and project coverage change: -0.01% ⚠️

Comparison is base (b39eeb3) 86.61% compared to head (1a3c968) 86.60%.
Report is 57 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #8635      +/-   ##
==========================================
- Coverage   86.61%   86.60%   -0.01%     
==========================================
  Files         174      175       +1     
  Lines       25590    25607      +17     
==========================================
+ Hits        22164    22177      +13     
- Misses       3426     3430       +4     
Flag Coverage Δ
integration 83.46% <100.00%> (-0.02%) ⬇️
unit 65.12% <78.57%> (+0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files Changed Coverage Δ
core/dbt/contracts/graph/model_config.py 92.18% <100.00%> (+0.06%) ⬆️
core/dbt/contracts/graph/nodes.py 96.13% <100.00%> (+<0.01%) ⬆️
core/dbt/parser/base.py 93.57% <100.00%> (+0.12%) ⬆️
core/dbt/parser/schemas.py 92.05% <100.00%> (-0.70%) ⬇️

... and 9 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@gshank gshank marked this pull request as ready for review September 13, 2023 14:46
@gshank gshank added the user docs [docs.getdbt.com] Needs better documentation label Sep 13, 2023
Copy link
Member

@aranke aranke left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mostly LGTM, few nits

Comment on lines 339 to 340
and "access" in config_dict
and config_dict["access"]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: simplify to `config_dict.get('access', None)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

and "access" in config_dict
and config_dict["access"]
):
if AccessType.is_valid(config_dict["access"]):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need nested if statements here, or can we combine into 1 level?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

They can't be combined, the logic would be wrong.

manifest = run_dbt(["parse"])
# ['model.test.another_model', 'model.test.my_model', 'model.test.ref_my_model', 'model.test.people_model', 'model.test.metricflow_time_spine']
assert len(manifest.nodes) == 5
assert manifest.nodes["model.test.my_model"].access == AccessType.Private
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: can we parameterize this test to avoid repeating code?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not really. It's not doing the same test multiple times with different inputs, it's checking multiple things in the output of one test.

Copy link
Member

@emmyoop emmyoop left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, please update the changelog before merging though.

@@ -0,0 +1,6 @@
kind: Features
body: Allow setting access in config
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you make this a bit more clear so that someone reading it without context will have a better idea what it means?

@gshank gshank merged commit f5baeee into main Sep 14, 2023
49 of 50 checks passed
@gshank gshank deleted the 8383-access_configs branch September 14, 2023 15:09
@FishtownBuildBot
Copy link
Collaborator

Opened a new issue in dbt-labs/docs.getdbt.com: dbt-labs/docs.getdbt.com#4075

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cla:yes user docs [docs.getdbt.com] Needs better documentation
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[CT-2968] [ModelGov3] Allow access to be set as config, including in dbt_project.yml
4 participants