Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Prefix macro calls #25

Closed
1 of 6 tasks
emmahunt opened this issue Jan 31, 2024 · 4 comments · Fixed by #28
Closed
1 of 6 tasks

Prefix macro calls #25

emmahunt opened this issue Jan 31, 2024 · 4 comments · Fixed by #28
Assignees
Labels
category:macros Related to the macros in the package. priority:low Not on the roadmap. status:completed Completed - but might not be released yet. type:bug Bugs or weaknesses. The issue has to contain steps to reproduce.

Comments

@emmahunt
Copy link

emmahunt commented Jan 31, 2024

Describe the bug

The unified package calls snowow-utils macros without a prefix, which throws errors in some circumstances e.g. overwriting custom models without snowplow-utils locally installed

-- currently
 {{ web_only_fields("ev") }}

-- I believe it should be
 {{ snowplow_unifiied.web_only_fields("ev") }}

Steps to reproduce

Create a copy of the snowplow_unified_sessions_this_run model locally, disable the unified version of the model and run dbt run or dbt compile.

Expected results

The custom model (copy) runs without issues

Actual results

An error message such as: 'platform_independent_fields' is undefined. This can happen when calling a macro that does not exist. Check for typos and/or install package dependencies with "dbt deps".

Screenshots and log output

(dbt-env) emmahunt@Emmas-MacBook-Air dbt % dbt compile --full-refresh --vars 'snowplow__allow_refresh: true'

09:11:04  Running with dbt=1.7.3
09:11:04  Registered adapter: databricks=1.7.2
09:11:05  Found 22 models, 96 tests, 3 seeds, 3 operations, 17 sources, 0 exposures, 0 metrics, 935 macros, 0 groups, 0 semantic models
09:11:05  
09:11:32  Concurrency: 1 threads (target='prod')
09:11:32  
09:11:51  09:11:51 + Snowplow: Standard incremental run
09:12:00  Encountered an error:
Runtime Error
  Compilation Error in model snowplow_unified_sessions_this_run (models/sessions/scratch/snowplow_unified_sessions_this_run.sql)
    'platform_independent_fields' is undefined. This can happen when calling a macro that does not exist. Check for typos and/or install package dependencies with "dbt deps".

System information

The contents of your packages.yml file:

packages:
  - package: snowplow/snowplow_unified
    version: [">=0.1.2", "<0.2.0"]

Which database are you using dbt with?

  • postgres
  • redshift
  • bigquery
  • snowflake
  • databricks
  • other (specify: ____________)

The output of dbt --version:

Core:
  - installed: 1.7.3
  - latest:    1.7.6 - Update available!

The operating system you're using:
MacOS 14.3 (23D56)

The output of python --version:
Python 3.11.5

Additional context

N/A

Are you interested in contributing towards the fix?

Happy to contribute

@emmahunt emmahunt added the type:bug Bugs or weaknesses. The issue has to contain steps to reproduce. label Jan 31, 2024
@github-actions github-actions bot added the status:needs_triage Needs maintainer triage. label Jan 31, 2024
@georgewoodhead
Copy link
Contributor

Hi @emmahunt, thanks for raising this issue! You will find you encounter this behaviour across all of our dbt packages. Whilst we could add the package name prefix to all the macros, we have no real way of enforcing this with future updates and package development work.

I think it’s fair to say that if a user is customising any of the models, they should be expected to update these macros themselves. The returned error message is quite helpful here in identifying what is the issue, but perhaps we could add a note in our docs as something to keep in mind when customising any of the models.

There might be other opinions here so will keep this ticket open for now.

@emmahunt
Copy link
Author

thanks for the reply and explanation George! I see your point. Just want to loop @rlh1994 into the conversation as I was discussing this with him as well 🙂

@rlh1994
Copy link
Contributor

rlh1994 commented Feb 1, 2024

@georgewoodhead I think we should support this where possible, especially because we are use them so heavily throughout the package - I just need to remind myself if this impacts the ability of a user to overwrite the macro or not...

@rlh1994
Copy link
Contributor

rlh1994 commented Feb 1, 2024

Just verified, this does work still even with the prefix, so I think we should prefix them all to make it easier for users to make custom models then only overwrite the required ones in the future. It's not a high priority item but we should get round to it.

@rlh1994 rlh1994 added priority:low Not on the roadmap. category:macros Related to the macros in the package. and removed status:needs_triage Needs maintainer triage. labels Feb 1, 2024
@rlh1994 rlh1994 mentioned this issue Feb 8, 2024
20 tasks
rlh1994 added a commit that referenced this issue Feb 8, 2024
@rlh1994 rlh1994 added the status:has_pr A PR exists for this issue. label Feb 8, 2024
@rlh1994 rlh1994 self-assigned this Feb 8, 2024
rlh1994 added a commit that referenced this issue Feb 12, 2024
rlh1994 added a commit that referenced this issue Feb 12, 2024
rlh1994 added a commit that referenced this issue Feb 12, 2024
@rlh1994 rlh1994 added status:completed Completed - but might not be released yet. and removed status:has_pr A PR exists for this issue. labels Feb 12, 2024
@rlh1994 rlh1994 reopened this Feb 21, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
category:macros Related to the macros in the package. priority:low Not on the roadmap. status:completed Completed - but might not be released yet. type:bug Bugs or weaknesses. The issue has to contain steps to reproduce.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants