Use cookiecutter and delete irrelevant code

[clrcrl]

Was working on a Discourse article to pair with a Coalesce talk, went to use this tool to quickly create a repo in labs, got annoyed with it, updated it to use the cookiecutter framework rather than use my custom code 😅

[jtcohen6]

nice one

tried running per new installation instructions, after pip install cookiecutter in a virtualenv:

$ cookiecutter gh:fishtown-analytics/dbt-init
A valid repository for "https://github.com/fishtown-analytics/dbt-init.git" could not be found in the following locations:
/Users/jerco/.cookiecutters/dbt-init

let me know if I'm missing something obvious?

[clrcrl]

Oh yah, you'd need to run it from the branch!

$ cookiecutter gh:fishtown-analytics/dbt-init --checkout use-cookiecutter

[jtcohen6]

ok — this is cool, even kinda fun

small things:

• do you feel strongly about name, project_name, profile_name, and client_name being separate inputs? I feel like I'd be happy with one consistent name. only downside is that project_name has to be snake case, and I know we both like our dir + repo names to be kebab case
• require-dbt-version is set to v0.16.0, should upgrade. is there a sane way to make this dynamic?
• should probably remove future grants from our Snowflake grants macro

cool idea for future:

• an input for connection method, based on their selected database (is this kind of pathing possible with cookiecutter?)

[clrcrl]

Overall thoughts

This was a super fun project to work on as I was looking to learn some more python, but I don't think we've really used it that much as a team, or externally. I think eventually I'd like to see some of this logic moved into the real $ dbt init. Until then, I think we should err on the side of making this less clever :)

Answering your questions

(with that in mind)

do you feel strongly about name, project_name, profile_name, and client_name being separate inputs? I feel like I'd be happy with one consistent name. only downside is that project_name has to be snake case, and I know we both like our dir + repo names to be kebab case

One thing to note: project_name is a cookiecutter requirement — it needs a directory named project_name to know what to replicate. Previously we referred to this variable as dir_name.

I think I originally had separate inputs because I found the resultant templates easier to read: profile: {{ profile_name }} felt more logical than profile: {{ name }}. And the former implementation had --profile_name as an optional flag, so most people weren't likely to use it (whereas cookiecutter prompts regardless).

I think I will rationalize the existing variables:

{
  "name": "e.g. jaffle_shop",
  "warehouse": ["snowflake", "redshift", "bigquery", "postgres"],
  "client_name": "{{ cookiecutter.name.replace('_', '-') }}",
  "project_name": "{{ cookiecutter.name.replace('_', '-') }}-dbt",
  "profile_name": "{{ cookiecutter.name }}"
}

To this instead:

{
  "name": "e.g. jaffle_shop",
  "project_name": "{{ cookiecutter.name.replace('_', '-') }}-dbt",
  "warehouse": ["snowflake", "redshift", "bigquery", "postgres"]
}

(and update the templates where required)

require-dbt-version is set to v0.16.0, should upgrade. is there a sane way to make this dynamic?

We could make that an input, but then there's all the other changes that come with a version upgrade, that would be hard to capture in a template (e.g. config-version, nesting of vars). IMO we should simply do regular upgrades to the repo if it's a thing we want to keep alive :)

should probably remove future grants from our Snowflake grants macro

Yup, I might do another comb through of code — incidentally I also removed the Redshift warehouse maintenance macros.

an input for connection method, based on their selected database (is this kind of pathing possible with cookiecutter?)

Hmm I'm not sure if it's possible — my initial search (and tinkering) indicates it's not. I think we'll simply have to err on the side of being less-clever here and maybe throw in some links to docs in various places

[jtcohen6]

that all makes sense! I got excited playing around with this / cookiecutter for the first time; I think you've got the right balance of effort + impact.