Skip to content
This repository has been archived by the owner on Nov 11, 2024. It is now read-only.

Use cookiecutter and delete irrelevant code #32

Open
wants to merge 2 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 9 additions & 13 deletions .circleci/config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,31 +13,27 @@ jobs:
command: |
python3 -m venv venv
. venv/bin/activate
pip install -e .
pip install cookiecutter
- run:
name: "Create Starter Project - BigQuery"
name: "Create Starter Project - Snowflake"
command: |
mkdir jaffle_shop_bigquery
. venv/bin/activate
dbt-init --client jaffle-shop --warehouse bigquery --target-dir ./jaffle_shop_bigquery
cookiecutter --no-input gh:fishtown-analytics/dbt-init --checkout ${CIRCLE_BRANCH} name=jaffle_shop project_name=jaffle_shop_snowflake warehouse=snowflake
- run:
name: "Create Starter Project - Postgres"
name: "Create Starter Project - Redshift"
command: |
mkdir jaffle_shop_postgres
. venv/bin/activate
dbt-init --client jaffle-shop --warehouse postgres --target-dir ./jaffle_shop_postgres
cookiecutter --no-input gh:fishtown-analytics/dbt-init --checkout ${CIRCLE_BRANCH} name=jaffle_shop project_name=jaffle_shop_redshift warehouse=redshift
- run:
name: "Create Starter Project - Redshift"
name: "Create Starter Project - BigQuery"
command: |
mkdir jaffle_shop_redshift
. venv/bin/activate
dbt-init --client jaffle-shop --warehouse redshift --target-dir ./jaffle_shop_redshift
cookiecutter --no-input gh:fishtown-analytics/dbt-init --checkout ${CIRCLE_BRANCH} name=jaffle_shop project_name=jaffle_shop_bigquery warehouse=bigquery
- run:
name: "Create Starter Project - Snowflake"
name: "Create Starter Project - Postgres"
command: |
mkdir jaffle_shop_snowflake
. venv/bin/activate
dbt-init --client jaffle-shop --warehouse snowflake --target-dir ./jaffle_shop_snowflake
cookiecutter --no-input gh:fishtown-analytics/dbt-init --checkout ${CIRCLE_BRANCH} name=jaffle_shop project_name=jaffle_shop_postgres warehouse=postgres
- save_cache:
key: deps1-{{ .Branch }}
paths:
Expand Down
51 changes: 22 additions & 29 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,15 +1,26 @@
# dbt-init
A tool to create dbt projects for consulting.
A cookiecutter template to create dbt projects for consulting.
dbt-init will create a project as a subdirectory within the target directory you
provide it, and populate as much of the dbt project as possible

## Installation & usage
1. Install using `pip install dbt-init`
2. To create a new client project run a command like following:
## Usage
1. Install [cookiecutter](https://github.com/cookiecutter/cookiecutter) (preferably in a virtual environemnt) `pip install cookiecutter`
2. Then run `cookiecutter gh:fishtown-analytics/dbt-init`. This will give you a number of prompts — hitting `enter` will use the default value indicated in the `[]` parentheses.

```bash
$ dbt-init --client jaffle-shop --warehouse snowflake --target-dir ~/clients/
$ cookiecutter gh:fishtown-analytics/dbt-init
name [e.g. jaffle_shop]: jaffle_shop
Select warehouse:
1 - snowflake
2 - redshift
3 - bigquery
4 - postgres
Choose from 1, 2, 3, 4 [1]: 1
client_name [jaffle-shop]:
project_name [jaffle-shop-dbt]:
profile_name [jaffle_shop]:
```
You can also check the available arguments with `dbt-init --help`
3. `cd` into your newly created dbt project!

## Once you've created your project
1. Update `sample.profile.yml` to contain the correct profile details for your
Expand All @@ -24,29 +35,11 @@ statements in the post-run hooks (defined in `dbt_project.yml`).
If you're interested in helping build out the starter project, here is a list of
variables you can use – a lot of them have defaults based on the client name.
```
{{ project.name }}: The name of the project, as defined in `dbt_project.yml`, e.g. jaffle_shop.
{{ project.warehouse }}: The warehouse that a client is using.
{{ project.client_name }}: The name of the client, e.g. jaffle-shop.
{{ project.dir_name }}: The name of the directory this project is in, e.g. jaffle-shop-dbt.
{{ project.profile_name }}: The name of the profile used by this project, e.g. jaffle_shop.
```

## Testing out the changes
If you're just making simple changes to the starter project, testing out the
changes is optional. If you want to improve the script, or just get familiar
with virtual environments, this is a good idea!
1. Clone this repo and `cd` into it
2. Create a new virtual environment `dbt-init-dev` and activate it. Make sure
your virtual environment uses python 3.
3. Run `pip install -r requirements-dev.txt`
4. You should now have a development version of `dbt-init` installed. Test your
changes by creating a sample project and inspecting the results (I know, we
should build real tests), e.g.:
```
$ dbt-init --client test --target-dir ~/clrcrl/ --warehouse bigquery
New dbt project for test created at /Users/claire/clrcrl/test-dbt! 🎉

$ open /Users/claire/clrcrl/test-dbt
{{ cookiecutter.name }}: The name of the project, as defined in `dbt_project.yml`, e.g. jaffle_shop.
{{ cookiecutter.warehouse }}: The warehouse that a client is using.
{{ cookiecutter.client_name }}: The name of the client, e.g. jaffle-shop.
{{ cookiecutter.project_name }}: The name of the directory this project is in, e.g. jaffle-shop-dbt (this has to be called project name for cookiecutter reasons)
{{ cookiecutter.profile_name }}: The name of the profile used by this project, e.g. jaffle_shop.
```

## To-do:
Expand Down
9 changes: 0 additions & 9 deletions RELEASING.md

This file was deleted.

7 changes: 7 additions & 0 deletions cookiecutter.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
{
"name": "e.g. jaffle_shop",
"warehouse": ["snowflake", "redshift", "bigquery", "postgres"],
"client_name": "{{ cookiecutter.name.replace('_', '-') }}",
"project_name": "{{ cookiecutter.name.replace('_', '-') }}-dbt",
"profile_name": "{{ cookiecutter.name }}"
}
Empty file removed core/__init__.py
Empty file.
186 changes: 0 additions & 186 deletions core/main.py

This file was deleted.

5 changes: 0 additions & 5 deletions requirements-dev.txt

This file was deleted.

1 change: 0 additions & 1 deletion requirements.txt

This file was deleted.

50 changes: 0 additions & 50 deletions setup.py

This file was deleted.

17 changes: 0 additions & 17 deletions starter-project/models/admin/warehouse_operation.sql

This file was deleted.

Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,7 @@ addressed, and remove any items that are not relevant to this PR.
- [ ] I have materialized my models appropriately.
- [ ] I have added appropriate tests and documentation to any new models.
- [ ] I have updated the README file.
{%- if project.warehouse == 'redshift' %}
{%- if cookiecutter.warehouse == 'redshift' %}
- [ ] I have added sort and dist keys to models materialized as tables.
- [ ] I have validated the SQL in any late-binding views.
{% endif %}
Loading