Skip to content

Commit

Permalink
updates to the project docs naming and organization + wrote new proje…
Browse files Browse the repository at this point in the history
…ct setup instructions for dbt cloud
  • Loading branch information
britt-allen committed Sep 13, 2024
1 parent cd9d622 commit 2234fec
Show file tree
Hide file tree
Showing 4 changed files with 36 additions and 560 deletions.
39 changes: 31 additions & 8 deletions docs/dbt.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,36 @@
# dbt on the Data Services and Engineering team

## New project setup

To set up a new project on dbt Cloud follow these steps:

1. Give your new project a name.
1. Click *Advanced settings* and in the *Project subdirectory* field, enter "transform"
1. Select a datawarehouse connection. (e.g. Snowflake, BigQuery, Redshift)
1. For the *Development credentials* section you'll want to:
1. Under *Auth method* select *Key pair*
1. Enter your data warehouse username
1. Enter the private key amd private keypassphrase
1. For more guidance, read [dbt's docs on connecting to Snowflake via key pair](https://docs.getdbt.com/docs/cloud/connect-data-platform/connect-snowflake#key-pair)
1. Finally click the *Test Connection* button.
1. Connect the apprioriate repository (usually GitHub). Read [dbt's docs on connecting to GitHub](https://docs.getdbt.com/docs/cloud/git/connect-github).

Once you're through the first five steps you can return to the dbt homepage and click the Settings button in the upper right corner. From there you can follow the steps to configure three environments for Continuous intergation - CI, development, and production. Read [dbt's docs on CI in dbt Cloud](https://docs.getdbt.com/docs/deploy/continuous-integration). Read [dbt's docs on creating production (deployment) environments](https://docs.getdbt.com/docs/deploy/deploy-environments) and [dbt's docs on creating and scheduling deploy jobs](https://docs.getdbt.com/docs/deploy/deploy-jobs#create-and-schedule-jobs).

You'll also want to [configure notifications for job failures](Configure notifications for job failures).

Pictured below is an example of environment variables you can set for each environment. For more guidance, read [dbt's docs on environment variables](https://docs.getdbt.com/docs/build/environment-variables).

![environment variables](images/environment_variables.png)

## Architecture

We broadly follow the architecture described in
[this dbt blog post](https://www.getdbt.com/blog/how-we-configure-snowflake/)
for our Snowflake dbt project.

It is described in more detail in our [Snowflake docs](./snowflake.md#architecture).

## Naming conventions

Models in a data warehouse do not follow the same naming conventions as [raw cloud resources](./naming-conventions.md#general-approach),
Expand All @@ -19,14 +50,6 @@ We may adopt additional conventions for denoting aggregations, column data types
If during the course of a project's model development we determine that simpler human-readable names
work better for our partners or downstream consumers, we may drop the above prefixing conventions.

## Architecture

We broadly follow the architecture described in
[this dbt blog post](https://www.getdbt.com/blog/how-we-configure-snowflake/)
for our Snowflake dbt project.

It is described in more detail in our [Snowflake docs](./snowflake.md#architecture).

## Custom schema names

dbt's default method for generating [custom schema names](https://docs.getdbt.com/docs/build/custom-schemas)
Expand Down
315 changes: 0 additions & 315 deletions docs/new-project-setup.md

This file was deleted.

Loading

0 comments on commit 2234fec

Please sign in to comment.