diff --git a/website/docs/docs/deploy/deploy-environments.md b/website/docs/docs/deploy/deploy-environments.md index 4c1467afe8b..e42d39bad82 100644 --- a/website/docs/docs/deploy/deploy-environments.md +++ b/website/docs/docs/deploy/deploy-environments.md @@ -16,15 +16,18 @@ A dbt Cloud project can have multiple deployment environments, providing you the To learn different approaches to managing dbt Cloud environments and recommendations for your organization's unique needs, read [dbt Cloud environment best practices](/guides/set-up-ci). ::: -This page reviews the different types of environments and how to configure your deployment environment in dbt Cloud. +Learn more about development vs. deployment environments in [dbt Cloud Environments](/docs/dbt-cloud-environments). -import CloudEnvInfo from '/snippets/_cloud-environments-info.md'; +There are three types of deployment environments that serve different needs: +- **Production**: Environment for transforming data and building pipelines for production use. +- **Staging**: Environment for working with production tools while limiting access to production data. +- **General**: General use environment for deployment development. - +We highly recommend using the `Production` environment type for the final, source of truth deployment data. There can be only one environment marked for final production workflows and we don't recommend using a `General` environment for this purpose. ## Create a deployment environment -To create a new dbt Cloud development environment, navigate to **Deploy** -> **Environments** and then click **Create Environment**. Select **Deployment** as the environment type. +To create a new dbt Cloud deployment environment, navigate to **Deploy** -> **Environments** and then click **Create Environment**. Select **Deployment** as the environment type. The option will be greyed out if you already have a development environment. @@ -38,7 +41,8 @@ In dbt Cloud, each project can have one designated deployment environment, which For Semantic Layer-eligible customers, the next section of environment settings is the Semantic Layer configurations. [The Semantic Layer setup guide](/docs/use-dbt-semantic-layer/setup-sl) has the most up-to-date setup instructions! -### Deployment connection + +## Deployment connection :::info Warehouse Connections @@ -186,6 +190,46 @@ This section allows you to determine the credentials that should be used when co +## Staging environment + +:::note +Currently in limited availability beta. Contact support or your account team if you're interested in beta access. +::: + +Staging environments are useful ways to grant developers access to deployment workflows and tools while controlling access to production data. They are configured with their own long-living branch (for example, `staging`) that may be very similar to `main` in many ways while potentially limiting the data the developers can access. + +Ideally, the workflows would move upstream from the Development environment -> Staging environment -> Production environment with developer branches feeding into the staging branch, then ultimately `main`. In many cases, the `main` and `staging` branches will be identical after a merge and remain until the next batch of changes from the `development` branches are ready to be elevated. We recommend setting branch protection rules on `staging` similar to `main`. + +### Create a staging environment + +In the dbt Cloud, navigate to **Deploy** -> **Environments** and then click **Create Environment**. Select **Deployment** as the environment type. The option will be greyed out if you already have a development environment. + + + + +Follow the steps outlined in [deployment credentials](#deployment-connection) to complete the remainder of the environment setup. + +We recommend that the data warehouse credentials be for a dedicated user or service principal. + +### Why use a staging environment + +There are two primary motivations for using a Staging environment: +1. An additional validation layer before changes are deployed into Production. You can deploy, test, and explore your dbt models in Staging. +2. Clear isolation between development workflows and production data. It enables developers to work in metadata-powered ways, using features like deferral and cross-project references, without accessing data in production deployments. + +:::info Coming soon: environment-level permissions +Provide developers with the ability to create, edit, and trigger ad hoc jobs in the Staging environment, while keeping the Production environment locked down. +::: + +Let's say you have `Project B` downstream of `Project A` with cross-project refs configured in the models. When developers work in the IDE for `Project B`, cross-project refs will resolve to the Staging environment of `Project A`, rather than production. You'll get the same results with those refs when jobs are run in the Staging environment. Only the Production environment will reference the Production data, keeping the data and access isolated without needing separate projects. + +If `Project B` also has a Staging deployment, then references to unbuilt upstream models within `Project B` will resolve to that environment, using [deferral](/docs/cloud/about-cloud-develop-defer), rather than resolving to the models in Production. This saves developers time and warehouse spend, while preserving clear separation of environments. + +Finally, the Staging environment has its own view in [dbt Explorer](/docs/collaborate/explore-projects), giving you a full view of your prod and pre-prod data. + + + + ## Related docs - [dbt Cloud environment best practices](/guides/set-up-ci) diff --git a/website/snippets/_cloud-environments-info.md b/website/snippets/_cloud-environments-info.md index 929337f86d4..4822c8a5eae 100644 --- a/website/snippets/_cloud-environments-info.md +++ b/website/snippets/_cloud-environments-info.md @@ -1,15 +1,19 @@ ## Types of environments In dbt Cloud, there are two types of environments: -- Deployment environment — Determines the settings used when jobs created within that environment are executed. -- Development environment — Determines the settings used in the dbt Cloud IDE or dbt Cloud CLI, for that particular project. +- **Deployment environment** — Determines the settings used when jobs created within that environment are executed.

+ Types of deployment environments: + - General + - Staging + - Production +- **Development environment** — Determines the settings used in the dbt Cloud IDE or dbt Cloud CLI, for that particular project. Each dbt Cloud project can only have a single development environment but can have any number of deployment environments. -| | Development Environments | Deployment Environments | -| --- | --- | --- | -| Determines settings for | dbt Cloud IDE or dbt Cloud CLI | dbt Cloud Job runs | -| How many can I have in my project? | 1 | Any number | +|| Development | Staging | Deployment | +|------| --- | --- | --- | +| **Determines settings for** | dbt Cloud IDE or dbt Cloud CLI | dbt Cloud Job runs | dbt Cloud Job runs | +| **How many can I have in my project?** | 1 | Any number | Any number | :::note For users familiar with development on dbt Core, each environment is roughly analogous to an entry in your `profiles.yml` file, with some additional information about your repository to ensure the proper version of code is executed. More info on dbt core environments [here](/docs/core/dbt-core-environments). diff --git a/website/static/img/docs/collaborate/dbt-explorer/explore-staging-env.png b/website/static/img/docs/collaborate/dbt-explorer/explore-staging-env.png new file mode 100644 index 00000000000..ef218f962e0 Binary files /dev/null and b/website/static/img/docs/collaborate/dbt-explorer/explore-staging-env.png differ diff --git a/website/static/img/docs/dbt-cloud/cloud-configuring-dbt-cloud/create-staging-environment.png b/website/static/img/docs/dbt-cloud/cloud-configuring-dbt-cloud/create-staging-environment.png new file mode 100644 index 00000000000..b7fde51b7d2 Binary files /dev/null and b/website/static/img/docs/dbt-cloud/cloud-configuring-dbt-cloud/create-staging-environment.png differ