From d82d910a822461c4cd4528fd4c3b30dcf9be0c7a Mon Sep 17 00:00:00 2001 From: mirnawong1 Date: Tue, 7 Nov 2023 15:47:02 +0000 Subject: [PATCH 01/27] clarify use cases for packages --- website/docs/docs/build/packages.md | 10 ++---- .../govern/project-dependencies.md | 22 +++---------- website/snippets/_packages_or_dependencies.md | 32 +++++++++++++++++++ 3 files changed, 39 insertions(+), 25 deletions(-) create mode 100644 website/snippets/_packages_or_dependencies.md diff --git a/website/docs/docs/build/packages.md b/website/docs/docs/build/packages.md index 8d18a55e949..b60b4ba5b5e 100644 --- a/website/docs/docs/build/packages.md +++ b/website/docs/docs/build/packages.md @@ -25,15 +25,9 @@ dbt _packages_ are in fact standalone dbt projects, with models and macros that * It's important to note that defining and installing dbt packages is different from [defining and installing Python packages](/docs/build/python-models#using-pypi-packages) -:::info `dependencies.yml` has replaced `packages.yml` -Starting from dbt v1.6, `dependencies.yml` has replaced `packages.yml`. This file can now contain both types of dependencies: "package" and "project" dependencies. -- "Package" dependencies lets you add source code from someone else's dbt project into your own, like a library. -- "Project" dependencies provide a different way to build on top of someone else's work in dbt. Refer to [Project dependencies](/docs/collaborate/govern/project-dependencies) for more info. -- -You can rename `packages.yml` to `dependencies.yml`, _unless_ you need to use Jinja within your packages specification. This could be necessary, for example, if you want to add an environment variable with a git token in a private git package specification. - -::: +import UseCaseInfo from '/snippets/_packages_or_dependencies.md'; + ## How do I add a package to my project? 1. Add a file named `dependencies.yml` or `packages.yml` to your dbt project. This should be at the same level as your `dbt_project.yml` file. diff --git a/website/docs/docs/collaborate/govern/project-dependencies.md b/website/docs/docs/collaborate/govern/project-dependencies.md index 9a1d8b59b68..f45f6a2f2a1 100644 --- a/website/docs/docs/collaborate/govern/project-dependencies.md +++ b/website/docs/docs/collaborate/govern/project-dependencies.md @@ -22,8 +22,12 @@ This year, dbt Labs is introducing an expanded notion of `dependencies` across m - **Packages** — Familiar and pre-existing type of dependency. You take this dependency by installing the package's full source code (like a software library). - **Projects** — A _new_ way to take a dependency on another project. Using a metadata service that runs behind the scenes, dbt Cloud resolves references on-the-fly to public models defined in other projects. You don't need to parse or run those upstream models yourself. Instead, you treat your dependency on those models as an API that returns a dataset. The maintainer of the public model is responsible for guaranteeing its quality and stability. +import UseCaseInfo from '/snippets/_packages_or_dependencies.md'; + + + +Refer to the [FAQs](#faqs) for more info. -Starting in dbt v1.6 or higher, `packages.yml` has been renamed to `dependencies.yml`. However, if you need use Jinja within your packages config, such as an environment variable for your private package, you need to keep using `packages.yml` for your packages for now. Refer to the [FAQs](#faqs) for more info. ## Prerequisites @@ -33,22 +37,6 @@ In order to add project dependencies and resolve cross-project `ref`, you must: - Have a successful run of the upstream ("producer") project - Have a multi-tenant or single-tenant [dbt Cloud Enterprise](https://www.getdbt.com/pricing) account (Azure ST is not supported but coming soon) - ## Example As an example, let's say you work on the Marketing team at the Jaffle Shop. The name of your team's project is `jaffle_marketing`: diff --git a/website/snippets/_packages_or_dependencies.md b/website/snippets/_packages_or_dependencies.md new file mode 100644 index 00000000000..d0831f1072d --- /dev/null +++ b/website/snippets/_packages_or_dependencies.md @@ -0,0 +1,32 @@ + +## Use cases + +Starting from dbt v1.6, `dependencies.yml` has replaced `packages.yml`. The `dependencies.yml` file can now contain both types of dependencies: "package" and "project" dependencies. +- "Package" dependencies lets you add source code from someone else's dbt project into your own, like a library. +- "Project" dependencies provide a different way to build on top of someone else's work in dbt. Refer to [Project dependencies](/docs/collaborate/govern/project-dependencies) for more info. +- You can rename your existing `packages.yml` to `dependencies.yml` unless you need to use Jinja within your packages specification. This is necessary if you want to add an environment variable with a Git token in a private Git package specification. + +There are some important differences between using a `dependencies.yml` compared to a `packages.yml` file: + + + + +`dependencies.yml` is designed for the [dbt Mesh](/guides/best-practices/how-we-mesh/mesh-1-intro) and cross-project reference workflow. Consider using it in the following scenarios: + +- When you need to set up cross-project references between different dbt projects, especially in a dbt Mesh setup. +- When you want to include both projects and non-private dbt packages in your project's dependencies. + - Note that private packages are not supported yet in `dependencies.yml`. +- For organization and maintainability. `dependencies.yml` can help maintain your project's organization by allowing you to specify hub packages like `dbt_utils`. This reduces the need for multiple YAML files to manage dependencies. +- Keep in mind that `dependencies.yml` does not currently support Jinja rendering or conditional configuration. This is something important to consider if you need to configure your packages with dynamic or conditional settings. + + + + +Packages allows you to add source code from someone else's dbt project into your own, like a library. Consider using it in the following scenarios: + +- Use `packages.yml` when you want to download dbt packages, such as dbt projects, into your root or parent dbt project. Something to note is that it doesn't contribute to the dbt Mesh workflow. +- Use `packages.yml` to include packages, including private packages, in your project's dependencies. If you have private packages that you need to reference, `packages.yml` is the way to go. +- It supports Jinja rendering, which can be useful if you need to insert values, like a Git token from an environment variable, into your package specifications. + + + From cc8985522cb5400dc13542b8ee4e4f8b5b2f9a20 Mon Sep 17 00:00:00 2001 From: mirnawong1 Date: Tue, 7 Nov 2023 16:10:54 +0000 Subject: [PATCH 02/27] tweaks --- website/snippets/_packages_or_dependencies.md | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/website/snippets/_packages_or_dependencies.md b/website/snippets/_packages_or_dependencies.md index d0831f1072d..94a3d041795 100644 --- a/website/snippets/_packages_or_dependencies.md +++ b/website/snippets/_packages_or_dependencies.md @@ -13,20 +13,20 @@ There are some important differences between using a `dependencies.yml` compared `dependencies.yml` is designed for the [dbt Mesh](/guides/best-practices/how-we-mesh/mesh-1-intro) and cross-project reference workflow. Consider using it in the following scenarios: -- When you need to set up cross-project references between different dbt projects, especially in a dbt Mesh setup. -- When you want to include both projects and non-private dbt packages in your project's dependencies. - - Note that private packages are not supported yet in `dependencies.yml`. -- For organization and maintainability. `dependencies.yml` can help maintain your project's organization by allowing you to specify hub packages like `dbt_utils`. This reduces the need for multiple YAML files to manage dependencies. -- Keep in mind that `dependencies.yml` does not currently support Jinja rendering or conditional configuration. This is something important to consider if you need to configure your packages with dynamic or conditional settings. +- Use `dependencies.yml` when you need to set up cross-project references between different dbt projects, especially in a dbt Mesh setup. +- Use `dependencies.yml` when you want to include both projects and non-private dbt packages in your project's dependencies. + - Private packages are not supported in `dependencies.yml` because it intentionally doesn't support Jinja rendering or conditional configuration. This is to maintain static and predictable configuration, and ensures compatibility with other services, like dbt Cloud. +- Use `dependencies.yml` for organization and maintainability. It can help maintain your project's organization by allowing you to specify hub packages like `dbt_utils`. This reduces the need for multiple YAML files to manage dependencies. + Packages allows you to add source code from someone else's dbt project into your own, like a library. Consider using it in the following scenarios: - Use `packages.yml` when you want to download dbt packages, such as dbt projects, into your root or parent dbt project. Something to note is that it doesn't contribute to the dbt Mesh workflow. - Use `packages.yml` to include packages, including private packages, in your project's dependencies. If you have private packages that you need to reference, `packages.yml` is the way to go. -- It supports Jinja rendering, which can be useful if you need to insert values, like a Git token from an environment variable, into your package specifications. - +- `packages.yml` supports Jinja rendering for historical reasons, allowing dynamic configurations. This can be useful if you need to insert values, like a Git token from an environment variable, into your package specifications. + From 8290de6cd61b15845981ee8c0aeaf6ec4180442e Mon Sep 17 00:00:00 2001 From: mirnawong1 <89008547+mirnawong1@users.noreply.github.com> Date: Wed, 8 Nov 2023 11:10:17 +0000 Subject: [PATCH 03/27] Update _packages_or_dependencies.md --- website/snippets/_packages_or_dependencies.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/website/snippets/_packages_or_dependencies.md b/website/snippets/_packages_or_dependencies.md index 94a3d041795..7052428ddd3 100644 --- a/website/snippets/_packages_or_dependencies.md +++ b/website/snippets/_packages_or_dependencies.md @@ -15,14 +15,14 @@ There are some important differences between using a `dependencies.yml` compared - Use `dependencies.yml` when you need to set up cross-project references between different dbt projects, especially in a dbt Mesh setup. - Use `dependencies.yml` when you want to include both projects and non-private dbt packages in your project's dependencies. - - Private packages are not supported in `dependencies.yml` because it intentionally doesn't support Jinja rendering or conditional configuration. This is to maintain static and predictable configuration, and ensures compatibility with other services, like dbt Cloud. -- Use `dependencies.yml` for organization and maintainability. It can help maintain your project's organization by allowing you to specify hub packages like `dbt_utils`. This reduces the need for multiple YAML files to manage dependencies. + - Private packages are not supported in `dependencies.yml` because they intentionally don't support Jinja rendering or conditional configuration. This is to maintain static and predictable configuration and ensures compatibility with other services, like dbt Cloud. +- Use `dependencies.yml` for organization and maintainability. It can help maintain your project's organization by allowing you to specify [dbt Hub packages](https://hub.getdbt.com/) like `dbt_utils`. This reduces the need for multiple YAML files to manage dependencies. -Packages allows you to add source code from someone else's dbt project into your own, like a library. Consider using it in the following scenarios: +Packages allow you to add source code from someone else's dbt project into your own, like a library. Consider using it in the following scenarios: - Use `packages.yml` when you want to download dbt packages, such as dbt projects, into your root or parent dbt project. Something to note is that it doesn't contribute to the dbt Mesh workflow. - Use `packages.yml` to include packages, including private packages, in your project's dependencies. If you have private packages that you need to reference, `packages.yml` is the way to go. From 253943bdf7368bf8ab8b700ba8f8a79e69eb58d5 Mon Sep 17 00:00:00 2001 From: mirnawong1 <89008547+mirnawong1@users.noreply.github.com> Date: Tue, 14 Nov 2023 12:24:35 +0000 Subject: [PATCH 04/27] Update website/snippets/_packages_or_dependencies.md --- website/snippets/_packages_or_dependencies.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/snippets/_packages_or_dependencies.md b/website/snippets/_packages_or_dependencies.md index 7052428ddd3..0618da233fe 100644 --- a/website/snippets/_packages_or_dependencies.md +++ b/website/snippets/_packages_or_dependencies.md @@ -11,7 +11,7 @@ There are some important differences between using a `dependencies.yml` compared -`dependencies.yml` is designed for the [dbt Mesh](/guides/best-practices/how-we-mesh/mesh-1-intro) and cross-project reference workflow. Consider using it in the following scenarios: +`dependencies.yml` is designed for the [dbt Mesh](/best-practices/how-we-mesh/mesh-1-intro) and cross-project reference workflow. Consider using it in the following scenarios: - Use `dependencies.yml` when you need to set up cross-project references between different dbt projects, especially in a dbt Mesh setup. - Use `dependencies.yml` when you want to include both projects and non-private dbt packages in your project's dependencies. From ff5a401a8bb90bf52f5b6fb2121d9a17dc25d2c2 Mon Sep 17 00:00:00 2001 From: mirnawong1 Date: Fri, 17 Nov 2023 12:39:05 +0000 Subject: [PATCH 05/27] move sidebar --- website/docs/docs/about-setup.md | 8 +- .../docs/docs/build/build-metrics-intro.md | 2 +- .../docs/docs/build/metricflow-commands.md | 2 +- .../docs/docs/cloud/about-cloud-develop.md | 33 ------- website/docs/docs/cloud/about-develop-dbt.md | 42 +++++++++ website/docs/docs/connect-adapters.md | 2 +- website/docs/docs/core/about-core-setup.md | 8 +- website/docs/docs/core/about-dbt-core.md | 25 ----- website/docs/docs/core/docker-install.md | 2 +- .../docs/docs/core/installation-overview.md | 23 ++++- .../core-upgrade/00-upgrading-to-v1.7.md | 2 +- .../core-upgrade/01-upgrading-to-v1.6.md | 2 +- .../core-upgrade/02-upgrading-to-v1.5.md | 2 +- .../core-upgrade/05-upgrading-to-v1.3.md | 2 +- .../core-upgrade/07-upgrading-to-v1.1.md | 2 +- .../core-upgrade/08-upgrading-to-v1.0.md | 2 +- .../docs/docs/dbt-versions/core-versions.md | 2 +- website/docs/guides/dremio-lakehouse.md | 4 +- website/docs/guides/manual-install-qs.md | 2 +- website/sidebars.js | 91 ++++++++++--------- website/vercel.json | 5 + 21 files changed, 137 insertions(+), 126 deletions(-) delete mode 100644 website/docs/docs/cloud/about-cloud-develop.md create mode 100644 website/docs/docs/cloud/about-develop-dbt.md delete mode 100644 website/docs/docs/core/about-dbt-core.md diff --git a/website/docs/docs/about-setup.md b/website/docs/docs/about-setup.md index ceb34a5ccbb..1021c1b65ac 100644 --- a/website/docs/docs/about-setup.md +++ b/website/docs/docs/about-setup.md @@ -21,14 +21,14 @@ To begin configuring dbt now, select the option that is right for you. diff --git a/website/docs/docs/build/build-metrics-intro.md b/website/docs/docs/build/build-metrics-intro.md index cdac51224ed..24af2a0864a 100644 --- a/website/docs/docs/build/build-metrics-intro.md +++ b/website/docs/docs/build/build-metrics-intro.md @@ -14,7 +14,7 @@ Use MetricFlow in dbt to centrally define your metrics. As a key component of th MetricFlow allows you to: - Intuitively define metrics in your dbt project -- Develop from your preferred environment, whether that's the [dbt Cloud CLI](/docs/cloud/cloud-cli-installation), [dbt Cloud IDE](/docs/cloud/dbt-cloud-ide/develop-in-the-cloud), or [dbt Core](/docs/core/installation) +- Develop from your preferred environment, whether that's the [dbt Cloud CLI](/docs/cloud/cloud-cli-installation), [dbt Cloud IDE](/docs/cloud/dbt-cloud-ide/develop-in-the-cloud), or [dbt Core](/docs/core/installation-overview) - Use [MetricFlow commands](/docs/build/metricflow-commands) to query and test those metrics in your development environment - Harness the true magic of the universal dbt Semantic Layer and dynamically query these metrics in downstream tools (Available for dbt Cloud [Team or Enterprise](https://www.getdbt.com/pricing/) accounts only). diff --git a/website/docs/docs/build/metricflow-commands.md b/website/docs/docs/build/metricflow-commands.md index 4d2477ad2ed..0b9e9aa10b2 100644 --- a/website/docs/docs/build/metricflow-commands.md +++ b/website/docs/docs/build/metricflow-commands.md @@ -8,7 +8,7 @@ tags: [Metrics, Semantic Layer] Once you define metrics in your dbt project, you can query metrics, dimensions, and dimension values, and validate your configs using the MetricFlow commands. -MetricFlow allows you to define and query metrics in your dbt project in the [dbt Cloud CLI](/docs/cloud/cloud-cli-installation), [dbt Cloud IDE](/docs/cloud/dbt-cloud-ide/develop-in-the-cloud), or [dbt Core](/docs/core/installation). To experience the power of the universal [dbt Semantic Layer](/docs/use-dbt-semantic-layer/dbt-sl) and dynamically query those metrics in downstream tools, you'll need a dbt Cloud [Team or Enterprise](https://www.getdbt.com/pricing/) account. +MetricFlow allows you to define and query metrics in your dbt project in the [dbt Cloud CLI](/docs/cloud/cloud-cli-installation), [dbt Cloud IDE](/docs/cloud/dbt-cloud-ide/develop-in-the-cloud), or [dbt Core](/docs/core/installation-overview). To experience the power of the universal [dbt Semantic Layer](/docs/use-dbt-semantic-layer/dbt-sl) and dynamically query those metrics in downstream tools, you'll need a dbt Cloud [Team or Enterprise](https://www.getdbt.com/pricing/) account. MetricFlow is compatible with Python versions 3.8, 3.9, 3.10, and 3.11. diff --git a/website/docs/docs/cloud/about-cloud-develop.md b/website/docs/docs/cloud/about-cloud-develop.md deleted file mode 100644 index 90abbb98bf4..00000000000 --- a/website/docs/docs/cloud/about-cloud-develop.md +++ /dev/null @@ -1,33 +0,0 @@ ---- -title: About developing in dbt Cloud -id: about-cloud-develop -description: "Learn how to develop your dbt projects using dbt Cloud." -sidebar_label: "About developing in dbt Cloud" -pagination_next: "docs/cloud/cloud-cli-installation" -hide_table_of_contents: true ---- - -dbt Cloud offers a fast and reliable way to work on your dbt project. It runs dbt Core in a hosted (single or multi-tenant) environment. You can develop in your browser using an integrated development environment (IDE) or in a dbt Cloud-powered command line interface (CLI): - -
- - - - - -

- -The following sections provide detailed instructions on setting up the dbt Cloud CLI and dbt Cloud IDE. To get started with dbt development, you'll need a [developer](/docs/cloud/manage-access/seats-and-users) account. For a more comprehensive guide about developing in dbt, refer to our [quickstart guides](/guides). - - ---------- -**Note**: The dbt Cloud CLI and the open-sourced dbt Core are both command line tools that let you run dbt commands. The key distinction is the dbt Cloud CLI is tailored for dbt Cloud's infrastructure and integrates with all its [features](/docs/cloud/about-cloud/dbt-cloud-features). - diff --git a/website/docs/docs/cloud/about-develop-dbt.md b/website/docs/docs/cloud/about-develop-dbt.md new file mode 100644 index 00000000000..6c96eb13963 --- /dev/null +++ b/website/docs/docs/cloud/about-develop-dbt.md @@ -0,0 +1,42 @@ +--- +title: About developing in dbt +id: about-develop-dbt +description: "Learn how to develop your dbt projects using dbt Cloud." +sidebar_label: "About developing in dbt" +pagination_next: "docs/cloud/cloud-cli-installation" +hide_table_of_contents: false +--- + +Develop dbt projects using dbt Cloud or dbt Core. There are a few key differences between the two options: + +## dbt Cloud + +- dbt Cloud offers a fast and reliable way to work on your dbt project. It runs dbt Core in a hosted (single or multi-tenant) environment. +- You can develop in your browser using an integrated development environment (IDE) or in a dbt Cloud-powered command line interface (CLI). + +
+ + + + + +

+ +To get started with dbt development, you'll need a [developer](/docs/cloud/manage-access/seats-and-users) account. For a more comprehensive guide about developing in dbt, refer to our [quickstart guides](/guides). + +## dbt Core + +- Install dbt Core, the open-sourced version of dbt, on your local machine to develop your dbt project locally. +- You can install dbt Core on the command line with pip install, Homebrew, Docker image, or from source. For more information, see [About dbt Core installation](/docs/core/installation-overview). + +--------- +**Note**: The dbt Cloud CLI and the open-sourced dbt Core are both command line tools that let you run dbt commands. The key distinction is the dbt Cloud CLI is tailored for dbt Cloud's infrastructure and integrates with all its [features](/docs/cloud/about-cloud/dbt-cloud-features). + diff --git a/website/docs/docs/connect-adapters.md b/website/docs/docs/connect-adapters.md index e301cfc237e..0ccc1c4420b 100644 --- a/website/docs/docs/connect-adapters.md +++ b/website/docs/docs/connect-adapters.md @@ -15,7 +15,7 @@ Explore the fastest and most reliable way to deploy dbt using dbt Cloud, a hoste Install dbt Core, an open-source tool, locally using the command line. dbt communicates with a number of different data platforms by using a dedicated adapter plugin for each. When you install dbt Core, you'll also need to install the specific adapter for your database, [connect to dbt Core](/docs/core/about-core-setup), and set up a `profiles.yml` file. -With a few exceptions [^1], you can install all [Verified adapters](/docs/supported-data-platforms) from PyPI using `pip install adapter-name`. For example to install Snowflake, use the command `pip install dbt-snowflake`. The installation will include `dbt-core` and any other required dependencies, which may include both other dependencies and even other adapter plugins. Read more about [installing dbt](/docs/core/installation). +With a few exceptions [^1], you can install all [Verified adapters](/docs/supported-data-platforms) from PyPI using `pip install adapter-name`. For example to install Snowflake, use the command `pip install dbt-snowflake`. The installation will include `dbt-core` and any other required dependencies, which may include both other dependencies and even other adapter plugins. Read more about [installing dbt](/docs/core/installation-overview). [^1]: Here are the two different adapters. Use the PyPI package name when installing with `pip` diff --git a/website/docs/docs/core/about-core-setup.md b/website/docs/docs/core/about-core-setup.md index 64e7694b793..fa90e3e55bd 100644 --- a/website/docs/docs/core/about-core-setup.md +++ b/website/docs/docs/core/about-core-setup.md @@ -3,7 +3,7 @@ title: About dbt Core setup id: about-core-setup description: "Configuration settings for dbt Core." sidebar_label: "About dbt Core setup" -pagination_next: "docs/core/about-dbt-core" +pagination_next: "docs/core/dbt-core-environments" pagination_prev: null --- @@ -11,9 +11,11 @@ dbt Core is an [open-source](https://github.com/dbt-labs/dbt-core) tool that ena This section of our docs will guide you through various settings to get started: -- [About dbt Core](/docs/core/about-dbt-core) -- [Installing dbt](/docs/core/installation) - [Connecting to a data platform](/docs/core/connect-data-platform/profiles.yml) - [How to run your dbt projects](/docs/running-a-dbt-project/run-your-dbt-projects) If you need a more detailed first-time setup guide for specific data platforms, read our [quickstart guides](https://docs.getdbt.com/guides). + + +To learn about developing dbt projects in dbt Cloud or dbt Core, refer to [Develop dbt](/docs/cloud/about-develop-dbt). +- **Note** — dbt Cloud provides a command line interface with the [dbt Cloud CLI](/docs/cloud/cloud-cli-installation). Both the open-sourced dbt Core and the dbt Cloud CLI are command line tools that let you run dbt commands. The key distinction is the dbt Cloud CLI is tailored for dbt Cloud's infrastructure and integrates with all its [features](/docs/cloud/about-cloud/dbt-cloud-features). diff --git a/website/docs/docs/core/about-dbt-core.md b/website/docs/docs/core/about-dbt-core.md deleted file mode 100644 index a35d92420f3..00000000000 --- a/website/docs/docs/core/about-dbt-core.md +++ /dev/null @@ -1,25 +0,0 @@ ---- -title: "About dbt Core" -id: "about-dbt-core" -sidebar_label: "About dbt Core" ---- - -[dbt Core](https://github.com/dbt-labs/dbt-core) is an open sourced project where you can develop from the command line and run your dbt project. - -To use dbt Core, your workflow generally looks like: - -1. **Build your dbt project in a code editor —** popular choices include VSCode and Atom. - -2. **Run your project from the command line —** macOS ships with a default Terminal program, however you can also use iTerm or the command line prompt within a code editor to execute dbt commands. - -:::info How we set up our computers for working on dbt projects - -We've written a [guide](https://discourse.getdbt.com/t/how-we-set-up-our-computers-for-working-on-dbt-projects/243) for our recommended setup when running dbt projects using dbt Core. - -::: - -If you're using the command line, we recommend learning some basics of your terminal to help you work more effectively. In particular, it's important to understand `cd`, `ls` and `pwd` to be able to navigate through the directory structure of your computer easily. - -You can find more information on installing and setting up the dbt Core [here](/docs/core/installation). - -**Note** — dbt supports a dbt Cloud CLI and dbt Core, both command line interface tools that enable you to run dbt commands. The key distinction is the dbt Cloud CLI is tailored for dbt Cloud's infrastructure and integrates with all its [features](/docs/cloud/about-cloud/dbt-cloud-features). diff --git a/website/docs/docs/core/docker-install.md b/website/docs/docs/core/docker-install.md index dfb2a669e34..9185bd12698 100644 --- a/website/docs/docs/core/docker-install.md +++ b/website/docs/docs/core/docker-install.md @@ -11,7 +11,7 @@ You might also be able to use Docker to install and develop locally if you don't ### Prerequisites * You've installed Docker. For more information, see the [Docker](https://docs.docker.com/) site. -* You understand which database adapter(s) you need. For more information, see [About dbt adapters](/docs/core/installation#about-dbt-adapters). +* You understand which database adapter(s) you need. For more information, see [About dbt adapters](docs/core/installation-overview#about-dbt-data-platforms-and-adapters). * You understand how dbt Core is versioned. For more information, see [About dbt Core versions](/docs/dbt-versions/core). * You have a general understanding of the dbt, dbt workflow, developing locally in the command line interface (CLI). For more information, see [About dbt](/docs/introduction#how-do-i-use-dbt). diff --git a/website/docs/docs/core/installation-overview.md b/website/docs/docs/core/installation-overview.md index cb1df26b0f8..34594b0a92c 100644 --- a/website/docs/docs/core/installation-overview.md +++ b/website/docs/docs/core/installation-overview.md @@ -1,11 +1,28 @@ --- -title: "About installing dbt" -id: "installation" +title: "About dbt Core and installation" description: "You can install dbt Core using a few different tested methods." pagination_next: "docs/core/homebrew-install" pagination_prev: null --- +[dbt Core](https://github.com/dbt-labs/dbt-core) is an open sourced project where you can develop from the command line and run your dbt project. + +To use dbt Core, your workflow generally looks like: + +1. **Build your dbt project in a code editor —** popular choices include VSCode and Atom. + +2. **Run your project from the command line —** macOS ships with a default Terminal program, however you can also use iTerm or the command line prompt within a code editor to execute dbt commands. + +:::info How we set up our computers for working on dbt projects + +We've written a [guide](https://discourse.getdbt.com/t/how-we-set-up-our-computers-for-working-on-dbt-projects/243) for our recommended setup when running dbt projects using dbt Core. + +::: + +If you're using the command line, we recommend learning some basics of your terminal to help you work more effectively. In particular, it's important to understand `cd`, `ls` and `pwd` to be able to navigate through the directory structure of your computer easily. + +## Install dbt Core + You can install dbt Core on the command line by using one of these methods: - [Use pip to install dbt](/docs/core/pip-install) (recommended) @@ -13,6 +30,8 @@ You can install dbt Core on the command line by using one of these methods: - [Use a Docker image to install dbt](/docs/core/docker-install) - [Install dbt from source](/docs/core/source-install) +**Note** — The [dbt Cloud CLI](/docs/cloud/cloud-cli-installation) and the open-sourced dbt Core are both command line tools that let you run dbt commands. The key distinction is the dbt Cloud CLI is tailored for dbt Cloud's infrastructure and integrates with all its [features](/docs/cloud/about-cloud/dbt-cloud-features). + :::tip Pro tip: Using the --help flag Most command-line tools, including dbt, have a `--help` flag that you can use to show available commands and arguments. For example, you can use the `--help` flag with dbt in two ways:

diff --git a/website/docs/docs/dbt-versions/core-upgrade/00-upgrading-to-v1.7.md b/website/docs/docs/dbt-versions/core-upgrade/00-upgrading-to-v1.7.md index 9ebd3c64cf3..fc8848e5b42 100644 --- a/website/docs/docs/dbt-versions/core-upgrade/00-upgrading-to-v1.7.md +++ b/website/docs/docs/dbt-versions/core-upgrade/00-upgrading-to-v1.7.md @@ -12,7 +12,7 @@ import UpgradeMove from '/snippets/_upgrade-move.md'; ## Resources - [Changelog](https://github.com/dbt-labs/dbt-core/blob/8aaed0e29f9560bc53d9d3e88325a9597318e375/CHANGELOG.md) -- [CLI Installation guide](/docs/core/installation) +- [CLI Installation guide](/docs/core/installation-overview) - [Cloud upgrade guide](/docs/dbt-versions/upgrade-core-in-cloud) - [Release schedule](https://github.com/dbt-labs/dbt-core/issues/8260) diff --git a/website/docs/docs/dbt-versions/core-upgrade/01-upgrading-to-v1.6.md b/website/docs/docs/dbt-versions/core-upgrade/01-upgrading-to-v1.6.md index d36cc544814..36146246d3a 100644 --- a/website/docs/docs/dbt-versions/core-upgrade/01-upgrading-to-v1.6.md +++ b/website/docs/docs/dbt-versions/core-upgrade/01-upgrading-to-v1.6.md @@ -17,7 +17,7 @@ dbt Core v1.6 has three significant areas of focus: ## Resources - [Changelog](https://github.com/dbt-labs/dbt-core/blob/1.6.latest/CHANGELOG.md) -- [CLI Installation guide](/docs/core/installation) +- [CLI Installation guide](/docs/core/installation-overview - [Cloud upgrade guide](/docs/dbt-versions/upgrade-core-in-cloud) - [Release schedule](https://github.com/dbt-labs/dbt-core/issues/7481) diff --git a/website/docs/docs/dbt-versions/core-upgrade/02-upgrading-to-v1.5.md b/website/docs/docs/dbt-versions/core-upgrade/02-upgrading-to-v1.5.md index dded8a690fe..e739caa477a 100644 --- a/website/docs/docs/dbt-versions/core-upgrade/02-upgrading-to-v1.5.md +++ b/website/docs/docs/dbt-versions/core-upgrade/02-upgrading-to-v1.5.md @@ -16,7 +16,7 @@ dbt Core v1.5 is a feature release, with two significant additions: ## Resources - [Changelog](https://github.com/dbt-labs/dbt-core/blob/1.5.latest/CHANGELOG.md) -- [CLI Installation guide](/docs/core/installation) +- [CLI Installation guide](/docs/core/installation-overview) - [Cloud upgrade guide](/docs/dbt-versions/upgrade-core-in-cloud) - [Release schedule](https://github.com/dbt-labs/dbt-core/issues/6715) diff --git a/website/docs/docs/dbt-versions/core-upgrade/05-upgrading-to-v1.3.md b/website/docs/docs/dbt-versions/core-upgrade/05-upgrading-to-v1.3.md index f66d9bb9706..d9d97f17dc5 100644 --- a/website/docs/docs/dbt-versions/core-upgrade/05-upgrading-to-v1.3.md +++ b/website/docs/docs/dbt-versions/core-upgrade/05-upgrading-to-v1.3.md @@ -12,7 +12,7 @@ import UpgradeMove from '/snippets/_upgrade-move.md'; ### Resources - [Changelog](https://github.com/dbt-labs/dbt-core/blob/1.3.latest/CHANGELOG.md) -- [CLI Installation guide](/docs/core/installation) +- [CLI Installation guide](/docs/core/installation-overview) - [Cloud upgrade guide](/docs/dbt-versions/upgrade-core-in-cloud) ## What to know before upgrading diff --git a/website/docs/docs/dbt-versions/core-upgrade/07-upgrading-to-v1.1.md b/website/docs/docs/dbt-versions/core-upgrade/07-upgrading-to-v1.1.md index 403264a46e6..12f0f42354a 100644 --- a/website/docs/docs/dbt-versions/core-upgrade/07-upgrading-to-v1.1.md +++ b/website/docs/docs/dbt-versions/core-upgrade/07-upgrading-to-v1.1.md @@ -12,7 +12,7 @@ import UpgradeMove from '/snippets/_upgrade-move.md'; ### Resources - [Changelog](https://github.com/dbt-labs/dbt-core/blob/1.1.latest/CHANGELOG.md) -- [CLI Installation guide](/docs/core/installation) +- [CLI Installation guide](/docs/core/installation-overview) - [Cloud upgrade guide](/docs/dbt-versions/upgrade-core-in-cloud) ## What to know before upgrading diff --git a/website/docs/docs/dbt-versions/core-upgrade/08-upgrading-to-v1.0.md b/website/docs/docs/dbt-versions/core-upgrade/08-upgrading-to-v1.0.md index 3f45e44076c..436a60c18b9 100644 --- a/website/docs/docs/dbt-versions/core-upgrade/08-upgrading-to-v1.0.md +++ b/website/docs/docs/dbt-versions/core-upgrade/08-upgrading-to-v1.0.md @@ -13,7 +13,7 @@ import UpgradeMove from '/snippets/_upgrade-move.md'; - [Discourse](https://discourse.getdbt.com/t/3180) - [Changelog](https://github.com/dbt-labs/dbt-core/blob/1.0.latest/CHANGELOG.md) -- [CLI Installation guide](/docs/core/installation) +- [CLI Installation guide](/docs/core/installation-overview) - [Cloud upgrade guide](/docs/dbt-versions/upgrade-core-in-cloud) ## What to know before upgrading diff --git a/website/docs/docs/dbt-versions/core-versions.md b/website/docs/docs/dbt-versions/core-versions.md index 2467f3c946b..ff7f97f54fc 100644 --- a/website/docs/docs/dbt-versions/core-versions.md +++ b/website/docs/docs/dbt-versions/core-versions.md @@ -29,7 +29,7 @@ All dbt Core versions released prior to 1.0 and their version-specific documenta All dbt Core minor versions that have reached end-of-life (EOL) will have no new patch releases. This means they will no longer receive any fixes, including for known bugs that have been identified. Fixes for those bugs will instead be made in newer minor versions that are still under active support. -We recommend upgrading to a newer version in [dbt Cloud](/docs/dbt-versions/upgrade-core-in-cloud) or [dbt Core](/docs/core/installation#upgrading-dbt-core) to continue receiving support. +We recommend upgrading to a newer version in [dbt Cloud](/docs/dbt-versions/upgrade-core-in-cloud) or [dbt Core](/docs/core/installation-overview#upgrading-dbt-core) to continue receiving support. All dbt Core v1.0 and later are available in dbt Cloud until further notice. In the future, we intend to align dbt Cloud availability with dbt Core ongoing support. You will receive plenty of advance notice before any changes take place. diff --git a/website/docs/guides/dremio-lakehouse.md b/website/docs/guides/dremio-lakehouse.md index 1c59c04d175..6115f8c79c7 100644 --- a/website/docs/guides/dremio-lakehouse.md +++ b/website/docs/guides/dremio-lakehouse.md @@ -20,7 +20,7 @@ This guide will demonstrate how to build a data lakehouse with dbt Core 1.5 or n * You must have a [Dremio Cloud](https://docs.dremio.com/cloud/) account. * You must have Python 3 installed. -* You must have dbt Core v1.5 or newer [installed](/docs/core/installation). +* You must have dbt Core v1.5 or newer [installed](//docs/core/installation-overview). * You must have the Dremio adapter 1.5.0 or newer [installed and configured](/docs/core/connect-data-platform/dremio-setup) for Dremio Cloud. * You must have basic working knowledge of Git and the command line interface (CLI). @@ -193,4 +193,4 @@ GROUP BY vendor_id -This completes the integration setup and data is ready for business consumption. \ No newline at end of file +This completes the integration setup and data is ready for business consumption. diff --git a/website/docs/guides/manual-install-qs.md b/website/docs/guides/manual-install-qs.md index 61796fe008a..c74d30db51c 100644 --- a/website/docs/guides/manual-install-qs.md +++ b/website/docs/guides/manual-install-qs.md @@ -15,7 +15,7 @@ When you use dbt Core to work with dbt, you will be editing files locally using ### Prerequisites * To use dbt Core, it's important that you know some basics of the Terminal. In particular, you should understand `cd`, `ls` and `pwd` to navigate through the directory structure of your computer easily. -* Install dbt Core using the [installation instructions](/docs/core/installation) for your operating system. +* Install dbt Core using the [installation instructions](/docs/core/installation-overview) for your operating system. * Complete [Setting up (in BigQuery)](/guides/bigquery?step=2) and [Loading data (BigQuery)](/guides/bigquery?step=3). * [Create a GitHub account](https://github.com/join) if you don't already have one. diff --git a/website/sidebars.js b/website/sidebars.js index 66ba731fb1b..0f4e9adaa45 100644 --- a/website/sidebars.js +++ b/website/sidebars.js @@ -33,6 +33,49 @@ const sidebarSettings = { href: `/guides`, }, { + type: "category", + label: "Develop dbt", + collapsed: true, + link: { type: "doc", id: "docs/cloud/about-develop-dbt" }, + items: [ + "docs/cloud/about-develop-dbt", + "docs/cloud/about-cloud-develop-defer", + { + type: "category", + label: "dbt Cloud CLI", + collapsed: true, + link: { type: "doc", id: "docs/cloud/cloud-cli-installation" }, + items: [ + "docs/cloud/cloud-cli-installation", + "docs/cloud/configure-cloud-cli", + ], + }, + { + type: "category", + label: "dbt Cloud IDE", + link: { type: "doc", id: "docs/cloud/dbt-cloud-ide/develop-in-the-cloud" }, + items: [ + "docs/cloud/dbt-cloud-ide/develop-in-the-cloud", + "docs/cloud/dbt-cloud-ide/ide-user-interface", + "docs/cloud/dbt-cloud-ide/lint-format", + "docs/cloud/dbt-cloud-ide/dbt-cloud-tips", + ], + }, + { + type: "category", + label: "dbt Core", + link: { type: "doc", id: "docs/core/installation-overview", }, + items: [ + "docs/core/installation-overview", + "docs/core/homebrew-install", + "docs/core/pip-install", + "docs/core/docker-install", + "docs/core/source-install", + ], + }, + ], + }, + { type: "category", label: "Set up dbt", collapsed: true, @@ -120,35 +163,6 @@ const sidebarSettings = { }, ], }, // Supported Git providers - { - type: "category", - label: "Develop in dbt Cloud", - link: { type: "doc", id: "docs/cloud/about-cloud-develop" }, - items: [ - "docs/cloud/about-cloud-develop", - "docs/cloud/about-cloud-develop-defer", - { - type: "category", - label: "dbt Cloud CLI", - link: { type: "doc", id: "docs/cloud/cloud-cli-installation" }, - items: [ - "docs/cloud/cloud-cli-installation", - "docs/cloud/configure-cloud-cli", - ], - }, - { - type: "category", - label: "dbt Cloud IDE", - link: { type: "doc", id: "docs/cloud/dbt-cloud-ide/develop-in-the-cloud" }, - items: [ - "docs/cloud/dbt-cloud-ide/develop-in-the-cloud", - "docs/cloud/dbt-cloud-ide/ide-user-interface", - "docs/cloud/dbt-cloud-ide/lint-format", - "docs/cloud/dbt-cloud-ide/dbt-cloud-tips", - ], - }, - ], - }, // dbt Cloud develop directory { type: "category", label: "Secure your tenant", @@ -174,20 +188,7 @@ const sidebarSettings = { link: { type: "doc", id: "docs/core/about-core-setup" }, items: [ "docs/core/about-core-setup", - "docs/core/about-dbt-core", "docs/core/dbt-core-environments", - { - type: "category", - label: "Install dbt", - link: { type: "doc", id: "docs/core/installation" }, - items: [ - "docs/core/installation", - "docs/core/homebrew-install", - "docs/core/pip-install", - "docs/core/docker-install", - "docs/core/source-install", - ], - }, { type: "category", label: "Connect data platform", @@ -955,11 +956,11 @@ const sidebarSettings = { type: "category", label: "Database Permissions", items: [ - "reference/database-permissions/about-database-permissions", + "reference/database-permissions/about-database-permissions", "reference/database-permissions/databricks-permissions", "reference/database-permissions/postgres-permissions", - "reference/database-permissions/redshift-permissions", - "reference/database-permissions/snowflake-permissions", + "reference/database-permissions/redshift-permissions", + "reference/database-permissions/snowflake-permissions", ], }, ], diff --git a/website/vercel.json b/website/vercel.json index 149aaaeb09a..6cb562a19a7 100644 --- a/website/vercel.json +++ b/website/vercel.json @@ -2,6 +2,11 @@ "cleanUrls": true, "trailingSlash": false, "redirects": [ + { + "source": "/docs/core/about-dbt-core", + "destination": "/docs/core/installation-overview", + "permanent": true + }, { "source": "/docs/about/overview", "destination": "/docs/introduction", From aac0f6b094476f2003343db180d716faedd8ea9d Mon Sep 17 00:00:00 2001 From: mirnawong1 Date: Fri, 17 Nov 2023 12:50:49 +0000 Subject: [PATCH 06/27] add redirects and adjust links --- website/docs/docs/cloud/about-cloud-setup.md | 3 +-- website/docs/docs/cloud/about-develop-dbt.md | 2 +- website/docs/docs/core/about-core-setup.md | 5 ++--- .../docs/docs/running-a-dbt-project/run-your-dbt-projects.md | 2 +- website/vercel.json | 5 +++++ 5 files changed, 10 insertions(+), 7 deletions(-) diff --git a/website/docs/docs/cloud/about-cloud-setup.md b/website/docs/docs/cloud/about-cloud-setup.md index 5c8e5525bf1..7daf33a4684 100644 --- a/website/docs/docs/cloud/about-cloud-setup.md +++ b/website/docs/docs/cloud/about-cloud-setup.md @@ -13,14 +13,13 @@ dbt Cloud is the fastest and most reliable way to deploy your dbt jobs. It conta - Configuring access to [GitHub](/docs/cloud/git/connect-github), [GitLab](/docs/cloud/git/connect-gitlab), or your own [git repo URL](/docs/cloud/git/import-a-project-by-git-url). - [Managing users and licenses](/docs/cloud/manage-access/seats-and-users) - [Configuring secure access](/docs/cloud/manage-access/about-user-access) -- Configuring the [dbt Cloud IDE](/docs/cloud/about-cloud-develop) -- Installing and configuring the [dbt Cloud CLI](/docs/cloud/cloud-cli-installation) These settings are intended for dbt Cloud administrators. If you need a more detailed first-time setup guide for specific data platforms, read our [quickstart guides](/guides). If you want a more in-depth learning experience, we recommend taking the dbt Fundamentals on our [dbt Learn online courses site](https://courses.getdbt.com/). ## Prerequisites + - To set up dbt Cloud, you'll need to have a dbt Cloud account with administrator access. If you still need to create a dbt Cloud account, [sign up today](https://getdbt.com) on our North American servers or [contact us](https://getdbt.com/contact) for international options. - To have the best experience using dbt Cloud, we recommend you use modern and up-to-date web browsers like Chrome, Safari, Edge, and Firefox. diff --git a/website/docs/docs/cloud/about-develop-dbt.md b/website/docs/docs/cloud/about-develop-dbt.md index 6c96eb13963..89a969fdd60 100644 --- a/website/docs/docs/cloud/about-develop-dbt.md +++ b/website/docs/docs/cloud/about-develop-dbt.md @@ -30,7 +30,7 @@ Develop dbt projects using dbt Cloud or dbt Core. There are a few key difference
-To get started with dbt development, you'll need a [developer](/docs/cloud/manage-access/seats-and-users) account. For a more comprehensive guide about developing in dbt, refer to our [quickstart guides](/guides). +To get started with dbt development, you'll need a [dbt Cloud](https://www.getdbt.com/signup) account and developer seat. For a more comprehensive guide about developing in dbt, refer to our [quickstart guides](/guides). ## dbt Core diff --git a/website/docs/docs/core/about-core-setup.md b/website/docs/docs/core/about-core-setup.md index fa90e3e55bd..2f6c077ba7d 100644 --- a/website/docs/docs/core/about-core-setup.md +++ b/website/docs/docs/core/about-core-setup.md @@ -14,8 +14,7 @@ dbt Core is an [open-source](https://github.com/dbt-labs/dbt-core) tool that ena - [Connecting to a data platform](/docs/core/connect-data-platform/profiles.yml) - [How to run your dbt projects](/docs/running-a-dbt-project/run-your-dbt-projects) -If you need a more detailed first-time setup guide for specific data platforms, read our [quickstart guides](https://docs.getdbt.com/guides). - - To learn about developing dbt projects in dbt Cloud or dbt Core, refer to [Develop dbt](/docs/cloud/about-develop-dbt). - **Note** — dbt Cloud provides a command line interface with the [dbt Cloud CLI](/docs/cloud/cloud-cli-installation). Both the open-sourced dbt Core and the dbt Cloud CLI are command line tools that let you run dbt commands. The key distinction is the dbt Cloud CLI is tailored for dbt Cloud's infrastructure and integrates with all its [features](/docs/cloud/about-cloud/dbt-cloud-features). + +If you need a more detailed first-time setup guide for specific data platforms, read our [quickstart guides](https://docs.getdbt.com/guides). diff --git a/website/docs/docs/running-a-dbt-project/run-your-dbt-projects.md b/website/docs/docs/running-a-dbt-project/run-your-dbt-projects.md index b3b6ffb3e45..7de1d4df7df 100644 --- a/website/docs/docs/running-a-dbt-project/run-your-dbt-projects.md +++ b/website/docs/docs/running-a-dbt-project/run-your-dbt-projects.md @@ -11,7 +11,7 @@ You can run your dbt projects with [dbt Cloud](/docs/cloud/about-cloud/dbt-cloud - Share your [dbt project's documentation](/docs/collaborate/build-and-view-your-docs) with your team. - Integrates with the dbt Cloud IDE, allowing you to run development tasks and environment in the dbt Cloud UI for a seamless experience. - The dbt Cloud CLI to develop and run dbt commands against your dbt Cloud development environment from your local command line. - - For more details, refer to [Develop in the Cloud](/docs/cloud/about-cloud-develop). + - For more details, refer to [Develop dbt](/docs/cloud/about-develop-dbt). - **dbt Core**: An open source project where you can develop from the [command line](/docs/core/about-dbt-core). diff --git a/website/vercel.json b/website/vercel.json index 6cb562a19a7..a65234c8f52 100644 --- a/website/vercel.json +++ b/website/vercel.json @@ -2,6 +2,11 @@ "cleanUrls": true, "trailingSlash": false, "redirects": [ + { + "source": "/docs/cloud/about-cloud-develop", + "destination": "/docs/cloud/about-develop-dbt", + "permanent": true + }, { "source": "/docs/core/about-dbt-core", "destination": "/docs/core/installation-overview", From c1c3b46bb00b5deebfd337d94915548a654f8af0 Mon Sep 17 00:00:00 2001 From: mirnawong1 Date: Fri, 17 Nov 2023 12:53:14 +0000 Subject: [PATCH 07/27] fix links --- website/docs/docs/core/connect-data-platform/profiles.yml.md | 2 +- website/docs/docs/deploy/job-commands.md | 2 +- .../docs/docs/running-a-dbt-project/run-your-dbt-projects.md | 2 +- website/docs/reference/dbt-commands.md | 4 ++-- 4 files changed, 5 insertions(+), 5 deletions(-) diff --git a/website/docs/docs/core/connect-data-platform/profiles.yml.md b/website/docs/docs/core/connect-data-platform/profiles.yml.md index 97254dda1c4..f8acb65f3d2 100644 --- a/website/docs/docs/core/connect-data-platform/profiles.yml.md +++ b/website/docs/docs/core/connect-data-platform/profiles.yml.md @@ -3,7 +3,7 @@ title: "About profiles.yml" id: profiles.yml --- -If you're using [dbt Core](/docs/core/about-dbt-core), you'll need a `profiles.yml` file that contains the connection details for your data platform. When you run dbt Core from the command line, it reads your `dbt_project.yml` file to find the `profile` name, and then looks for a profile with the same name in your `profiles.yml` file. This profile contains all the information dbt needs to connect to your data platform. +If you're using [dbt Core](/docs/core/installation-overview), you'll need a `profiles.yml` file that contains the connection details for your data platform. When you run dbt Core from the command line, it reads your `dbt_project.yml` file to find the `profile` name, and then looks for a profile with the same name in your `profiles.yml` file. This profile contains all the information dbt needs to connect to your data platform. For detailed info, you can refer to the [Connection profiles](/docs/core/connect-data-platform/connection-profiles). diff --git a/website/docs/docs/deploy/job-commands.md b/website/docs/docs/deploy/job-commands.md index db284c78a05..205e5c119b4 100644 --- a/website/docs/docs/deploy/job-commands.md +++ b/website/docs/docs/deploy/job-commands.md @@ -41,7 +41,7 @@ For every job, you have the option to select the [Generate docs on run](/docs/co ### Command list -You can add or remove as many [dbt commands](/reference/dbt-commands) as necessary for every job. However, you need to have at least one dbt command. There are few commands listed as "dbt Core" in the [dbt Command reference doc](/reference/dbt-commands) page. This means they are meant for use in [dbt Core](/docs/core/about-dbt-core) only and are not available in dbt Cloud. +You can add or remove as many [dbt commands](/reference/dbt-commands) as necessary for every job. However, you need to have at least one dbt command. There are few commands listed as "dbt Core" in the [dbt Command reference doc](/reference/dbt-commands) page. This means they are meant for use in [dbt Core](/docs/core/installation-overview) only and are not available in dbt Cloud. :::tip Using selectors diff --git a/website/docs/docs/running-a-dbt-project/run-your-dbt-projects.md b/website/docs/docs/running-a-dbt-project/run-your-dbt-projects.md index 7de1d4df7df..f1e631f0d78 100644 --- a/website/docs/docs/running-a-dbt-project/run-your-dbt-projects.md +++ b/website/docs/docs/running-a-dbt-project/run-your-dbt-projects.md @@ -13,7 +13,7 @@ You can run your dbt projects with [dbt Cloud](/docs/cloud/about-cloud/dbt-cloud - The dbt Cloud CLI to develop and run dbt commands against your dbt Cloud development environment from your local command line. - For more details, refer to [Develop dbt](/docs/cloud/about-develop-dbt). -- **dbt Core**: An open source project where you can develop from the [command line](/docs/core/about-dbt-core). +- **dbt Core**: An open source project where you can develop from the [command line](/docs/core/installation-overview). The dbt Cloud CLI and dbt Core are both command line tools that enable you to run dbt commands. The key distinction is the dbt Cloud CLI is tailored for dbt Cloud's infrastructure and integrates with all its [features](/docs/cloud/about-cloud/dbt-cloud-features). diff --git a/website/docs/reference/dbt-commands.md b/website/docs/reference/dbt-commands.md index d5f0bfcd2ad..2bd9c63682b 100644 --- a/website/docs/reference/dbt-commands.md +++ b/website/docs/reference/dbt-commands.md @@ -5,7 +5,7 @@ title: "dbt Command reference" You can run dbt using the following tools: - In your browser with the [dbt Cloud IDE](/docs/cloud/dbt-cloud-ide/develop-in-the-cloud) -- On the command line interface using the [dbt Cloud CLI](/docs/cloud/cloud-cli-installation) or open-source [dbt Core](/docs/core/about-dbt-core), both of which enable you to execute dbt commands. The key distinction is the dbt Cloud CLI is tailored for dbt Cloud's infrastructure and integrates with all its [features](/docs/cloud/about-cloud/dbt-cloud-features). +- On the command line interface using the [dbt Cloud CLI](/docs/cloud/cloud-cli-installation) or open-source [dbt Core](/docs/cloud/about-develop-dbt), both of which enable you to execute dbt commands. The key distinction is the dbt Cloud CLI is tailored for dbt Cloud's infrastructure and integrates with all its [features](/docs/cloud/about-cloud/dbt-cloud-features). The following sections outline the commands supported by dbt and their relevant flags. For information about selecting models on the command line, consult the docs on [Model selection syntax](/reference/node-selection/syntax). @@ -71,7 +71,7 @@ Use the following dbt commands in the [dbt Cloud IDE](/docs/cloud/dbt-cloud-ide/ -Use the following dbt commands in [dbt Core](/docs/core/about-dbt-core) and use the `dbt` prefix. For example, to run the `test` command, type `dbt test`. +Use the following dbt commands in [dbt Core](/docs/core/installation-overview) and use the `dbt` prefix. For example, to run the `test` command, type `dbt test`. - [build](/reference/commands/build): build and test all selected resources (models, seeds, snapshots, tests) - [clean](/reference/commands/clean): deletes artifacts present in the dbt project From 4df3bf600628feeb5ab9fe97383306b9c04effca Mon Sep 17 00:00:00 2001 From: mirnawong1 Date: Fri, 17 Nov 2023 12:54:49 +0000 Subject: [PATCH 08/27] fix link --- website/docs/reference/dbt-commands.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/reference/dbt-commands.md b/website/docs/reference/dbt-commands.md index 2bd9c63682b..4cb20051ea2 100644 --- a/website/docs/reference/dbt-commands.md +++ b/website/docs/reference/dbt-commands.md @@ -5,7 +5,7 @@ title: "dbt Command reference" You can run dbt using the following tools: - In your browser with the [dbt Cloud IDE](/docs/cloud/dbt-cloud-ide/develop-in-the-cloud) -- On the command line interface using the [dbt Cloud CLI](/docs/cloud/cloud-cli-installation) or open-source [dbt Core](/docs/cloud/about-develop-dbt), both of which enable you to execute dbt commands. The key distinction is the dbt Cloud CLI is tailored for dbt Cloud's infrastructure and integrates with all its [features](/docs/cloud/about-cloud/dbt-cloud-features). +- On the command line interface using the [dbt Cloud CLI](/docs/cloud/cloud-cli-installation) or open-source [dbt Core](/docs/core/installation-overview), both of which enable you to execute dbt commands. The key distinction is the dbt Cloud CLI is tailored for dbt Cloud's infrastructure and integrates with all its [features](/docs/cloud/about-cloud/dbt-cloud-features). The following sections outline the commands supported by dbt and their relevant flags. For information about selecting models on the command line, consult the docs on [Model selection syntax](/reference/node-selection/syntax). From 6c6a891ad030006386154582da84ef90a66876ac Mon Sep 17 00:00:00 2001 From: mirnawong1 Date: Fri, 17 Nov 2023 12:57:45 +0000 Subject: [PATCH 09/27] rephrase --- website/docs/docs/deploy/job-commands.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/docs/deploy/job-commands.md b/website/docs/docs/deploy/job-commands.md index 205e5c119b4..26fe1931db6 100644 --- a/website/docs/docs/deploy/job-commands.md +++ b/website/docs/docs/deploy/job-commands.md @@ -41,7 +41,7 @@ For every job, you have the option to select the [Generate docs on run](/docs/co ### Command list -You can add or remove as many [dbt commands](/reference/dbt-commands) as necessary for every job. However, you need to have at least one dbt command. There are few commands listed as "dbt Core" in the [dbt Command reference doc](/reference/dbt-commands) page. This means they are meant for use in [dbt Core](/docs/core/installation-overview) only and are not available in dbt Cloud. +You can add or remove as many dbt commands as necessary for every job. However, you need to have at least one dbt command. There are few commands listed as "dbt Cloud CLI" or "dbt Core" in the [dbt Command reference page](/reference/dbt-commands) page. This means they are meant for use in dbt Core or dbt Cloud CLI, and not in dbt Cloud IDE. :::tip Using selectors From 9e65fc6a2057577fb574dd169b05753720d9e2e6 Mon Sep 17 00:00:00 2001 From: mirnawong1 Date: Fri, 17 Nov 2023 13:09:09 +0000 Subject: [PATCH 10/27] fix links --- .../docs/docs/dbt-versions/core-upgrade/04-upgrading-to-v1.4.md | 2 +- .../docs/docs/dbt-versions/core-upgrade/06-upgrading-to-v1.2.md | 2 +- website/docs/docs/dbt-versions/core-versions.md | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/website/docs/docs/dbt-versions/core-upgrade/04-upgrading-to-v1.4.md b/website/docs/docs/dbt-versions/core-upgrade/04-upgrading-to-v1.4.md index 6c6d96b2326..a946bdf369b 100644 --- a/website/docs/docs/dbt-versions/core-upgrade/04-upgrading-to-v1.4.md +++ b/website/docs/docs/dbt-versions/core-upgrade/04-upgrading-to-v1.4.md @@ -12,7 +12,7 @@ import UpgradeMove from '/snippets/_upgrade-move.md'; ### Resources - [Changelog](https://github.com/dbt-labs/dbt-core/blob/1.4.latest/CHANGELOG.md) -- [CLI Installation guide](/docs/core/installation) +- [CLI Installation guide](/docs/core/installation-overview) - [Cloud upgrade guide](/docs/dbt-versions/upgrade-core-in-cloud) **Final release:** January 25, 2023 diff --git a/website/docs/docs/dbt-versions/core-upgrade/06-upgrading-to-v1.2.md b/website/docs/docs/dbt-versions/core-upgrade/06-upgrading-to-v1.2.md index 16825ff4e2b..72a3e0c82ad 100644 --- a/website/docs/docs/dbt-versions/core-upgrade/06-upgrading-to-v1.2.md +++ b/website/docs/docs/dbt-versions/core-upgrade/06-upgrading-to-v1.2.md @@ -12,7 +12,7 @@ import UpgradeMove from '/snippets/_upgrade-move.md'; ### Resources - [Changelog](https://github.com/dbt-labs/dbt-core/blob/1.2.latest/CHANGELOG.md) -- [CLI Installation guide](/docs/core/installation) +- [CLI Installation guide](/docs/core/installation-overview) - [Cloud upgrade guide](/docs/dbt-versions/upgrade-core-in-cloud) ## What to know before upgrading diff --git a/website/docs/docs/dbt-versions/core-versions.md b/website/docs/docs/dbt-versions/core-versions.md index ff7f97f54fc..74970b437e7 100644 --- a/website/docs/docs/dbt-versions/core-versions.md +++ b/website/docs/docs/dbt-versions/core-versions.md @@ -18,7 +18,7 @@ dbt Labs provides different support levels for different versions, which may inc ### Further reading - To learn how you can use dbt Core versions in dbt Cloud, see [Choosing a dbt Core version](/docs/dbt-versions/upgrade-core-in-cloud). -- To learn about installing dbt Core, see "[How to install dbt Core](/docs/core/installation)." +- To learn about installing dbt Core, see "[How to install dbt Core](/docs/core/installation-overview)." - To restrict your project to only work with a range of dbt Core versions, or use the currently running dbt Core version, see [`require-dbt-version`](/reference/project-configs/require-dbt-version) and [`dbt_version`](/reference/dbt-jinja-functions/dbt_version). ## Version support prior to v1.0 From a2a2b17f6e0ad0db599020e9cd93a3213af46bce Mon Sep 17 00:00:00 2001 From: mirnawong1 Date: Fri, 17 Nov 2023 13:40:09 +0000 Subject: [PATCH 11/27] fix next --- website/docs/docs/cloud/about-develop-dbt.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/docs/cloud/about-develop-dbt.md b/website/docs/docs/cloud/about-develop-dbt.md index 89a969fdd60..6aa2a0e96dd 100644 --- a/website/docs/docs/cloud/about-develop-dbt.md +++ b/website/docs/docs/cloud/about-develop-dbt.md @@ -3,7 +3,7 @@ title: About developing in dbt id: about-develop-dbt description: "Learn how to develop your dbt projects using dbt Cloud." sidebar_label: "About developing in dbt" -pagination_next: "docs/cloud/cloud-cli-installation" +pagination_next: "docs/cloud/about-cloud-develop-defer" hide_table_of_contents: false --- From f1e08bc9dab00b27968b40e5b2ad64b11f2b1116 Mon Sep 17 00:00:00 2001 From: mirnawong1 Date: Mon, 27 Nov 2023 06:39:00 -0500 Subject: [PATCH 12/27] folding in jerco's feedback --- website/snippets/_packages_or_dependencies.md | 22 ++++++++++--------- 1 file changed, 12 insertions(+), 10 deletions(-) diff --git a/website/snippets/_packages_or_dependencies.md b/website/snippets/_packages_or_dependencies.md index 0618da233fe..5cc4c67e63c 100644 --- a/website/snippets/_packages_or_dependencies.md +++ b/website/snippets/_packages_or_dependencies.md @@ -2,16 +2,17 @@ ## Use cases Starting from dbt v1.6, `dependencies.yml` has replaced `packages.yml`. The `dependencies.yml` file can now contain both types of dependencies: "package" and "project" dependencies. -- "Package" dependencies lets you add source code from someone else's dbt project into your own, like a library. -- "Project" dependencies provide a different way to build on top of someone else's work in dbt. Refer to [Project dependencies](/docs/collaborate/govern/project-dependencies) for more info. -- You can rename your existing `packages.yml` to `dependencies.yml` unless you need to use Jinja within your packages specification. This is necessary if you want to add an environment variable with a Git token in a private Git package specification. +- ["Package" dependencies](/docs/build/packages) lets you add source code from someone else's dbt project into your own, like a library. +- ["Project" dependencies](/docs/collaborate/govern/project-dependencies) provide a different way to build on top of someone else's work in dbt. -There are some important differences between using a `dependencies.yml` compared to a `packages.yml` file: +If your dbt project doesn't require the use of Jinja within the package specifications, you can simply rename your existing `packages.yml` to `dependencies.yml`. However, something to note is if your project's package specifications use Jinja, particularly for scenarios like adding an environment variable or a [Git token method](/docs/build/packages#git-token-method) in a private Git package specification, you should continue using the `packages.yml` file name. + +There are some important differences between Package dependencies and Project dependencies: - + -`dependencies.yml` is designed for the [dbt Mesh](/best-practices/how-we-mesh/mesh-1-intro) and cross-project reference workflow. Consider using it in the following scenarios: +Project dependencies are designed for the [dbt Mesh](/best-practices/how-we-mesh/mesh-1-intro) and [cross-project reference](/docs/collaborate/govern/project-dependencies#how-to-use-ref) workflow: - Use `dependencies.yml` when you need to set up cross-project references between different dbt projects, especially in a dbt Mesh setup. - Use `dependencies.yml` when you want to include both projects and non-private dbt packages in your project's dependencies. @@ -20,13 +21,14 @@ There are some important differences between using a `dependencies.yml` compared - + -Packages allow you to add source code from someone else's dbt project into your own, like a library. Consider using it in the following scenarios: +Package dependencies allow you to add source code from someone else's dbt project into your own, like a library: - Use `packages.yml` when you want to download dbt packages, such as dbt projects, into your root or parent dbt project. Something to note is that it doesn't contribute to the dbt Mesh workflow. - Use `packages.yml` to include packages, including private packages, in your project's dependencies. If you have private packages that you need to reference, `packages.yml` is the way to go. -- `packages.yml` supports Jinja rendering for historical reasons, allowing dynamic configurations. This can be useful if you need to insert values, like a Git token from an environment variable, into your package specifications. - +- `packages.yml` supports Jinja rendering for historical reasons, allowing dynamic configurations. This can be useful if you need to insert values, like a [Git token method](/docs/build/packages#git-token-method) from an environment variable, into your package specifications. + +Currently, to use private git repositories in dbt, you need to use a workaround that involves embedding a git token with Jinja. This is not ideal as it requires extra steps like creating a user and sharing a git token. We're planning to introduce a simpler method soon that won't require Jinja-embedded secret environment variables. For that reason, `dependencies.yml` does not support Jinja. From e65822c6e9a4a07c7839b8d737d3d94b3b3272ef Mon Sep 17 00:00:00 2001 From: mirnawong1 <89008547+mirnawong1@users.noreply.github.com> Date: Mon, 27 Nov 2023 13:19:59 -0500 Subject: [PATCH 13/27] Update website/sidebars.js --- website/sidebars.js | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/sidebars.js b/website/sidebars.js index 0f4e9adaa45..8145e278e5a 100644 --- a/website/sidebars.js +++ b/website/sidebars.js @@ -34,7 +34,7 @@ const sidebarSettings = { }, { type: "category", - label: "Develop dbt", + label: "Develop with dbt", collapsed: true, link: { type: "doc", id: "docs/cloud/about-develop-dbt" }, items: [ From 6c6c06382c3ca8dad84da01e4aa09f84a4380704 Mon Sep 17 00:00:00 2001 From: mirnawong1 Date: Mon, 27 Nov 2023 13:31:22 -0500 Subject: [PATCH 14/27] tweak language --- .../docs/docs/core/installation-overview.md | 19 +++++++++---------- 1 file changed, 9 insertions(+), 10 deletions(-) diff --git a/website/docs/docs/core/installation-overview.md b/website/docs/docs/core/installation-overview.md index 34594b0a92c..8c139012667 100644 --- a/website/docs/docs/core/installation-overview.md +++ b/website/docs/docs/core/installation-overview.md @@ -29,16 +29,7 @@ You can install dbt Core on the command line by using one of these methods: - [Use Homebrew to install dbt](/docs/core/homebrew-install) - [Use a Docker image to install dbt](/docs/core/docker-install) - [Install dbt from source](/docs/core/source-install) - -**Note** — The [dbt Cloud CLI](/docs/cloud/cloud-cli-installation) and the open-sourced dbt Core are both command line tools that let you run dbt commands. The key distinction is the dbt Cloud CLI is tailored for dbt Cloud's infrastructure and integrates with all its [features](/docs/cloud/about-cloud/dbt-cloud-features). - -:::tip Pro tip: Using the --help flag - -Most command-line tools, including dbt, have a `--help` flag that you can use to show available commands and arguments. For example, you can use the `--help` flag with dbt in two ways:

-— `dbt --help`: Lists the commands available for dbt
-— `dbt run --help`: Lists the flags available for the `run` command - -::: +- You can also develop locally using the [dbt Cloud CLI](/docs/cloud/cloud-cli-installation). The dbt Cloud CLI and dbt Core are both command line tools that let you run dbt commands. The key distinction is the dbt Cloud CLI is tailored for dbt Cloud's infrastructure and integrates with all its [features](/docs/cloud/about-cloud/dbt-cloud-features). ## Upgrading dbt Core @@ -51,3 +42,11 @@ dbt provides a number of resources for understanding [general best practices](/b ## About dbt data platforms and adapters dbt works with a number of different data platforms (databases, query engines, and other SQL-speaking technologies). It does this by using a dedicated _adapter_ for each. When you install dbt Core, you'll also want to install the specific adapter for your database. For more details, see [Supported Data Platforms](/docs/supported-data-platforms). + +:::tip Pro tip: Using the --help flag + +Most command-line tools, including dbt, have a `--help` flag that you can use to show available commands and arguments. For example, you can use the `--help` flag with dbt in two ways:

+— `dbt --help`: Lists the commands available for dbt
+— `dbt run --help`: Lists the flags available for the `run` command + +::: From bceb671b3e7c66bcda6650b973947cfcca4ce96e Mon Sep 17 00:00:00 2001 From: mirnawong1 Date: Mon, 27 Nov 2023 13:35:07 -0500 Subject: [PATCH 15/27] fix link --- website/docs/docs/connect-adapters.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/docs/connect-adapters.md b/website/docs/docs/connect-adapters.md index 6ccc1b4f376..56ff538dc9b 100644 --- a/website/docs/docs/connect-adapters.md +++ b/website/docs/docs/connect-adapters.md @@ -15,7 +15,7 @@ Explore the fastest and most reliable way to deploy dbt using dbt Cloud, a hoste Install dbt Core, an open-source tool, locally using the command line. dbt communicates with a number of different data platforms by using a dedicated adapter plugin for each. When you install dbt Core, you'll also need to install the specific adapter for your database, [connect to dbt Core](/docs/core/about-core-setup), and set up a `profiles.yml` file. -With a few exceptions [^1], you can install all [Verified adapters](/docs/supported-data-platforms) from PyPI using `python -m pip install adapter-name`. For example to install Snowflake, use the command `python -m pip install dbt-snowflake`. The installation will include `dbt-core` and any other required dependencies, which may include both other dependencies and even other adapter plugins. Read more about [installing dbt](/docs/core/installation). +With a few exceptions [^1], you can install all [Verified adapters](/docs/supported-data-platforms) from PyPI using `python -m pip install adapter-name`. For example to install Snowflake, use the command `python -m pip install dbt-snowflake`. The installation will include `dbt-core` and any other required dependencies, which may include both other dependencies and even other adapter plugins. Read more about [installing dbt](/docs/core/installation-overview). [^1]: Here are the two different adapters. Use the PyPI package name when installing with `pip` From 1af2d2ed0b318e2eb330e32c19a237487eb7b3db Mon Sep 17 00:00:00 2001 From: mirnawong1 Date: Tue, 28 Nov 2023 05:36:47 -0500 Subject: [PATCH 16/27] updating table --- website/snippets/core-version-support.md | 2 +- website/snippets/core-versions-table.md | 29 ++++++++++++++++++++---- 2 files changed, 26 insertions(+), 5 deletions(-) diff --git a/website/snippets/core-version-support.md b/website/snippets/core-version-support.md index ff9fa94ff8c..4ec976d4df6 100644 --- a/website/snippets/core-version-support.md +++ b/website/snippets/core-version-support.md @@ -2,4 +2,4 @@ - **[Active](/docs/dbt-versions/core#ongoing-patches)** — We will patch regressions, new bugs, and include fixes for older bugs / quality-of-life improvements. We implement these changes when we have high confidence that they're narrowly scoped and won't cause unintended side effects. - **[Critical](/docs/dbt-versions/core#ongoing-patches)** — Newer minor versions transition the previous minor version into "Critical Support" with limited "security" releases for critical security and installation fixes. - **[End of Life](/docs/dbt-versions/core#eol-version-support)** — Minor versions that have reached EOL no longer receive new patch releases. -- **Deprecated** — dbt-core versions older than v1.0 are no longer maintained by dbt Labs, nor supported in dbt Cloud. +- **Deprecated** — dbt Core versions older than v1.0 are no longer maintained by dbt Labs, nor supported in dbt Cloud. diff --git a/website/snippets/core-versions-table.md b/website/snippets/core-versions-table.md index f1241d8301b..f5cd8e1c2f0 100644 --- a/website/snippets/core-versions-table.md +++ b/website/snippets/core-versions-table.md @@ -1,3 +1,23 @@ +### Latest releases + +| dbt Core | Initial release | Support status and end date | +|:----------------------------------------------------:|:---------------:|:-------------------------------------:| +| [**v1.7**](/docs/dbt-versions/core-upgrade/upgrading-to-v1.7) | Nov 2, 2023 | Active — Nov 1, 2024 | +| [**v1.6**](/docs/dbt-versions/core-upgrade/upgrading-to-v1.6) | Jul 31, 2023 | Critical — Jul 30, 2024 | +| [**v1.5**](/docs/dbt-versions/core-upgrade/upgrading-to-v1.5) | Apr 27, 2023 | Critical — Apr 27, 2024 | +| [**v1.4**](/docs/dbt-versions/core-upgrade/upgrading-to-v1.4) | Jan 25, 2023 | Critical — Jan 25, 2024 | +| [**v1.3**](/docs/dbt-versions/core-upgrade/upgrading-to-v1.3) | Oct 12, 2022 | End of Life* ⚠️ | +| [**v1.2**](/docs/dbt-versions/core-upgrade/upgrading-to-v1.2) | Jul 26, 2022 | End of Life* ⚠️ | +| [**v1.1**](/docs/dbt-versions/core-upgrade/upgrading-to-v1.1) | Apr 28, 2022 | End of Life* ⚠️ | +| [**v1.0**](/docs/dbt-versions/core-upgrade/upgrading-to-v1.0) | Dec 3, 2021 | End of Life* ⚠️ | +| **v0.X** ⛔️ | (Various dates) | Deprecated ⛔️ | Deprecated ⛔️ | +_*All versions of dbt Core since v1.0 are available in dbt Cloud until further notice. Versions that are EOL do not receive any fixes. For the best support, we recommend upgrading to a version released within the past 12 months._ + + + + + From c6e444f40d0e5f9a558c45117cf9400cc734b3f1 Mon Sep 17 00:00:00 2001 From: mirnawong1 Date: Tue, 28 Nov 2023 09:57:51 -0500 Subject: [PATCH 17/27] adjust --- website/docs/docs/cloud/about-develop-dbt.md | 18 ++-------- website/sidebars.js | 36 ++++++++++---------- 2 files changed, 21 insertions(+), 33 deletions(-) diff --git a/website/docs/docs/cloud/about-develop-dbt.md b/website/docs/docs/cloud/about-develop-dbt.md index 6aa2a0e96dd..2249a46da76 100644 --- a/website/docs/docs/cloud/about-develop-dbt.md +++ b/website/docs/docs/cloud/about-develop-dbt.md @@ -4,15 +4,12 @@ id: about-develop-dbt description: "Learn how to develop your dbt projects using dbt Cloud." sidebar_label: "About developing in dbt" pagination_next: "docs/cloud/about-cloud-develop-defer" -hide_table_of_contents: false +hide_table_of_contents: true --- -Develop dbt projects using dbt Cloud or dbt Core. There are a few key differences between the two options: +Develop dbt projects using dbt Cloud, which offers a fast and reliable way to work on your dbt project. It runs dbt Core in a hosted (single or multi-tenant) environment. -## dbt Cloud - -- dbt Cloud offers a fast and reliable way to work on your dbt project. It runs dbt Core in a hosted (single or multi-tenant) environment. -- You can develop in your browser using an integrated development environment (IDE) or in a dbt Cloud-powered command line interface (CLI). +You can develop in your browser using an integrated development environment (IDE) or in a dbt Cloud-powered command line interface (CLI).
@@ -31,12 +28,3 @@ Develop dbt projects using dbt Cloud or dbt Core. There are a few key difference

To get started with dbt development, you'll need a [dbt Cloud](https://www.getdbt.com/signup) account and developer seat. For a more comprehensive guide about developing in dbt, refer to our [quickstart guides](/guides). - -## dbt Core - -- Install dbt Core, the open-sourced version of dbt, on your local machine to develop your dbt project locally. -- You can install dbt Core on the command line with pip install, Homebrew, Docker image, or from source. For more information, see [About dbt Core installation](/docs/core/installation-overview). - ---------- -**Note**: The dbt Cloud CLI and the open-sourced dbt Core are both command line tools that let you run dbt commands. The key distinction is the dbt Cloud CLI is tailored for dbt Cloud's infrastructure and integrates with all its [features](/docs/cloud/about-cloud/dbt-cloud-features). - diff --git a/website/sidebars.js b/website/sidebars.js index e1c33d1c849..6643ca767ac 100644 --- a/website/sidebars.js +++ b/website/sidebars.js @@ -27,14 +27,9 @@ const sidebarSettings = { "docs/cloud/about-cloud/browsers", ], }, // About dbt Cloud directory - { - type: "link", - label: "Guides", - href: `/guides`, - }, { type: "category", - label: "Develop with dbt", + label: "Develop with dbt Cloud", collapsed: true, link: { type: "doc", id: "docs/cloud/about-develop-dbt" }, items: [ @@ -61,20 +56,13 @@ const sidebarSettings = { "docs/cloud/dbt-cloud-ide/dbt-cloud-tips", ], }, - { - type: "category", - label: "dbt Core", - link: { type: "doc", id: "docs/core/installation-overview", }, - items: [ - "docs/core/installation-overview", - "docs/core/homebrew-install", - "docs/core/pip-install", - "docs/core/docker-install", - "docs/core/source-install", - ], - }, ], }, + { + type: "link", + label: "Guides", + href: `/guides`, + }, { type: "category", label: "Set up dbt", @@ -190,6 +178,18 @@ const sidebarSettings = { items: [ "docs/core/about-core-setup", "docs/core/dbt-core-environments", + { + type: "category", + label: "Install dbt Core", + link: { type: "doc", id: "docs/core/installation-overview", }, + items: [ + "docs/core/installation-overview", + "docs/core/homebrew-install", + "docs/core/pip-install", + "docs/core/docker-install", + "docs/core/source-install", + ], + }, { type: "category", label: "Connect data platform", From 186174fbd9a3f799bee4904fd638345cd7812567 Mon Sep 17 00:00:00 2001 From: mirnawong1 <89008547+mirnawong1@users.noreply.github.com> Date: Tue, 28 Nov 2023 11:02:03 -0500 Subject: [PATCH 18/27] Update vercel.json --- website/vercel.json | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/website/vercel.json b/website/vercel.json index 82e01200b8f..4749a5a8701 100644 --- a/website/vercel.json +++ b/website/vercel.json @@ -10,8 +10,11 @@ { "source": "/docs/core/about-dbt-core", "destination": "/docs/core/installation-overview", + "permanent": true + }, + { "source": "/docs/deploy/airgapped", - "destination": "/docs/cloud/about-cloud/tenancy" + "destination": "/docs/cloud/about-cloud/tenancy", "permanent": true }, { From 65937cd175e5d10f6e49826430257fa41ecf7a38 Mon Sep 17 00:00:00 2001 From: mirnawong1 Date: Tue, 28 Nov 2023 12:25:36 -0500 Subject: [PATCH 19/27] adjust ia based on feedback --- website/docs/docs/cloud/about-develop-dbt.md | 2 +- website/sidebars.js | 72 ++++++++++---------- 2 files changed, 37 insertions(+), 37 deletions(-) diff --git a/website/docs/docs/cloud/about-develop-dbt.md b/website/docs/docs/cloud/about-develop-dbt.md index 2249a46da76..a71c32d5352 100644 --- a/website/docs/docs/cloud/about-develop-dbt.md +++ b/website/docs/docs/cloud/about-develop-dbt.md @@ -2,7 +2,7 @@ title: About developing in dbt id: about-develop-dbt description: "Learn how to develop your dbt projects using dbt Cloud." -sidebar_label: "About developing in dbt" +sidebar_label: "About developing in dbt" pagination_next: "docs/cloud/about-cloud-develop-defer" hide_table_of_contents: true --- diff --git a/website/sidebars.js b/website/sidebars.js index cf4960f1ffa..ea7c0c90814 100644 --- a/website/sidebars.js +++ b/website/sidebars.js @@ -1,6 +1,11 @@ const sidebarSettings = { docs: [ "docs/introduction", + { + type: "link", + label: "Guides", + href: `/guides`, + }, { type: "category", label: "Supported data platforms", @@ -27,42 +32,6 @@ const sidebarSettings = { "docs/cloud/about-cloud/browsers", ], }, // About dbt Cloud directory - { - type: "category", - label: "Develop with dbt Cloud", - collapsed: true, - link: { type: "doc", id: "docs/cloud/about-develop-dbt" }, - items: [ - "docs/cloud/about-develop-dbt", - "docs/cloud/about-cloud-develop-defer", - { - type: "category", - label: "dbt Cloud CLI", - collapsed: true, - link: { type: "doc", id: "docs/cloud/cloud-cli-installation" }, - items: [ - "docs/cloud/cloud-cli-installation", - "docs/cloud/configure-cloud-cli", - ], - }, - { - type: "category", - label: "dbt Cloud IDE", - link: { type: "doc", id: "docs/cloud/dbt-cloud-ide/develop-in-the-cloud" }, - items: [ - "docs/cloud/dbt-cloud-ide/develop-in-the-cloud", - "docs/cloud/dbt-cloud-ide/ide-user-interface", - "docs/cloud/dbt-cloud-ide/lint-format", - "docs/cloud/dbt-cloud-ide/dbt-cloud-tips", - ], - }, - ], - }, - { - type: "link", - label: "Guides", - href: `/guides`, - }, { type: "category", label: "Set up dbt", @@ -250,6 +219,37 @@ const sidebarSettings = { "docs/running-a-dbt-project/using-threads", ], }, + { + type: "category", + label: "Develop with dbt Cloud", + collapsed: true, + link: { type: "doc", id: "docs/cloud/about-develop-dbt" }, + items: [ + "docs/cloud/about-develop-dbt", + "docs/cloud/about-cloud-develop-defer", + { + type: "category", + label: "dbt Cloud CLI", + collapsed: true, + link: { type: "doc", id: "docs/cloud/cloud-cli-installation" }, + items: [ + "docs/cloud/cloud-cli-installation", + "docs/cloud/configure-cloud-cli", + ], + }, + { + type: "category", + label: "dbt Cloud IDE", + link: { type: "doc", id: "docs/cloud/dbt-cloud-ide/develop-in-the-cloud" }, + items: [ + "docs/cloud/dbt-cloud-ide/develop-in-the-cloud", + "docs/cloud/dbt-cloud-ide/ide-user-interface", + "docs/cloud/dbt-cloud-ide/lint-format", + "docs/cloud/dbt-cloud-ide/dbt-cloud-tips", + ], + }, + ], + }, { type: "category", label: "Build dbt projects", From 5212f0b3b1e22b0d9099057a79ab09fe5548534c Mon Sep 17 00:00:00 2001 From: Ly Nguyen Date: Tue, 28 Nov 2023 10:16:42 -0800 Subject: [PATCH 20/27] Add date in YAML front matter --- .../release-notes/02-Nov-2023/explorer-updates-rn.md | 1 + .../release-notes/02-Nov-2023/job-notifications-rn.md | 1 + .../release-notes/02-Nov-2023/microsoft-fabric-support-rn.md | 1 + 3 files changed, 3 insertions(+) diff --git a/website/docs/docs/dbt-versions/release-notes/02-Nov-2023/explorer-updates-rn.md b/website/docs/docs/dbt-versions/release-notes/02-Nov-2023/explorer-updates-rn.md index 4adefb7f6ca..8b829311d81 100644 --- a/website/docs/docs/dbt-versions/release-notes/02-Nov-2023/explorer-updates-rn.md +++ b/website/docs/docs/dbt-versions/release-notes/02-Nov-2023/explorer-updates-rn.md @@ -4,6 +4,7 @@ description: "November 2023: New features and UI changes to dbt Explorer, includ sidebar_label: "Enhancement: New features and UI changes to dbt Explorer" sidebar_position: 08 tags: [Nov-2023] +date: 2023-11-28 --- dbt Labs is excited to announce the latest features and UI updates to dbt Explorer! diff --git a/website/docs/docs/dbt-versions/release-notes/02-Nov-2023/job-notifications-rn.md b/website/docs/docs/dbt-versions/release-notes/02-Nov-2023/job-notifications-rn.md index 660129513d7..02fe2e037df 100644 --- a/website/docs/docs/dbt-versions/release-notes/02-Nov-2023/job-notifications-rn.md +++ b/website/docs/docs/dbt-versions/release-notes/02-Nov-2023/job-notifications-rn.md @@ -4,6 +4,7 @@ description: "November 2023: New quality-of-life improvements for setting up and sidebar_label: "Enhancement: Job notifications" sidebar_position: 10 tags: [Nov-2023] +date: 2023-11-28 --- There are new quality-of-life improvements in dbt Cloud for email and Slack notifications about your jobs: diff --git a/website/docs/docs/dbt-versions/release-notes/02-Nov-2023/microsoft-fabric-support-rn.md b/website/docs/docs/dbt-versions/release-notes/02-Nov-2023/microsoft-fabric-support-rn.md index 13aefa80ffc..9c4b81e9b94 100644 --- a/website/docs/docs/dbt-versions/release-notes/02-Nov-2023/microsoft-fabric-support-rn.md +++ b/website/docs/docs/dbt-versions/release-notes/02-Nov-2023/microsoft-fabric-support-rn.md @@ -4,6 +4,7 @@ description: "November 2023: Public Preview now available for Microsoft Fabric i sidebar_label: "New: Public Preview of Microsoft Fabric support" sidebar_position: 09 tags: [Nov-2023] +date: 2023-11-28 --- Public Preview is now available in dbt Cloud for Microsoft Fabric! From e2473c58c93c157559a6482a0283ab858b7a01db Mon Sep 17 00:00:00 2001 From: mirnawong1 <89008547+mirnawong1@users.noreply.github.com> Date: Wed, 29 Nov 2023 05:20:53 -0500 Subject: [PATCH 21/27] Update core-versions-table.md small fix --- website/snippets/core-versions-table.md | 22 +--------------------- 1 file changed, 1 insertion(+), 21 deletions(-) diff --git a/website/snippets/core-versions-table.md b/website/snippets/core-versions-table.md index f5cd8e1c2f0..fc7b054bc0a 100644 --- a/website/snippets/core-versions-table.md +++ b/website/snippets/core-versions-table.md @@ -1,6 +1,6 @@ ### Latest releases -| dbt Core | Initial release | Support status and end date | +| dbt Core | Initial release | Support level and end date | |:----------------------------------------------------:|:---------------:|:-------------------------------------:| | [**v1.7**](/docs/dbt-versions/core-upgrade/upgrading-to-v1.7) | Nov 2, 2023 | Active — Nov 1, 2024 | | [**v1.6**](/docs/dbt-versions/core-upgrade/upgrading-to-v1.6) | Jul 31, 2023 | Critical — Jul 30, 2024 | @@ -13,23 +13,3 @@ | **v0.X** ⛔️ | (Various dates) | Deprecated ⛔️ | Deprecated ⛔️ | _*All versions of dbt Core since v1.0 are available in dbt Cloud until further notice. Versions that are EOL do not receive any fixes. For the best support, we recommend upgrading to a version released within the past 12 months._ - - - - From 7214b6c1595dda446741f5090272186979fdc4b6 Mon Sep 17 00:00:00 2001 From: mirnawong1 Date: Wed, 29 Nov 2023 07:14:11 -0500 Subject: [PATCH 22/27] removing redundant landing page --- .../docs/cloud/dbt-cloud-ide/dbt-cloud-ide.md | 37 ------------------- website/docs/docs/core/about-core-setup.md | 4 +- website/vercel.json | 5 +++ 3 files changed, 7 insertions(+), 39 deletions(-) delete mode 100644 website/docs/docs/cloud/dbt-cloud-ide/dbt-cloud-ide.md diff --git a/website/docs/docs/cloud/dbt-cloud-ide/dbt-cloud-ide.md b/website/docs/docs/cloud/dbt-cloud-ide/dbt-cloud-ide.md deleted file mode 100644 index 3c41432bc62..00000000000 --- a/website/docs/docs/cloud/dbt-cloud-ide/dbt-cloud-ide.md +++ /dev/null @@ -1,37 +0,0 @@ ---- -title: "dbt Cloud IDE" -description: "Learn how to configure Git in dbt Cloud" -pagination_next: "docs/cloud/dbt-cloud-ide/develop-in-the-cloud" -pagination_prev: null ---- - -
- - - - - -
-
-
- - - - -
\ No newline at end of file diff --git a/website/docs/docs/core/about-core-setup.md b/website/docs/docs/core/about-core-setup.md index 2f6c077ba7d..690cd885d3b 100644 --- a/website/docs/docs/core/about-core-setup.md +++ b/website/docs/docs/core/about-core-setup.md @@ -14,7 +14,7 @@ dbt Core is an [open-source](https://github.com/dbt-labs/dbt-core) tool that ena - [Connecting to a data platform](/docs/core/connect-data-platform/profiles.yml) - [How to run your dbt projects](/docs/running-a-dbt-project/run-your-dbt-projects) -To learn about developing dbt projects in dbt Cloud or dbt Core, refer to [Develop dbt](/docs/cloud/about-develop-dbt). -- **Note** — dbt Cloud provides a command line interface with the [dbt Cloud CLI](/docs/cloud/cloud-cli-installation). Both the open-sourced dbt Core and the dbt Cloud CLI are command line tools that let you run dbt commands. The key distinction is the dbt Cloud CLI is tailored for dbt Cloud's infrastructure and integrates with all its [features](/docs/cloud/about-cloud/dbt-cloud-features). +To learn about developing dbt projects in dbt Cloud, refer to [Develop dbt](/docs/cloud/about-develop-dbt). + - dbt Cloud provides a command line interface with the [dbt Cloud CLI](/docs/cloud/cloud-cli-installation). Both dbt Core and the dbt Cloud CLI are command line tools that let you run dbt commands. The key distinction is the dbt Cloud CLI is tailored for dbt Cloud's infrastructure and integrates with all its [features](/docs/cloud/about-cloud/dbt-cloud-features). If you need a more detailed first-time setup guide for specific data platforms, read our [quickstart guides](https://docs.getdbt.com/guides). diff --git a/website/vercel.json b/website/vercel.json index 4749a5a8701..3490c661f1a 100644 --- a/website/vercel.json +++ b/website/vercel.json @@ -2,6 +2,11 @@ "cleanUrls": true, "trailingSlash": false, "redirects": [ + { + "source": "/docs/cloud/dbt-cloud-ide", + "destination": "/docs/cloud/dbt-cloud-ide/develop-in-the-cloud", + "permanent": true + }, { "source": "/docs/cloud/about-cloud-develop", "destination": "/docs/cloud/about-develop-dbt", From bcccc1d2accbf87f537f9a697b0b87fac24f3550 Mon Sep 17 00:00:00 2001 From: mirnawong1 <89008547+mirnawong1@users.noreply.github.com> Date: Wed, 29 Nov 2023 07:55:04 -0500 Subject: [PATCH 23/27] removing limitation bullet removing `metric_time` limitation as this was resolved in https://github.com/dbt-labs/metricflow/issues/825 --- website/docs/docs/build/cumulative-metrics.md | 5 +---- 1 file changed, 1 insertion(+), 4 deletions(-) diff --git a/website/docs/docs/build/cumulative-metrics.md b/website/docs/docs/build/cumulative-metrics.md index 708045c1f3e..e4c3e1b9cbc 100644 --- a/website/docs/docs/build/cumulative-metrics.md +++ b/website/docs/docs/build/cumulative-metrics.md @@ -38,10 +38,7 @@ metrics: ## Limitations Cumulative metrics are currently under active development and have the following limitations: - -1. You can only use the [`metric_time` dimension](/docs/build/dimensions#time) to check cumulative metrics. If you don't use `metric_time` in the query, the cumulative metric will return incorrect results because it won't perform the time spine join. This means you cannot reference time dimensions other than the `metric_time` in the query. -2. If you use `metric_time` in your query filter but don't include "start_time" and "end_time," cumulative metrics will left-censor the input data. For example, if you query a cumulative metric with a 7-day window with the filter `{{ TimeDimension('metric_time') }} BETWEEN '2023-08-15' AND '2023-08-30' `, the values for `2023-08-15` to `2023-08-20` return missing or incomplete data. This is because we apply the `metric_time` filter to the aggregation input. To avoid this, you must use `start_time` and `end_time` in the query filter. - +- You can only use the [`metric_time` dimension](/docs/build/dimensions#time) to check cumulative metrics. If you don't use `metric_time` in the query, the cumulative metric will return incorrect results because it won't perform the time spine join. This means you cannot reference time dimensions other than the `metric_time` in the query. ## Cumulative metrics example From d2613429f0c29f096bf31678d3a850cdc432bd19 Mon Sep 17 00:00:00 2001 From: Jason Karlavige Date: Wed, 29 Nov 2023 08:38:53 -0500 Subject: [PATCH 24/27] update banner copy --- website/docusaurus.config.js | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docusaurus.config.js b/website/docusaurus.config.js index 13c284dd557..b4b758e7744 100644 --- a/website/docusaurus.config.js +++ b/website/docusaurus.config.js @@ -71,7 +71,7 @@ var siteSettings = { }, announcementBar: { id: "biweekly-demos", - content: "Join our weekly demos and dbt Cloud in action!", + content: "Join our weekly demos and see dbt Cloud in action!", backgroundColor: "#047377", textColor: "#fff", isCloseable: true, From d4ace15b15e009687fe4954b9638c602b5c35cbb Mon Sep 17 00:00:00 2001 From: mirnawong1 Date: Wed, 29 Nov 2023 12:51:41 -0500 Subject: [PATCH 25/27] adding core redirect --- website/vercel.json | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/website/vercel.json b/website/vercel.json index 4749a5a8701..a881dc0f097 100644 --- a/website/vercel.json +++ b/website/vercel.json @@ -2,6 +2,12 @@ "cleanUrls": true, "trailingSlash": false, "redirects": [ + { + "source": "/docs/core/installation", + "destination": "/docs/core/installation-overview", + "permanent": true + + }, { "source": "/docs/cloud/about-cloud-develop", "destination": "/docs/cloud/about-develop-dbt", From 09ccb02e0a8e5286fce44d196b50d2f0ad111fc8 Mon Sep 17 00:00:00 2001 From: mirnawong1 <89008547+mirnawong1@users.noreply.github.com> Date: Wed, 29 Nov 2023 13:04:17 -0500 Subject: [PATCH 26/27] Update website/docs/docs/build/cumulative-metrics.md --- website/docs/docs/build/cumulative-metrics.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/docs/build/cumulative-metrics.md b/website/docs/docs/build/cumulative-metrics.md index e4c3e1b9cbc..45a136df751 100644 --- a/website/docs/docs/build/cumulative-metrics.md +++ b/website/docs/docs/build/cumulative-metrics.md @@ -38,7 +38,7 @@ metrics: ## Limitations Cumulative metrics are currently under active development and have the following limitations: -- You can only use the [`metric_time` dimension](/docs/build/dimensions#time) to check cumulative metrics. If you don't use `metric_time` in the query, the cumulative metric will return incorrect results because it won't perform the time spine join. This means you cannot reference time dimensions other than the `metric_time` in the query. +- You are required to use [`metric_time` dimension](/docs/build/dimensions#time) when querying cumulative metrics. If you don't use `metric_time` in the query, the cumulative metric will return incorrect results because it won't perform the time spine join. This means you cannot reference time dimensions other than the `metric_time` in the query. ## Cumulative metrics example From e9c7e5bb520a6ce5beccab569c73316e7545ec61 Mon Sep 17 00:00:00 2001 From: mirnawong1 <89008547+mirnawong1@users.noreply.github.com> Date: Wed, 29 Nov 2023 13:27:59 -0500 Subject: [PATCH 27/27] Update website/docs/docs/core/about-core-setup.md Co-authored-by: Matt Shaver <60105315+matthewshaver@users.noreply.github.com> --- website/docs/docs/core/about-core-setup.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/docs/core/about-core-setup.md b/website/docs/docs/core/about-core-setup.md index 690cd885d3b..8b170ba70d4 100644 --- a/website/docs/docs/core/about-core-setup.md +++ b/website/docs/docs/core/about-core-setup.md @@ -14,7 +14,7 @@ dbt Core is an [open-source](https://github.com/dbt-labs/dbt-core) tool that ena - [Connecting to a data platform](/docs/core/connect-data-platform/profiles.yml) - [How to run your dbt projects](/docs/running-a-dbt-project/run-your-dbt-projects) -To learn about developing dbt projects in dbt Cloud, refer to [Develop dbt](/docs/cloud/about-develop-dbt). +To learn about developing dbt projects in dbt Cloud, refer to [Develop with dbt Cloud](/docs/cloud/about-develop-dbt). - dbt Cloud provides a command line interface with the [dbt Cloud CLI](/docs/cloud/cloud-cli-installation). Both dbt Core and the dbt Cloud CLI are command line tools that let you run dbt commands. The key distinction is the dbt Cloud CLI is tailored for dbt Cloud's infrastructure and integrates with all its [features](/docs/cloud/about-cloud/dbt-cloud-features). If you need a more detailed first-time setup guide for specific data platforms, read our [quickstart guides](https://docs.getdbt.com/guides).