Skip to content

Commit

Permalink
dbt Explorer FAQs (#5237)
Browse files Browse the repository at this point in the history
## What are you changing in this pull request and why?

Per
[slack](https://dbt-labs.slack.com/archives/C02NCQ9483C/p1711406797155509),
create FAQs page for dbt Explorer

## Checklist
- [x] Review the [Content style
guide](https://github.com/dbt-labs/docs.getdbt.com/blob/current/contributing/content-style-guide.md)
so my content adheres to these guidelines.
- [x] For [docs
versioning](https://github.com/dbt-labs/docs.getdbt.com/blob/current/contributing/single-sourcing-content.md#about-versioning),
review how to [version a whole
page](https://github.com/dbt-labs/docs.getdbt.com/blob/current/contributing/single-sourcing-content.md#adding-a-new-version)
and [version a block of
content](https://github.com/dbt-labs/docs.getdbt.com/blob/current/contributing/single-sourcing-content.md#versioning-blocks-of-content).
- [x] Needs review from PMM

Adding pages:
- [x] Add page in `website/sidebars.js`
- [x] Provide a unique filename for new pages
- [x] Run link testing locally with `npm run build` to update the links
that point to deleted pages
  • Loading branch information
nghi-ly authored Apr 5, 2024
2 parents 26b8e3e + 4fcf4aa commit 22ee0da
Show file tree
Hide file tree
Showing 6 changed files with 184 additions and 5 deletions.
5 changes: 1 addition & 4 deletions website/docs/docs/collaborate/column-level-lineage.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,15 +3,12 @@ title: "Column-level lineage"
description: "Use dbt Explorer's column-level lineage to gain insights about your data at a granular level."
---

# Column-level lineage <Lifecycle status='beta' />
# Column-level lineage <Lifecycle status='public preview' />

dbt Explorer now offers column-level lineage (CLL) for the resources in your dbt project. Analytics engineers can quickly and easily gain insight into the provenance of their data products at a more granular level. For each column in a resource (model, source, or snapshot) in a dbt project, Explorer provides end-to-end lineage for the data in that column given how it's used.

CLL is available to dbt Cloud Enterprise accounts that can use Explorer. It’s also available through the [Discovery API](/docs/dbt-cloud-apis/discovery-api).

:::tip Check out our beta
Explorer's CLL is currently available as a [public beta](/docs/dbt-versions/product-lifecycles#dbt-cloud) for Enterprise plan accounts. Please check it out!
:::

## Access the column-level lineage

Expand Down
176 changes: 176 additions & 0 deletions website/docs/docs/collaborate/dbt-explorer-faqs.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,176 @@
---
title: "dbt Explorer FAQs"
sidebar_label: "dbt Explorer FAQs"
description: "Learn more with the FAQs about dbt Explorer, how it works, how to interact with it, and more."
pagination_next: null
---

[dbt Explorer](/docs/collaborate/explore-projects) is dbt Cloud’s new knowledge base and lineage visualization experience. It offers an interactive and high-level view of your company’s entire data estate, where you can dive deep into the context you need to understand and improve lineage so your teams can trust the data they’re using to make decisions.

## Overview

<expandable alt_header="How does dbt Explorer help with data quality?" >

dbt Explorer makes it easy and intuitive to understand your entire lineage &mdash; from data source to the reporting layer &mdash; so you can troubleshoot, improve, and optimize your pipelines. With built-in features like project recommendations and model performance analysis, you can be sure you have appropriate test and documentation coverage across your estate and quickly spot and remediate slow-running models. With column-level lineage, you can quickly identify the potential downstream impacts of table changes or work backwards to quickly understand the root cause of an incident. dbt Explorer gives teams the insights they need to improve data quality proactively, ensuring pipelines stay performant and data trust remains solid.

</expandable>

<expandable alt_header="How is dbt Explorer priced?" >

With the current Preview, dbt Explorer is available to everyone on the dbt Cloud Enterprise and Team plans. Certain features within dbt Explorer, such as multi-project lineage and column-level lineage, are only available on the Enterprise plan. dbt Explorer can be accessed by users with developer and read-only seats during the Preview period.

</expandable>

<expandable alt_header="What happened to dbt Docs?" >

dbt Explorer is the default documentation experience for dbt Cloud customers. dbt Docs is still available but doesn't offer the same speed, metadata, or visibility as dbt Explorer and will become a legacy feature.

</expandable>

## How dbt Explorer works

<expandable alt_header="Can I use dbt Explorer on-premise or with my self-hosted dbt Core deployment?" >

No. dbt Explorer and all of its features are only available as a dbt Cloud user experience. dbt Explorer reflects the metadata from your dbt Cloud project(s) and their runs.

</expandable>

<expandable alt_header="How does dbt Explorer support dbt Cloud environments?" >

dbt Explorer defaults to the latest production state of a project. Support for staging and development (Cloud CLI and IDE) environments is coming soon. Users can only assign a single production and staging environment per dbt Cloud project.

</expandable>

<expandable alt_header="How do I get started in Explorer? How does it update?" >

Simply select **Explore** from the dbt Cloud top navigation bar. dbt Explorer automatically updates after each dbt Cloud run in the given project’s environment (production, by default). The dbt commands you run within the environment will generate and update the metadata in dbt Explorer, so make sure to run the correct combination of commands within the jobs of the environment; for more details, refer to [Generate metadata](/docs/collaborate/explore-projects#generate-metadata).

</expandable>

<expandable alt_header="Is it possible to export dbt lineage to an external system or catalog?" >

Yes. The lineage that powers dbt Explorer is also available through the Discovery API.

</expandable>

<expandable alt_header="How does dbt Explorer integrate with third-party tools to show end-to-end lineage?" >

dbt Explorer reflects all the lineage defined within the dbt project. Our vision for dbt Explorer is to incorporate additional metadata from external tools like data loaders (sources) and BI/analytics tools (exposures) integrated with dbt Cloud, all seamlessly incorporated into the lineage of the dbt Cloud project.

</expandable>


## Key features

<expandable alt_header="Does dbt Explorer support multi-project discovery (dbt Mesh)?" >

Yes. Refer to [Explore multiple projects](/docs/collaborate/explore-multiple-projects) to learn more.

</expandable>

<expandable alt_header="What kind of search capabilities does dbt Explorer support?" >

Resource search capabilities include using keywords, partial strings (fuzzy search), and set operators like `OR`. Meanwhile, lineage search supports using dbt selectors. For details, refer to [Keyword search](/docs/collaborate/explore-projects#search-resources).

</expandable>

<expandable alt_header="Can I view model execution information for a job that is currently being run?" >

dbt Cloud updates the performance charts and metrics after a job run.

</expandable>

<expandable alt_header="Can I analyze the number of successful model runs within a month?" >

A chart of models built by month is available in the dbt Cloud dashboard.

</expandable>

<expandable alt_header="Can model or column descriptions be edited within dbt Cloud?" >

Yes. Today, you can edit descriptions in the dbt Cloud IDE or CLI by changing the YAML files within the dbt project. In the future, dbt Explorer will support more ways of editing descriptions.

</expandable>

<expandable alt_header="Where do recommendations come from? Can they be customized?" >

Recommendations largely mirror the best practice rules from the `dbt_project_evaluator` package. At this time, recommendations can’t be customized. In the future, dbt Explorer will likely support recommendation customization capabilities (for example, in project code).

</expandable>

## Column-level lineage

<expandable alt_header="What are the best use cases for column-level lineage in dbt Explorer?" >

Column-level lineage in dbt Explorer can be used to improve many data development workflows, including:

- **Audit** &mdash; Visualize how data moves through and is used in your dbt project
- **Root cause** &mdash; Improve time to detect and resolve data quality issues, tracking back to the source
- **Impact analysis** &mdash; Trace transformations and usage to avoid introducing issues for consumers
- **Efficiency** &mdash; Prune unnecessary columns to reduce costs and data team overhead

</expandable>

<expandable alt_header="Does the column-level lineage remain functional even if column names vary between models?" >

Yes. Column-level lineage can handle name changes across instances of the column in the dbt project.

</expandable>

<expandable alt_header="Can multiple projects leverage the same column definition?" >

No. Cross-project column lineage is supported in the sense of viewing how a public model is used across projects, but not on a column-level.

</expandable>

<expandable alt_header="Is column-level lineage data available through the API?" >

Yes. Column-level lineage is available through a beta endpoint using the [Discovery API](/docs/dbt-cloud-apis/discovery-api) to dbt Cloud Enterprise plan customers.

</expandable>

<expandable alt_header="Can column descriptions be propagated down in downstream lineage automatically?" >

Not currently, but this type of functionality is planned for future releases.

</expandable>

<expandable alt_header="Is column-level lineage also available in the development tab?" >

Not currently, but we plan to incorporate column-level awareness across features in dbt Cloud in the future.

</expandable>



## Availability, access, and permissions

<expandable alt_header="How can non-developers interact with dbt Explorer?" >

Read-only users can consume metadata in dbt Explorer. More bespoke experiences and exploration avenues for analysts and less-technical contributors will be provided in the future.

</expandable>

<expandable alt_header="Does dbt Explorer require a specific dbt Cloud plan?" >

dbt Explorer is available on the dbt Cloud Team and Enterprise plans. Certain features within dbt Explorer, like multi-project lineage and column-level lineage, are only available on the Enterprise plan.

</expandable>

<expandable alt_header="Will dbt Core users be able to leverage any of these new dbt Explorer features?" >

No. dbt Explorer is a dbt Cloud-only product experience.

</expandable>

<expandable alt_header="Is it possible to access dbt Explorer using a read-only license?" >

Yes, during the current Preview period.

</expandable>

<expandable alt_header="Is there an easy way to share useful dbt Explorer content with people outside of dbt Cloud?" >

The ability to embed and share views is being evaluated as a potential future capability.

</expandable>
3 changes: 2 additions & 1 deletion website/docs/docs/collaborate/explore-multiple-projects.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,10 @@
title: "Explore multiple projects"
sidebar_label: "Explore multiple projects"
description: "Learn about project-level lineage in dbt Explorer and its uses."
pagination_next: null
---

# Explore multiple projects <Lifecycle status='public preview' />

You can also view all the different projects and public models in the account, where the public models are defined, and how they are used to gain a better understanding about your cross-project resources.

The resource-level lineage graph for a given project displays the cross-project relationships in the DAG. The different icons indicate whether you’re looking at an upstream producer project (parent) or a downstream consumer project (child).
Expand Down
2 changes: 2 additions & 0 deletions website/docs/docs/collaborate/model-performance.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,8 @@ sidebar_label: "Model performance"
description: "Learn about the performance of your models so you can make improvements to save time and money."
---

# Model performance <Lifecycle status='public preview' />

dbt Explorer provides metadata on dbt Cloud runs for in-depth model performance and quality analysis. This feature assists in reducing infrastructure costs and saving time for data teams by highlighting where to fine-tune projects and deployments &mdash; such as model refactoring or job configuration adjustments.

<LoomVideo id='98f33b3b7a374df0b7c04747eae6ef44' />
Expand Down
2 changes: 2 additions & 0 deletions website/docs/docs/collaborate/project-recommendations.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,8 @@ title: "Project recommendations"
sidebar_label: "Project recommendations"
description: "dbt Explorer provides recommendations that you can take to improve the quality of your dbt project."
---

# Project recommendations <Lifecycle status='public preview' />

dbt Explorer provides recommendations about your project from the `dbt_project_evaluator` [package](https://hub.getdbt.com/dbt-labs/dbt_project_evaluator/latest/) using metadata from the Discovery API.

Expand Down
1 change: 1 addition & 0 deletions website/sidebars.js
Original file line number Diff line number Diff line change
Expand Up @@ -455,6 +455,7 @@ const sidebarSettings = {
"docs/collaborate/model-performance",
"docs/collaborate/project-recommendations",
"docs/collaborate/explore-multiple-projects",
"docs/collaborate/dbt-explorer-faqs",
],
},
{
Expand Down

0 comments on commit 22ee0da

Please sign in to comment.