Skip to content

Commit

Permalink
introduce "trusted" tier for adapters (#3758)
Browse files Browse the repository at this point in the history
## What are you changing in this pull request and why?

Introducing Trusted adapters.

## Pages Changed

### Supported Data Platforms


[preview](https://deploy-preview-3758--docs-getdbt-com.netlify.app/docs/supported-data-platforms)

- Added bullet point about "Trusted adapters"
- Added Trusted adapter cards below the verified adapter cards

### Trusted Adapters


[preview](https://deploy-preview-3758--docs-getdbt-com.netlify.app/docs/trusted-adapters)

- brand new page that gives overview
- links out to a new adapter dev guide

### Building a Trusted Adapter


[preview](https://deploy-preview-3758--docs-getdbt-com.netlify.app/guides/dbt-ecosystem/adapter-development/8-building-a-trusted-adapter)

- details what Trusted adapter means
- enumerates the relevant guidelines
- explains how to become Trusted

### New Issue Template

[Link to
`.yaml`](https://github.com/dbt-labs/docs.getdbt.com/pull/3758/files#diff-ef4a0e442faae77613880dfc0527a87b5f0444e3512597c3d622eaed7ce5ffaf)
(there's no way to preview this AFAICT)

This serves as both a webform and also a "contract"


### Other changes

- Make the cards view for Verified and Trusted adapters into snippets
for re-usability
- add Verified Adapters overview page to sidebar
- remove outdated reference on "Connect Adapters" page
- small cleanups

## Checklist

- [x] Review the [Content style
guide](https://github.com/dbt-labs/docs.getdbt.com/blob/current/contributing/content-style-guide.md)
and [About
versioning](https://github.com/dbt-labs/docs.getdbt.com/blob/current/contributing/single-sourcing-content.md#adding-a-new-version)
so my content adheres to these guidelines.
- [ ] internal stakeholders approve

Adding new pages (delete if not applicable):
- [x] Add page to `website/sidebars.js`
- [ ] Provide a unique filename for the new page
  • Loading branch information
dataders authored Aug 31, 2023
2 parents 17f697c + 2f02c83 commit bfc18a4
Show file tree
Hide file tree
Showing 10 changed files with 285 additions and 108 deletions.
51 changes: 51 additions & 0 deletions .github/ISSUE_TEMPLATE/add-adapter-to-trusted-list.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
name: Add adapter to Trusted list
description: >
For adapter maintainers who wish to have theirs added to the list of [Trusted adapters](https://docs.getdbt.com/docs/trusted-adapters)
labels: ["adapter maintainers"]
body:
- type: markdown
attributes:
value: |
We're excited that you'd like to support your adapter formally as "Trusted"! This template will ensure that you are aware of the process and the guidelines. Additionally, that you can vouch that your adapter currently meets the standards of a Trusted adapter
- type: input
id: adapter-repo
attributes:
label: Link to adapter repo
description: Please link to the GitHub repo
validations:
required: true

- type: input
id: contact
attributes:
label: Contact Details
description: How can we get in touch with you?
placeholder: your preferred email and/or dbt Slack handle
validations:
required: true

- type: checkboxes
id: author_type
attributes:
label: Which of these best describes you?
options:
- label: I am a dbt Community member
- label: I work for the vendor on top of which the dbt adapter functions

- type: checkboxes
id: read-program-guide
attributes:
label: Please agree to the each of the following
options:
- label: I am a maintainer of the adapter being submited for Trusted status
- label: I have read both the [Trusted adapters](https://docs.getdbt.com/docs/trusted-adapters) and [Building a Trusted Adapter](https://docs.getdbt.com/guides/dbt-ecosystem/adapter-development/8-building-a-trusted-adapter) pages.
- label: I believe that the adapter currently meets the expectations given above
- label: I will ensure this adapter stays in compliance with the guidelines
- label: I understand that dbt Labs reserves the right to remove an adapter from the trusted adapter list at any time, should any of the below guidelines not be met
validations:
required: true




20 changes: 3 additions & 17 deletions website/docs/docs/connect-adapters.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,32 +5,18 @@ id: "connect-adapters"

Adapters are an essential component of dbt. At their most basic level, they are how dbt connects with the various supported data platforms. At a higher-level, adapters strive to give analytics engineers more transferrable skills as well as standardize how analytics projects are structured. Gone are the days where you have to learn a new language or flavor of SQL when you move to a new job that has a different data platform. That is the power of adapters in dbt — for more detail, read the [What are adapters](/guides/dbt-ecosystem/adapter-development/1-what-are-adapters) guide.

This section provides more details on different ways you can connect dbt to an adapter, and explains what a maintainer is.
This section provides more details on different ways you can connect dbt to an adapter, and explains what a maintainer is.

### Set up in dbt Cloud

Explore the fastest and most reliable way to deploy dbt using dbt Cloud, a hosted architecture that runs dbt Core across your organization. dbt Cloud lets you seamlessly [connect](/docs/cloud/about-cloud-setup) with a variety of [verified](/docs/supported-data-platforms) data platform providers directly in the dbt Cloud UI.

dbt Cloud supports data platforms that are verified and [maintained](#maintainers) by dbt Labs or partners. This level of support ensures that users can trust certain adapters for use in production.
Explore the fastest and most reliable way to deploy dbt using dbt Cloud, a hosted architecture that runs dbt Core across your organization. dbt Cloud lets you seamlessly [connect](/docs/cloud/about-cloud-setup) with a variety of [verified](/docs/supported-data-platforms) data platform providers directly in the dbt Cloud UI.

### Install using the CLI

Install dbt Core, which is an open-source tool, locally using the CLI. dbt communicates with a number of different data platforms by using a dedicated adapter plugin for each. When you install dbt Core, you'll also need to install the specific adapter for your database, [connect to dbt Core](/docs/core/about-core-setup), and set up a `profiles.yml` file.

Data platforms supported in dbt Core may be verified or unverified, and are [maintained](#maintainers) by dbt Labs, partners, or community members.
Install dbt Core, which is an open-source tool, locally using the CLI. dbt communicates with a number of different data platforms by using a dedicated adapter plugin for each. When you install dbt Core, you'll also need to install the specific adapter for your database, [connect to dbt Core](/docs/core/about-core-setup), and set up a `profiles.yml` file.

With a few exceptions [^1], you can install all [Verified adapters](/docs/supported-data-platforms) from PyPI using `pip install adapter-name`. For example to install Snowflake, use the command `pip install dbt-snowflake`. The installation will include `dbt-core` and any other required dependencies, which may include both other dependencies and even other adapter plugins. Read more about [installing dbt](/docs/core/installation).


## Maintainers

Who made and maintains an adapter is certainly relevant, but we recommend using an adapter's verification status to determine the quality and health of an adapter. So far there are three categories of maintainers:

| Supported by | Maintained By |
| ------------ | ---------------- |
| dbt Labs | dbt Labs maintains a set of adapter plugins for some of the most common databases, warehouses, and platforms. As for why particular data platforms were chosen, see ["Why Verify an Adapter"](/guides/dbt-ecosystem/adapter-development/7-verifying-a-new-adapter#why-verify-an-adapter) |
| Partner | These adapter plugins are built and maintained by the same people who build and maintain the complementary data technology. |
| Community | These adapter plugins are contributed and maintained by members of the community. 🌱 |
[^1]: Here are the two different adapters. Use the PyPI package name when installing with `pip`

| Adapter repo name | PyPI package name |
Expand Down
93 changes: 25 additions & 68 deletions website/docs/docs/supported-data-platforms.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,78 +6,35 @@ description: "Connect dbt to any data platform in dbt Cloud or dbt Core, using a
hide_table_of_contents: true
---

dbt connects to and runs SQL against your database, warehouse, lake, or query engine. These SQL-speaking platforms are collectively referred to as _data platforms_. dbt connects with data platforms by using a dedicated adapter plugin for each. Plugins are built as Python modules that dbt Core discovers if they are installed on your system. Read [What are Adapters](/guides/dbt-ecosystem/adapter-development/1-what-are-adapters) for more info.
dbt connects to and runs SQL against your database, warehouse, lake, or query engine. These SQL-speaking platforms are collectively referred to as _data platforms_. dbt connects with data platforms by using a dedicated adapter plugin for each. Plugins are built as Python modules that dbt Core discovers if they are installed on your system. Read [What are Adapters](/guides/dbt-ecosystem/adapter-development/1-what-are-adapters) for more info.

You can [connect](/docs/connect-adapters) to adapters and data platforms either directly in the dbt Cloud user interface (UI) or install them manually using the command line (CLI). There are two types of adapters available and to evaluate quality and maintenance, we recommend you consider their verification status. You can also [further configure](/reference/resource-configs/postgres-configs) your specific data platform to optimize performance.
You can [connect](/docs/connect-adapters) to adapters and data platforms either directly in the dbt Cloud user interface (UI) or install them manually using the command line (CLI).

- **Verified** — dbt Labs' strict [adapter program](/guides/dbt-ecosystem/adapter-development/7-verifying-a-new-adapter) assures users of trustworthy, tested, and regularly updated adapters for production use. Verified adapters earn a "Verified" status, providing users with trust and confidence.
- **Community** — [Community adapters](/docs/community-adapters) are open-source and maintained by community members.
You can also further customize how dbt works with your specific data platform via configuration: see [Configuring Postgres](/reference/resource-configs/postgres-configs) for an example.

## Types of Adapters

There are three types of adapters available today:

- **Verified** — [Verified adapters](verified-adapters) are those that have completed a rigorous verification process in collaboration with dbt Labs.
- **Trusted** — [Trusted adapters](trusted-adapters) are those where the adapter maintainers have agreed to meet a higher standard of quality.
- **Community** — [Community adapters](community-adapters) are open-source and maintained by community members.

### Verified adapters

The following are **Verified adapters** ✓ you can connect to either in dbt Cloud or dbt Core:

<div className="grid--4-col">

<Card
title="AlloyDB"
body="<a href='/docs/cloud/connect-data-platform/connect-redshift-postgresql-alloydb'><img src='/img/icons/dbt-bit.svg' width='7%'/>Set up in dbt Cloud </a> <br /><a href='/docs/core/connect-data-platform/alloydb-setup'><img src='/img/icons/dbt-bit.svg' width='7%'/>Install using the CLI </a> <br /><br /><a href=https://badge.fury.io/py/dbt-postgres><img src=https://badge.fury.io/py/dbt-postgres.svg/></a>"
icon="alloydb"/>
<Card
title="BigQuery"
body="<a href='/docs/cloud/connect-data-platform/connect-bigquery'><img src='/img/icons/dbt-bit.svg' width='7%'/>Set up in dbt Cloud </a> <br /><a href='/docs/core/connect-data-platform/bigquery-setup'><img src='/img/icons/dbt-bit.svg' width='7%'/>Install using the CLI </a> <br /><br /> <a href=https://badge.fury.io/py/dbt-bigquery><img src=https://badge.fury.io/py/dbt-bigquery.svg/></a>"
icon="bigquery"/>

<Card
title="Databricks"
body="<a href='/docs/cloud/connect-data-platform/connect-databricks'><img src='/img/icons/dbt-bit.svg' width='7%'/>Set up in dbt Cloud </a> <br /><a href='/docs/core/connect-data-platform/databricks-setup'><img src='/img/icons/dbt-bit.svg' width='7%'/>Install using the CLI </a> <br /><br /> <a href=https://badge.fury.io/py/dbt-databricks><img src=https://badge.fury.io/py/dbt-databricks.svg/></a>"
icon="databricks"/>

<Card
title="Dremio*"
body="<a href='/docs/core/connect-data-platform/dremio-setup'><img src='/img/icons/dbt-bit.svg' width='7%'/>Install using the CLI </a> <br /><br /> <a href=https://badge.fury.io/py/dbt-dremio><img src=https://badge.fury.io/py/dbt-dremio.svg/><br />"
icon="dremio"/>

<Card
title="Postgres"
body="<a href='/docs/cloud/connect-data-platform/connect-redshift-postgresql-alloydb'><img src='/img/icons/dbt-bit.svg' width='7%'/>Set up in dbt Cloud </a> <br /><a href='/docs/core/connect-data-platform/postgres-setup'><img src='/img/icons/dbt-bit.svg' width='7%'/>Install using the CLI </a> <br /><br /> <a href=https://badge.fury.io/py/dbt-postgres><img src=https://badge.fury.io/py/dbt-postgres.svg/></a>"
icon="redshift"/>

<Card
title="Redshift"
body="<a href='/docs/cloud/connect-data-platform/connect-redshift-postgresql-alloydb'><img src='/img/icons/dbt-bit.svg' width='7%'/>Set up in dbt Cloud </a> <br /><a href='/docs/core/connect-data-platform/redshift-setup'><img src='/img/icons/dbt-bit.svg' width='7%'/>Install using the CLI </a> <br /><br /> <a href=https://badge.fury.io/py/dbt-redshift><img src=https://badge.fury.io/py/dbt-redshift.svg/></a>"
icon="redshift"/>

<Card
title="Snowflake"
body="<a href='/docs/cloud/connect-data-platform/connect-snowflake'><img src='/img/icons/dbt-bit.svg' width='7%'/>Set up in dbt Cloud </a> <br /><a href='/docs/core/connect-data-platform/snowflake-setup'><img src='/img/icons/dbt-bit.svg' width='7%'/>Install using the CLI </a> <br /><br /> <a href=https://badge.fury.io/py/dbt-snowflake><img src=https://badge.fury.io/py/dbt-snowflake.svg/></a>"
icon="snowflake"/>

<Card
title="Spark"
body="<a href='/docs/cloud/connect-data-platform/connect-apache-spark'><img src='/img/icons/dbt-bit.svg' width='7%'/>Set up in dbt Cloud </a> <br /><a href='/docs/core/connect-data-platform/spark-setup'><img src='/img/icons/dbt-bit.svg' width='7%'/>Install using the CLI </a> <br /><br /> <a href=https://badge.fury.io/py/dbt-spark><img src=https://badge.fury.io/py/dbt-spark.svg/></a>"
icon="databricks"/>

<Card
title="Starburst/Trino"
body="<a href='/docs/cloud/connect-data-platform/connect-starburst-trino'><img src='/img/icons/dbt-bit.svg' width='7%'/>Set up in dbt Cloud </a> <br /><a href='/docs/core/connect-data-platform/trino-setup'><img src='/img/icons/dbt-bit.svg' width='7%'/>Install using the CLI </a> <br /> <br /> <a href=https://badge.fury.io/py/dbt-trino><img src=https://badge.fury.io/py/dbt-trino.svg/>"
icon="starburst"/>

<Card
title="Fabric Synapse*"
body="<a href='/docs/core/connect-data-platform/fabric-setup'><img src='/img/icons/dbt-bit.svg' width='7%'/>Install using the CLI </a> <br /><br /> <a href=https://badge.fury.io/py/dbt-fabric><img src=https://badge.fury.io/py/dbt-fabric.svg/><br /> 🚧 Verification in progress"
icon="rocket"/>
<Card
title="Azure Synapse*"
body="<a href='/docs/core/connect-data-platform/azuresynapse-setup'><img src='/img/icons/dbt-bit.svg' width='7%'/>Install using the CLI </a> <br /><br /> <a href=https://badge.fury.io/py/dbt-synapse><img src=https://badge.fury.io/py/dbt-synapse.svg/><br /> 🚧 Verification in progress"
icon="rocket"/>

<Card
title="Teradata*"
body="<a href='/docs/core/connect-data-platform/teradata-setup'><img src='/img/icons/dbt-bit.svg' width='7%'/>Install using the CLI </a> <br /><br /> <a href=https://badge.fury.io/py/dbt-teradata><img src=https://badge.fury.io/py/dbt-teradata.svg/><br />"
icon="teradata"/>

</div>

<br />
* Install these adapters using the CLI as they're not currently supported in dbt Cloud. <br />
import AdaptersVerified from '/snippets/_adapters-verified.md';

<AdaptersVerified />

### Trusted adapters

The following are **Trusted adapters** ✓ you can connect to in dbt Core:

import AdaptersTrusted from '/snippets/_adapters-trusted.md';

<AdaptersTrusted />

<br/> * Install these adapters using the CLI as they're not currently supported in dbt Cloud. <br />

40 changes: 40 additions & 0 deletions website/docs/docs/trusted-adapters.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
---
title: "Trusted adapters"
id: "trusted-adapters"
---

Trusted adapters are adapters not maintained by dbt Labs, that we feel comfortable recommending to users for use in production.

Free and open-source tools for the data professional are increasingly abundant. This is by-and-large a *good thing*, however it requires due dilligence that wasn't required in a paid-license, closed-source software world. As a user, there are questions to answer important before taking a dependency on an open-source project. The trusted adapter designation is meant to streamline this process for end users.

<details><summary>Considerations for depending on an open-source project</summary>

1. Does it work?
2. Does anyone "own" the code, or is anyone liable for ensuring it works?
3. Do bugs get fixed quickly?
4. Does it stay up-to-date with new Core features?
5. Is the usage substantial enough to self-sustain?
pendency on this library?

</details>

### Trusted adapter specifications

See [Building a Trusted Adapter](/guides/dbt-ecosystem/adapter-development/8-building-a-trusted-adapter) for more information, particularly if you are an adapter maintainer considering having your adapter be added to the trusted list.

### Trusted vs Verified

The Verification program exists to highlight adapters that meets both of the following criteria:

- the guidelines given in the Trusted program,
- formal agreements required for integration with dbt Cloud

For more information on the Verified Adapter program, reach out the [dbt Labs partnerships team](mailto:[email protected])

### Trusted adapters

The following are **Trusted adapters** ✓ you can connect to in dbt Core:

import AdaptersTrusted from '/snippets/_adapters-trusted.md';

<AdaptersTrusted />
23 changes: 6 additions & 17 deletions website/docs/docs/verified-adapters.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,27 +4,16 @@ id: "verified-adapters"
---


The dbt Labs has a rigorous verified adapter program which provides reassurance to users about which adapters can be trusted to use in production, has been tested, and is actively maintained and updated. The process covers aspects of development, documentation, user experience, and maintenance.
The dbt Labs has a rigorous verified adapter program which provides reassurance to users about which adapters can be trusted to use in production, has been tested, and is actively maintained and updated. The process covers aspects of development, documentation, user experience, and maintenance.

These adapters then earn a "Verified" status so that users can have a certain level of trust and expectation when they use them. The adapters also have maintainers and we recommend using the adapter's verification status to determine its quality and health.

Here's the list of the verified data platforms that can connect to dbt and its latest version.

| dbt Cloud setup | CLI installation | latest verified version |
| ---------------- | ----------------------------------------- | ------------------------ |
| [Setup AlloyDB](/docs/cloud/connect-data-platform/connect-redshift-postgresql-alloydb) | [Install AlloyDB](/docs/core/connect-data-platform/alloydb-setup) | (same as `dbt-postgres`) |
| Not supported | [Install Azure Synapse](/docs/core/connect-data-platform/azuresynapse-setup) | 1.3 :construction: |
| [Set up BigQuery](/docs/cloud/connect-data-platform/connect-bigquery) | [Install BigQuery](/docs/core/connect-data-platform/bigquery-setup) | 1.4 |
| [Set up Databricks ](/docs/cloud/connect-data-platform/connect-databricks)| [ Install Databricks](/docs/core/connect-data-platform/databricks-setup) | 1.4 |
| Not supported | [Install Dremio](/docs/core/connect-data-platform/dremio-setup) | 1.4 :construction: |
| [Set up Postgres](/docs/cloud/connect-data-platform/connect-redshift-postgresql-alloydb) | [Install Postgres](/docs/core/connect-data-platform/postgres-setup) | 1.4 |
| [Set up Redshift](/docs/cloud/connect-data-platform/connect-redshift-postgresql-alloydb) | [Install Redshift](/docs/core/connect-data-platform/redshift-setup) | 1.4 |
| [Set up Snowflake](/docs/cloud/connect-data-platform/connect-snowflake) | [ Install Snowflake](/docs/core/connect-data-platform/snowflake-setup) | 1.4 |
| [Set up Spark](/docs/cloud/connect-data-platform/connect-apache-spark) | [Install Spark](/docs/core/connect-data-platform/spark-setup) | 1.4 |
| [Set up Starburst & Trino](/docs/cloud/connect-data-platform/connect-starburst-trino)| [Installl Starburst & Trino](/docs/core/connect-data-platform/trino-setup) | 1.4 |

:construction:: Verification in progress
The verification process serves as the on-ramp to integration with dbt Cloud. As such, we restrict applicants to data platform vendors with whom we are already engaged.

To learn more, see [Verifying a new adapter](/guides/dbt-ecosystem/adapter-development/7-verifying-a-new-adapter).

Here's the list of the verified data platforms that can connect to dbt and its latest version.

import AdaptersVerified from '/snippets/_adapters-verified.md';

<AdaptersVerified />
Loading

0 comments on commit bfc18a4

Please sign in to comment.