Skip to content

Commit

Permalink
Merge branch 'master' into ele-966-add-exposure-data
Browse files Browse the repository at this point in the history
  • Loading branch information
NoyaArie committed Jun 26, 2023
2 parents 094310c + 84e2fbf commit edf0bca
Show file tree
Hide file tree
Showing 64 changed files with 375,825 additions and 374,948 deletions.
22 changes: 12 additions & 10 deletions docs/_snippets/faq/question-tests-configuration-priorities.mdx
Original file line number Diff line number Diff line change
@@ -1,20 +1,22 @@
<Accordion title="How test configurations are prioritized?">

The configuration of Elementary is dbt native and follows the same priorities of `dbt configuration`.
The more granular and specific configuration overrides the less granular one.
The configuration of Elementary is dbt native and follows the same priorities and inheritance.
The more granular and specific configuration overrides the less granular one.

Elementary searches and prioritizes configuration in the following order:

For models:
**For models tests:**
1. Test arguments.
2. Model configuration.
3. Global vars in `dbt_project.yml`.
2. Tests path configuration under `tests` key in `dbt_project.yml`.
3. Model configuration.
4. Path configuration under `models` key in `dbt_project.yml`.
5. Global vars in `dbt_project.yml`.

For sources:
**For sources tests:**
1. Test arguments.
2. Table configuration.
3. Source configuration.
4. Global vars in `dbt_project.yml`.

2. Tests path configuration under `tests` key in `dbt_project.yml`.
3. Table configuration.
4. Source configuration.
5. Global vars in `dbt_project.yml`.

</Accordion>
1 change: 1 addition & 0 deletions docs/_snippets/profiles/redshift-profile.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ elementary:
user: [username]
password: [password]

port: 5439
dbname: [database name]
schema: [schema name]_elementary
threads: 4
Expand Down
6 changes: 3 additions & 3 deletions docs/_snippets/quickstart-package-install.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ Make sure to copy the packages that are relevant to your dbt version.
```yml packages.yml
packages:
- package: elementary-data/elementary
version: 0.8.0
version: 0.8.1
## Docs: https://docs.elementary-data.com
```

Expand All @@ -55,7 +55,7 @@ packages:
```yml packages.yml
packages:
- package: elementary-data/elementary
version: 0.8.0
version: 0.8.1
## Docs: https://docs.elementary-data.com

## !! Important !! For dbt >=1.2.0 \<1.3.0 ##
Expand All @@ -69,7 +69,7 @@ packages:
```yml packages.yml
packages:
- package: elementary-data/elementary
version: 0.8.0
version: 0.8.1
## Docs: https://docs.elementary-data.com

## !! Important !! For dbt <1.2.0 ##
Expand Down
6 changes: 5 additions & 1 deletion docs/cloud/general/security-and-privacy.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -45,9 +45,13 @@ To avoid this sampling, set the var `test_sample_rows_count: 0` in your `dbt_pro

## Compliance

<Check>
**SOC 2 certification:** Elementary is currently in the process of obtaining SOC2 and ISO27001 compliance.
</Check>

[Contact us](mailto:[email protected]) for auditing reports and penetration testing results.

## Have more questions?

We would be happy to answer!
Reach out to us on [email](mailto:[email protected]) or [Slack](https://join.slack.com/t/elementary-community/shared_invite/zt-1b9vogqmq-y~IRhc2396CbHNBXLsrXcA).
Reach out to us on [email](mailto:[email protected]) or [Slack](https://join.slack.com/t/elementary-community/shared_invite/zt-1b9vogqmq-y~IRhc2396CbHNBXLsrXcA).
17 changes: 3 additions & 14 deletions docs/cloud/introduction.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,7 @@ title: "Introduction"

Elementary Cloud is the easiest and fastest way to get the most out of Elementary.

<Card title="Signup for details" href="https://www.elementary-data.com/cloud-beta">
_The service is currently in private beta_
<Card title="Signup to Elementary Cloud" href="https://t2taztilhde.typeform.com/to/ObfMbxB5?utm_source=docs&utm_medium=cloud_docs&utm_content=v1">
</Card>

##
Expand Down Expand Up @@ -39,15 +38,5 @@ alt="Elementary Managed high level flow"
1. [Install the Elementary dbt package in your project](/cloud/onboarding/quickstart-dbt-package).
2. [Signup and setup integrations](/cloud/onboarding/signup).

<Card title="Signup for details" href="https://www.elementary-data.com/cloud-beta">
_The service is currently in private beta_
</Card>

## Security and privacy

Elementary cloud requires access only to the Elementary schema and the tables in it.
The data in the schema in full is stored in the client's data warehouse.

We secure Elementary cloud infrastructure with the highest standards.
You can delete your account at any time, and all your configuration and reports will be deleted immediately and permanently from Elementary servers.
For details, refer to our [Terms of Service](https://www.elementary-data.com/terms-of-service).
<Card title="Signup to Elementary Cloud" href="https://t2taztilhde.typeform.com/to/ObfMbxB5?utm_source=docs&utm_medium=cloud_docs&utm_content=v1">
</Card>
13 changes: 3 additions & 10 deletions docs/cloud/manage-team.mdx
Original file line number Diff line number Diff line change
@@ -1,23 +1,16 @@
---
title: "Quickstart: Invite and remove users"
sidebarTitle: "Team settings"
title: "Invite and remove users"
sidebarTitle: "5. Invite users"
---

### Invite users

After you signup, you could invite team members to join you! 🎉

On the top left buttun select `Account settings`, and you can invite users on the `Team` screen.
On the top left button select `Account settings`, and you can invite users on the `Team` screen.

Users you invite will recieve an Email saying you invited them, and will need to accept and activate their account.

<Frame>
<img src="/pics/cloud/invite_users.png" />
</Frame>


### Remove users

On the top left buttun select `Account settings`, and select the `Team` screen.

You can remove users by clicking selecting this option under the user options.
43 changes: 12 additions & 31 deletions docs/cloud/onboarding/connect-data-warehouse.mdx
Original file line number Diff line number Diff line change
@@ -1,46 +1,27 @@
---
title: "Connect your data warehouse"
sidebarTitle: "Data warehouse"
sidebarTitle: "4. Connect data warehouse"
---

You can connect Elementary to a data warehouse that has an Elementary schema (created by the [Elementary dbt package](/cloud/onboarding/quickstart-dbt-package)).

Here are the steps needed to enable the connection:
Elementary Cloud needs:
- [`profiles.yml`](/cloud/onboarding/create-profile) with connection details
- Read permissions to the Elementary schema (and not the rest of your data)
- Network access (might require to allowlist Elementary IP address)

### Authentication and IP Allowlist

Elementary needs authentication details, permissions to read the Elementary schema (and not the rest of your data), and network access enabled by adding the cloud IPs to your data warehouse allowlist.

Here are the guides on how to configure these on each supported data warehouse:

- Bigquery
- Snowflake
- Redshift
- Databricks
- Postgres

Elementary IP for allowlist: `3.126.156.226`

### Create a `profiles.yml` file

You will need to provide the connection and authentication details by uploading a YML file with a connection profile named `elementary`.
The profile needs to point at the database and schema name where your elementary tables are.

The easiest way to generate the profile is to run the following command within the dbt project where you deployed the elementary dbt package (works in dbt cloud as well):
### Connect Elementary cloud

```shell
dbt run-operation elementary.generate_elementary_cli_profile
```
On the `Account settings` under `Integrations`, press `Connect` on the "Connect Your data warehouse" section.

Save the output to a YML file, update the missing details, and you are ready.
Provide an environment name, select a data warehouse type, and upload the `profiles.yml` file with the `elementary` profile.

Here are the formats of profile for each supported data warehouse:

<Snippet file="profiles/all-profiles.mdx" />
### Allowlist Elementary IP

Elementary IP for allowlist: `3.126.156.226`

### Connect Elementary cloud

On the `Account settings` under `Integrations`, press `Connect` on the "Connect Your data warehouse" section.
### Need help with onboarding?

Provide an env name, select a data warehouse type, and upload the `profiles.yml` file with the `elementary` profile.
We can provide [support on Slack](https://join.slack.com/t/elementary-community/shared_invite/zt-1b9vogqmq-y~IRhc2396CbHNBXLsrXcA) or hop on an [onboarding call](https://savvycal.com/MaayanSa/df29881c).
133 changes: 133 additions & 0 deletions docs/cloud/onboarding/create-profile.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,133 @@
---
title: "Create `profiles.yml` file"
sidebarTitle: "2. Create profiles.yml"
---

You will need to provide Elementary cloud a `profiles.yml` file with a connection profile named `elementary`.

- The profile needs to point at the database and schema name where your elementary tables are.
- The provided credentials need to have read permissions to the elementary schema.

The easiest way to generate the profile is:
1. Run the following command in the dbt project where elementary dbt package is deployed (works in dbt cloud as well):

```shell
dbt run-operation elementary.generate_elementary_cli_profile
```

2. Copy and save the output to a `profiles.yml` file, update the missing details, and you are ready.

### Permissions and security

**Elementary cloud doesn't need permissions to your sensitive data.**

It is recommended to create a read only user for the elementary schema only, and provide it to Elementary Cloud in the profile.
For more details, refer to [security and privacy](/cloud/security-and-privacy).

### `profiles.yml` examples

Here is the format of `profiles.yml` for each supported data warehouse:

<CodeGroup>

```yml Snowflake
## SNOWFLAKE ##
## Configure the database and schema of elementary models.

elementary:
outputs:
default:
type: snowflake
account: [account id]

## User/password auth ##
user: [username]
password: [password]

port: 5439
role: [user role]
database: [database name]
warehouse: [warehouse name]
schema: [schema name]_elementary
threads: 4

```

```yml BigQuery
## BIGQUERY ##
## Configure the database and schema of elementary models.

elementary:
outputs:
default:
type: bigquery

## Service account auth ##
method: service-account
keyfile: empty

project: [project id]
dataset: [dataset name] # elementary dataset, usually [dataset name]_elementary
threads: 4
location: [dataset location]
priority: interactive
```
```yml Redshift
## REDSHIFT ##
## Configure the database and schema of elementary models.

elementary:
outputs:
default:
type: redshift
host: [hostname, like hostname.region.redshift.amazonaws.com]

## User/password auth ##
user: [username]
password: [password]

dbname: [database name]
schema: [schema name] # elementary schema, usually [schema name]_elementary
threads: 4
```
```yml Databricks
## DATABRICKS ##
## Configure the database and schema of elementary models.

elementary:
outputs:
default:
type: databricks
host: [hostname, like <ID>.cloud.databricks.com]
http_path: [like /sql/1.0/endpoints/<ID>]
schema: [schema name] # elementary schema, usually [schema name]_elementary
token: [token]
threads: [number of threads like 8]
```
```yml Postgres
## POSTGRES ##
## Configure the database and schema of elementary models.

elementary:
outputs:
default:
type: postgres
host: [hostname]
user: [username]
password: [password]
port: [port]
dbname: [database name]
schema: [schema name] # elementary schema, usually [schema name]_elementary
threads: [1 or more]

```

</CodeGroup>

### What's next?

1. [Singup to Elementary cloud](/cloud/sonboarding/signup).
2. [Connect your Elementary schema to Elementary cloud](/cloud/onboarding/connect-data-warehouse).
9 changes: 5 additions & 4 deletions docs/cloud/onboarding/quickstart-dbt-package.mdx
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
title: "Quickstart: Install Elementary dbt package"
sidebarTitle: "Install dbt package"
title: "Install Elementary dbt package"
sidebarTitle: "1. Install dbt package"
---

<Snippet file="quickstart-package-install.mdx" />
Expand Down Expand Up @@ -52,5 +52,6 @@ If you see data in these models you completed the package deployment (Congrats!

### What's next?

1. [Singup to Elementary cloud](/cloud/saas-onboarding/signup).
2. [Connect your Elementary schema to Elementary cloud](/cloud/saas-onboarding/connect-data-warehouse).
1. [Create a connection profile](/cloud/onboarding/create-profile).
2. [Singup to Elementary cloud](/cloud/sonboarding/signup).
3. [Connect your Elementary schema to Elementary cloud](/cloud/onboarding/connect-data-warehouse).
6 changes: 3 additions & 3 deletions docs/cloud/onboarding/signup.mdx
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
title: "Quickstart: Signup and connect"
sidebarTitle: "Signup and connect"
title: "Signup and login"
sidebarTitle: "3. Signup and login"
---

### Signup to Elementary cloud
Expand Down Expand Up @@ -28,4 +28,4 @@ After you connect a data warehouse with an Elementary schema in it, you can star

### What's next?

[Connect your Elementary schema to Elementary cloud](/cloud/saas-onboarding/connect-data-warehouse).
[Connect your Elementary schema to Elementary cloud](/cloud/onboarding/connect-data-warehouse).
Loading

0 comments on commit edf0bca

Please sign in to comment.