-
Notifications
You must be signed in to change notification settings - Fork 160
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
* Update data-observability-dashboard.mdx * automated monitors * fix * Update multi-env.mdx * Update elementary-alerts.mdx * Update key-features.mdx * Update config-as-code.mdx * Update elementary-alerts.mdx * Update looker.mdx Added the `develop` permission. * remove dependency on git access for exposures from docs * Tableau server * security * exclude elementary tests * Improved webhook jobs description * change CTAs * Docs update - structure and comparison table (#1456) * WIP * added cloud vs oss * fixes * added integrations * Update troubleshoot.mdx * Update teams.mdx (#1457) * Fix typo * Integrations pages change (#1469) * WIP, BI * WIP * WIP dwh integrations * WIP bi integrations * WIP integrations * WIP alerts * WIP alerts * orchestration * Update exposures-lineage.mdx * Eplicit explenation about the output of generate_schema_baseline_test * added output example * fix alert links * fix links (#1477) * video lineage * minor docs fixes * fix orchestrators links * change offering names * change offering names * updates * add empty ui troubleshooting section * quick update for model override * Update volume-anomalies.mdx (#1526) Use regular dash instead of EM dash so it will work when copying * fixes to dimensions documentation * fixes to dimensions documentation * Snowflake summit * github instructions * Fixed incorrect test name 'all_column_anomalies' in docs. * Cloud docs revamp (#1567) * fixes to dimensions documentation * business case and how does elementary work * fixes to dimensions documentation * fixes to dimensions documentation * how it works snippet * how it works snippet * Lineage * Lineage changes * anomaly detection * data tests * data tests * data tests * performance monitoring * navigation changes * links * typos * broken links --------- Co-authored-by: ronisaban <[email protected]> * Minor typo in docs: how-anomaly-detection-works.mdx (#1564) Used "it's" instead of "its". * New alert integrations (#1569) * alert integrations pages * alert integrations pages * Slack private message * integrations * integrations * integrations * Incident management docs (#1573) * incidents * incidents * Docs-fixes-June-2 (#1574) * Typos and fixes on new docs * Typos on fixes --------- Co-authored-by: Maayan Salom <[email protected]> * Grammar mistake in Update introduction.mdx (#1570) One instance of "it's" should be "its" Co-authored-by: Maayan Salom <[email protected]> --------- Co-authored-by: Alex Alves <[email protected]> Co-authored-by: IDoneShaveIt <[email protected]> Co-authored-by: ellakz <[email protected]> Co-authored-by: Elon Gliksberg <[email protected]> Co-authored-by: Ofek Weiss <[email protected]> Co-authored-by: Ofek Weiss <[email protected]> Co-authored-by: quickcoffee <[email protected]> Co-authored-by: Noy Arie <[email protected]> Co-authored-by: Itamar Hartstein <[email protected]> Co-authored-by: noakurman <[email protected]> Co-authored-by: ronisaban <[email protected]> Co-authored-by: Georgios Tzanakis <[email protected]> Co-authored-by: ronisaban2024 <[email protected]>
- Loading branch information
14 people
committed
Jul 2, 2024
1 parent
81ab16e
commit d5a7b1c
Showing
75 changed files
with
1,188 additions
and
261 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,81 @@ | ||
|
|
||
| ### Detection & Coverage | ||
|
|
||
| Elementary integrates powerful anomaly detection and dbt tests into a unified detection strategy. | ||
| Effective detection of data issues requires a comprehensive approach, | ||
| including both pipeline and data monitoring, validation tests, | ||
| anomaly detection for unexpected behavior, and a single interface to manage it all at scale. | ||
|
|
||
| <CardGroup cols={2}> | ||
| <Card title="Anomaly Detection" href="/features/anomaly-detection/monitors-overview"> | ||
| ML-powered monitors automatically detect data quality issues. | ||
| Out-of-the-box for volume and freshness, and opt-in for data quality metrics. | ||
| </Card> | ||
| <Card title="Data Tests" href="/features/anomaly-detection/data-tests-overview"> | ||
| Validate data and track the results of dbt tests, dbt packages tests (dbt-utils, dbt-expectations, elementary) and custom SQL tests. | ||
| </Card> | ||
| <Card title="Schema Validation" href="/features/data-tests/schema-validation-test"> | ||
| Validate there are no breaking changes in tables schema, JSON schema, and downstream exposures such as dashboards. | ||
| </Card> | ||
| <Card title="Runs & Performance" href="/features/performance-monitoring/performance-monitoring"> | ||
| Track failures and runs of jobs, models, and tests overtime. | ||
| Pipeline failures and performance issues can cause data incidents, and create unneceserry costs. | ||
| </Card> | ||
| <Card title="Code & UI Configuration"> | ||
| Configure Elementary in code, or via the UI for non-technical users or for adding tests in bulk. | ||
| The platform opens PRs to your repo, saving hours of tedious YAML edits. | ||
| </Card> | ||
| <Card title="Test Recommendations"> | ||
| Coming soon! | ||
| </Card> | ||
| </CardGroup> | ||
|
|
||
| ### Triage & Response | ||
|
|
||
| Detecting issues is just the first step to achieve data reliability. | ||
| Elementary offers tools to create an effective response plan, for faster recovery. | ||
| This includes investigating the root cause and impact of issues, communicating issues to the relevant people, assigning owners to fix issues, keeping track of open incidents and more. | ||
|
|
||
| <CardGroup cols={2}> | ||
| <Card title="Data Lineage" href="/features/data-lineage/lineage"> | ||
| Column-level lineage that spans through sources, models and BI tools, enriched with monitoring results. Enables granular root cause and impact analysis. | ||
| </Card> | ||
| <Card title="Ownership" href="/features/data-governance/define-ownership"> | ||
| Define clear ownership of data assets and enable owners to be informed and accountable for the health and status of their data. | ||
| </Card> | ||
| <Card title="Actionable Alerts" href="/features/alerts-and-incidents/alerts-and-incidents-overview"> | ||
| Distribute highly configurable alerts to different channels and integrations. | ||
| Automatically tag owners, and enable setting status and assigns at the alert level. | ||
| </Card> | ||
| <Card title="Automated Grouping" href="/features/alerts-and-incidents/incidents"> | ||
| Different failures related to the same issue are grouped automatically to a single incident. | ||
| This accelerates triage and response, and reduces alerts fautigue. | ||
| </Card> | ||
| <Card title="Incident Management" href="/features/alerts-and-incidents/incident-management"> | ||
| Manage all open incidents in a single interface, with a clear view of status and assignees. | ||
| Track historical incidents and high-level incidents metrics. | ||
| </Card> | ||
| </CardGroup> | ||
|
|
||
| ### Collaboration & Communication | ||
|
|
||
| The data team doesn’t live in a silo - you have many stakeholders. | ||
| The only way to improve data trust is by bringing in more team members, users and stakeholders to the data health process. | ||
| Elementary fosters collaboration by allowing you to easily share and communicate the status of issues, | ||
| the overall health of the data platform and progress made to improve it with the broader organization. | ||
|
|
||
| <CardGroup cols={2}> | ||
| <Card title="Data Health Dashboard" href="/features/collaboration-and-communication/data-observability-dashboard"> | ||
| Up to date dashboard with current status and trends of data issues. | ||
| Share the dashboard with others, enable them to slice results and stay informed. | ||
| </Card> | ||
| <Card title="Assets Tags" href="/features/data-governance/leverage-tags"> | ||
| Enable effective collaboration and communication by grouping related data assets and tests by business domains, data products, priority, etc. | ||
| </Card> | ||
| <Card title="Data Catalog" href="/features/collaboration-and-communication/catalog"> | ||
| Search and explore your datasets information - descriptions, columns, column descriptions, compiled code, datasets health and more. | ||
| </Card> | ||
| <Card title="Data Health Score"> | ||
| Coming soon! | ||
| </Card> | ||
| </CardGroup> |
7 changes: 7 additions & 0 deletions
7
docs/_snippets/cloud/features/alerts-and-incidents/alert-types.mdx
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,7 @@ | ||
| Elementary can be configured to send alerts on: | ||
|
|
||
| - Model run failures | ||
| - Failures and/or warnings of dbt tests (including Elementary dbt package and other packages) | ||
| - Failures and/or warnings Elementary Anomaly Detection monitors | ||
| - Failures and/or warning of custom SQL tests | ||
| - dbt source freshness failures |
10 changes: 10 additions & 0 deletions
10
docs/_snippets/cloud/features/anomaly-detection/automated-monitors-cards.mdx
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,10 @@ | ||
| <CardGroup cols={2}> | ||
| <Card title="Automated Freshness" href="/features/anomaly-detection/automated-freshness"> | ||
| Monitors updates to tables and how frequently a table is updated, | ||
| and fails if there is an unexpected delay. | ||
| </Card> | ||
| <Card title="Automated Volume" href="/features/anomaly-detection/automated-volume"> | ||
| Monitors how many rows were added or removed to a table on each update, | ||
| and fails if there is an unexpected drop or spike in rows. | ||
| </Card> | ||
| </CardGroup> |
5 changes: 5 additions & 0 deletions
5
docs/_snippets/cloud/features/anomaly-detection/automated-monitors-intro.mdx
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,5 @@ | ||
| Out-of-the-box ML-powered monitoring for freshness and volume issues on all production tables. | ||
| The automated monitors feature provides broad coverage and detection of critical pipeline issues, without any configuration effort. | ||
|
|
||
| These monitors track updates to tables, and will detect data delays, incomplete updates, and significant volume changes. | ||
| Additionally, there will be no increase in compute costs as the monitors leverage only warehouse metadata (e.g. information schema, query history). |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,12 @@ | ||
| 1. You install the Elementary dbt package in your dbt project and configure it to write to it's own schema, the Elementary schema. | ||
| 2. The package writes test results, run results, logs and metadata to the Elementary schema. | ||
| 3. The cloud service only requires `read access` to the Elementary schema, not to schemas where your sensitive data is stored. | ||
| 4. The cloud service connects to sync the Elementary schema using an **encrypted connection** and a **static IP address** that you will need to add to your allowlist. | ||
|
|
||
| <Frame> | ||
| <img | ||
| src="https://res.cloudinary.com/diuctyblm/image/upload/v1719157574/Cloud_platform_architecture_dogyrm.png" | ||
| alt="Elementary cloud security" | ||
| width="700" | ||
| /> | ||
| </Frame> |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.