Skip to content

Commit

Permalink
Merge branch 'current' into update_docs_on_overrides_ut
Browse files Browse the repository at this point in the history
  • Loading branch information
matthewshaver authored Feb 23, 2024
2 parents a17d878 + 9fcc45b commit 35dc061
Show file tree
Hide file tree
Showing 7 changed files with 70 additions and 6 deletions.
3 changes: 3 additions & 0 deletions website/docs/docs/build/custom-aliases.md
Original file line number Diff line number Diff line change
Expand Up @@ -127,6 +127,9 @@ The default implementation of `generate_alias_name` simply uses the supplied `al

</VersionBlock>

import WhitespaceControl from '/snippets/_whitespace-control.md';

<WhitespaceControl/>

### Dispatch macro - SQL alias management for databases and dbt packages

Expand Down
4 changes: 4 additions & 0 deletions website/docs/docs/build/custom-databases.md
Original file line number Diff line number Diff line change
Expand Up @@ -85,6 +85,10 @@ The default implementation of `generate_database_name` simply uses the supplied

</File>

import WhitespaceControl from '/snippets/_whitespace-control.md';

<WhitespaceControl/>

<VersionBlock firstVersion="1.6">

### Managing different behaviors across packages
Expand Down
11 changes: 8 additions & 3 deletions website/docs/docs/build/custom-schemas.md
Original file line number Diff line number Diff line change
Expand Up @@ -83,6 +83,12 @@ The following code represents the default macro's logic:
{%- endmacro %}
```
<br />

import WhitespaceControl from '/snippets/_whitespace-control.md';

<WhitespaceControl/>


## Changing the way dbt generates a schema name

Expand All @@ -92,8 +98,7 @@ To customize this macro, copy the example code in the section [How does dbt gene

Be careful. dbt will ignore any custom `generate_schema_name` macros included in installed packages.

<details>
<summary>❗️ Warning: Don't replace <code>default_schema</code> in the macro.</summary>
<expandable alt_header="Warning: Don't replace `default_schema` in the macro">

If you're modifying how dbt generates schema names, don't just replace ```{{ default_schema }}_{{ custom_schema_name | trim }}``` with ```{{ custom_schema_name | trim }}``` in the ```generate_schema_name``` macro.

Expand All @@ -119,7 +124,7 @@ If you remove ```{{ default_schema }}```, it causes developers to override each
```

</details>
</expandable>

### generate_schema_name arguments

Expand Down
3 changes: 3 additions & 0 deletions website/docs/docs/build/jinja-macros.md
Original file line number Diff line number Diff line change
Expand Up @@ -124,6 +124,9 @@ from app_data.payments

</File>

import WhitespaceControl from '/snippets/_whitespace-control.md';

<WhitespaceControl/>

### Using a macro from a package
A number of useful macros have also been grouped together into [packages](/docs/build/packages) — our most popular package is [dbt-utils](https://hub.getdbt.com/dbt-labs/dbt_utils/latest/).
Expand Down
45 changes: 43 additions & 2 deletions website/docs/docs/build/python-models.md
Original file line number Diff line number Diff line change
Expand Up @@ -153,6 +153,26 @@ with upstream_python_model as (
Referencing [ephemeral](/docs/build/materializations#ephemeral) models is currently not supported (see [feature request](https://github.com/dbt-labs/dbt-core/issues/7288))
:::

<VersionBlock firstVersion="1.8">

From dbt version 1.8, Python models also support dynamic configurations within Python f-strings. This allows for more nuanced and dynamic model configurations directly within your Python code. For example:

<File name='models/my_python_model.py'>

```python
# Previously, attempting to access a configuration value like this would result in None
print(f"{dbt.config.get('my_var')}") # Output before change: None
# Now you can access the actual configuration value
# Assuming 'my_var' is configured to 5 for the current model
print(f"{dbt.config.get('my_var')}") # Output after change: 5
```

This also means you can use `dbt.config.get()` within Python models to ensure that configuration values are effectively retrievable and usable within Python f-strings.

</File>
</VersionBlock>

## Configuring Python models

Just like SQL models, there are three ways to configure Python models:
Expand All @@ -173,7 +193,7 @@ def model(dbt, session):

</File>

There's a limit to how complex you can get with the `dbt.config()` method. It accepts _only_ literal values (strings, booleans, and numeric types). Passing another function or a more complex data structure is not possible. The reason is that dbt statically analyzes the arguments to `config()` while parsing your model without executing your Python code. If you need to set a more complex configuration, we recommend you define it using the [`config` property](/reference/resource-properties/config) in a YAML file.
There's a limit to how complex you can get with the `dbt.config()` method. It accepts _only_ literal values (strings, booleans, and numeric types) and dynamic configuration. Passing another function or a more complex data structure is not possible. The reason is that dbt statically analyzes the arguments to `config()` while parsing your model without executing your Python code. If you need to set a more complex configuration, we recommend you define it using the [`config` property](/reference/resource-properties/config) in a YAML file.

#### Accessing project context

Expand All @@ -184,7 +204,7 @@ Out of the box, the `dbt` class supports:
- Accessing the database location of the current model: `dbt.this()` (also: `dbt.this.database`, `.schema`, `.identifier`)
- Determining if the current model's run is incremental: `dbt.is_incremental`

It is possible to extend this context by "getting" them via `dbt.config.get()` after they are configured in the [model's config](/reference/model-configs). This includes inputs such as `var`, `env_var`, and `target`. If you want to use those values to power conditional logic in your model, we require setting them through a dedicated `.yml` file config:
It is possible to extend this context by "getting" them with `dbt.config.get()` after they are configured in the [model's config](/reference/model-configs). Starting from dbt v1.8, the `dbt.config.get()` method supports dynamic access to configurations within Python models, enhancing flexibility in model logic. This includes inputs such as `var`, `env_var`, and `target`. If you want to use those values for the conditional logic in your model, we require setting them through a dedicated YAML file config:

<File name='models/config.yml'>

Expand Down Expand Up @@ -221,6 +241,27 @@ def model(dbt, session):

</File>

<VersionBlock firstVersion="1.8">

#### Dynamic configurations

In addition to the existing methods of configuring Python models, you also have dynamic access to configuration values set with `dbt.config()` within Python models using f-strings. This increases the possibilities for custom logic and configuration management.

<File name='models/my_python_model.py'>

```python
def model(dbt, session):
dbt.config(materialized="table")
# Dynamic configuration access within Python f-strings,
# which allows for real-time retrieval and use of configuration values.
# Assuming 'my_var' is set to 5, this will print: Dynamic config value: 5
print(f"Dynamic config value: {dbt.config.get('my_var')}")
```

</File>
</VersionBlock>

### Materializations

Python models support these materializations:
Expand Down
2 changes: 1 addition & 1 deletion website/docs/docs/build/unit-tests.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ For dbt Core, unit tests will be available in v1.8, planned for late April 2024.

:::

Historically, dbt's test coverage was confined to [“data” tests](/docs/build/data-tests), assessing the quality of input data or resulting datasets' structure. However, these tests could only be executed _after_ a building a model.
Historically, dbt's test coverage was confined to [“data” tests](/docs/build/data-tests), assessing the quality of input data or resulting datasets' structure. However, these tests could only be executed _after_ building a model.

Now, we are introducing a new type of test to dbt - unit tests. In software programming, unit tests validate small portions of your functional code, and they work much the same way here. Unit tests allow you to validate your SQL modeling logic on a small set of static inputs _before_ you materialize your full model in production. Unit tests enable test-driven development, benefiting developer efficiency and code reliability.

Expand Down
8 changes: 8 additions & 0 deletions website/snippets/_whitespace-control.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
<details>
<summary>💡 Use Jinja's whitespace control to tidy your macros!</summary>

When you're modifying macros in your project, you might notice extra white space in your code in the `target/compiled` folder.

You can remove unwanted spaces and lines with Jinja's [whitespace control](/faqs/Jinja/jinja-whitespace) by using a minus sign. For example, use `{{- ... -}}` or `{%- ... %}` around your macro definitions (such as `{%- macro generate_schema_name(...) -%} ... {%- endmacro -%}`).

</details>

0 comments on commit 35dc061

Please sign in to comment.