More models and knowledge
This commit is contained in:
parent
23a70ed74e
commit
ab581b0fb5
5 changed files with 83 additions and 3 deletions
25
notes/8.md
25
notes/8.md
|
|
@ -16,5 +16,30 @@ Models can be related between themselves to map dependencies.
|
|||
- Incremental: also a table, but can only create new records, not update
|
||||
- Ephemeral: it's actually NOT materializing. The model can be used by dependents, but it won't be materialized in the DB. It will truly only be a CTE that gets used by other models. Mostly for intermediate states in transformations.
|
||||
|
||||
Materializations can be defined at the model level, folder level and project level. This can be modified in the `dbt_project.yml` file, under the `models` key.
|
||||
|
||||
To set materialization config at the model level, one must make a jinja tag at the start of the file and call the `config` dbt function. See an example below:
|
||||
|
||||
```python
|
||||
{{
|
||||
config(
|
||||
materialized = 'incremental',
|
||||
on_schema_change = 'fail'
|
||||
)
|
||||
}}
|
||||
```
|
||||
|
||||
Incremental materializations need to a block that defines the logic to apply in incremental loads (as opposed to the 'normal' logic, that gets apply on first runs). See below an example:
|
||||
|
||||
```SQL
|
||||
[... rest of query ...]
|
||||
WHERE
|
||||
review_text IS NOT NULL
|
||||
{% if is_incremental() %}
|
||||
AND review_date > (SELECT MAX(review_date) FROM {{ this }})
|
||||
{% endif %}
|
||||
```
|
||||
|
||||
Bear in mind that how to define the strategy to determine what should be loaded is up to the engineer. Any SQL can be placed within the `if is_incremental()` block. In the example above, we have a date field that easily signals what's the most recent date the table has currently seen.
|
||||
|
||||
##
|
||||
Loading…
Add table
Add a link
Reference in a new issue