45 lines
No EOL
1.7 KiB
Markdown
45 lines
No EOL
1.7 KiB
Markdown
## Models
|
|
|
|
Models are the fundamental concept behind dbt.
|
|
|
|
They are stored as SQL files in the `models` folder.
|
|
|
|
Models can be related between themselves to map dependencies.
|
|
|
|
|
|
|
|
## Materializations
|
|
|
|
- Ways in which a model can be stored in the database. There are 4:
|
|
- View: it's just a view
|
|
- Table: the model gets stored as a table
|
|
- Incremental: also a table, but can only create new records, not update
|
|
- Ephemeral: it's actually NOT materializing. The model can be used by dependents, but it won't be materialized in the DB. It will truly only be a CTE that gets used by other models. Mostly for intermediate states in transformations.
|
|
|
|
Materializations can be defined at the model level, folder level and project level. This can be modified in the `dbt_project.yml` file, under the `models` key.
|
|
|
|
To set materialization config at the model level, one must make a jinja tag at the start of the file and call the `config` dbt function. See an example below:
|
|
|
|
```python
|
|
{{
|
|
config(
|
|
materialized = 'incremental',
|
|
on_schema_change = 'fail'
|
|
)
|
|
}}
|
|
```
|
|
|
|
Incremental materializations need to a block that defines the logic to apply in incremental loads (as opposed to the 'normal' logic, that gets apply on first runs). See below an example:
|
|
|
|
```SQL
|
|
[... rest of query ...]
|
|
WHERE
|
|
review_text IS NOT NULL
|
|
{% if is_incremental() %}
|
|
AND review_date > (SELECT MAX(review_date) FROM {{ this }})
|
|
{% endif %}
|
|
```
|
|
|
|
Bear in mind that how to define the strategy to determine what should be loaded is up to the engineer. Any SQL can be placed within the `if is_incremental()` block. In the example above, we have a date field that easily signals what's the most recent date the table has currently seen.
|
|
|
|
## |