83 lines
2.9 KiB
Markdown
83 lines
2.9 KiB
Markdown
|
# Reproducing versioning bug in dbt
|
|||
|
|
|
|||
|
|
Related:
|
|||
|
|
|
|||
|
|
- Task related to this page: https://guardhog.visualstudio.com/Data/_boards/board/t/Data%20Team/Stories?workitem=30882
|
|||
|
|
- Original incident: [20240913-01 - `dbt run` blocked by “not in the graph” error](20240913-01%20-%20dbt%20run%20blocked%20by%20%E2%80%9Cnot%20in%20the%20graph%201030446ff9c980c291f1d57751f443ee.md)
|
|||
|
|
- Link to Github bug: https://github.com/dbt-labs/dbt-core/issues/8872
|
|||
|
|
- Recent version upgrade in our project to 1.9.8
|
|||
|
|
- PR: https://guardhog.visualstudio.com/Data/_git/data-dwh-dbt-project/pullrequest/5455
|
|||
|
|
- Task: https://guardhog.visualstudio.com/Data/_boards/board/t/Data%20Team/Stories?workitem=30881
|
|||
|
|
- Page: [dbt 1.9.1 to 1.9.8 upgrade](dbt%201%209%201%20to%201%209%208%20upgrade%202100446ff9c980cbaa01e84c22bdd13c.md)
|
|||
|
|
|
|||
|
|
—
|
|||
|
|
|
|||
|
|
# Summary
|
|||
|
|
|
|||
|
|
- I confirmed the bug still happens in `1.9.8`
|
|||
|
|
- I realised it won’t be fixed until dbt `1.10.0`, so we just wait until then.
|
|||
|
|
|
|||
|
|
—
|
|||
|
|
|
|||
|
|
I want to reproduce the bug properly to confirm that it happens in `1.9.1` but not in `1.9.8`.
|
|||
|
|
|
|||
|
|
Steps:
|
|||
|
|
|
|||
|
|
- Set up env with `1.9.1`
|
|||
|
|
- Reproduce issue with script
|
|||
|
|
- Set up env with `1.9.8`
|
|||
|
|
- Repeat script, confirm the error doesn’t apper
|
|||
|
|
|
|||
|
|
Helpers:
|
|||
|
|
|
|||
|
|
Downgrade to `1.9.1`
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
pip uninstall dbt-core -y
|
|||
|
|
pip install dbt-core==1.9.1
|
|||
|
|
dbt --version
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
Upgrade to `1.9.8`
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
pip uninstall dbt-core -y
|
|||
|
|
pip install dbt-core==1.9.8
|
|||
|
|
dbt --version
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
Cause the issue
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
# Delete target to begin from a clean state
|
|||
|
|
rm -rf target/*
|
|||
|
|
# Create a new model
|
|||
|
|
echo "SELECT 1" > models/debug_model.sql
|
|||
|
|
# Add a dependant
|
|||
|
|
echo "SELECT * FROM {{ref('debug_model')}}" > models/debug_dependant.sql
|
|||
|
|
# Build to cause a parse. Should run fine.
|
|||
|
|
dbt build --select debug_model
|
|||
|
|
# Create new version of same model and add yaml bits
|
|||
|
|
echo "SELECT 2" > models/debug_model_v2.sql
|
|||
|
|
# Add schema.yml contents
|
|||
|
|
echo -e "version: 2\n\nmodels:\n - name: debug_model\n description: \"Debug model\"\n latest_version: 1\n versions:\n - v: 1\n - v: 2" > models/schema.yml
|
|||
|
|
# Compile again, causes the bug
|
|||
|
|
dbt build --select debug_model
|
|||
|
|
|
|||
|
|
# Optional cleanup
|
|||
|
|
rm models/debug_model.sql; rm models/debug_dependant.sql; rm models/debug_model_v2.sql; rm models/schema.yml; rm -rf target/*
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
Okay, funky…
|
|||
|
|
|
|||
|
|
Using the above, I’m able to reproduce the issue in `1.9.8`, which I was not expecting at all.
|
|||
|
|
|
|||
|
|
I’ve made [this comment](https://github.com/dbt-labs/dbt-core/issues/8872#issuecomment-2967223325) on [the original issue](https://github.com/dbt-labs/dbt-core/issues/8872). If it doesn’t get attention, I’ll probably open a new one.
|
|||
|
|
|
|||
|
|
## Final explainer
|
|||
|
|
|
|||
|
|
Okay, I found out what’s going on.
|
|||
|
|
|
|||
|
|
It’s ok that `1.9.8` does not fix the bug. The PR that contains the bug fix was included in the branch that is working towards version `1.10.0`, and the dbt guys decided not to include it in any patch for `1.9.8`. That confirms that (1) it is not fixed in `1.9.8` and (2) that’s expected, not an issue really.
|
|||
|
|
|
|||
|
|
We will have to wait to `1.10.0` to fix this.
|