sh-notion/notion_data_team_no_files/Reproducing versioning bug in dbt 2100446ff9c98034902fe1c7080b3698.md

Reproducing versioning bug in dbt

Related:

• Task related to this page: https://guardhog.visualstudio.com/Data/_boards/board/t/Data%20Team/Stories?workitem=30882
• Original incident: 20240913-01 - dbt run blocked by “not in the graph” error
• Link to Github bug: https://github.com/dbt-labs/dbt-core/issues/8872
• Recent version upgrade in our project to 1.9.8
  • PR: https://guardhog.visualstudio.com/Data/_git/data-dwh-dbt-project/pullrequest/5455
  • Task: https://guardhog.visualstudio.com/Data/_boards/board/t/Data%20Team/Stories?workitem=30881
  • Page: dbt 1.9.1 to 1.9.8 upgrade

—

Summary

• I confirmed the bug still happens in 1.9.8
• I realised it won’t be fixed until dbt 1.10.0, so we just wait until then.

—

I want to reproduce the bug properly to confirm that it happens in 1.9.1 but not in 1.9.8.

Steps:

• Set up env with 1.9.1
• Reproduce issue with script
• Set up env with 1.9.8
• Repeat script, confirm the error doesn’t apper

Helpers:

Downgrade to 1.9.1

pip uninstall dbt-core -y
pip install dbt-core==1.9.1
dbt --version

Upgrade to 1.9.8

pip uninstall dbt-core -y
pip install dbt-core==1.9.8
dbt --version

Cause the issue

# Delete target to begin from a clean state
rm -rf target/*
# Create a new model
echo "SELECT 1" > models/debug_model.sql
# Add a dependant
echo "SELECT * FROM {{ref('debug_model')}}" > models/debug_dependant.sql
# Build to cause a parse. Should run fine.
dbt build --select debug_model
# Create new version of same model and add yaml bits
echo "SELECT 2" > models/debug_model_v2.sql
# Add schema.yml contents
echo -e "version: 2\n\nmodels:\n  - name: debug_model\n    description: \"Debug model\"\n    latest_version: 1\n    versions:\n      - v: 1\n      - v: 2" > models/schema.yml
# Compile again, causes the bug
dbt build --select debug_model

# Optional cleanup
rm models/debug_model.sql; rm models/debug_dependant.sql; rm models/debug_model_v2.sql; rm models/schema.yml; rm -rf target/*

Okay, funky…

Using the above, I’m able to reproduce the issue in 1.9.8, which I was not expecting at all.

I’ve made this comment on the original issue. If it doesn’t get attention, I’ll probably open a new one.

Final explainer

Okay, I found out what’s going on.

It’s ok that 1.9.8 does not fix the bug. The PR that contains the bug fix was included in the branch that is working towards version 1.10.0, and the dbt guys decided not to include it in any patch for 1.9.8. That confirms that (1) it is not fixed in 1.9.8 and (2) that’s expected, not an issue really.

We will have to wait to 1.10.0 to fix this.