sh-notion/notion_data_team_no_files/dbt 1 7 to 1 9 upgrade 1740446ff9c98054915fd620df86339a.md

# dbt 1.7 to 1.9 upgrade

On Jan ‘25, we set ourselves to upgrade our dbt project version. This page tracks the task.

## Starting details

On commit `04a10cf9c52ad849ef6f61b133e605efc813e33d`, we hold the following versions in our `requirements.txt` file:

```sql
dbt-core~=1.7.6
dbt-postgres~=1.7.6
```

Furthermore, in the Airbyte production machine, we have these versions installed in the `venv` dedicated to dbt:

```sql
dbt-core==1.7.9
dbt-postgres==1.7.9
```

## Goal

To bump versions into the highest `1.9` patch, ensure everything works, provide instructions for all analysts and also inform on new features available.

At the time of writing this, the highest `1.9` patch for dbt is `1.9.1` (https://github.com/dbt-labs/dbt-core/releases/tag/v1.9.1)

As for the Postgres adapter, the most recent version is `1.9.0`.

## Steps

- [x]  Backup `pip freeze` output of production dbt deployment.
    - Output here
        
        ```python
        agate==1.7.1
        annotated-types==0.6.0
        attrs==23.2.0
        Babel==2.14.0
        certifi==2024.2.2
        cffi==1.16.0
        charset-normalizer==3.3.2
        click==8.1.7
        colorama==0.4.6
        dbt-core==1.7.9
        dbt-extractor==0.5.1
        dbt-postgres==1.7.9
        dbt-semantic-interfaces==0.4.4
        idna==3.6
        importlib-metadata==6.11.0
        isodate==0.6.1
        Jinja2==3.1.3
        jsonschema==4.21.1
        jsonschema-specifications==2023.12.1
        leather==0.4.0
        Logbook==1.5.3
        MarkupSafe==2.1.5
        mashumaro==3.12
        minimal-snowplow-tracker==0.0.2
        more-itertools==10.2.0
        msgpack==1.0.8
        networkx==3.2.1
        packaging==23.2
        parsedatetime==2.6
        pathspec==0.11.2
        protobuf==4.25.3
        psycopg2-binary==2.9.9
        pycparser==2.21
        pydantic==2.6.3
        pydantic_core==2.16.3
        python-dateutil==2.9.0.post0
        python-slugify==8.0.4
        pytimeparse==1.1.8
        pytz==2024.1
        PyYAML==6.0.1
        referencing==0.33.0
        requests==2.31.0
        rpds-py==0.18.0
        six==1.16.0
        sqlparse==0.4.4
        text-unidecode==1.3
        typing_extensions==4.10.0
        urllib3==1.26.18
        zipp==3.17.0
        ```
        
- [x]  Upgrade package versions in production dbt deployment.
    - New pip freeze here.
        
        ```python
        agate==1.7.1
        annotated-types==0.6.0
        attrs==23.2.0
        Babel==2.14.0
        certifi==2024.2.2
        cffi==1.16.0
        charset-normalizer==3.3.2
        click==8.1.7
        colorama==0.4.6
        daff==1.3.46
        dbt-adapters==1.13.0
        dbt-common==1.14.0
        dbt-core==1.9.1
        dbt-extractor==0.5.1
        dbt-postgres==1.9.0
        dbt-semantic-interfaces==0.7.4
        deepdiff==7.0.1
        idna==3.6
        importlib-metadata==6.11.0
        isodate==0.6.1
        Jinja2==3.1.3
        jsonschema==4.21.1
        jsonschema-specifications==2023.12.1
        leather==0.4.0
        Logbook==1.5.3
        MarkupSafe==2.1.5
        mashumaro==3.12
        minimal-snowplow-tracker==0.0.2
        more-itertools==10.2.0
        msgpack==1.0.8
        networkx==3.2.1
        ordered-set==4.1.0
        packaging==23.2
        parsedatetime==2.6
        pathspec==0.11.2
        protobuf==5.29.2
        psycopg2-binary==2.9.9
        pycparser==2.21
        pydantic==2.6.3
        pydantic_core==2.16.3
        python-dateutil==2.9.0.post0
        python-slugify==8.0.4
        pytimeparse==1.1.8
        pytz==2024.1
        PyYAML==6.0.1
        referencing==0.33.0
        requests==2.31.0
        rpds-py==0.18.0
        six==1.16.0
        snowplow-tracker==1.0.4
        sqlparse==0.5.3
        text-unidecode==1.3
        types-requests==2.32.0.20241016
        typing_extensions==4.10.0
        urllib3==2.3.0
        zipp==3.17.0
        ```
        
- [x]  Attempt to run our usual dbt run. Check if everything works and logs look good.
    - [x]  If shit hits the fan, rollback, study issues and go back to step 1. Do not continue down this list.
    - Shit did hit the fan
        - We started to get this error when running any dbt cli command: `ModuleNotFoundError: No module named 'dbt.adapters.factory'`
        - We fixed it by applying this good gentleman’s advice: https://github.com/dbt-labs/dbt-core/issues/10135#issuecomment-2113728550
- [x]  If all is well, open PR to bump versions in git repo.
    - PR here: https://guardhog.visualstudio.com/Data/_git/data-dwh-dbt-project/pullrequest/3970
- [x]  Create instructions for team to upgrade their local environments and make sure to communicate thoroughly, ask everyone to ACK back once done.
    - Instructions below in this page.
- [x]  Make TLDR on cool features we have obtained and reference to docs for further detail.

## Instructions for analysts

Team, we’ve upgraded our version of `dbt` to 1.9. This is already applied in our production deployment, and [this PR](https://guardhog.visualstudio.com/Data/_git/data-dwh-dbt-project/pullrequest/3970) is ready to apply it on the project level.

We also need you to apply this version upgrade in your laptops so that versions are in sync across environments and stuff fits nicely. It’s very simple, you can find below the steps:

- Open your VSCode workspace for the dbt project.
- Open up a terminal and make sure it has the project virtual environment activated.
- Make a backup of your python packages in case things go wrong: `pip freeze > my_packages_backup.txt`
- Run the following sequence of commands to get things installed:
    - `pip uninstall -y dbt-adapters`
    - `pip install dbt-core==1.9.1 --upgrade`
    - `pip install dbt-postgres==1.9.0 --upgrade`
- To check that stuff works, just try to use dbt. You can begin with a humble `dbt --version`, which should show the new version that is installed. If that works fine, move into using dbt as usual in your local env.

And that’s it! Welcome to dbt 1.9.