Merged PR 1396: Update instructions on dbt packages and deployment

This commit is contained in:
Pablo Martín 2024-03-06 10:32:06 +00:00
commit 316dfc9c1c

View file

@ -90,14 +90,19 @@ To deploy:
- On the VM, set up git creds for the project (for example, with an ssh key) and clone the git project in the `azureuser` home dir. And checkout main.
- Create a virtual environment for the project with `python3 -m venv venv`.
- Activate the virtual environment and run `pip install -r requirements.txt`
- Also run `dbt deps` to install the dbt packages required by the project.
- Create an entry for this project `profiles.yml` file at `~/.dbt/profiles.yml`. You have a suggested template at `profiles.yml.example`. Make sure that the `profiles.yml` host and port settings are consistent with whatever networking approach you've taken.
- There's a script in the root of this project called `run_dbt.sh`. Place it in `~/run_dbt.sh`. Adjust the paths of the script if you want/need to.
- Create a cron entry with `crontab -e` that runs the script. For example: `0 2 * * * /bin/bash /home/azureuser/run_dbt.sh` to run the dbt models every day at 2AM.
To monitor
To monitor:
- The script writes output to a `dbt_run.log` file. You can check the contents to see what happened in the past runs. The exact location of the log file depends on how you set up the `run_dbt.sh` script. If you are unsure of where your logs are being written, check the script to find out.
To maintain:
- Remember to update dbt package dependencies when including new packages.
## Stuff that we haven't done but we would like to
- Automate formatting with git pre-commit.