From 40a12df950b662b08c9f48d4ffdee36d8bf443db Mon Sep 17 00:00:00 2001 From: Pablo Martin Date: Tue, 28 May 2024 14:29:31 +0200 Subject: [PATCH] a few tiny changes in readme and profiles template --- README.md | 8 +++++--- profiles.yml.example | 6 +++--- 2 files changed, 8 insertions(+), 6 deletions(-) diff --git a/README.md b/README.md index 047959e..d8b7951 100644 --- a/README.md +++ b/README.md @@ -18,9 +18,11 @@ Welcome to Superhog's DWH dbt project. Here we model the entire DWH. - Activate the virtual environment and run `pip install -r requirements.txt` - Create an entry for this project `profiles.yml` file at `~/.dbt/profiles.yml`. You have a suggested template at `profiles.yml.example` - Make sure that the `profiles.yml` host and port settings are consistent with whatever networking approach you've taken. + - Run `chmod 600 ~/.dbt/profiles.yml` to secure your profiles file. + - Run `dbt deps` to install dbt dependencies - Check - Ensure you are running in the project venv, either by setting VSCode Python interpreter to the one created by `poetry`, or by running `poetry shell` in the console when in the root dir. - - Turn on your tunnel to `dev` and run `dbt debug`. If it runs well, you are all set. If it fails, there's something wrong with your set up. Grab the terminal output and pull the thread. + - Run `dbt debug`. If it runs well, you are all set. If it fails, there's something wrong with your set up. Grab the terminal output and pull the thread. - Complements - If you are in VSCode, you most probably want to have this extension installed: [dbt Power User](https://marketplace.visualstudio.com/items?itemName=innoverio.vscode-dbt-power-user) - It is advised to use [this autoformatter](https://sqlfmt.com/) and to automatically [run it on save](https://docs.sqlfmt.com/integrations/vs-code). @@ -38,9 +40,9 @@ The only missing bit to make your local deployment be like the production DWH is For example, if you are working on models that use data from Core, you can dump and restore from your terminal with something roughly like this: ```bash -pg_dump -h superhog-dwh-prd.postgres.database.azure.com -U airbyte_user -W -F plain dwh -n sync_core > core.dump +pg_dump -h superhog-dwh-prd.postgres.database.azure.com -U airbyte_user -W -F t dwh -n sync_xero_superhog_limited > xero.dump -pg_restore -h localhost -U postgres -W -d dwh core.dump +pg_restore -h localhost -U postgres -W -d dwh xero.dump ``` ## Branching strategy diff --git a/profiles.yml.example b/profiles.yml.example index f34c570..001e2ef 100644 --- a/profiles.yml.example +++ b/profiles.yml.example @@ -6,7 +6,7 @@ dwh_dbt: user: pass: port: - schema: working + schema: staging threads: 4 type: postgres @@ -16,8 +16,8 @@ dwh_dbt: user: pass: port: - schema: working - threads: 4 + schema: staging + threads: 2 type: postgres target: dev \ No newline at end of file