counterweight/data-dwh-dbt-project
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

atypos

Browse source
This commit is contained in:
Pablo Martin 2024-10-04 15:34:47 +02:00
parent bcd4ceec86
commit 8fcf1a2966
1 changed files with 3 additions and 3 deletions
Download patch file Download diff file Expand all files Collapse all files

6
dev-env/local_dwh.md
View file

@ -22,7 +22,7 @@ The setup will consist on preparing:
### Pre-requisites ### Pre-requisites
- Before you begin, you will need to have a personal user in the production DWH with the `modeler` role granted. If you don't have one, ask Pablo to get you one. - Before you begin, you will need to have a personal user in the production DWH with the `modeler` role granted. If you don't have one, ask Pablo to get you one.
- You will also need to have the WSL and Docker Dekstop enable, and to enable the integration between both. You can read more in this page: <https://www.notion.so/knowyourguest-superhog/How-to-set-up-WSL-and-Docker-Desktop-4771651ae49a455dac98d7071abcd66d?pvs=4> - You will also need to have the WSL and Docker Desktop enabled, and to enable the integration between both. You can read more in this page: <https://www.notion.so/knowyourguest-superhog/How-to-set-up-WSL-and-Docker-Desktop-4771651ae49a455dac98d7071abcd66d?pvs=4>
### Creating your server ### Creating your server
@ -190,13 +190,13 @@ You might be wondering why we have set up two different databases in your docker
### Filling `dwh` with copied data ### Filling `dwh` with copied data
Given the previous discussion, sometimes it might be the case you want to work fully locally in the `dwh` database with copies of data instead of references a la `dwh_hybrid. Given the previous discussion, sometimes it might be the case you want to work fully locally in the `dwh` database with copies of data instead of references a la `dwh_hybrid`.
There are many ways to populate the data. Historically, we used to do a dump from the production database with what we needed and then restored it in the local Postgres. That's an option and you can still do it. There are many ways to populate the data. Historically, we used to do a dump from the production database with what we needed and then restored it in the local Postgres. That's an option and you can still do it.
You can also leverage the same Foreign Data Wrappers extension we saw in earlier sections to copy data into your local dwh instead of just referencing it like we do in `dwh_hybrid`. This requires a bit of setup, but once you have done that, it is extremely convenient and fast. The rest of this section will detail how to set it up. You can also leverage the same Foreign Data Wrappers extension we saw in earlier sections to copy data into your local dwh instead of just referencing it like we do in `dwh_hybrid`. This requires a bit of setup, but once you have done that, it is extremely convenient and fast. The rest of this section will detail how to set it up.
To set things up, we will create similar but not exactly equal procedures as in `dwh_hybrid`. Begin by running connecting to `dwh` and running the following code to create two stored procedures: To set things up, we will create similar but not exactly equal procedures as in `dwh_hybrid`. Begin by connecting to `dwh` and running the following code to create two stored procedures:
```sql ```sql
CREATE OR REPLACE PROCEDURE setup_fdw_for_el( CREATE OR REPLACE PROCEDURE setup_fdw_for_el(