Backup stuff

This commit is contained in:
Pablo Martin 2024-02-12 17:44:49 +01:00
parent 783f3e4641
commit 8057ab96ce

View file

@ -608,8 +608,37 @@ Follow this to deploy the entire data infra.
## 7. dbt ## 7. dbt
## 8. Status monitoring - Our dbt project (https://guardhog.visualstudio.com/Data/_git/data-dwh-dbt-project) can be deployed on any linux VM within the virtual network. The instructions on how to deploy and schedule it are in the project repository.
- You can opt to deploy it in the same machine where airbyte is stored, since that machine is probably fairly underutilized.
## 8. Monitoring
### 8.1 Infra monitoring
WIP: we are planning on using Azure Dashboards with metrics.
### 8.2 Service status
WIP: we need support to learn how to use statuspage.io
## 9. Backups ## 9. Backups
- If you are working on a dev or staging environment, you might want to skip this section. - If you are working on a dev or staging environment, you might want to skip this section.
## 9.1 DWH
- Backups are managed with Azure. In the Azure Portal page for the PostgreSQL service, visit section `Backup and restore`. Production servers should have 14 days as a retention period.
## 9.2 Jumphost
- Jumphosts barely hold any data at all. Although it's quite tempting to forget about this and simply raise another VM if something goes wrong, it would be annoying to have to regenerate the keys of both the VPN server and other clients.
- To solve this, make a habit of making regular copies of the Wireguard config file in another machine. Theoretically, only making a copy everytime it gets modified should be enough.
## 9.3 Airbyte
- Our strategy for backing up Airbyte is to backup the entire VM.
- WIP
## 9.4 PBI Gateway
- The PBI Gateway is pretty much stateless. Given this, if there are any issues or disasters on the current VM, simply create another one and set up the gateway again.