diff --git a/human-script.md b/human-script.md index c6d44b4..dc739c6 100644 --- a/human-script.md +++ b/human-script.md @@ -461,6 +461,45 @@ Follow this to deploy the entire data infra. ## 5. Airbyte +### 5.1 Deploying Airbyte VM + +- Airbyte lives on its own VM. To do so, create a new VM following these steps. + - Basic settings + - Name it: `airbyte-` + - Use Ubuntu Server 22.04 + - Use Size: `Standard_DS1_v2` + - Use username: `azureuser` + - Use the SSH Key: `superhog-data--general-ssh` + - Select the option `None` for Public inbound ports. + - Disk settings + - Increasing the data disk to at least 64gb as a starting point is recommended. Airbyte can be a bit of a disk hog, and running low on space might lead to obscure errors happening. Start with 64gb and monitor as you increase usage. + - Networking + - Attach to the virtual network `superhog-data-vnet-` + - Attach to the subnet `services-subnet` + - Assign no public IP. + - For setting `NIC network security group` select option `None` + - Management settings + - Defaults are fine. + - Monitoring + - Defaults are fine. + - Advanced + - Defaults are fine. + - Add tags: + - `team: data` + - `environment: ` + - `project: airbyte` +- Once the VM is running, you should be able to ssh into the machine when your VPN is active. + +### 5.2 Deploying Airbyte + +!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! +CONTINUE HERE +Script out these instructions https://docs.airbyte.com/deploying-airbyte/on-azure-vm-cloud-shell + +### 5.3 Test connection to DWH + +- This step is optional. If you are deploying production, you should probably skip it to avoid making the DWH dirty. + ## 6. Power BI ## 7. dbt