No description
1d981fc728
With the arrival of new members to the data team, I've redesigned a bit the game of roles and users in the DWH. This PR modifies the infra script to include the commands that should be run to end up in today's state of the DWH should we ever have to redeploy from scratch. It's not fully perfect since it requires some statefulness (personal users of data team members, existing `sync_`schemas made by airbyte, etc). But it does create all necessary roles and the most important users. And it explains the general philosophy. Related work items: #16911 |
||
|---|---|---|
| human-script.md | ||
| platform-overview.md | ||
| README.md | ||
Data Infra Script
This repository contains our documentation and scripts on how to deploy the Data team's infrastructure stack.
Content is structured as follows:
human-script.mdis a deployment script for you to read and follow. It guides you through all actions you should take.architecture-overview.mdis an overview of the final architecture that results of following the humna script. If you are not familiar with our architecture, it probably makes sense to read this first so you know what you are setting up.monitoring-and-administration.mdcontains guidelines on how to keep the lights on on several of the components.
The following contents are not covered in this repository:
- Application-level configuration for DWH, Airbyte, Power BI, dbt, etc.
- Instance types, disk sizes and other elements will come with defaults. But these are values that you should adapt to the deployment according to volumen and needs, and probably will also change with time. Treat the defaults with skepticism and make sure you adapt to your needs.
- No IaaC definitions exist as of today. This guide expects you to deploy things manually.