This PR adapts our infra script to a new pattern. We now have a dedicated, small VM that acts as a gateway for web-server delivered contents. The PR:
- Adds the instructions to set up the web gateway VM.
- Adapts some networking and firewall configs (lock down web access from outside the VNET to only the web gateway).
- Updates the Airbyte deployment instructions so that the UI gets served through the web gateway instead of directly from the Airbyte VM.
Related work items: #23999
With the arrival of new members to the data team, I've redesigned a bit the game of roles and users in the DWH.
This PR modifies the infra script to include the commands that should be run to end up in today's state of the DWH should we ever have to redeploy from scratch.
It's not fully perfect since it requires some statefulness (personal users of data team members, existing `sync_`schemas made by airbyte, etc). But it does create all necessary roles and the most important users. And it explains the general philosophy.
Related work items: #16911
