61 lines
2.7 KiB
Markdown
61 lines
2.7 KiB
Markdown
# Anaxi
|
|
|
|
`Anaxi` is Superhog's tool to perform Extract-Load (EL) syncs between our multiple Cosmos DB databases and our DWH.
|
|
|
|
## How to use the tool
|
|
|
|
*Note: the app has only been used so far in a Linux environment. Windows support is dubious.*
|
|
|
|
### Install
|
|
|
|
- Ensure you have Python 3.10>= and `poetry` installed.
|
|
- Run `poetry install` to install dependencies.
|
|
- Activate the project's virtual environment. You can use `poetry shell`.
|
|
- Test that everything is working by running `anaxi smoke-test`. You should see a happy pig.
|
|
|
|
### Set up credentials
|
|
|
|
`anaxi` needs a few configs and secrets to run.
|
|
|
|
Regarding Cosmos DB databases: `anaxi` expects to find a file called `cosmos-db.yml` in the path `~/.anaxi/cosmos-db.yml`. The file should specify one or more Cosmos DB databases, along with the required secrets to interact with them. You can check the example file in this repo named `example-cosmos-db.yml` to understand how to build this file. Once you've done that, you can check if any database is reachable with the `cosmos-db-healthcheck` command. See more in the `General Usage` section below.
|
|
|
|
For Postgres databases: `anaxi` expects to find a file called `postgres.yml` in the path `~/.anaxi/postgres.yml`. The file should specify one or more Postgres databases, along with the required secrets to interact with them. You can check the example file in this repo named `example-postgres.yml` to understand how to build this file. Once you've done that, you can check if any database is reachable with the `postgres-healthcheck` command. See more in the `General Usage` section below.
|
|
|
|
### General Usage
|
|
|
|
You can run a healthcheck against any Cosmos DB database like this:
|
|
|
|
```bash
|
|
anaxis cosmos-db-healthcheck --cosmos-db-id <your-db-id>
|
|
```
|
|
|
|
You can run a healthcheck against Postgres databases like this:
|
|
|
|
```bash
|
|
anaxis postgres-healthcheck --postgres-database <your-db-name>
|
|
```
|
|
|
|
## Incremental scopes
|
|
|
|
- [X] Callable CLI app
|
|
- [X] Healthchecks against Cosmos DB doable
|
|
- [X] Healthchecks against DWH doable
|
|
- [ ] Reading from Cosmos DB
|
|
- [ ] Writing into DWH
|
|
- [ ] Refactors and improvements
|
|
|
|
## Development
|
|
|
|
### Local Cosmos DB
|
|
|
|
Microsoft provides tools to run a local emulator of Cosmos DB. The bad news is we have been unable to make it work so far, it always breaks for one reason or another.
|
|
|
|
You can find instructions here:
|
|
|
|
- <https://learn.microsoft.com/en-us/azure/cosmos-db/emulator>
|
|
- <https://learn.microsoft.com/en-us/azure/cosmos-db/how-to-develop-emulator>
|
|
|
|
|
|
## What's with the name
|
|
|
|
`Anaxi` is short for Anaximander. [Anaximander of Miletus](https://en.wikipedia.org/wiki/Anaximander) was a pre-Socratic Greek philosopher who lived in Miletus. He is often called the "Father of Cosmology" and founder of astronomy.
|