counterweight/data-xexe
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

some updates in readme

Browse source
This commit is contained in:
Pablo Martin 2024-06-13 18:00:48 +02:00
parent 029514d80b
commit cf24605e44
1 changed files with 23 additions and 2 deletions
Download patch file Download diff file Expand all files Collapse all files

25
README.md
View file

@ -17,7 +17,7 @@
To use `xexe`, you will need to have credentials for the `xe.com` API. Specifically, you need an account id and it's matching api key. To use `xexe`, you will need to have credentials for the `xe.com` API. Specifically, you need an account id and it's matching api key.
To write into the DWH, you will also need to pass credentials to connect it to it. To write into the DWH, you will also need to pass credentials to connect to it.
To set up your environment, you should create a `.env` file and place it in `~/.xexe/.env`. You will have to run `xexe` as the right user to ensure the `.env` file is found. You can use the `.env-example` file as a reference. We also recommend running `chmod 400` or `chmod 600` on it for safety. To set up your environment, you should create a `.env` file and place it in `~/.xexe/.env`. You will have to run `xexe` as the right user to ensure the `.env` file is found. You can use the `.env-example` file as a reference. We also recommend running `chmod 400` or `chmod 600` on it for safety.
@ -34,7 +34,7 @@ To be able to write rates into the DWH, take these points into consideration:
- `xexe` should run with a user that has permission to write into `dwh/sync_xedotcom_currencies` and to create tables. It will create the right tables if it can't find them. - `xexe` should run with a user that has permission to write into `dwh/sync_xedotcom_currencies` and to create tables. It will create the right tables if it can't find them.
- These details are hardcoded in the `constants` module. You might want to refactor them into run-time configuration options if you find yourself having to change them often. - These details are hardcoded in the `constants` module. You might want to refactor them into run-time configuration options if you find yourself having to change them often.
### Using ### General Usage
Remember to activate the project virtual environment. Remember to activate the project virtual environment.
@ -134,6 +134,27 @@ A few more details:
- Trying to place an `end-date` before a `start-date` will cause an exception. - Trying to place an `end-date` before a `start-date` will cause an exception.
- Running with the option `--dry-run` will run against a mock of the xe.com API. Format will be valid, but all rates will be fixed. This is for testing purposes. - Running with the option `--dry-run` will run against a mock of the xe.com API. Format will be valid, but all rates will be fixed. This is for testing purposes.
### Deploying for Superhog infra
This tool was made specifically to feed our DWH. These are the steps to perform to deploy it from scratch.
- Setup
- Prepare a Linux VM.
- Run the steps described in this readme in sections:
- `How to Use the tool > Install`
- `How to Use the tool > Set up credentials`
- `How to Use the tool > DWH pre-requisites`
- Schedule
- Finally, schedule the execution in the Linux VM to fit your needs.
- Specifics are up to you and your circunstances.
- A general pattern would be to create a little bash script that calls the tool with the right parameters on it.
- Remember to use the `--ignore-warnings` flag if necessary to allow large, automated runs without manually interaction.
- Backfilling
- If you are loading rates for the first time, you might need to backfill long periods of time manually at first.
- The tool is flexible enough. You can probably figure out the right commands by taking a look at `How to Use the tool > General Usage`
- Be careful since you might hit consumption limits set by xe.com.
## Testing ## Testing
This CLI app has three groups of tests: This CLI app has three groups of tests: