Added a couple of usage examples
This commit is contained in:
parent
b360256713
commit
7d6fabe206
1 changed files with 49 additions and 2 deletions
51
README.md
51
README.md
|
|
@ -2,12 +2,59 @@
|
|||
|
||||
Lolafect is a collection of Python bits that help us build our Prefect flows.
|
||||
|
||||
## Quickstart
|
||||
|
||||
You can find below examples of how to leverage `lolafect` in your flows.
|
||||
|
||||
**Let the `LolaConfig` object do the boilerplate env stuff for you**
|
||||
|
||||
```python
|
||||
from lolafect.lolaconfig import build_lolaconfig
|
||||
|
||||
lolaconfig = build_lolaconfig(
|
||||
flow_name="some-flow",
|
||||
env_s3_bucket="bucket",
|
||||
kubernetes_labels=["some_label"],
|
||||
kubernetes_image="the-image:latest",
|
||||
)
|
||||
|
||||
# Now you can access all the env stuff from here
|
||||
lolaconfig.FLOW_NAME
|
||||
lolaconfig.FLOW_NAME_UDCS
|
||||
lolaconfig.STORAGE
|
||||
lolaconfig.KUBERNETES_IMAGE
|
||||
lolaconfig.KUBERNETES_LABELS
|
||||
lolaconfig.SLACK_WEBHOOKS
|
||||
# etc
|
||||
```
|
||||
|
||||
**Send a warning message to slack if your tasks fails**
|
||||
|
||||
```python
|
||||
from prefect.triggers import any_failed
|
||||
from lolafect.slack import SendSlackMessageTask
|
||||
|
||||
send_warning_message_on_any_failure = SendSlackMessageTask(trigger=any_failed) # You can generate other tasks with
|
||||
#different triggers. For example, you can send a message when all tasks fail, or all tasks succeed
|
||||
|
||||
with Flow(...) as flow:
|
||||
crucial_task_result = some_crucial_task()
|
||||
|
||||
send_warning_message_on_any_failure(
|
||||
webhook_url="the-channel-webhook", # You can probably try to fetch this from lolaconfig.SLACK_WEBHOOKS
|
||||
text_to_send="Watchout, the flow failed!",
|
||||
upstream_tasks=[crucial_task_result]
|
||||
)
|
||||
```
|
||||
|
||||
## How to test
|
||||
|
||||
IDE-agnostic:
|
||||
|
||||
1. Set up a virtual environment which contains both `lolafect` and the dependencies listed in `requirements-dev.txt`.
|
||||
2. Run: `pytests tests`
|
||||
|
||||
In Pycharm: if you configure `pytest` as the project test runner, Pycharm will most probably autodetect the test
|
||||
folder and allow you to run the test suite within the IDE.
|
||||
In Pycharm:
|
||||
|
||||
- If you configure `pytest` as the project test runner, Pycharm will most probably autodetect the test
|
||||
folder and allow you to run the test suite within the IDE.
|
||||
Loading…
Add table
Add a link
Reference in a new issue