Merged PR 1952: Automated Slack Alerts

This PR:
- Improves the `run_dbt.sh` script to monitor for failures and send success/failure messages.
- Also updates the `README.md` to explain how to configure the Slack Webhook URLs.

Besides that, I've already created the right slack channels.

Related work items: #16948

README.md

@@ -97,6 +97,7 @@ To deploy:
 - Also run `dbt deps` to install the dbt packages required by the project.
 - Create an entry for this project `profiles.yml` file at `~/.dbt/profiles.yml`. You have a suggested template at `profiles.yml.example`. Make sure that the `profiles.yml` host and port settings are consistent with whatever networking approach you've taken.
 - There's a script in the root of this project called `run_dbt.sh`. Place it in `~/run_dbt.sh`. Adjust the paths of the script if you want/need to.
+- The script is designed to send both success and failure messages to slack channels upon completion. To properly set this up, you will need to place a file called `slack_webhook_urls.txt` on the same path you drop `run_dbt.sh`. The file should have two lines: `SLACK_ALERT_WEBHOOK_URL=<url-of-webhook-for-failures>` and `SLACK_RECEIPT_WEBHOOK_URL=<url-of-webhook-for-successful-runs>`. Setting up the slack channels  and webhooks is outside of the scope of this readme.
 - Create a cron entry with `crontab -e` that runs the script. For example: `0 2 * * * /bin/bash /home/azureuser/run_dbt.sh` to run the dbt models every day at 2AM.
 
 To monitor:

run_dbt.sh

@@ -1,5 +1,23 @@
 #!/bin/bash
 
+# Define the Slack webhook URL
+script_dir=$(dirname "$0")
+webhooks_file="slack_webhook_urls.txt"
+env_file="$script_dir/$webhooks_file"
+if [ -f "$env_file" ]; then
+  export $(grep -v '^#' "$env_file" | xargs)
+else
+  echo "Error: $webhooks_file file not found in the script directory."
+  exit 1
+fi
+
+# Messages to be sent to Slack
+slack_failure_message=":rotating_light::rotating_light::rotating_light: One or more failures during scheduled dbt run in production. :rotating_light::rotating_light::rotating_light:"
+slack_success_message=":white_check_mark::white_check_mark::white_check_mark: dbt run executed successfully in production. :white_check_mark::white_check_mark::white_check_mark:"
+
+# Initialize the failure flag
+has_any_step_failed=0
+
 cd /home/azureuser/data-dwh-dbt-project
 
 # Update from git
@@ -13,15 +31,35 @@ source venv/bin/activate
 # Run seeds
 echo "Triggering dbt seed" | while IFS= read -r line; do printf '%s %s\n' "$(date)" "$line"; done >> /home/azureuser/dbt_run.log 2>&1
 dbt seed | while IFS= read -r line; do printf '%s %s\n' "$(date)" "$line"; done >> /home/azureuser/dbt_run.log 2>&1
+if [ $? -ne 0 ]; then
+  has_any_step_failed=1
+fi
 
 # Run staging layer
 echo "Triggering dbt run: Staging" | while IFS= read -r line; do printf '%s %s\n' "$(date)" "$line"; done >> /home/azureuser/dbt_run.log 2>&1
 dbt run -s models/staging | while IFS= read -r line; do printf '%s %s\n' "$(date)" "$line"; done >> /home/azureuser/dbt_run.log 2>&1
+if [ $? -ne 0 ]; then
+  has_any_step_failed=1
+fi
 
 # Run intermediate layer
 echo "Triggering dbt run: Intermediate" | while IFS= read -r line; do printf '%s %s\n' "$(date)" "$line"; done >> /home/azureuser/dbt_run.log 2>&1
 dbt run -s models/intermediate | while IFS= read -r line; do printf '%s %s\n' "$(date)" "$line"; done >> /home/azureuser/dbt_run.log 2>&1
+if [ $? -ne 0 ]; then
+  has_any_step_failed=1
+fi
 
 # Run reporting layer
 echo "Triggering dbt run: Reporting" | while IFS= read -r line; do printf '%s %s\n' "$(date)" "$line"; done >> /home/azureuser/dbt_run.log 2>&1
 dbt run -s models/reporting | while IFS= read -r line; do printf '%s %s\n' "$(date)" "$line"; done >> /home/azureuser/dbt_run.log 2>&1
+if [ $? -ne 0 ]; then
+  has_any_step_failed=1
+fi
+
+# Check if any step failed and send a Slack message
+if [ $has_any_step_failed -eq 1 ]; then
+  curl -X POST -H 'Content-type: application/json' --data "{\"text\":\"$slack_failure_message\"}" $SLACK_ALERT_WEBHOOK_URL
+fi
+if [ $has_any_step_failed -eq 0 ]; then
+  curl -X POST -H 'Content-type: application/json' --data "{\"text\":\"$slack_success_message\"}" $SLACK_RECEIPT_WEBHOOK_URL
+fi