Merged PR 4309: Cancelled Bookings KPIs rework

# Description

Exposes new metrics:
* Created Bookings (Excl. Cancelled)
* Cancelled Created Bookings
* Check Out Bookings (Excl. Cancelled)
* Cancelled Check Out Bookings

Re-naming of existing metrics:
* Created Bookings -> Total Created Bookings
* Checkout Bookings -> Total Check Out Bookings

Removes exposure of previous Cancelled Bookings. In the monthly by deal model, it's hardcoded still - need to change PBI to remove safely. This will be done later on once we remove the Cancelled Bookings models.

Adapts the existing tests on KPIs to accommodate for the changes by including new metrics. Also, I've set the detector to 5 (from 8) since it's been a while this has not triggered thus might be worth the effort to have more detection capabilities.

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #24637

models/intermediate/cross/int_monthly_aggregated_metrics_history_by_deal.sql

@@ -92,9 +92,19 @@ select
     -- DEAL LIFECYCLE --
     daily_deal_lifecycle.deal_lifecycle_state,
 
-    -- BOOKINGS --
+    -- CREATED BOOKINGS --
     created_bookings.created_bookings,
+    created_bookings.cancelled_created_bookings,
+    created_bookings.not_cancelled_created_bookings,
+    created_bookings.cancelled_created_bookings_rate,
+
+    -- CHECK OUT BOOKINGS --
     check_out_bookings.check_out_bookings,
+    check_out_bookings.cancelled_check_out_bookings,
+    check_out_bookings.not_cancelled_check_out_bookings,
+    check_out_bookings.cancelled_check_out_bookings_rate,
+
+    -- OTHER BOOKINGS --
     cancelled_bookings.cancelled_bookings,
     billable_bookings.billable_bookings,
 
@@ -157,7 +167,7 @@ select
             "cast(host_resolutions.xero_host_resolution_payment_count as decimal)
             / created_bookings.created_bookings",
             -1,
-            1
+            1,
         )
     }}
     as host_resolution_payment_per_created_booking_ratio,
@@ -227,16 +237,17 @@ select
                 + coalesce(invoiced_revenue.xero_operator_net_fees_in_gbp, 0)
                 + coalesce(invoiced_revenue.xero_apis_net_fees_in_gbp, 0)
                 + coalesce(invoiced_revenue.xero_waiver_paid_back_to_host_in_gbp, 0), 
-                0
+                0) / 
-            ) / nullif(
+             nullif(
                 coalesce(guest_payments.total_guest_payments_in_gbp, 0)
                 + coalesce(invoiced_revenue.xero_operator_net_fees_in_gbp, 0)
                 + coalesce(invoiced_revenue.xero_apis_net_fees_in_gbp, 0),
-                0
+                0)",
-            )",
             -1,
-            1
+            1,
-        )}} as revenue_retained_ratio,
+        )
+    }}
+    as revenue_retained_ratio,
 
     -- REVENUE RETAINED POST RESOLUTIONS--
     nullif(
@@ -255,17 +266,16 @@ select
                 + coalesce(invoiced_revenue.xero_apis_net_fees_in_gbp, 0)                 
                 + coalesce(invoiced_revenue.xero_waiver_paid_back_to_host_in_gbp, 0)                
                 + coalesce(host_resolutions.xero_host_resolution_amount_paid_in_gbp, 0),                 
-                0
+                0) / 
-            ) / nullif(
+             nullif(coalesce(guest_payments.total_guest_payments_in_gbp, 0)
-                coalesce(guest_payments.total_guest_payments_in_gbp, 0)
                 + coalesce(invoiced_revenue.xero_operator_net_fees_in_gbp, 0)
                 + coalesce(invoiced_revenue.xero_apis_net_fees_in_gbp, 0),
-                0
+                0)",
-            )",
             -1,
-            1
+            1,
         )
-    }} as revenue_retained_post_resolutions_ratio
+    }}
+    as revenue_retained_post_resolutions_ratio
 
 from int_kpis__agg_dates_main_kpis d
 left join int_kpis__dimension_deals ikdd on d.dimension_value = ikdd.id_deal

models/intermediate/cross/int_mtd_aggregated_metrics.sql

@@ -1,7 +1,25 @@
 {% set metrics = [
     {
         "order_by": 1,
-        "metric": "Created Bookings",
+        "metric": "Created Bookings (Excl. Cancelled)",
+        "value": "not_cancelled_created_bookings",
+        "previous_year_value": "previous_year_not_cancelled_created_bookings",
+        "relative_increment": "relative_increment_not_cancelled_created_bookings",
+        "number_format": "integer",
+        "increment_sign_format": "positive",
+    },
+    {
+        "order_by": 2,
+        "metric": "Cancelled Created Bookings",
+        "value": "cancelled_created_bookings",
+        "previous_year_value": "previous_year_cancelled_created_bookings",
+        "relative_increment": "relative_increment_cancelled_created_bookings",
+        "number_format": "integer",
+        "increment_sign_format": "negative",
+    },
+    {
+        "order_by": 3,
+        "metric": "Total Created Bookings",
         "value": "created_bookings",
         "previous_year_value": "previous_year_created_bookings",
         "relative_increment": "relative_increment_created_bookings",
@@ -9,17 +27,26 @@
         "increment_sign_format": "positive",
     },
     {
-        "order_by": 2,
+        "order_by": 4,
-        "metric": "Cancelled Bookings",
+        "metric": "Check Out Bookings (Excl. Cancelled)",
-        "value": "cancelled_bookings",
+        "value": "not_cancelled_check_out_bookings",
-        "previous_year_value": "previous_year_cancelled_bookings",
+        "previous_year_value": "previous_year_not_cancelled_check_out_bookings",
-        "relative_increment": "relative_increment_cancelled_bookings",
+        "relative_increment": "relative_increment_not_cancelled_check_out_bookings",
+        "number_format": "integer",
+        "increment_sign_format": "positive",
+    },
+    {
+        "order_by": 5,
+        "metric": "Cancelled Check Out Bookings",
+        "value": "cancelled_check_out_bookings",
+        "previous_year_value": "previous_year_cancelled_check_out_bookings",
+        "relative_increment": "relative_increment_cancelled_check_out_bookings",
         "number_format": "integer",
         "increment_sign_format": "negative",
     },
     {
-        "order_by": 3,
+        "order_by": 6,
-        "metric": "Checkout Bookings",
+        "metric": "Total Check Out Bookings",
         "value": "check_out_bookings",
         "previous_year_value": "previous_year_check_out_bookings",
         "relative_increment": "relative_increment_check_out_bookings",
@@ -27,7 +54,7 @@
         "increment_sign_format": "positive",
     },
     {
-        "order_by": 4,
+        "order_by": 9,
         "metric": "Est. Billable Bookings",
         "value": "billable_bookings",
         "previous_year_value": "previous_year_billable_bookings",
@@ -208,6 +235,24 @@
     },
     {
         "order_by": 110,
+        "metric": "Created Booking Cancellation Rate",
+        "value": "cancelled_created_bookings_rate",
+        "previous_year_value": "previous_year_cancelled_created_bookings_rate",
+        "relative_increment": "relative_increment_cancelled_created_bookings_rate",
+        "number_format": "percentage",
+        "increment_sign_format": "negative",
+    },
+    {
+        "order_by": 111,
+        "metric": "Check Out Booking Cancellation Rate",
+        "value": "cancelled_check_out_bookings_rate",
+        "previous_year_value": "previous_year_cancelled_check_out_bookings_rate",
+        "relative_increment": "relative_increment_cancelled_check_out_bookings_rate",
+        "number_format": "percentage",
+        "increment_sign_format": "negative",
+    },
+    {
+        "order_by": 120,
         "metric": "Guest Journey Start Rate",
         "value": "start_rate_guest_journey",
         "previous_year_value": "previous_year_start_rate_guest_journey",
@@ -216,7 +261,7 @@
         "increment_sign_format": "positive",
     },
     {
-        "order_by": 111,
+        "order_by": 121,
         "metric": "Guest Journey Completion Rate",
         "value": "completion_rate_guest_journey",
         "previous_year_value": "previous_year_completion_rate_guest_journey",
@@ -225,7 +270,7 @@
         "increment_sign_format": "positive",
     },
     {
-        "order_by": 112,
+        "order_by": 122,
         "metric": "Guest Journey Incompletion Rate",
         "value": "incompletion_rate_guest_journey",
         "previous_year_value": "previous_year_incompletion_rate_guest_journey",
@@ -234,7 +279,7 @@
         "increment_sign_format": "negative",
     },
     {
-        "order_by": 113,
+        "order_by": 123,
         "metric": "Guest Journey Payment Rate",
         "value": "payment_rate_guest_journey",
         "previous_year_value": "previous_year_payment_rate_guest_journey",
@@ -243,7 +288,7 @@
         "increment_sign_format": "positive",
     },
     {
-        "order_by": 120,
+        "order_by": 130,
         "metric": "Revenue Churn Rate",
         "value": "total_revenue_churn_average_contribution",
         "previous_year_value": "previous_year_total_revenue_churn_average_contribution",
@@ -252,7 +297,7 @@
         "increment_sign_format": "negative",
     },
     {
-        "order_by": 121,
+        "order_by": 131,
         "metric": "Bookings Churn Rate",
         "value": "created_bookings_churn_average_contribution",
         "previous_year_value": "previous_year_created_bookings_churn_average_contribution",
@@ -261,7 +306,7 @@
         "increment_sign_format": "negative",
     },
     {
-        "order_by": 122,
+        "order_by": 132,
         "metric": "Listings Churn Rate",
         "value": "listings_booked_in_month_churn_average_contribution",
         "previous_year_value": "previous_year_listings_booked_in_month_churn_average_contribution",

models/intermediate/cross/int_mtd_vs_previous_year_metrics.sql

@@ -192,9 +192,19 @@ with
             d.dimension,
             d.dimension_value,
 
-            -- BOOKINGS --
+            -- CREATED BOOKINGS --
             created_bookings.created_bookings,
+            created_bookings.cancelled_created_bookings,
+            created_bookings.not_cancelled_created_bookings,
+            created_bookings.cancelled_created_bookings_rate,
+
+            -- CHECK OUT BOOKINGS --
             check_out_bookings.check_out_bookings,
+            check_out_bookings.cancelled_check_out_bookings,
+            check_out_bookings.not_cancelled_check_out_bookings,
+            check_out_bookings.cancelled_check_out_bookings_rate,
+
+            -- OTHER BOOKINGS --
             cancelled_bookings.cancelled_bookings,
             billable_bookings.billable_bookings,
 
@@ -269,10 +279,9 @@ with
             as host_resolution_amount_paid_per_created_booking,
             {{
                 return_capped_value(
-                    "cast(host_resolutions.xero_host_resolution_payment_count as decimal)
+                    "cast(host_resolutions.xero_host_resolution_payment_count as decimal)                         / created_bookings.created_bookings",
-                        / created_bookings.created_bookings",
                     -1,
-                    1
+                    1,
                 )
             }}
             as host_resolution_payment_per_created_booking_ratio,
@@ -350,22 +359,12 @@ with
             ) as revenue_retained_in_gbp,
             {{
                 return_capped_value(
-                    "nullif(
+                    "nullif(                         coalesce(guest_payments.total_guest_payments_in_gbp, 0)                         + coalesce(invoiced_revenue.xero_operator_net_fees_in_gbp, 0)                         + coalesce(invoiced_revenue.xero_apis_net_fees_in_gbp, 0)                         + coalesce(invoiced_revenue.xero_waiver_paid_back_to_host_in_gbp, 0),                         0                     ) / nullif(                         coalesce(guest_payments.total_guest_payments_in_gbp, 0)                         + coalesce(invoiced_revenue.xero_operator_net_fees_in_gbp, 0)                         + coalesce(invoiced_revenue.xero_apis_net_fees_in_gbp, 0),                         0                     )",
-                        coalesce(guest_payments.total_guest_payments_in_gbp, 0)
-                        + coalesce(invoiced_revenue.xero_operator_net_fees_in_gbp, 0)
-                        + coalesce(invoiced_revenue.xero_apis_net_fees_in_gbp, 0)
-                        + coalesce(invoiced_revenue.xero_waiver_paid_back_to_host_in_gbp, 0),
-                        0
-                    ) / nullif(
-                        coalesce(guest_payments.total_guest_payments_in_gbp, 0)
-                        + coalesce(invoiced_revenue.xero_operator_net_fees_in_gbp, 0)
-                        + coalesce(invoiced_revenue.xero_apis_net_fees_in_gbp, 0),
-                        0
-                    )",
                     -1,
-                    1
+                    1,
                 )
-            }} as revenue_retained_ratio,
+            }}
+            as revenue_retained_ratio,
 
             -- INCOME RETAINED POST RESOLUTIONS--
             nullif(
@@ -378,23 +377,12 @@ with
             ) as revenue_retained_post_resolutions_in_gbp,
             {{
                 return_capped_value(
-                    "nullif(
+                    "nullif(                         coalesce(guest_payments.total_guest_payments_in_gbp, 0)                         + coalesce(invoiced_revenue.xero_operator_net_fees_in_gbp, 0)                         + coalesce(invoiced_revenue.xero_apis_net_fees_in_gbp, 0)                         + coalesce(invoiced_revenue.xero_waiver_paid_back_to_host_in_gbp, 0)                         + coalesce(host_resolutions.xero_host_resolution_amount_paid_in_gbp, 0),                         0                     ) / nullif(                         coalesce(guest_payments.total_guest_payments_in_gbp, 0)                         + coalesce(invoiced_revenue.xero_operator_net_fees_in_gbp, 0)                         + coalesce(invoiced_revenue.xero_apis_net_fees_in_gbp, 0),                         0                     )",
-                        coalesce(guest_payments.total_guest_payments_in_gbp, 0)
-                        + coalesce(invoiced_revenue.xero_operator_net_fees_in_gbp, 0)
-                        + coalesce(invoiced_revenue.xero_apis_net_fees_in_gbp, 0)
-                        + coalesce(invoiced_revenue.xero_waiver_paid_back_to_host_in_gbp, 0)
-                        + coalesce(host_resolutions.xero_host_resolution_amount_paid_in_gbp, 0),
-                        0
-                    ) / nullif(
-                        coalesce(guest_payments.total_guest_payments_in_gbp, 0)
-                        + coalesce(invoiced_revenue.xero_operator_net_fees_in_gbp, 0)
-                        + coalesce(invoiced_revenue.xero_apis_net_fees_in_gbp, 0),
-                        0
-                    )",
                     -1,
-                    1
+                    1,
                 )
-            }} as revenue_retained_post_resolutions_ratio,
+            }}
+            as revenue_retained_post_resolutions_ratio,
 
             -- ONBOARDING MRR METRIC --
             onboarding_mrr.expected_mrr_per_deal,
@@ -496,9 +484,19 @@ select
     current.dimension_value,
     previous_year.date as previous_year_date,
 
-    -- BOOKINGS --
+    -- CREATED BOOKINGS --
     {{ calculate_safe_relative_increment("created_bookings") }},
+    {{ calculate_safe_relative_increment("cancelled_created_bookings") }},
+    {{ calculate_safe_relative_increment("not_cancelled_created_bookings") }},
+    {{ calculate_safe_relative_increment("cancelled_created_bookings_rate") }},
+
+    -- CHECK OUT BOOKINGS --
     {{ calculate_safe_relative_increment("check_out_bookings") }},
+    {{ calculate_safe_relative_increment("cancelled_check_out_bookings") }},
+    {{ calculate_safe_relative_increment("not_cancelled_check_out_bookings") }},
+    {{ calculate_safe_relative_increment("cancelled_check_out_bookings_rate") }},
+
+    -- OTHER BOOKINGS --
     {{ calculate_safe_relative_increment("cancelled_bookings") }},
     {{ calculate_safe_relative_increment("billable_bookings") }},
 

models/reporting/general/monthly_aggregated_metrics_history_by_deal.sql

@@ -13,8 +13,14 @@ select
     main_billing_country_iso_3_per_deal as main_billing_country_iso_3_per_deal,
     deal_lifecycle_state as deal_lifecycle_state,
     created_bookings as created_bookings,
+    cancelled_created_bookings as cancelled_created_bookings,
+    not_cancelled_created_bookings as not_cancelled_created_bookings,
+    cancelled_created_bookings_rate as cancelled_created_bookings_rate,
     check_out_bookings as check_out_bookings,
-    cancelled_bookings as cancelled_bookings,
+    cancelled_check_out_bookings as cancelled_check_out_bookings,
+    not_cancelled_check_out_bookings as not_cancelled_check_out_bookings,
+    cancelled_check_out_bookings_rate as cancelled_check_out_bookings_rate,
+    0 as cancelled_bookings,  -- to be removed
     billable_bookings as billable_bookings,
     created_guest_journeys as created_guest_journeys,
     started_guest_journeys as started_guest_journeys,

tests/kpis_additive_metrics_per_dimension_are_consistent.sql

@@ -15,12 +15,15 @@ than the value reported in the Global dimension.
 {% set tolerance_threshold = 0.000001 %}
 
 {% set additive_metric_names = (
-    "Cancelled Bookings",
+    "Bookings Churn Rate",
+    "Cancelled Check Out Bookings",
+    "Cancelled Created Bookings",
     "Check-In Hero Revenue",
-    "Checkout Bookings",
+    "Check Out Bookings (Excl. Cancelled)",
     "Churning Deals",
     "Churning Listings",
-    "Created Bookings",
+    "Created Bookings (Excl. Cancelled)",
+    "Damage Host-Waiver Payments",
     "Deals Booked in 12 Months",
     "Deals Booked in 6 Months",
     "Deals Booked in Month",
@@ -36,9 +39,9 @@ than the value reported in the Global dimension.
     "Host Resolutions Amount Paid",
     "Host Resolutions Payment Count",
     "Invoiced APIs Revenue",
+    "Invoiced Athena Revenue",
     "Invoiced Booking Fees Revenue",
     "Invoiced E-Deposit Revenue",
-    "Invoiced Athena Revenue",
     "Invoiced Listing Fees Revenue",
     "Invoiced Operator Revenue",
     "Invoiced Verification Fees Revenue",
@@ -47,13 +50,14 @@ than the value reported in the Global dimension.
     "Listings Booked in Month",
     "New Deals",
     "New Listings",
+    "Revenue Churn Rate",
+    "Revenue Retained",
+    "Revenue Retained Post-Resolutions",
+    "Total Check Out Bookings",
+    "Total Created Bookings",
     "Total Revenue",
-    "Damage Host-Waiver Payments",
     "Waiver Revenue",
     "Waiver Retained",
-    "Revenue Churn Rate",
-    "Bookings Churn Rate",
-    "Listings Churn Rate",
 ) %}
 
 with

tests/kpis_global_metrics_outlier_detection.sql

@@ -13,9 +13,11 @@ point it becomes too sensitive, just adapt the following parameters.
 -- since there will be no history to check against. 
 -- Do NOT include rates/percentages/ratios.
 {% set metric_names = (
-    "Cancelled Bookings",
+    "Cancelled Check Out Bookings",
-    "Checkout Bookings",
+    "Cancelled Created Bookings",
-    "Created Bookings",
+    "Check Out Bookings (Excl. Cancelled)",
+    "Created Bookings (Excl. Cancelled)",
+    "Damage Host-Waiver Payments",
     "Deposit Fees Revenue",
     "Est. Billable Bookings",
     "First Time Booked Deals",
@@ -28,16 +30,19 @@ point it becomes too sensitive, just adapt the following parameters.
     "Host Resolutions Amount Paid",
     "Host Resolutions Payment Count",
     "Invoiced APIs Revenue",
+    "Invoiced Athena Revenue",
     "Invoiced Booking Fees Revenue",
     "Invoiced E-Deposit Revenue",
-    "Invoiced Athena Revenue",
     "Invoiced Listing Fees Revenue",
     "Invoiced Operator Revenue",
     "Invoiced Verification Fees Revenue",
     "New Deals",
     "New Listings",
+    "Revenue Retained",
+    "Revenue Retained Post-Resolutions",
+    "Total Check Out Bookings",
+    "Total Created Bookings",
     "Total Revenue",
-    "Damage Host-Waiver Payments",
     "Waiver Revenue",
     "Waiver Retained",
 ) %}
@@ -52,7 +57,7 @@ point it becomes too sensitive, just adapt the following parameters.
 -- thus it will be more tolerant.
 -- A lower value means that the chances of detecting outliers 
 -- and false positives will be higher. Recommended around 10.
-{% set detector_tolerance = 8 %}
+{% set detector_tolerance = 5 %}
 
 -- Specify here the number of days in the past that will be used
 -- to compare against. Keep in mind that we only keep the daily