counterweight/data-dwh-dbt-project
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Merged PR 3124: 1/3 - Revenue renaming Main KPIs - MTD scope

Browse source 
# Description

Adapts revenue figures in Main KPIs - MTD scope or global view. This includes MTD, Monthly Overview, Global Evolution over Time, Detail by Category. In essence, everything that is not by deal.

The changes are mainly 2:
* Remove the line that deducts the `Waiver Amount Paid Back to Hosts` in all metrics except the `Waiver Net Fees`. This effectively means that the previous `Guest Revenue` = `Guest Payments`, thus I dropped all 3 `Guest Payments` metrics.
* Do a renaming at metric display level, but not in the code. This means that I remove the computation of `guest_revenue_in_gbp` for instance and keep `guest_payments_in_gbp`, and apply the renaming later on, since the modelisation already accounts for defining metric names differently from those of the fields. For the rest of metrics, I revised all metrics name and did changes based on the [whiteboard](https://whiteboard.office.com/me/whiteboards/p/c3BvOmh0dHBzOi8vZ3VhcmRob2ctbXkuc2hhcmVwb2ludC5jb20vcGVyc29uYWwvcGFibG9fbWFydGluX3N1cGVyaG9nX2NvbQ%3d%3d/b!T2D3opQuBECSDnhuFZrUacFu3TxvSvdIsnI4Dxsh2IuaB1AigbciRqkqte61I4wz/01H5SI4J4L7HTPJGUT7JGYKTOSQYYWACXU). I also changed the dedicated data tests in Main KPIs to ensure it's working. I also changed the exclusion logic in reporting based on the name of the metric to not display metrics that depend on the invoicing cycle unless it's 2 months ago or before.

To keep in mind:
* Merging this will automatically display the new figures/naming in production. Might be wise to communicate to stakeholders since some key metrics (namely, Guest Revenue / Total Revenue) will change the meaning.
* We also need to do these changes in the metrics by deal part of the computation. I'd do first the removal of these fields in the PBI report (and take the opportunity to change the Data Catalogue) and then do the PR in DWH to change the logic. Before that though let's check that the names included in this PR are the correct ones :)

# Checklist

- [X] The edited models and dependants run properly with production data.
- [NA] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [NA] I have checked for DRY opportunities with other models and docs.
- [NA] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #22688
This commit is contained in:
Oriol Roqué Paniagua 2024-10-10 13:46:59 +00:00
parent bf3d1344a3
commit 745f00bad2
5 changed files with 110 additions and 178 deletions
Download patch file Download diff file Expand all files Collapse all files

65
models/intermediate/cross/int_mtd_aggregated_metrics.sql
View file

@ -280,7 +280,7 @@
},
{
"order_by": 225,
"metric": "Invoiced Booking Fees",
"metric": "Invoiced Booking Fees Revenue",
"value": "xero_booking_net_fees_in_gbp",
"previous_year_value": "previous_year_xero_booking_net_fees_in_gbp",
"relative_increment": "relative_increment_xero_booking_net_fees_in_gbp",
@ -289,7 +289,7 @@
},
{
"order_by": 226,
"metric": "Invoiced Listing Fees",
"metric": "Invoiced Listing Fees Revenue",
"value": "xero_listing_net_fees_in_gbp",
"previous_year_value": "previous_year_xero_listing_net_fees_in_gbp",
"relative_increment": "relative_increment_xero_listing_net_fees_in_gbp",
@ -298,7 +298,7 @@
},
{
"order_by": 227,
"metric": "Invoiced Verification Fees",
"metric": "Invoiced Verification Fees Revenue",
"value": "xero_verification_net_fees_in_gbp",
"previous_year_value": "previous_year_xero_verification_net_fees_in_gbp",
"relative_increment": "relative_increment_xero_verification_net_fees_in_gbp",
@ -316,7 +316,7 @@
},
{
"order_by": 235,
"metric": "Invoiced Guesty Fees",
"metric": "Invoiced Athena Revenue",
"value": "xero_guesty_net_fees_in_gbp",
"previous_year_value": "previous_year_xero_guesty_net_fees_in_gbp",
"relative_increment": "relative_increment_xero_guesty_net_fees_in_gbp",
@ -325,7 +325,7 @@
},
{
"order_by": 236,
"metric": "Invoiced E-Deposit Fees",
"metric": "Invoiced E-Deposit Revenue",
"value": "xero_e_deposit_net_fees_in_gbp",
"previous_year_value": "previous_year_xero_e_deposit_net_fees_in_gbp",
"relative_increment": "relative_increment_xero_e_deposit_net_fees_in_gbp",
@ -335,33 +335,33 @@
{
"order_by": 250,
"metric": "Guest Revenue",
"value": "total_guest_revenue_in_gbp",
"previous_year_value": "previous_year_total_guest_revenue_in_gbp",
"relative_increment": "relative_increment_total_guest_revenue_in_gbp",
"value": "total_guest_payments_in_gbp",
"previous_year_value": "previous_year_total_guest_payments_in_gbp",
"relative_increment": "relative_increment_total_guest_payments_in_gbp",
"number_format": "currency_gbp",
"increment_sign_format": "positive",
},
{
"order_by": 251,
"metric": "Guest Revenue per Guest Journey Completed",
"value": "guest_revenue_per_completed_guest_journey",
"previous_year_value": "previous_year_guest_revenue_per_completed_guest_journey",
"relative_increment": "relative_increment_guest_revenue_per_completed_guest_journey",
"value": "guest_payments_per_completed_guest_journey",
"previous_year_value": "previous_year_guest_payments_per_completed_guest_journey",
"relative_increment": "relative_increment_guest_payments_per_completed_guest_journey",
"number_format": "converted_metric_currency_gbp",
"increment_sign_format": "positive",
},
{
"order_by": 252,
"metric": "Guest Revenue per Guest Journey with Payment",
"value": "guest_revenue_per_paid_guest_journey",
"previous_year_value": "previous_year_guest_revenue_per_paid_guest_journey",
"relative_increment": "relative_increment_guest_revenue_per_paid_guest_journey",
"value": "guest_payments_per_paid_guest_journey",
"previous_year_value": "previous_year_guest_payments_per_paid_guest_journey",
"relative_increment": "relative_increment_guest_payments_per_paid_guest_journey",
"number_format": "converted_metric_currency_gbp",
"increment_sign_format": "positive",
},
{
"order_by": 260,
"metric": "Deposit Fees",
"metric": "Deposit Fees Revenue",
"value": "deposit_fees_in_gbp",
"previous_year_value": "previous_year_deposit_fees_in_gbp",
"relative_increment": "relative_increment_deposit_fees_in_gbp",
@ -370,7 +370,7 @@
},
{
"order_by": 262,
"metric": "Waiver Amount Paid by Guests",
"metric": "Waiver Revenue",
"value": "waiver_payments_in_gbp",
"previous_year_value": "previous_year_waiver_payments_in_gbp",
"relative_increment": "relative_increment_waiver_payments_in_gbp",
@ -379,7 +379,7 @@
},
{
"order_by": 263,
"metric": "Waiver Amount Paid back to Hosts",
"metric": "Damage Host-Waiver Payments",
"value": "xero_waiver_paid_back_to_host_in_gbp",
"previous_year_value": "previous_year_xero_waiver_paid_back_to_host_in_gbp",
"relative_increment": "relative_increment_xero_waiver_paid_back_to_host_in_gbp",
@ -388,7 +388,7 @@
},
{
"order_by": 264,
"metric": "Waiver Net Fees",
"metric": "Waiver Retained",
"value": "waiver_net_fees_in_gbp",
"previous_year_value": "previous_year_waiver_net_fees_in_gbp",
"relative_increment": "relative_increment_waiver_net_fees_in_gbp",
@ -397,7 +397,7 @@
},
{
"order_by": 265,
"metric": "Check-In Hero Amount Paid by Guests",
"metric": "Check-In Hero Revenue",
"value": "checkin_cover_fees_in_gbp",
"previous_year_value": "previous_year_checkin_cover_fees_in_gbp",
"relative_increment": "relative_increment_checkin_cover_fees_in_gbp",
@ -413,33 +413,6 @@
"number_format": "currency_gbp",
"increment_sign_format": "negative",
},
{
"order_by": 290,
"metric": "Guest Payments",
"value": "total_guest_payments_in_gbp",
"previous_year_value": "previous_year_total_guest_payments_in_gbp",
"relative_increment": "relative_increment_total_guest_payments_in_gbp",
"number_format": "currency_gbp",
"increment_sign_format": "positive",
},
{
"order_by": 291,
"metric": "Guest Payments per Guest Journey Completed",
"value": "guest_payments_per_completed_guest_journey",
"previous_year_value": "previous_year_guest_payments_per_completed_guest_journey",
"relative_increment": "relative_increment_guest_payments_per_completed_guest_journey",
"number_format": "converted_metric_currency_gbp",
"increment_sign_format": "positive",
},
{
"order_by": 292,
"metric": "Guest Payments per Guest Journey with Payment",
"value": "guest_payments_per_paid_guest_journey",
"previous_year_value": "previous_year_guest_payments_per_paid_guest_journey",
"relative_increment": "relative_increment_guest_payments_per_paid_guest_journey",
"number_format": "converted_metric_currency_gbp",
"increment_sign_format": "positive",
},
] %}
with
int_mtd_vs_previous_year_metrics as (

31
models/intermediate/cross/int_mtd_vs_previous_year_metrics.sql
View file

@ -114,16 +114,10 @@ with
) as waiver_net_fees_in_gbp,
guest_payments.checkin_cover_fees_in_gbp,
guest_payments.total_guest_payments_in_gbp,
nullif(
coalesce(guest_payments.total_guest_payments_in_gbp, 0)
+ coalesce(invoicing.xero_waiver_paid_back_to_host_in_gbp, 0),
0
) as total_guest_revenue_in_gbp,
-- TOTAL REVENUE --
nullif(
coalesce(guest_payments.total_guest_payments_in_gbp, 0)
+ coalesce(invoicing.xero_waiver_paid_back_to_host_in_gbp, 0)
+ coalesce(invoicing.xero_operator_net_fees_in_gbp, 0)
+ coalesce(invoicing.xero_apis_net_fees_in_gbp, 0),
0
@ -136,25 +130,10 @@ with
guest_payments.total_guest_payments_in_gbp / nullif(
guest_journeys.paid_guest_journeys, 0
) as guest_payments_per_paid_guest_journey,
nullif(
coalesce(guest_payments.total_guest_payments_in_gbp, 0)
+ coalesce(invoicing.xero_waiver_paid_back_to_host_in_gbp, 0),
0
) / nullif(
guest_journeys.completed_guest_journeys, 0
) as guest_revenue_per_completed_guest_journey,
nullif(
coalesce(guest_payments.total_guest_payments_in_gbp, 0)
+ coalesce(invoicing.xero_waiver_paid_back_to_host_in_gbp, 0),
0
) / nullif(
guest_journeys.paid_guest_journeys, 0
) as guest_revenue_per_paid_guest_journey,
-- TOTAL REVENUE WEIGHTED METRICS --
(
coalesce(guest_payments.total_guest_payments_in_gbp, 0)
+ coalesce(invoicing.xero_waiver_paid_back_to_host_in_gbp, 0)
+ coalesce(invoicing.xero_operator_net_fees_in_gbp, 0)
+ coalesce(invoicing.xero_apis_net_fees_in_gbp, 0)
) / nullif(
@ -162,7 +141,6 @@ with
) as total_revenue_per_created_booking,
(
coalesce(guest_payments.total_guest_payments_in_gbp, 0)
+ coalesce(invoicing.xero_waiver_paid_back_to_host_in_gbp, 0)
+ coalesce(invoicing.xero_operator_net_fees_in_gbp, 0)
+ coalesce(invoicing.xero_apis_net_fees_in_gbp, 0)
) / nullif(
@ -170,7 +148,6 @@ with
) as total_revenue_per_created_guest_journey,
(
coalesce(guest_payments.total_guest_payments_in_gbp, 0)
+ coalesce(invoicing.xero_waiver_paid_back_to_host_in_gbp, 0)
+ coalesce(invoicing.xero_operator_net_fees_in_gbp, 0)
+ coalesce(invoicing.xero_apis_net_fees_in_gbp, 0)
) / nullif(
@ -178,7 +155,6 @@ with
) as total_revenue_per_deals_booked_in_month,
(
coalesce(guest_payments.total_guest_payments_in_gbp, 0)
+ coalesce(invoicing.xero_waiver_paid_back_to_host_in_gbp, 0)
+ coalesce(invoicing.xero_operator_net_fees_in_gbp, 0)
+ coalesce(invoicing.xero_apis_net_fees_in_gbp, 0)
) / nullif(
@ -299,27 +275,24 @@ select
{{ calculate_safe_relative_increment("xero_host_resolution_amount_paid_in_gbp") }},
{{ calculate_safe_relative_increment("xero_host_resolution_payment_count") }},
-- GUEST REVENUE AND PAYMENTS --
-- GUEST REVENUE --
{{ calculate_safe_relative_increment("deposit_fees_in_gbp") }},
{{ calculate_safe_relative_increment("waiver_payments_in_gbp") }},
{{ calculate_safe_relative_increment("xero_waiver_paid_back_to_host_in_gbp") }},
{{ calculate_safe_relative_increment("waiver_net_fees_in_gbp") }},
{{ calculate_safe_relative_increment("checkin_cover_fees_in_gbp") }},
{{ calculate_safe_relative_increment("total_guest_payments_in_gbp") }},
{{ calculate_safe_relative_increment("total_guest_revenue_in_gbp") }},
-- TOTAL REVENUE --
{{ calculate_safe_relative_increment("total_revenue_in_gbp") }},
-- GUEST REVENUE AND PAYMENTS WEIGHTED METRICS --
-- GUEST REVENUE WEIGHTED METRICS --
{{
calculate_safe_relative_increment(
"guest_payments_per_completed_guest_journey"
)
}},
{{ calculate_safe_relative_increment("guest_payments_per_paid_guest_journey") }},
{{ calculate_safe_relative_increment("guest_revenue_per_completed_guest_journey") }},
{{ calculate_safe_relative_increment("guest_revenue_per_paid_guest_journey") }},
-- TOTAL REVENUE WEIGHTED METRICS --
{{ calculate_safe_relative_increment("total_revenue_per_created_booking") }},

12
models/reporting/general/mtd_aggregated_metrics.sql
View file

@ -59,20 +59,20 @@ where
(
-- Not show current + previous month if the metric depends on invoicing cycle
(
lower(metric) like '%revenue%'
lower(metric) like '%total revenue%'
or lower(metric) like '%resolutions%'
or lower(metric) like '%invoiced%'
or lower(metric) like '%back to host%'
or lower(metric) like '%waiver net fees%'
or lower(metric) like '%retained%'
or lower(metric) like '%damage host%'
)
and {{ is_date_before_previous_month("date") }}
)
-- Keep all history for the rest of metrics
or not
(
lower(metric) like '%revenue%'
lower(metric) like '%total revenue%'
or lower(metric) like '%resolutions%'
or lower(metric) like '%invoiced%'
or lower(metric) like '%back to host%'
or lower(metric) like '%waiver net fees%'
or lower(metric) like '%retained%'
or lower(metric) like '%damage host%'
)

105
tests/kpis_additive_metrics_per_dimension_are_consistent.sql
View file

@ -11,76 +11,61 @@ the aggregation might not match the Global value on a given metric.
However, the aggregation cannot be higher than the value reported in
the Global dimension.
*/
{% set additive_metric_names = (
'Cancelled Bookings',
'Check-In Hero Amount Paid by Guests',
'Checkout Bookings',
'Churning Deals',
'Churning Listings',
'Created Bookings',
'Deals Booked in 12 Months',
'Deals Booked in 6 Months',
'Deals Booked in Month',
'Deposit Fees',
'Est. Billable Bookings',
'First Time Booked Deals',
'First Time Booked Listings',
'Guest Journey Completed',
'Guest Journey Created',
'Guest Journey Started',
'Guest Journey with Payment',
'Guest Payments',
'Guest Revenue',
'Host Resolutions Amount Paid',
'Host Resolutions Payment Count',
'Invoiced APIs Revenue',
'Invoiced Booking Fees',
'Invoiced E-Deposit Fees',
'Invoiced Guesty Fees',
'Invoiced Listing Fees',
'Invoiced Operator Revenue',
'Invoiced Verification Fees',
'Listings Booked in 12 Months',
'Listings Booked in 6 Months',
'Listings Booked in Month',
'New Deals',
'New Listings',
'Total Revenue',
'Waiver Amount Paid back to Hosts',
'Waiver Amount Paid by Guests',
'Waiver Net Fees')
%}
"Cancelled Bookings",
"Check-In Hero Revenue",
"Checkout Bookings",
"Churning Deals",
"Churning Listings",
"Created Bookings",
"Deals Booked in 12 Months",
"Deals Booked in 6 Months",
"Deals Booked in Month",
"Deposit Fees Revenue",
"Est. Billable Bookings",
"First Time Booked Deals",
"First Time Booked Listings",
"Guest Journey Completed",
"Guest Journey Created",
"Guest Journey Started",
"Guest Journey with Payment",
"Guest Revenue",
"Host Resolutions Amount Paid",
"Host Resolutions Payment Count",
"Invoiced APIs Revenue",
"Invoiced Booking Fees Revenue",
"Invoiced E-Deposit Revenue",
"Invoiced Athena Revenue",
"Invoiced Listing Fees Revenue",
"Invoiced Operator Revenue",
"Invoiced Verification Fees Revenue",
"Listings Booked in 12 Months",
"Listings Booked in 6 Months",
"Listings Booked in Month",
"New Deals",
"New Listings",
"Total Revenue",
"Damage Host-Waiver Payments",
"Waiver Revenue",
"Waiver Retained",
) %}
with
dimensions_total_metric_values as (
select
date,
dimension,
metric,
number_format,
sum(value) as total_metric_value
select date, dimension, metric, number_format, sum(value) as total_metric_value
from {{ ref("mtd_aggregated_metrics") }}
where
date in (select max(date) from {{ ref("mtd_aggregated_metrics") }}) and
metric in {{ additive_metric_names }}
date in (select max(date) from {{ ref("mtd_aggregated_metrics") }})
and metric in {{ additive_metric_names }}
group by date, dimension, metric, number_format
),
global_dimension_metric_values as (
select date,
dimension,
metric,
number_format,
total_metric_value
select date, dimension, metric, number_format, total_metric_value
from dimensions_total_metric_values
where dimension = 'Global'
),
other_dimension_metric_values as (
select date,
dimension,
metric,
number_format,
total_metric_value
select date, dimension, metric, number_format, total_metric_value
from dimensions_total_metric_values
where dimension != 'Global'
),
@ -93,9 +78,11 @@ select
abs(g.total_metric_value) as global_metric_value,
abs(o.total_metric_value) as dimension_metric_value,
abs(o.total_metric_value) - abs(g.total_metric_value) as abs_diff,
abs(o.total_metric_value) / nullif(abs(g.total_metric_value),0) - 1 as rel_diff
abs(o.total_metric_value) / nullif(abs(g.total_metric_value), 0)
- 1 as rel_diff
from global_dimension_metric_values as g
left join other_dimension_metric_values as o
left join
other_dimension_metric_values as o
on g.date = o.date
and g.metric = o.metric
)

19
tests/kpis_global_metrics_outlier_detection.sql
View file

@ -16,7 +16,7 @@ point it becomes too sensitive, just adapt the following parameters.
"Cancelled Bookings",
"Checkout Bookings",
"Created Bookings",
"Deposit Fees",
"Deposit Fees Revenue",
"Est. Billable Bookings",
"First Time Booked Deals",
"First Time Booked Listings",
@ -24,23 +24,22 @@ point it becomes too sensitive, just adapt the following parameters.
"Guest Journey Created",
"Guest Journey Started",
"Guest Journey with Payment",
"Guest Payments",
"Guest Revenue",
"Host Resolutions Amount Paid",
"Host Resolutions Payment Count",
"Invoiced APIs Revenue",
"Invoiced Booking Fees",
"Invoiced E-Deposit Fees",
"Invoiced Guesty Fees",
"Invoiced Listing Fees",
"Invoiced Booking Fees Revenue",
"Invoiced E-Deposit Revenue",
"Invoiced Athena Revenue",
"Invoiced Listing Fees Revenue",
"Invoiced Operator Revenue",
"Invoiced Verification Fees",
"Invoiced Verification Fees Revenue",
"New Deals",
"New Listings",
"Total Revenue",
"Waiver Amount Paid back to Hosts",
"Waiver Amount Paid by Guests",
"Waiver Net Fees",
"Damage Host-Waiver Payments",
"Waiver Revenue",
"Waiver Retained",
) %}
-- Specify here the day of the month that will start to be considered