data-dwh-dbt-project/models/reporting/general/mtd_aggregated_metrics.sql

102 lines
3.9 KiB
MySQL
Raw Normal View History

Merged PR 2607: Propagates and exposes multiple dimension handling for KPIs # Description This PR ensures the propagation of the dimensions for KPIs across the key aggregating and exposing models. Additionally, provides these 2 new fields in reporting while **not affecting the current data display**, thus it's safe to work in the PBI report without needing to work in 2 PRs in parallel. **Changes:** **1 - Intermediate, `int_mtd_vs_previous_year_metrics`:** * Removes the temporary filter on `where dimension in ({{ production_dimensions }})`. This will be applied directly to reporting later. This ensures that the new dimension on customer segmentation is fully available only within intermediate. * Adds `dimension` and `dimension_value` granularity. This includes: 1) adding these fields, 2) joining by these fields with all the source CTEs containing the source models with metrics - which in turn needs the change of the dates model - and 3) joining by these fields in the self-join to compute the incremental vs. previous year. * Changes on the schema file **2 - Intermediate, `int_mtd_aggregated_metrics`:** * Adds `dimension` and `dimension_value` granularity. This includes only adding these fields. * Changes on the schema file **3 - Reporting, `mtd_aggregated_metrics`:** * Adds the filter removed on `int_mtd_vs_previous_year_metrics`. This ensures that only the Global dimension is available for the reporting, thus **no changes from user POV**. * Adds `dimension` and `dimension_value` granularity. This includes only adding these fields * Changes on the schema file # Checklist - [X] The edited models and dependants run properly with production data. - [X] The edited models are sufficiently documented. - [X] The edited models contain PK tests, and I've ran and passed them. - [X] I have checked for DRY opportunities with other models and docs. - [X] I've picked the right materialization for the affected models. # Other - [ ] Check if a full-refresh is required after this PR is merged. Related work items: #19325
2024-08-20 15:42:27 +00:00
{% set production_dimensions = get_kpi_dimensions_for_production() %}
{{
config(
materialized="table",
indexes=[
{"columns": ["dimension"]},
{"columns": ["dimension", "date"]},
],
)
}}
with
dimensions as (
{% for dimension in production_dimensions %}
2024-09-20 14:53:43 +02:00
select
{{ dimension.dimension }} as dimension,
{{ dimension.dimension_display }} as dimension_display
{% if not loop.last %}
union all
{% endif %}
{% endfor %}
),
int_mtd_aggregated_metrics as (
2024-09-20 14:53:43 +02:00
select m.*, d.dimension_display
from {{ ref("int_mtd_aggregated_metrics") }} m
Merged PR 2607: Propagates and exposes multiple dimension handling for KPIs # Description This PR ensures the propagation of the dimensions for KPIs across the key aggregating and exposing models. Additionally, provides these 2 new fields in reporting while **not affecting the current data display**, thus it's safe to work in the PBI report without needing to work in 2 PRs in parallel. **Changes:** **1 - Intermediate, `int_mtd_vs_previous_year_metrics`:** * Removes the temporary filter on `where dimension in ({{ production_dimensions }})`. This will be applied directly to reporting later. This ensures that the new dimension on customer segmentation is fully available only within intermediate. * Adds `dimension` and `dimension_value` granularity. This includes: 1) adding these fields, 2) joining by these fields with all the source CTEs containing the source models with metrics - which in turn needs the change of the dates model - and 3) joining by these fields in the self-join to compute the incremental vs. previous year. * Changes on the schema file **2 - Intermediate, `int_mtd_aggregated_metrics`:** * Adds `dimension` and `dimension_value` granularity. This includes only adding these fields. * Changes on the schema file **3 - Reporting, `mtd_aggregated_metrics`:** * Adds the filter removed on `int_mtd_vs_previous_year_metrics`. This ensures that only the Global dimension is available for the reporting, thus **no changes from user POV**. * Adds `dimension` and `dimension_value` granularity. This includes only adding these fields * Changes on the schema file # Checklist - [X] The edited models and dependants run properly with production data. - [X] The edited models are sufficiently documented. - [X] The edited models contain PK tests, and I've ran and passed them. - [X] I have checked for DRY opportunities with other models and docs. - [X] I've picked the right materialization for the affected models. # Other - [ ] Check if a full-refresh is required after this PR is merged. Related work items: #19325
2024-08-20 15:42:27 +00:00
-- The following clause limits the display execution
-- to only include those dimensions configured to
-- appear for production purposes
2024-09-20 14:53:43 +02:00
inner join dimensions d on m.dimension = d.dimension
)
select
year as year,
month as month,
day as day,
case when is_end_of_month then 1 else 0 end as is_end_of_month,
case when is_current_month then 1 else 0 end as is_current_month,
case
when is_end_of_month_or_yesterday then 1 else 0
end as is_end_of_month_or_yesterday,
first_day_month as first_day_month,
date as date,
dimension_display as dimension,
Merged PR 2607: Propagates and exposes multiple dimension handling for KPIs # Description This PR ensures the propagation of the dimensions for KPIs across the key aggregating and exposing models. Additionally, provides these 2 new fields in reporting while **not affecting the current data display**, thus it's safe to work in the PBI report without needing to work in 2 PRs in parallel. **Changes:** **1 - Intermediate, `int_mtd_vs_previous_year_metrics`:** * Removes the temporary filter on `where dimension in ({{ production_dimensions }})`. This will be applied directly to reporting later. This ensures that the new dimension on customer segmentation is fully available only within intermediate. * Adds `dimension` and `dimension_value` granularity. This includes: 1) adding these fields, 2) joining by these fields with all the source CTEs containing the source models with metrics - which in turn needs the change of the dates model - and 3) joining by these fields in the self-join to compute the incremental vs. previous year. * Changes on the schema file **2 - Intermediate, `int_mtd_aggregated_metrics`:** * Adds `dimension` and `dimension_value` granularity. This includes only adding these fields. * Changes on the schema file **3 - Reporting, `mtd_aggregated_metrics`:** * Adds the filter removed on `int_mtd_vs_previous_year_metrics`. This ensures that only the Global dimension is available for the reporting, thus **no changes from user POV**. * Adds `dimension` and `dimension_value` granularity. This includes only adding these fields * Changes on the schema file # Checklist - [X] The edited models and dependants run properly with production data. - [X] The edited models are sufficiently documented. - [X] The edited models contain PK tests, and I've ran and passed them. - [X] I have checked for DRY opportunities with other models and docs. - [X] I've picked the right materialization for the affected models. # Other - [ ] Check if a full-refresh is required after this PR is merged. Related work items: #19325
2024-08-20 15:42:27 +00:00
dimension_value as dimension_value,
previous_year_date as previous_year_date,
order_by as order_by,
number_format as number_format,
metric as metric,
value as value,
previous_year_value as previous_year_value,
relative_increment as relative_increment,
relative_increment_with_sign_format as relative_increment_with_sign_format
from int_mtd_aggregated_metrics
/*
The following where condition is applied to avoid displaying revenue metrics
in the MTD for the current month and the previous month. The main reason is
that we have a time delay between when the guest does a payment vs. when we
invoice or credit hosts (Xero). Same applies for Host Resolutions.
This is specially tricky for the Host-takes-waiver revenue: guests payments
happen in a timely fashion, and we get all waiver money from the guests. Once
the month is finished, Finance will start to invoice hosts, and in this case,
all hosts that have the Host-takes-waiver need to be payed back the amount
coming from the guest.
Not having this filter would mean that, in the current month, the figures
displayed of guest revenue would be much larger than they actually will be
because we still need to pay back to the hosts these waivers.
For a more current-month evaluation, Guest Payments should be a good proxy
metric to follow.
*/
where
(
(
-- Not show current + previous month if the metric depends on invoicing
-- cycle and it is before the 20th of the month, if it is the 20th of
-- the month or after, only exclude the current month.
(
lower(metric) like '%total revenue%'
or lower(metric) like '%resolutions%'
or lower(metric) like '%invoiced%'
or lower(metric) like '%retained%'
or lower(metric) like '%damage host%'
)
and {{ is_date_before_20th_of_previous_month("date") }}
)
-- Keep all history for the rest of metrics
or not
(
Merged PR 3124: 1/3 - Revenue renaming Main KPIs - MTD scope # Description Adapts revenue figures in Main KPIs - MTD scope or global view. This includes MTD, Monthly Overview, Global Evolution over Time, Detail by Category. In essence, everything that is not by deal. The changes are mainly 2: * Remove the line that deducts the `Waiver Amount Paid Back to Hosts` in all metrics except the `Waiver Net Fees`. This effectively means that the previous `Guest Revenue` = `Guest Payments`, thus I dropped all 3 `Guest Payments` metrics. * Do a renaming at metric display level, but not in the code. This means that I remove the computation of `guest_revenue_in_gbp` for instance and keep `guest_payments_in_gbp`, and apply the renaming later on, since the modelisation already accounts for defining metric names differently from those of the fields. For the rest of metrics, I revised all metrics name and did changes based on the [whiteboard](https://whiteboard.office.com/me/whiteboards/p/c3BvOmh0dHBzOi8vZ3VhcmRob2ctbXkuc2hhcmVwb2ludC5jb20vcGVyc29uYWwvcGFibG9fbWFydGluX3N1cGVyaG9nX2NvbQ%3d%3d/b!T2D3opQuBECSDnhuFZrUacFu3TxvSvdIsnI4Dxsh2IuaB1AigbciRqkqte61I4wz/01H5SI4J4L7HTPJGUT7JGYKTOSQYYWACXU). I also changed the dedicated data tests in Main KPIs to ensure it's working. I also changed the exclusion logic in reporting based on the name of the metric to not display metrics that depend on the invoicing cycle unless it's 2 months ago or before. To keep in mind: * Merging this will automatically display the new figures/naming in production. Might be wise to communicate to stakeholders since some key metrics (namely, Guest Revenue / Total Revenue) will change the meaning. * We also need to do these changes in the metrics by deal part of the computation. I'd do first the removal of these fields in the PBI report (and take the opportunity to change the Data Catalogue) and then do the PR in DWH to change the logic. Before that though let's check that the names included in this PR are the correct ones :) # Checklist - [X] The edited models and dependants run properly with production data. - [NA] The edited models are sufficiently documented. - [X] The edited models contain PK tests, and I've ran and passed them. - [NA] I have checked for DRY opportunities with other models and docs. - [NA] I've picked the right materialization for the affected models. # Other - [ ] Check if a full-refresh is required after this PR is merged. Related work items: #22688
2024-10-10 13:46:59 +00:00
lower(metric) like '%total revenue%'
or lower(metric) like '%resolutions%'
or lower(metric) like '%invoiced%'
Merged PR 3124: 1/3 - Revenue renaming Main KPIs - MTD scope # Description Adapts revenue figures in Main KPIs - MTD scope or global view. This includes MTD, Monthly Overview, Global Evolution over Time, Detail by Category. In essence, everything that is not by deal. The changes are mainly 2: * Remove the line that deducts the `Waiver Amount Paid Back to Hosts` in all metrics except the `Waiver Net Fees`. This effectively means that the previous `Guest Revenue` = `Guest Payments`, thus I dropped all 3 `Guest Payments` metrics. * Do a renaming at metric display level, but not in the code. This means that I remove the computation of `guest_revenue_in_gbp` for instance and keep `guest_payments_in_gbp`, and apply the renaming later on, since the modelisation already accounts for defining metric names differently from those of the fields. For the rest of metrics, I revised all metrics name and did changes based on the [whiteboard](https://whiteboard.office.com/me/whiteboards/p/c3BvOmh0dHBzOi8vZ3VhcmRob2ctbXkuc2hhcmVwb2ludC5jb20vcGVyc29uYWwvcGFibG9fbWFydGluX3N1cGVyaG9nX2NvbQ%3d%3d/b!T2D3opQuBECSDnhuFZrUacFu3TxvSvdIsnI4Dxsh2IuaB1AigbciRqkqte61I4wz/01H5SI4J4L7HTPJGUT7JGYKTOSQYYWACXU). I also changed the dedicated data tests in Main KPIs to ensure it's working. I also changed the exclusion logic in reporting based on the name of the metric to not display metrics that depend on the invoicing cycle unless it's 2 months ago or before. To keep in mind: * Merging this will automatically display the new figures/naming in production. Might be wise to communicate to stakeholders since some key metrics (namely, Guest Revenue / Total Revenue) will change the meaning. * We also need to do these changes in the metrics by deal part of the computation. I'd do first the removal of these fields in the PBI report (and take the opportunity to change the Data Catalogue) and then do the PR in DWH to change the logic. Before that though let's check that the names included in this PR are the correct ones :) # Checklist - [X] The edited models and dependants run properly with production data. - [NA] The edited models are sufficiently documented. - [X] The edited models contain PK tests, and I've ran and passed them. - [NA] I have checked for DRY opportunities with other models and docs. - [NA] I've picked the right materialization for the affected models. # Other - [ ] Check if a full-refresh is required after this PR is merged. Related work items: #22688
2024-10-10 13:46:59 +00:00
or lower(metric) like '%retained%'
or lower(metric) like '%damage host%'
)
)
2025-01-30 16:34:27 +01:00
-- If metric is Churn Rate or Expected MRR, do not show month in progress.
-- Unlike other revenue metrics, Expected MRR is calculated for the next month,
-- so it is not affected by the invoicing cycle.
2025-01-30 16:30:03 +01:00
and not (
(lower(metric) like '%churn rate%' or lower(metric) like '%mrr%')
and is_current_month = true
)