Merged PR 3329: First version of KPIs refactored - created bookings
# Description Creates skeleton for new KPIs data flow for created_bookings metric. Details are accessible [here](https://www.notion.so/knowyourguest-superhog/KPIs-Refactor-Let-s-go-daily-2024-10-23-1280446ff9c980dc87a3dc7453e95f06?pvs=4#12a0446ff9c98085bf4dfc77f6fc22f7) In essence: * Models are created in intermediate in a kpis folder. * Models have a daily segmentation. This includes `created_bookings` models, but also the daily lifecycle per listing and the segmentation. It also adds a `dimension_dates` model specific for KPIs. These have all the dimensions already in place and handle all the crazy logic. * Other time aggregation models simply read from existing daily models which are much easier (`int_kpis__metric_mtd_created_bookings` and `int_kpis__metric_monthly_created_bookings`). * Dimensionality aggregation can be easily added within a given timeframe (daily, mtd, monthly). For instance, I do it for mtd in the `int_kpis__aggregated_mtd_created_bookings` and for monthly in `int_kpis__aggregated_monthly_created_bookings` * Macro configuration for dimensions: Allows to set any specific dimension for `aggregated` models. By default, the subset of global, by billing country, by number of listings and by deal apply - since these are needed for Main KPIs. I added an example with Dash Source, that currently does not exist and it's currently configured as only appearing for created bookings. * Testing `aggregated` models completeness. A new macro called `assert_dimension_completeness` is available that ensures additive metrics are consistent vs. the global result, configurable at schema level. * Testing refactor impact. I'm aware that changing the lifecycle model to daily impacts the volumes for listing segments. For the rest, I added a `tmp` test that checks that the dimension and dimension value per date exactly match comparing new vs. old computation. Latest edits: * Changed naming convention * Split of MTD and Monthly. Now these are 2 different entities, as stated in `int_kpis__dimension_dates`. * Added start_date and end_date for models that contemplate a range (mtd, monthly). * Added a small readme entry in the kpis folders. Mostly it states nomenclature and some first conventions. Dbt docs:  # Checklist - [X] The edited models and dependants run properly with production data. - [X] The edited models are sufficiently documented. - [X] The edited models contain PK tests, and I've ran and passed them. - [ ] I have checked for DRY opportunities with other models and docs. **Likely we'll be able to add macros for mtd and dim_agg models. We will see later on.** - [ ] I've picked the right materialization for the affected models. **Models run ok except for the daily lifecycle of listings, which lasts several minutes in the first run. Model curr...
This commit is contained in:
parent
450975301a
commit
875f91be26
13 changed files with 1149 additions and 7 deletions
|
|
@ -5,12 +5,17 @@ Provides a general configuration for the Dimensions available for the KPIs.
|
|||
Please note that strings should be encoded with " ' your_value_here ' ",
|
||||
while fields from tables should be specified like " your_field_here "
|
||||
*/
|
||||
|
||||
{% macro get_kpi_dimensions() %}
|
||||
{% set dimensions = [
|
||||
{"dimension": "'global'", "dimension_value": "'global'"},
|
||||
{"dimension": "'by_number_of_listings'", "dimension_value": "active_accommodations_per_deal_segmentation"},
|
||||
{"dimension": "'by_billing_country'", "dimension_value": "main_billing_country_iso_3_per_deal"}
|
||||
{
|
||||
"dimension": "'by_number_of_listings'",
|
||||
"dimension_value": "active_accommodations_per_deal_segmentation",
|
||||
},
|
||||
{
|
||||
"dimension": "'by_billing_country'",
|
||||
"dimension_value": "main_billing_country_iso_3_per_deal",
|
||||
},
|
||||
] %}
|
||||
{{ return(dimensions) }}
|
||||
{% endmacro %}
|
||||
|
|
@ -25,12 +30,84 @@ This configuration ensures that working with new dimensions won't affect the dis
|
|||
Additionally, it provides a proper display name for reporting purposes.
|
||||
|
||||
*/
|
||||
|
||||
{% macro get_kpi_dimensions_for_production() %}
|
||||
{% set dimensions = [
|
||||
{"dimension": "'global'", "dimension_display": "'Global'"},
|
||||
{"dimension": "'by_number_of_listings'", "dimension_display": "'By # of Listings Booked in 12 Months'"},
|
||||
{"dimension": "'by_billing_country'", "dimension_display": "'By Billing Country'"}
|
||||
{
|
||||
"dimension": "'by_number_of_listings'",
|
||||
"dimension_display": "'By # of Listings Booked in 12 Months'",
|
||||
},
|
||||
{
|
||||
"dimension": "'by_billing_country'",
|
||||
"dimension_display": "'By Billing Country'",
|
||||
},
|
||||
] %}
|
||||
{{ return(dimensions) }}
|
||||
{% endmacro %}
|
||||
{% endmacro %}
|
||||
|
||||
/*
|
||||
The following lines specify for each dimension the field to be used in a
|
||||
standalone macro.
|
||||
Please note that strings should be encoded with " ' your_value_here ' ",
|
||||
while fields from tables should be specified like " your_field_here "
|
||||
*/
|
||||
{% macro dim_global() %}
|
||||
{{ return({"dimension": "'global'", "dimension_value": "'global'"}) }}
|
||||
{% endmacro %}
|
||||
{% macro dim_billing_country() %}
|
||||
{{
|
||||
return(
|
||||
{
|
||||
"dimension": "'by_billing_country'",
|
||||
"dimension_value": "main_billing_country_iso_3_per_deal",
|
||||
}
|
||||
)
|
||||
}}
|
||||
{% endmacro %}
|
||||
{% macro dim_number_of_listings() %}
|
||||
{{
|
||||
return(
|
||||
{
|
||||
"dimension": "'by_number_of_listings'",
|
||||
"dimension_value": "active_accommodations_per_deal_segmentation",
|
||||
}
|
||||
)
|
||||
}}
|
||||
{% endmacro %}
|
||||
{% macro dim_deal() %}
|
||||
{{ return({"dimension": "'by_deal'", "dimension_value": "id_deal"}) }}
|
||||
{% endmacro %}
|
||||
{% macro dim_dash() %}
|
||||
{{ return({"dimension": "'by_dash_source'", "dimension_value": "dash_source"}) }}
|
||||
{% endmacro %}
|
||||
|
||||
/*
|
||||
Macro: get_kpi_dimensions_per_model
|
||||
|
||||
Provides a general assignemnt for the Dimensions available for each KPI
|
||||
model. Keep in mind that these assignations need to be previously
|
||||
declared.
|
||||
|
||||
*/
|
||||
{% macro get_kpi_dimensions_per_model(entity_name) %}
|
||||
|
||||
{# Base dimensions shared by all models #}
|
||||
{% set base_dimensions = [
|
||||
dim_global(),
|
||||
dim_number_of_listings(),
|
||||
dim_billing_country(),
|
||||
dim_deal(),
|
||||
] %}
|
||||
|
||||
{# Initialize a list to hold any model-specific dimensions #}
|
||||
{% set additional_dimensions = [] %}
|
||||
|
||||
{# Add entity-specific dimensions #}
|
||||
{% if entity_name == "CREATED_BOOKINGS" %}
|
||||
{% set additional_dimensions = [dim_dash()] %}
|
||||
{% endif %}
|
||||
|
||||
{# Combine base dimensions with additional dimensions for the specific model #}
|
||||
{% set dimensions = base_dimensions + additional_dimensions %}
|
||||
{{ return(dimensions) }}
|
||||
{% endmacro %}
|
||||
|
|
|
|||
21
macros/tests/assert_dimension_completeness.sql
Normal file
21
macros/tests/assert_dimension_completeness.sql
Normal file
|
|
@ -0,0 +1,21 @@
|
|||
{% test assert_dimension_completeness(
|
||||
model, column_name, metric_column_name, global_dimension_name="global"
|
||||
) %}
|
||||
|
||||
with
|
||||
sum_by_dimension as (
|
||||
select {{ column_name }}, sum({{ metric_column_name }}) as sum_metric
|
||||
from {{ model }}
|
||||
group by {{ column_name }}
|
||||
),
|
||||
global_sum as (
|
||||
select sum({{ metric_column_name }}) as total_metric
|
||||
from {{ model }}
|
||||
where {{ column_name }} = '{{ global_dimension_name }}'
|
||||
)
|
||||
|
||||
select *
|
||||
from sum_by_dimension, global_sum
|
||||
where sum_by_dimension.sum_metric != global_sum.total_metric
|
||||
|
||||
{% endtest %}
|
||||
Loading…
Add table
Add a link
Reference in a new issue