counterweight/data-dwh-dbt-project
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Merged PR 3482: Adapts date dimension skeleton for Main KPIs

Browse source 
# Description

New model:
* int_kpis__agg_dates_main_kpis - Serves as the skeleton of dates and dimensions for Main KPIs. It's aggregated since it follows a similar aggregation strategy. It's a single model to feed both Main KPIs visualisations. Note boolean fields are real booleans (true/false) while before these were integers (1/0). This also affects downstream models.

Main KPIs flow adaptations to new skeleton model:
* int_monthly_aggregated_metrics_history_by_deal
* int_monthly_churn_metrics - additionally, calls usual KPIs macro instead of old one
* int_mtd_vs_previous_year_metrics

Reporting changes to ensure report is not down:
* mtd_aggregated_metrics - adaptations on booleans (true-1, false-0)

Cleaning:
* get_kpi_dimensions macro is no longer used
* int_dates_by_deal model and schema entry
* int_dates_mtd_by_dimension model and schema entry
* int_dates_mtd model and schema entry

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #23763
This commit is contained in:
Oriol Roqué Paniagua 2024-11-11 15:57:37 +00:00
parent 2f80642f6c
commit 9ba0edb82d
12 changed files with 197 additions and 416 deletions
Download patch file Download diff file Expand all files Collapse all files

243
models/intermediate/cross/schema.yml
View file

@ -175,151 +175,6 @@ models:
tests:
- not_null
- name: int_dates_mtd
description: |
This model provides Month-To-Date (MTD) necessary dates for MTD-based models to work.
- For month-to-month complete information, it retrieves all end month dates that have elapsed since 2020.
- For month-to-date information, it retrieves the days of the current month of this year up to yesterday.
Additionally, it also gets the days of its equivalent month from last year previous the current day of month of today.
Example:
Imagine we have are at 4th June 2024.
- We will get the dates for 1st, 2nd, 3rd of June 2024.
- We will also get the dates for 1st, 2nd, 3rd of June 2023.
- We will get all end of months from 2020 to yesterday,
i.e., 31st January 2020, 29th February 2020, ..., 30th April 2024, 31st May 2024.
deprecation_date: 2024-11-30
columns:
- name: year
data_type: int
description: Year number of the given date.
tests:
- not_null
- name: month
data_type: int
description: Month number of the given date.
tests:
- not_null
- name: day
data_type: int
description: Day monthly number of the given date.
tests:
- not_null
- name: is_end_of_month
data_type: boolean
description: Is end of month, 1 for yes, 0 for no.
tests:
- not_null
- name: is_current_month
data_type: boolean
description: |
Checks if the date is within the current executed month,
1 for yes, 0 for no.
tests:
- not_null
- name: first_day_month
data_type: date
description: |
First day of the month correspoding to the date field.
It comes from int_dates_mtd logic.
tests:
- not_null
- name: date
data_type: date
description: |
Main date for the computation, that is used for filters.
It's the primary key for this model.
tests:
- not_null
- unique
- name: int_dates_by_deal
description: |
This model provides the necessary dates for each deal for deal-based KPIs models to work.
It only considers those dates starting from when the host user of the deal was first available.
deprecation_date: 2024-11-30
tests:
- dbt_utils.unique_combination_of_columns:
combination_of_columns:
- date
- id_deal
columns:
- name: year
data_type: int
description: Year number of the given date.
tests:
- not_null
- name: month
data_type: int
description: Month number of the given date.
tests:
- not_null
- name: day
data_type: int
description: Day monthly number of the given date.
tests:
- not_null
- name: last_day_month
data_type: date
description: |
Last day of the month correspoding to the date field.
It comes from int_dates_mtd logic.
tests:
- not_null
- name: first_day_month
data_type: date
description: |
First day of the month correspoding to the date field.
It comes from int_dates_mtd logic.
tests:
- not_null
- name: date
data_type: date
description: |
Main date for the computation, that is used for filters.
It's the primary key for this model.
tests:
- not_null
- name: id_deal
data_type: string
description: |
Main identifier of the B2B clients. A deal can have multiple hosts.
A host should usually have a deal, but it does not happen on all cases.
In this KPI reporting we force that Deal is not null to avoid potential
data quality issues.
tests:
- not_null
- name: main_deal_name
data_type: string
description: |
Main name for this ID deal.
tests:
- not_null
- name: main_billing_country_iso_3_per_deal
data_type: string
description: |
ISO 3166-1 alpha-3 main country code in which the Deal is billed.
In some cases it's null.
- name: int_mtd_aggregated_metrics
description: |
The `int_mtd_aggregated_metrics` model aggregates multiple metrics on a year, month, and day basis.
@ -372,16 +227,14 @@ models:
- name: first_day_month
data_type: date
description: |
first day of the month correspoding to the date field.
It comes from int_dates_mtd logic.
first day of the month corresponding to the date field.
tests:
- not_null
- name: date
data_type: date
description: |
main date for the computation, that is used for filters.
It comes from int_dates_mtd logic.
main date for the computation, that is used for filters.
tests:
- not_null
@ -405,7 +258,7 @@ models:
data_type: date
description: |
corresponds to the date of the previous year, with respect to the field date.
It comes from int_dates_mtd logic. It's only displayed for information purposes,
It's only displayed for information purposes,
should not be needed for reporting.
- name: metric
@ -506,96 +359,6 @@ models:
ISO 3166-1 alpha-3 main country code in which the Deal is billed.
In some cases it's null.
- name: int_dates_mtd_by_dimension
description: |
This model provides Month-To-Date (MTD) necessary dates, dimension and dimension_values
for MTD-based models to work.
It provides the basic "empty" structure from which metrics will be built upon. This is, on
top of the Date that characterises int_dates_mtd, including the dimensions and their
respective values that should appear in any mtd metric model.
Example:
- For the "global" dimension, we will only have the "global" dimension value.
- For the "by_number_of_listing" dimension, we will have different values
according to the segments defined, ex: 0, 1-5, 6-20, etc.
... and so on and forth for any available dimension. These combinations should appear
for each date of the MTD models.
deprecation_date: 2024-11-30
tests:
- dbt_utils.unique_combination_of_columns:
combination_of_columns:
- date
- dimension
- dimension_value
columns:
- name: year
data_type: int
description: Year number of the given date.
tests:
- not_null
- name: month
data_type: int
description: Month number of the given date.
tests:
- not_null
- name: day
data_type: int
description: Day monthly number of the given date.
tests:
- not_null
- name: is_end_of_month
data_type: boolean
description: Is end of month, 1 for yes, 0 for no.
tests:
- not_null
- name: is_current_month
data_type: boolean
description: |
Checks if the date is within the current executed month,
1 for yes, 0 for no.
tests:
- not_null
- name: first_day_month
data_type: date
description: |
First day of the month correspoding to the date field.
It comes from int_dates_mtd logic.
tests:
- not_null
- name: date
data_type: date
description: |
Main date for the computation, metrics include monthly information
until this date.
tests:
- not_null
- name: dimension
data_type: string
description: The dimension or granularity of the metrics.
tests:
- accepted_values:
values:
- global
- by_number_of_listings
- by_billing_country
- name: dimension_value
data_type: string
description: The value or segment available for the selected dimension.
tests:
- not_null
- name: int_monthly_growth_score_by_deal
description: |
The main goal of this model is to provide a growth score by deal and month.