counterweight/data-dwh-dbt-project
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Merged PR 3373: Setting current KPIs models for deprecation

Browse source 
# Description

Sets up warning for KPIs models that will be deprecated, which are:
* 13 in core
* 2 in xero
* 5 in cross

I will keep alive the cross models that handle the final aggregations for Main KPIs for the time being, as well as the newly created Churn model that has a dependency on the monthly by deal to be filled into the mtd flow. I think handling exposure logic for Main KPIs could be a separated migration.

In other words, this is already quite a bit to migrate.

# Checklist

 **I just checked that dbt compiles correctly**

- [NA] The edited models and dependants run properly with production data.
- [NA] The edited models are sufficiently documented.
- [NA] The edited models contain PK tests, and I've ran and passed them.
- [NA] I have checked for DRY opportunities with other models and docs.
- [NA] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #23268
This commit is contained in:
Oriol Roqué Paniagua 2024-10-30 08:54:59 +00:00
parent 5c1f6f88f2
commit 450975301a
3 changed files with 70 additions and 30 deletions
Download patch file Download diff file Expand all files Collapse all files

26
models/intermediate/core/schema.yml
View file

@ -448,6 +448,8 @@ models:
- date
- id_deal
deprecation_date: 2024-11-30
columns:
- name: date
data_type: date
@ -472,6 +474,8 @@ models:
- date
- id_deal
deprecation_date: 2024-11-30
columns:
- name: date
data_type: date
@ -496,6 +500,8 @@ models:
- date
- id_deal
deprecation_date: 2024-11-30
columns:
- name: date
data_type: date
@ -516,6 +522,8 @@ models:
month and in the days necessary for the Month-to-Date computation of the
current month.
deprecation_date: 2024-11-30
tests:
- dbt_utils.unique_combination_of_columns:
combination_of_columns:
@ -553,6 +561,8 @@ models:
month and in the days necessary for the Month-to-Date computation of the
current month.
deprecation_date: 2024-11-30
tests:
- dbt_utils.unique_combination_of_columns:
combination_of_columns:
@ -590,6 +600,8 @@ models:
month and in the days necessary for the Month-to-Date computation of the
current month.
deprecation_date: 2024-11-30
tests:
- dbt_utils.unique_combination_of_columns:
combination_of_columns:
@ -627,6 +639,8 @@ models:
month and in the days necessary for the Month-to-Date computation of the
current month.
deprecation_date: 2024-11-30
tests:
- dbt_utils.unique_combination_of_columns:
combination_of_columns:
@ -663,6 +677,8 @@ models:
It's used for the business KPIs. Data is aggregated at the last day of the month and in the
days necessary for the Month-to-Date computation of the current month.
deprecation_date: 2024-11-30
tests:
- dbt_utils.unique_combination_of_columns:
combination_of_columns:
@ -699,6 +715,8 @@ models:
It's used for the business KPIs. Data is aggregated at the last day of the month and in the
days necessary for the Month-to-Date computation of the current month.
deprecation_date: 2024-11-30
tests:
- dbt_utils.unique_combination_of_columns:
combination_of_columns:
@ -735,6 +753,8 @@ models:
It's used for the business KPIs. Data is aggregated at the last day of the month and in the
days necessary for the Month-to-Date computation of the current month.
deprecation_date: 2024-11-30
tests:
- dbt_utils.unique_combination_of_columns:
combination_of_columns:
@ -776,6 +796,8 @@ models:
- date
- id_deal
deprecation_date: 2024-11-30
columns:
- name: date
data_type: date
@ -1338,6 +1360,8 @@ models:
- date
- id_accommodation
deprecation_date: 2024-11-30
columns:
- name: date
data_type: date
@ -1418,6 +1442,8 @@ models:
These segments are inspired from the ones RevOps team uses, but the associated deals and listings volume will differ
since the listing activity logic considered is different.
deprecation_date: 2024-11-30
tests:
- dbt_utils.unique_combination_of_columns:
combination_of_columns:

10
models/intermediate/cross/schema.yml
View file

@ -189,6 +189,8 @@ models:
- We will get all end of months from 2020 to yesterday,
i.e., 31st January 2020, 29th February 2020, ..., 30th April 2024, 31st May 2024.
deprecation_date: 2024-11-30
columns:
- name: year
data_type: int
@ -244,6 +246,8 @@ models:
This model provides the necessary dates for each deal for deal-based KPIs models to work.
It only considers those dates starting from when the host user of the deal was first available.
deprecation_date: 2024-11-30
tests:
- dbt_utils.unique_combination_of_columns:
combination_of_columns:
@ -518,6 +522,8 @@ models:
... and so on and forth for any available dimension. These combinations should appear
for each date of the MTD models.
deprecation_date: 2024-11-30
tests:
- dbt_utils.unique_combination_of_columns:
combination_of_columns:
@ -1444,6 +1450,8 @@ models:
this happens likely because of misconfiguration between Hubspot and Core. This should be reported to increase
data quality.
deprecation_date: 2024-11-30
tests:
- dbt_utils.unique_combination_of_columns:
combination_of_columns:
@ -1534,6 +1542,8 @@ models:
It's used for the business KPIs. Data is aggregated at the last day of the month and in the
days necessary for the Month-to-Date computation of the current month.
deprecation_date: 2024-11-30
tests:
- dbt_utils.unique_combination_of_columns:
combination_of_columns:

64
models/intermediate/xero/schema.yml
View file

@ -3,34 +3,34 @@ version: 2
models:
- name: int_xero__invoices
columns:
- name: id_invoice
tests:
- not_null
- unique
- name: id_invoice
tests:
- not_null
- unique
- name: int_xero__credit_notes
columns:
- name: id_credit_note
tests:
- not_null
- unique
- name: id_credit_note
tests:
- not_null
- unique
- name: int_xero__invoice_line_items
columns:
- name: id_line_item
tests:
- not_null
- unique
- name: id_line_item
tests:
- not_null
- unique
- name: int_xero__credit_note_line_items
columns:
- name: id_line_item
tests:
- not_null
- unique
- name: id_line_item
tests:
- not_null
- unique
- name: int_xero__bank_transactions
columns:
- name: id_bank_transaction
tests:
- not_null
- unique
- name: id_bank_transaction
tests:
- not_null
- unique
- name: int_xero__sales_denom_mart
description: |
@ -58,7 +58,7 @@ models:
Fields are not documented here: you can find more details in upstream
models.
columns:
- name: id_line_item
data_type: text
@ -79,9 +79,9 @@ models:
- not_null
- accepted_values:
values:
- 'invoice'
- 'credit note'
- "invoice"
- "credit note"
- name: id_account
data_type: text
description: ""
@ -265,12 +265,14 @@ models:
This model contains the historic information regarding invoicing metrics for each deal id.
It's used for the business KPIs in the view by deal id. Data is aggregated at the last
day of the month, or up to yesterday if it's the current month.
tests:
tests:
- dbt_utils.unique_combination_of_columns:
combination_of_columns:
- date
- id_deal
deprecation_date: 2024-11-30
columns:
- name: date
data_type: date
@ -280,7 +282,7 @@ models:
- name: id_deal
data_type: character varying
description: Id of the deal associated to the host.
description: Id of the deal associated to the host.
tests:
- not_null
@ -291,13 +293,15 @@ models:
It's used for the business KPIs. Data is aggregated at the last day of the month and in the
days necessary for the Month-to-Date computation of the current month.
tests:
deprecation_date: 2024-11-30
tests:
- dbt_utils.unique_combination_of_columns:
combination_of_columns:
- date
- dimension
- dimension_value
columns:
- name: date
data_type: date
@ -310,11 +314,11 @@ models:
description: The dimension or granularity of the metrics.
tests:
- accepted_values:
values:
values:
- global
- by_number_of_listings
- by_billing_country
- name: dimension_value
data_type: string
description: The value or segment available for the selected dimension.