counterweight/data-dwh-dbt-project
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Merged PR 2689: KPIs by Billing Country

Browse source 
# Description

Adds Billing Country dimension in KPIs, but does not expose them to reporting yet.
Silly thing, based on the macros I built, I cannot make incremental changes unless changing all models. This will need to be adapted, happy to hear your thoughts on how we do it.
Additionally, I have lack of performance of the model `mtd_guest_payments_metrics`. It takes around 5 min to execute, but technically the end-to-end runs in one shoot without breaking.
It's a complex PR because it changes many files, but you will see that:
* It mostly changes the join conditions for the dimensions or the schema tests,
* I tried to be very careful and add things step-by-step in the commits.

Goal is NOT to complete the PR yet until we see how we can improve performance. I can say though that data end-to-end looks ok to me, but would benefit from checking with production data for the new dimension

Update 30th Aug
* Added a new commit that includes `id_user_host` in `int_core__verification_payments`. Happy to discuss if it makes sense or not. But it changes the execution from ~600 sec to ~6 sec because it avoids a massive repeated join with `verification_requests`.

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models. **To check because of performance issues**

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #19082
This commit is contained in:
Oriol Roqué Paniagua 2024-09-04 10:17:12 +00:00
parent 940896824f
commit 556a52e991
12 changed files with 95 additions and 6 deletions
Download patch file Download diff file Expand all files Collapse all files

17
models/intermediate/xero/int_xero__mtd_invoicing_metrics.sql
View file

@ -34,6 +34,7 @@ with
select * from {{ ref("int_dates_mtd_by_dimension") }}
),
int_xero__contacts as (select * from {{ ref("int_xero__contacts") }}),
int_core__user_host as (select * from {{ ref("int_core__user_host") }}),
resolution_host_payment as (
{% for dimension in dimensions %}
select
@ -62,6 +63,12 @@ with
int_core__mtd_accommodation_segmentation mas
on c.id_deal = mas.id_deal
and d.date = mas.date
{% elif dimension.dimension == "'by_billing_country'" %}
inner join int_xero__contacts c on c.id_contact = bt.id_contact
inner join
int_core__user_host u
on c.id_deal = u.id_deal
and u.billing_country_iso_3 is not null
{% endif %}
group by 1, 2, 3
{% if not loop.last %}
@ -99,6 +106,11 @@ with
int_core__mtd_accommodation_segmentation mas
on sdm.id_deal = mas.id_deal
and d.date = mas.date
{% elif dimension.dimension == "'by_billing_country'" %}
inner join
int_core__user_host u
on sdm.id_deal = u.id_deal
and u.billing_country_iso_3 is not null
{% endif %}
where
upper(sdm.document_status) in {{ relevant_document_statuses }}
@ -156,6 +168,11 @@ with
int_core__mtd_accommodation_segmentation mas
on sdm.id_deal = mas.id_deal
and d.date = mas.date
{% elif dimension.dimension == "'by_billing_country'" %}
inner join
int_core__user_host u
on sdm.id_deal = u.id_deal
and u.billing_country_iso_3 is not null
{% endif %}
where
upper(sdm.document_status) in {{ relevant_document_statuses }}