counterweight/data-dwh-dbt-project
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Merged PR 3434: update assert_dimension_completeness test

Browse source 
# Description

Update assert_dimension_completeness test
I tested the models and everything seems to be working correctly
I tried to modify the test as you told me but I couldn't make it work Uri
`I think this could work with fewer repeated lines as well:
select * from discrepancy_{{ metric_column_name }}
{% if not loop.first %}
union all
{% endif %}`

# Checklist

- [x] The edited models and dependants run properly with production data.
- [x] The edited models are sufficiently documented.
- [x] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

update assert_dimension_completeness test
This commit is contained in:
Joaquin Ossa 2024-11-06 07:59:44 +00:00
commit 1cd9f88a45
2 changed files with 102 additions and 98 deletions
Download patch file Download diff file Expand all files Collapse all files

26
macros/tests/assert_dimension_completeness.sql
View file

@ -1,21 +1,35 @@
{% test assert_dimension_completeness( {% test assert_dimension_completeness(
model, column_name, metric_column_name, global_dimension_name="global" model, column_name, metric_column_names, global_dimension_name="global"
) %} ) %}
with with
sum_by_dimension as (
{% for metric_column_name in metric_column_names %}
sum_by_dimension_{{ metric_column_name }} as (
select {{ column_name }}, sum({{ metric_column_name }}) as sum_metric select {{ column_name }}, sum({{ metric_column_name }}) as sum_metric
from {{ model }} from {{ model }}
group by {{ column_name }} group by {{ column_name }}
), ),
global_sum as ( global_sum_{{ metric_column_name }} as (
select sum({{ metric_column_name }}) as total_metric select sum({{ metric_column_name }}) as total_metric
from {{ model }} from {{ model }}
where {{ column_name }} = '{{ global_dimension_name }}' where {{ column_name }} = '{{ global_dimension_name }}'
),
discrepancy_{{ metric_column_name }} as (
select *
from sum_by_dimension_{{ metric_column_name }}, global_sum_{{ metric_column_name }}
where sum_by_dimension_{{ metric_column_name }}.sum_metric != global_sum_{{ metric_column_name }}.total_metric
) )
{% if not loop.last %}, {% endif %}
{% endfor %}
select * {% for metric_column_name in metric_column_names %}
from sum_by_dimension, global_sum {% if loop.first %}
where sum_by_dimension.sum_metric != global_sum.total_metric select * from discrepancy_{{ metric_column_name }}
{% else %}
union all
select * from discrepancy_{{ metric_column_name }}
{% endif %}
{% endfor %}
{% endtest %} {% endtest %}

174
models/intermediate/kpis/schema.yml
View file

@ -513,7 +513,8 @@ models:
description: The dimension or granularity of the metrics. description: The dimension or granularity of the metrics.
tests: tests:
- assert_dimension_completeness: - assert_dimension_completeness:
metric_column_name: created_bookings metric_column_names:
- created_bookings
- accepted_values: - accepted_values:
values: values:
- global - global
@ -567,7 +568,8 @@ models:
description: The dimension or granularity of the metrics. description: The dimension or granularity of the metrics.
tests: tests:
- assert_dimension_completeness: - assert_dimension_completeness:
metric_column_name: created_bookings metric_column_names:
- created_bookings
- accepted_values: - accepted_values:
values: values:
- global - global
@ -817,7 +819,8 @@ models:
description: The dimension or granularity of the metrics. description: The dimension or granularity of the metrics.
tests: tests:
- assert_dimension_completeness: - assert_dimension_completeness:
metric_column_name: created_guest_journeys metric_column_names:
- created_guest_journeys
- accepted_values: - accepted_values:
values: values:
- global - global
@ -870,7 +873,8 @@ models:
description: The dimension or granularity of the metrics. description: The dimension or granularity of the metrics.
tests: tests:
- assert_dimension_completeness: - assert_dimension_completeness:
metric_column_name: created_guest_journeys metric_column_names:
- created_guest_journeys
- accepted_values: - accepted_values:
values: values:
- global - global
@ -1119,7 +1123,8 @@ models:
description: The dimension or granularity of the metrics. description: The dimension or granularity of the metrics.
tests: tests:
- assert_dimension_completeness: - assert_dimension_completeness:
metric_column_name: started_guest_journeys metric_column_names:
- started_guest_journeys
- accepted_values: - accepted_values:
values: values:
- global - global
@ -1172,7 +1177,8 @@ models:
description: The dimension or granularity of the metrics. description: The dimension or granularity of the metrics.
tests: tests:
- assert_dimension_completeness: - assert_dimension_completeness:
metric_column_name: started_guest_journeys metric_column_names:
- started_guest_journeys
- accepted_values: - accepted_values:
values: values:
- global - global
@ -1421,7 +1427,8 @@ models:
description: The dimension or granularity of the metrics. description: The dimension or granularity of the metrics.
tests: tests:
- assert_dimension_completeness: - assert_dimension_completeness:
metric_column_name: completed_guest_journeys metric_column_names:
- completed_guest_journeys
- accepted_values: - accepted_values:
values: values:
- global - global
@ -1474,7 +1481,8 @@ models:
description: The dimension or granularity of the metrics. description: The dimension or granularity of the metrics.
tests: tests:
- assert_dimension_completeness: - assert_dimension_completeness:
metric_column_name: completed_guest_journeys metric_column_names:
- completed_guest_journeys
- accepted_values: - accepted_values:
values: values:
- global - global
@ -1723,7 +1731,8 @@ models:
description: The dimension or granularity of the metrics. description: The dimension or granularity of the metrics.
tests: tests:
- assert_dimension_completeness: - assert_dimension_completeness:
metric_column_name: guest_journeys_with_payment metric_column_names:
- guest_journeys_with_payment
- accepted_values: - accepted_values:
values: values:
- global - global
@ -1776,7 +1785,8 @@ models:
description: The dimension or granularity of the metrics. description: The dimension or granularity of the metrics.
tests: tests:
- assert_dimension_completeness: - assert_dimension_completeness:
metric_column_name: guest_journeys_with_payment metric_column_names:
- guest_journeys_with_payment
- accepted_values: - accepted_values:
values: values:
- global - global
@ -2080,13 +2090,11 @@ models:
description: The dimension or granularity of the metrics. description: The dimension or granularity of the metrics.
tests: tests:
- assert_dimension_completeness: - assert_dimension_completeness:
metric_column_name: deposit_fees_in_gbp metric_column_names:
- assert_dimension_completeness: - deposit_fees_in_gbp
metric_column_name: waiver_payments_in_gbp - waiver_payments_in_gbp
- assert_dimension_completeness: - checkin_cover_fees_in_gbp
metric_column_name: checkin_cover_fees_in_gbp - total_guest_payments_in_gbp
- assert_dimension_completeness:
metric_column_name: total_guest_payments_in_gbp
- accepted_values: - accepted_values:
values: values:
- global - global
@ -2159,13 +2167,11 @@ models:
description: The dimension or granularity of the metrics. description: The dimension or granularity of the metrics.
tests: tests:
- assert_dimension_completeness: - assert_dimension_completeness:
metric_column_name: deposit_fees_in_gbp metric_column_names:
- assert_dimension_completeness: - deposit_fees_in_gbp
metric_column_name: waiver_payments_in_gbp - waiver_payments_in_gbp
- assert_dimension_completeness: - checkin_cover_fees_in_gbp
metric_column_name: checkin_cover_fees_in_gbp - total_guest_payments_in_gbp
- assert_dimension_completeness:
metric_column_name: total_guest_payments_in_gbp
- accepted_values: - accepted_values:
values: values:
- global - global
@ -2434,7 +2440,8 @@ models:
description: The dimension or granularity of the metrics. description: The dimension or granularity of the metrics.
tests: tests:
- assert_dimension_completeness: - assert_dimension_completeness:
metric_column_name: check_out_bookings metric_column_names:
- check_out_bookings
- accepted_values: - accepted_values:
values: values:
- global - global
@ -2487,7 +2494,8 @@ models:
description: The dimension or granularity of the metrics. description: The dimension or granularity of the metrics.
tests: tests:
- assert_dimension_completeness: - assert_dimension_completeness:
metric_column_name: check_out_bookings metric_column_names:
- check_out_bookings
- accepted_values: - accepted_values:
values: values:
- global - global
@ -2736,7 +2744,8 @@ models:
description: The dimension or granularity of the metrics. description: The dimension or granularity of the metrics.
tests: tests:
- assert_dimension_completeness: - assert_dimension_completeness:
metric_column_name: billable_bookings metric_column_names:
- billable_bookings
- accepted_values: - accepted_values:
values: values:
- global - global
@ -2789,7 +2798,8 @@ models:
description: The dimension or granularity of the metrics. description: The dimension or granularity of the metrics.
tests: tests:
- assert_dimension_completeness: - assert_dimension_completeness:
metric_column_name: billable_bookings metric_column_names:
- billable_bookings
- accepted_values: - accepted_values:
values: values:
- global - global
@ -3038,7 +3048,8 @@ models:
description: The dimension or granularity of the metrics. description: The dimension or granularity of the metrics.
tests: tests:
- assert_dimension_completeness: - assert_dimension_completeness:
metric_column_name: cancelled_bookings metric_column_names:
- cancelled_bookings
- accepted_values: - accepted_values:
values: values:
- global - global
@ -3091,7 +3102,8 @@ models:
description: The dimension or granularity of the metrics. description: The dimension or granularity of the metrics.
tests: tests:
- assert_dimension_completeness: - assert_dimension_completeness:
metric_column_name: cancelled_bookings metric_column_names:
- cancelled_bookings
- accepted_values: - accepted_values:
values: values:
- global - global
@ -3417,19 +3429,14 @@ models:
The dimension or granularity of the metrics. The dimension or granularity of the metrics.
tests: tests:
- assert_dimension_completeness: - assert_dimension_completeness:
metric_column_name: created_guest_journeys_not_cancelled metric_column_names:
- assert_dimension_completeness: - created_guest_journeys_not_cancelled
metric_column_name: started_guest_journeys_not_cancelled - started_guest_journeys_not_cancelled
- assert_dimension_completeness: - completed_guest_journeys_not_cancelled
metric_column_name: completed_guest_journeys_not_cancelled - created_guest_journeys
- assert_dimension_completeness: - started_guest_journeys
metric_column_name: created_guest_journeys - completed_guest_journeys
- assert_dimension_completeness: - count_csat_score
metric_column_name: started_guest_journeys
- assert_dimension_completeness:
metric_column_name: completed_guest_journeys
- assert_dimension_completeness:
metric_column_name: count_csat_score
- accepted_values: - accepted_values:
values: values:
- global - global
@ -3694,19 +3701,14 @@ models:
The dimension or granularity of the metrics. The dimension or granularity of the metrics.
tests: tests:
- assert_dimension_completeness: - assert_dimension_completeness:
metric_column_name: created_guest_journeys_not_cancelled metric_column_names:
- assert_dimension_completeness: - created_guest_journeys_not_cancelled
metric_column_name: started_guest_journeys_not_cancelled - started_guest_journeys_not_cancelled
- assert_dimension_completeness: - completed_guest_journeys_not_cancelled
metric_column_name: completed_guest_journeys_not_cancelled - created_guest_journeys
- assert_dimension_completeness: - started_guest_journeys
metric_column_name: created_guest_journeys - completed_guest_journeys
- assert_dimension_completeness: - count_csat_score
metric_column_name: started_guest_journeys
- assert_dimension_completeness:
metric_column_name: completed_guest_journeys
- assert_dimension_completeness:
metric_column_name: count_csat_score
- accepted_values: - accepted_values:
values: values:
- global - global
@ -4037,9 +4039,9 @@ models:
description: The dimension or granularity of the metrics. description: The dimension or granularity of the metrics.
tests: tests:
- assert_dimension_completeness: - assert_dimension_completeness:
metric_column_name: xero_host_resolution_payment_count metric_column_names:
- assert_dimension_completeness: - xero_host_resolution_payment_count
metric_column_name: xero_host_resolution_amount_paid_in_gbp - xero_host_resolution_amount_paid_in_gbp
- accepted_values: - accepted_values:
values: values:
- global - global
@ -4099,9 +4101,9 @@ models:
description: The dimension or granularity of the metrics. description: The dimension or granularity of the metrics.
tests: tests:
- assert_dimension_completeness: - assert_dimension_completeness:
metric_column_name: xero_host_resolution_payment_count metric_column_names:
- assert_dimension_completeness: - xero_host_resolution_payment_count
metric_column_name: xero_host_resolution_amount_paid_in_gbp - xero_host_resolution_amount_paid_in_gbp
- accepted_values: - accepted_values:
values: values:
- global - global
@ -4490,21 +4492,15 @@ models:
description: The dimension or granularity of the metrics. description: The dimension or granularity of the metrics.
tests: tests:
- assert_dimension_completeness: - assert_dimension_completeness:
metric_column_name: xero_booking_net_fees_in_gbp metric_column_names:
- assert_dimension_completeness: - xero_booking_net_fees_in_gbp
metric_column_name: xero_listing_net_fees_in_gbp - xero_listing_net_fees_in_gbp
- assert_dimension_completeness: - xero_verification_net_fees_in_gbp
metric_column_name: xero_verification_net_fees_in_gbp - xero_operator_net_fees_in_gbp
- assert_dimension_completeness: - xero_e_deposit_net_fees_in_gbp
metric_column_name: xero_operator_net_fees_in_gbp - xero_guesty_net_fees_in_gbp
- assert_dimension_completeness: - xero_apis_net_fees_in_gbp
metric_column_name: xero_e_deposit_net_fees_in_gbp - xero_waiver_paid_back_to_host_in_gbp
- assert_dimension_completeness:
metric_column_name: xero_guesty_net_fees_in_gbp
- assert_dimension_completeness:
metric_column_name: xero_apis_net_fees_in_gbp
- assert_dimension_completeness:
metric_column_name: xero_waiver_paid_back_to_host_in_gbp
- accepted_values: - accepted_values:
values: values:
- global - global
@ -4603,21 +4599,15 @@ models:
description: The dimension or granularity of the metrics. description: The dimension or granularity of the metrics.
tests: tests:
- assert_dimension_completeness: - assert_dimension_completeness:
metric_column_name: xero_booking_net_fees_in_gbp metric_column_names:
- assert_dimension_completeness: - xero_booking_net_fees_in_gbp
metric_column_name: xero_listing_net_fees_in_gbp - xero_listing_net_fees_in_gbp
- assert_dimension_completeness: - xero_verification_net_fees_in_gbp
metric_column_name: xero_verification_net_fees_in_gbp - xero_operator_net_fees_in_gbp
- assert_dimension_completeness: - xero_e_deposit_net_fees_in_gbp
metric_column_name: xero_operator_net_fees_in_gbp - xero_guesty_net_fees_in_gbp
- assert_dimension_completeness: - xero_apis_net_fees_in_gbp
metric_column_name: xero_e_deposit_net_fees_in_gbp - xero_waiver_paid_back_to_host_in_gbp
- assert_dimension_completeness:
metric_column_name: xero_guesty_net_fees_in_gbp
- assert_dimension_completeness:
metric_column_name: xero_apis_net_fees_in_gbp
- assert_dimension_completeness:
metric_column_name: xero_waiver_paid_back_to_host_in_gbp
- accepted_values: - accepted_values:
values: values: