counterweight/data-dwh-dbt-project
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Merged PR 3654: New Dash KPIs - Chargeable Services (Revenue)

Browse source 
# Description

This PR aims to do 2 things:

1. Create the first metric daily model for New Dash - Chargeable Services
* PK at Booking, Date and Service. I added a few more dimensions such as Accommodation and Business Type. The idea is that while Daily Unique Bookings/Accommodations Charged will be close (if not the same) as Charged Services, by having the ID we can compute Monthly/Weekly Unique Bookings/Accommodations Charged in a proper manner. Besides this, we would still compute additive metrics in the future such as the sum of Charged Services and the sum of Service Total Price in GBP.
* `IMPORTANT`: as discussed in the daily, I changed New Pricing models containing "Charged" columns to "Chargeable". This affects the new model for New Dash KPIs, but also `int_core__booking_summary` and `int_core__booking_service_detail`.

2. Small fixes on New Dash - Created Services.
* Mainly, there were some inconsistencies with what was actually written (and really applied) in the schema entry as for what was the PK of the model vs. what was stated in the model. I just re-ordered the columns and re-specified correctly the PK.

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #20809
This commit is contained in:
Oriol Roqué Paniagua 2024-11-26 10:14:37 +00:00
parent 645958239e
commit 2251a45540
8 changed files with 203 additions and 51 deletions
Download patch file Download diff file Expand all files Collapse all files

8
models/intermediate/core/int_core__booking_service_detail.sql
View file

@ -189,8 +189,8 @@ select
service_unit_price_local_curr,
service_unit_price_local_curr * er.rate as service_unit_price_in_gbp,
service_total_price_in_gbp,
first_chargeable_date_utc as service_first_charge_date_utc,
last_chargeable_date_utc as service_last_charge_date_utc,
first_chargeable_date_utc as service_first_chargeable_date_utc,
last_chargeable_date_utc as service_last_chargeable_date_utc,
is_missing_currency_code,
is_booking_cancelled,
case
@ -201,14 +201,14 @@ select
when first_chargeable_date_utc is null or last_chargeable_date_utc is null
then true
else false
end as is_missing_charge_date,
end as is_missing_chargeable_date,
case
when first_chargeable_date_utc is null or last_chargeable_date_utc is null
then null
when first_chargeable_date_utc = last_chargeable_date_utc
then true
else false
end as is_service_charged_in_a_single_date
end as is_service_chargeable_in_a_single_date
from service_aggregation bsd
left join
int_simple_exchange_rates er

14
models/intermediate/core/int_core__booking_summary.sql
View file

@ -31,11 +31,11 @@ select
bts.new_dash_version,
bts.user_in_new_dash_since_timestamp_at_utc,
sum(bsd.service_total_price_in_gbp) as booking_total_price_in_gbp,
min(bsd.service_first_charge_date_utc) as first_service_charge_date_utc,
max(bsd.service_last_charge_date_utc) as last_service_charge_date_utc,
min(bsd.service_detail_created_at_utc) as first_service_detail_created_at_utc,
max(bsd.service_detail_created_at_utc) as last_service_detail_created_at_utc,
max(bsd.service_detail_updated_at_utc) as last_service_detail_updated_at_utc,
min(bsd.service_first_chargeable_date_utc) as service_first_chargeable_date_utc,
max(bsd.service_last_chargeable_date_utc) as service_last_chargeable_date_utc,
min(bsd.service_detail_created_at_utc) as service_first_created_at_utc,
max(bsd.service_detail_created_at_utc) as service_last_created_at_utc,
max(bsd.service_detail_updated_at_utc) as service_last_updated_at_utc,
count(distinct bsd.id_booking_service_detail) as number_of_applied_services,
count(
distinct case
@ -50,10 +50,10 @@ select
case
when
sum(bsd.service_total_price_in_gbp) > 0
and min(bsd.service_first_charge_date_utc) is not null
and min(bsd.service_first_chargeable_date_utc) is not null
then true
else false
end as is_booking_charged,
end as is_booking_chargeable,
case
when sum(case when bsd.is_missing_currency_code then 1 else 0 end) > 0
then true

63
models/intermediate/core/schema.yml
View file

@ -3640,10 +3640,10 @@ models:
description: |
This is an informative field, use at your own risk.
Identifies the service unit price converted to GBP. If the date of
the charge exists, it will use the latest charge date for the exchange
the chargeable exists, it will use the latest chargeable date for the exchange
rate computation. If not, it will use the moment this service detail line
was created. It can be null. It can vary over time due to the currency rate
estimation in the future for charge dates, and can vary also because of
estimation in the future for chargeable dates, and can vary also because of
changes in the date used.
- name: service_total_price_in_gbp
@ -3652,24 +3652,24 @@ models:
Identifies the current total price of that service in a given
booking converted in GBP. Can be null. Can vary over time depending
on the service status, payments, etc, as well as it can vary over
time until the charge date due to the currency rate estimation in
time until the chargeable date due to the currency rate estimation in
the future.
- name: service_first_charge_date_utc
- name: service_first_chargeable_date_utc
data_type: date
description: |
Identifies the first date in which the billing item for this service
is supposed to be charged. It can be null if no billing item is
available. If there's only one billing item, this field will contain
the same value as service_last_charge_date_utc.
the same value as service_last_chargeable_date_utc.
- name: service_last_charge_date_utc
- name: service_last_chargeable_date_utc
data_type: date
description: |
Identifies the last date in which the billing item for this service
is supposed to be charged. It can be null if no billing item is
available. If there's only one billing item, this field will contain
the same value as service_first_charge_date_utc.
the same value as service_first_chargeable_date_utc.
- name: is_missing_currency_code
data_type: boolean
@ -3703,22 +3703,22 @@ models:
tests:
- not_null
- name: is_missing_charge_date
- name: is_missing_chargeable_date
data_type: boolean
description: |
Flag to identify if the service has no charge date.
Flag to identify if the service has no chargeable date.
Cannot be null.
tests:
- not_null
- name: is_service_charged_in_a_single_date
- name: is_service_chargeable_in_a_single_date
data_type: boolean
description: |
Flag to identify if the service is charged in a single date.
If True, it means that service_first_charge_date_utc contains
the same date as service_last_charge_date_utc. If False, these
Flag to identify if the service is chargeable in a single date.
If True, it means that service_first_chargeable_date_utc contains
the same date as service_last_chargeable_date_utc. If False, these
fields will contain different dates. It can be null, thus meaning
there's no charge date at all.
there's no chargeable date at all.
- name: int_core__booking_summary
description: |
@ -3912,22 +3912,22 @@ models:
Identifies the current total price of the booking by adding up the
prices of each service applied to this booking, converted in GBP.
Can be null. Can vary over time depending on the service status,
payments, etc, as well as it can vary over time until the charge
payments, etc, as well as it can vary over time until the chargeable
date due to the currency rate estimation in the future.
- name: first_service_charge_date_utc
- name: service_first_chargeable_date_utc
data_type: date
description: |
Identifies the first moment in time in which the first
service applied to this booking is charged.
service applied to this booking is supposed to be charged.
- name: last_service_charge_date_utc
- name: service_last_chargeable_date_utc
data_type: date
description: |
Identifies the last moment in time in which the last
service applied to this booking is charged.
service applied to this booking is supposed to be charged.
- name: first_service_detail_created_at_utc
- name: service_first_created_at_utc
data_type: timestamp
description: |
Timestamp corresponding to the first creation of a Service
@ -3935,7 +3935,7 @@ models:
tests:
- not_null
- name: last_service_detail_created_at_utc
- name: service_last_created_at_utc
data_type: timestamp
description: |
Timestamp corresponding to the latest creation of a Service
@ -3943,7 +3943,7 @@ models:
tests:
- not_null
- name: last_service_detail_updated_at_utc
- name: service_last_updated_at_utc
data_type: timestamp
description: |
Timestamp corresponding to the latest update on any Service
@ -3968,17 +3968,18 @@ models:
Total number of Services different from Basic Screening
applied to this Booking.
- name: is_booking_charged
- name: is_booking_chargeable
data_type: boolean
description: |
Flag to identify it the Booking is charged or not.
In essence, it solves the question: are we getting money out
of this booking, or not?
To be considered as charged, a charge date needs to exist as
well as the total price converted to GBP needs to be strictly
greater than 0. The fact that a booking is not charged does
not necessarily mean that it won't be in the future. It cannot
be null.
Flag to identify it the Booking is chargeable or not.
In essence, it solves the question: are we supposed to get
money out of this booking, or not?
To be considered as chargeable, a chargeable date needs to exist
as well as the total price converted to GBP needs to be strictly
greater than 0. The fact that a booking is not chargeable does
not necessarily mean that it won't be in the future. Similarly, if
the booking is chargeable it does not necessarily mean that is actually
charged. It cannot be null.
tests:
- not_null

6
models/intermediate/kpis/int_kpis__agg_daily_new_dash_created_services.sql
View file

@ -1,10 +1,6 @@
{% set dimensions = get_kpi_dimensions_per_model("NEW_DASH_CREATED_SERVICES") %}
{{
config(
materialized="table", unique_key=["end_date", "dimension", "dimension_value"]
)
}}
{{ config(materialized="table", unique_key=["date", "dimension", "dimension_value"]) }}
{% for dimension in dimensions %}

36
models/intermediate/kpis/int_kpis__metric_daily_new_dash_chargeable_services.sql Normal file
View file

@ -0,0 +1,36 @@
{{ config(materialized="table", unique_key=["id_booking", "service_name", "date"]) }}
select
-- Unique Key --
icbs.id_booking,
icbsd.service_last_chargeable_date_utc as date,
coalesce(icbsd.service_name) as service_name,
-- Dimensions --
coalesce(icbs.id_deal, 'UNSET') as id_deal,
icbs.id_accommodation,
case when icbsd.is_upgraded_service then 'YES' else 'NO' end as is_upgraded_service,
coalesce(icbsd.service_business_type, 'UNSET') as service_business_type,
coalesce(icbs.new_dash_version, 'UNSET') as new_dash_version,
coalesce(
icd.main_billing_country_iso_3_per_deal, 'UNSET'
) as main_billing_country_iso_3_per_deal,
coalesce(
icmas.active_accommodations_per_deal_segmentation, 'UNSET'
) as active_accommodations_per_deal_segmentation,
-- Metrics --
count(distinct icbsd.id_booking_service_detail) as chargeable_services,
sum(icbsd.service_total_price_in_gbp) as service_total_price_in_gbp
from {{ ref("int_core__booking_summary") }} as icbs
inner join
{{ ref("int_core__booking_service_detail") }} as icbsd
on icbs.id_booking = icbsd.id_booking
left join {{ ref("int_core__deal") }} as icd on icbs.id_deal = icd.id_deal
left join
{{ ref("int_kpis__dimension_daily_accommodation") }} as icmas
on icbs.id_deal = icmas.id_deal
and date(icbsd.service_last_chargeable_date_utc) = icmas.date
where
icbs.is_user_in_new_dash = true
and icbs.is_missing_id_deal = false
and icbsd.service_last_chargeable_date_utc
>= icbs.user_in_new_dash_since_timestamp_at_utc
group by 1, 2, 3, 4, 5, 6, 7, 8, 9, 10

4
models/intermediate/kpis/int_kpis__metric_monthly_new_dash_created_services.sql
View file

@ -4,7 +4,7 @@
unique_key=[
"end_date",
"service_name",
"new_dash_version",
"id_deal",
"active_accommodations_per_deal_segmentation",
],
)
@ -16,9 +16,9 @@ select
d.date as end_date,
cs.service_name,
cs.active_accommodations_per_deal_segmentation,
cs.id_deal,
-- Dimensions --
cs.new_dash_version,
cs.id_deal,
cs.is_upgraded_service,
cs.main_billing_country_iso_3_per_deal,
-- Metrics --

4
models/intermediate/kpis/int_kpis__metric_weekly_new_dash_created_services.sql
View file

@ -4,7 +4,7 @@
unique_key=[
"end_date",
"service_name",
"new_dash_version",
"id_deal",
"active_accommodations_per_deal_segmentation",
],
)
@ -16,9 +16,9 @@ select
d.date as end_date,
cs.service_name,
cs.active_accommodations_per_deal_segmentation,
cs.id_deal,
-- Dimensions --
cs.new_dash_version,
cs.id_deal,
cs.is_upgraded_service,
cs.main_billing_country_iso_3_per_deal,
-- Metrics --

119
models/intermediate/kpis/schema.yml
View file

@ -5842,3 +5842,122 @@ models:
This is an approximation to booking count since different services can
apply to the same booking and these do not need to be created in the same
time period. Therefore, it's not an additive metric.
- name: int_kpis__metric_daily_new_dash_chargeable_services
description: |
This model computes the Daily Chargeable Services at the deepest granularity.
It only retrieves services that come from users that are in New Dash, as well
as it only considers services chargeable after the user has moved to New Dash.
The unique key corresponds to the deepest granularity of the model,
in this case:
- date,
- id_booking,
- service_name
tests:
- dbt_utils.unique_combination_of_columns:
combination_of_columns:
- date
- id_booking
- service_name
columns:
- name: date
data_type: date
description: Date of when Services are supposed to be charged.
tests:
- not_null
- name: id_booking
data_type: bigint
description: Unique identifier of the Booking.
tests:
- not_null
- name: service_name
data_type: string
description: Name of the chargeable service.
tests:
- not_null
- name: id_deal
data_type: string
description: Unique identifier of an account.
tests:
- not_null
- name: id_accommodation
data_type: bigint
description: Unique identifier of an accommodation, or listing.
tests:
- not_null
- name: is_upgraded_service
data_type: string
description: |
Whether the service is an upgraded version of the
default. In other words, if it's not Basic Screening.
tests:
- not_null
- accepted_values:
values:
- "YES"
- "NO"
- name: service_business_type
data_type: string
description: |
Identifies the service type (Screening, Deposit Management, Protection)
according to New Pricing documentation.
Cannot be null.
tests:
- not_null
- accepted_values:
values:
- "SCREENING"
- "PROTECTION"
- "DEPOSIT_MANAGEMENT"
- "UNSET"
- name: new_dash_version
data_type: string
description: |
The version of the New Dash. It corresponds to the
release or migration phase from user point of view.
tests:
- not_null
- name: active_accommodations_per_deal_segmentation
data_type: string
description: |
Segment value based on the number of listings booked in 12 months
for a given deal and date.
tests:
- not_null
- accepted_values:
values:
- "0"
- "01-05"
- "06-20"
- "21-60"
- "61+"
- "UNSET"
- name: main_billing_country_iso_3_per_deal
data_type: string
description: |
Main billing country of the host aggregated at Deal level.
tests:
- not_null
- name: chargeable_services
data_type: integer
description: |
Count of daily chargeable services in a given date and per specified
dimension.
- name: service_total_price_in_gbp
data_type: decimal
description: |
Sum of the total prices of the chargeable services in a given date and
per specified dimension, in GBP.