counterweight/data-dwh-dbt-project
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Merged PR 3038: Adding Protection tables in the scope of New Pricing/Dashboard

Browse source 
# Description

This PR creates 3 new models in staging:
- `stg_core__protection_plan`: main information of a protection plan. Here I denormalise the info coming from Protection, meaning, Protection is not a standalone model.
- `stg_core__protection_plan_to_price`: How much does a protection service cost? It's very similar to the model product_service_to_price.
- `stg_core__protection_plan_cover`: How much does a protection service protects, i.e., covers? I took the liberty of modifying the original name since it was confusing to me. Let me know if you'd like to modify the name.

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #20809
This commit is contained in:
Oriol Roqué Paniagua 2024-10-03 11:05:05 +00:00
parent 3ba65e8ed5
commit 28c554de63
5 changed files with 438 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

8
models/staging/core/_core_sources.yml
View file

@ -245,3 +245,11 @@ sources:
identifier: InvoicingMethod
- name: PaymentType
identifier: PaymentType
- name: Protection
identifier: Protection
- name: ProtectionPlan
identifier: ProtectionPlan
- name: ProtectionPlanToPrice
identifier: ProtectionPlanToPrice
- name: ProtectionPlanToCurrency
identifier: ProtectionPlanToCurrency

278
models/staging/core/schema.yml
View file

@ -705,3 +705,281 @@ models:
Timestamp of when this data was extracted into DWH.
tests:
- not_null
- name: stg_core__protection_plan
description: |
Contains all the protection plans, also referred to protection services, operated
by Truvi (Superhog) in the scope of New Pricing.
tests:
- dbt_expectations.expect_column_pair_values_A_to_be_greater_than_B:
column_A: ends_at_utc
column_B: starts_at_utc
or_equal: True
columns:
- name: id_protection_plan
data_type: integer
description: |
Identifier of the product service to price. Acts as the primary key for this table.
tests:
- unique
- not_null
- name: protection_name
data_type: string
description: |
The name given to a protection service.
- name: protection_display_name
data_type: string
description: |
The name given to a protection service, usually better fit for visualisation purposes.
- name: required_product_services_binary_tier
data_type: integer
description: |
Required product service binary identifier (as powers of 2). It's a geometric progression with a
common ratio of 2. It can be used to uniquely identify which product services are required for a
given protection. The fact that it's a power of 2 is because it identifies if a product service
is applied (true) or not applied (false).
- name: starts_at_utc
data_type: timestamp
description: |
Timestamp of when this protection plan starts to be active.
tests:
- not_null
- name: ends_at_utc
data_type: timestamp
description: |
Timestamp of when this protection plan ends to be active.
It can be null, thus meaning it's currently active.
- name: has_no_end_date
data_type: boolean
description: |
True when ends_at_utc is not set, false otherwise.
- name: created_at_utc
data_type: timestamp
description: |
Timestamp of when this protection plan was created.
- name: created_date_utc
data_type: date
description: |
Date of when this protection plan was created.
- name: updated_at_utc
data_type: timestamp
description: |
Timestamp of when this protection plan was last updated.
- name: updated_date_utc
data_type: date
description: |
Date of when this product protection plan was last updated.
- name: dwh_extracted_at_utc
data_type: timestamp
description: |
Timestamp of when this data was extracted into DWH.
tests:
- not_null
- name: stg_core__protection_plan_to_price
description: |
Contains the relationship "this protection plan has this price in this currency"
in the scope of New Pricing. It accounts for the cost of the service, rather than
the amounts protected.
Each protection plan per currency can have different:
- Payment types (amount, percentage)
- Price computation methods (per booking, per night)
- Charging methods (post checkout, pre booking, at waiver payment, etc.)
Additionally, it's possible that a price is modified at user level. This is identifiable
by the id_user_product_bundle being set (custom price) vs. not set (default price).
Each protection plan and currency line can be active in a given time frame, and this table
also contains the history of previous, currently inactive, protection plan prices.
tests:
- dbt_expectations.expect_column_pair_values_A_to_be_greater_than_B:
column_A: ends_at_utc
column_B: starts_at_utc
or_equal: True
columns:
- name: id_protection_plan_to_price
data_type: integer
description: |
Identifier of the protection plan to price. Acts as the primary key for this table.
tests:
- unique
- not_null
- name: id_protection_plan
data_type: integer
description: |
Identifier of the protection plan.
tests:
- not_null
- name: id_currency
data_type: integer
description: |
Identifier of the currency.
tests:
- not_null
- name: payment_type
data_type: string
description: |
Type of the payment.
tests:
- not_null
- accepted_values:
values:
- "AMOUNT"
- "PERCENTAGE"
- name: price_base_unit
data_type: string
description: |
Represents how the price value should be taken into account, either
if the price is representing the total amount at booking level or
it's a nightly fee.
tests:
- not_null
- accepted_values:
values:
- "PER BOOKING"
- "PER NIGHT"
- name: invoicing_trigger
data_type: string
description: |
Represents at which moment in time this service should be invoiced.
tests:
- not_null
- accepted_values:
values:
- "PRE-BOOKING"
- "POST-CHECKOUT"
- "AT WAIVER PAYMENT"
- "AT DEPOSIT PAYMENT"
- "N/A"
- name: id_user_product_bundle
data_type: integer
description: |
Identifier of the user product bundle in which this protection plan to price
is applied. It can be null, thus referring to a product bundle per user that
uses a default price.
- name: amount_local_curr
data_type: decimal
description: |
Price amount of a given protection plan in the currency stated in id_currency.
tests:
- not_null
- name: protection_plan_price_name
data_type: string
description: |
The name given to a protection plan to price.
- name: starts_at_utc
data_type: timestamp
description: |
Timestamp of when this protection plan to price starts to be active.
tests:
- not_null
- name: ends_at_utc
data_type: timestamp
description: |
Timestamp of when this protection plan to price ends to be active.
It can be null, thus meaning it's currently active.
- name: has_no_end_date
data_type: boolean
description: |
True when ends_at_utc is not set, false otherwise.
- name: created_at_utc
data_type: timestamp
description: |
Timestamp of when this protection plan to price was created.
- name: created_date_utc
data_type: date
description: |
Date of when this protection plan to price was created.
- name: updated_at_utc
data_type: timestamp
description: |
Timestamp of when this protection plan to price was last updated.
- name: updated_date_utc
data_type: date
description: |
Date of when this protection plan to price was last updated.
- name: dwh_extracted_at_utc
data_type: timestamp
description: |
Timestamp of when this data was extracted into DWH.
tests:
- not_null
- name: stg_core__protection_plan_cover
description: |
Contains the relationship "this protection plan can protect these amounts in this currency"
in the scope of New Pricing. It accounts for the cover, i.e., how much we protect, rather than
the price of the protection service.
It's possible that a price is modified at user level. This is identifiable
by the id_user_host being set (custom price) vs. not set (default price).
columns:
- name: id_protection_plan_cover
data_type: integer
description: |
Identifier of the protection plan cover. Acts as the primary key for this table.
tests:
- unique
- not_null
- name: id_protection_plan
data_type: integer
description: |
Identifier of the protection plan.
tests:
- not_null
- name: id_currency
data_type: integer
description: |
Identifier of the currency.
tests:
- not_null
- name: id_user_host
data_type: string
description: |
Identifier of the user. It can be null, thus referring to a protection
cover that uses a default amount.
- name: baseline_protection_cover_local_curr
data_type: decimal
description: |
Baseline cover amount of a given protection plan in the currency stated in id_currency.
tests:
- not_null
- name: lower_protection_cover_local_curr
data_type: decimal
description: |
Lower cover amount of a given protection plan in the currency stated in id_currency.
tests:
- not_null
- name: maximum_protection_cover_local_curr
data_type: decimal
description: |
Maximum cover amount of a given protection plan in the currency stated in id_currency.
tests:
- not_null
- name: protection_plan_cover_name
data_type: string
description: |
The name given to a protection plan cover.
- name: created_at_utc
data_type: timestamp
description: |
Timestamp of when this protection plan cover was created.
- name: created_date_utc
data_type: date
description: |
Date of when this protection plan cover was created.
- name: updated_at_utc
data_type: timestamp
description: |
Timestamp of when this protection plan cover was last updated.
- name: updated_date_utc
data_type: date
description: |
Date of when this protection plan cover was last updated.
- name: dwh_extracted_at_utc
data_type: timestamp
description: |
Timestamp of when this data was extracted into DWH.
tests:
- not_null

48
models/staging/core/stg_core__protection_plan.sql Normal file
View file

@ -0,0 +1,48 @@
with
raw_protection as (
select
{{ adapter.quote("Id") }} as id,
{{ adapter.quote("Name") }} as protection_name,
{{ adapter.quote("DisplayName") }} as protection_display_name,
{{ adapter.quote("RequiredProductServices") }}
as required_product_services_binary_tier
from {{ source("core", "Protection") }}
),
raw_protection_plan as (
select
{{ adapter.quote("Id") }} as id_protection_plan,
{{ adapter.quote("ProtectionId") }} as id_protection,
{{ adapter.quote("StartDate") }} as starts_at_utc,
{{ adapter.quote("EndDate") }} as ends_at_utc,
case
when {{ adapter.quote("EndDate") }} is null then true else false
end as has_no_end_date,
{{ adapter.quote("CreatedDate") }} as created_at_utc,
cast({{ adapter.quote("CreatedDate") }} as date) as created_date_utc,
{{ adapter.quote("UpdatedDate") }} as updated_at_utc,
cast({{ adapter.quote("UpdatedDate") }} as date) as updated_date_utc,
{{ adapter.quote("_airbyte_extracted_at") }} as dwh_extracted_at_utc
from {{ source("core", "ProtectionPlan") }}
)
select
pp.id_protection_plan,
p.protection_name,
p.protection_display_name,
p.required_product_services_binary_tier,
pp.starts_at_utc,
pp.ends_at_utc,
pp.has_no_end_date,
pp.created_at_utc,
pp.created_date_utc,
pp.updated_at_utc,
pp.updated_date_utc,
pp.dwh_extracted_at_utc
from raw_protection_plan pp
left join raw_protection p on pp.id_protection = p.id

31
models/staging/core/stg_core__protection_plan_cover.sql Normal file
View file

@ -0,0 +1,31 @@
with
raw_protection_plan_cover as (
select * from {{ source("core", "ProtectionPlanToCurrency") }}
),
stg_core__protection_plan_cover as (
select
{{ adapter.quote("Id") }} as id_protection_plan_cover,
{{ adapter.quote("ProtectionPlanId") }} as id_protection_plan,
{{ adapter.quote("CurrencyId") }} as id_currency,
{{ adapter.quote("SuperhogUserId") }} as id_user_host,
{{ adapter.quote("DisplayName") }} as protection_plan_cover_name,
{{ adapter.quote("BaselineProtection") }}
as baseline_protection_cover_local_curr,
{{ adapter.quote("LowerProtection") }} as lower_protection_cover_local_curr,
{{ adapter.quote("MaximumProtection") }}
as maximum_protection_cover_local_curr,
{{ adapter.quote("CreatedDate") }} as created_at_utc,
cast({{ adapter.quote("CreatedDate") }} as date) as created_date_utc,
{{ adapter.quote("UpdatedDate") }} as updated_at_utc,
cast({{ adapter.quote("UpdatedDate") }} as date) as updated_date_utc,
{{ adapter.quote("_airbyte_extracted_at") }} as dwh_extracted_at_utc
from raw_protection_plan_cover
)
select *
from stg_core__protection_plan_cover

73
models/staging/core/stg_core__protection_plan_to_price.sql Normal file
View file

@ -0,0 +1,73 @@
with
raw_protection_plan_to_price as (
select * from {{ source("core", "ProtectionPlanToPrice") }}
),
raw_invoicing_trigger as (
select
{{ adapter.quote("Id") }} as id,
upper({{ adapter.quote("FullName") }}) as invoicing_trigger
from {{ source("core", "InvoicingMethod") }}
),
raw_price_base_unit as (
select
{{ adapter.quote("Id") }} as id,
upper({{ adapter.quote("FullName") }}) as price_base_unit
from {{ source("core", "BillingMethod") }}
),
raw_payment_type as (
select
{{ adapter.quote("Id") }} as id,
upper({{ adapter.quote("FullName") }}) as payment_type
from {{ source("core", "PaymentType") }}
),
stg_core__protection_plan_to_price as (
select
{{ adapter.quote("Id") }} as id_protection_plan_to_price,
{{ adapter.quote("ProtectionPlanId") }} as id_protection_plan,
{{ adapter.quote("CurrencyId") }} as id_currency,
{{ adapter.quote("PaymentTypeId") }} as id_payment_type,
{{ adapter.quote("BillingMethodId") }} as id_price_base_unit,
{{ adapter.quote("InvoicingMethodId") }} as id_invoicing_trigger,
{{ adapter.quote("UserProductBundleId") }} as id_user_product_bundle,
{{ adapter.quote("Amount") }} as amount_local_curr,
{{ adapter.quote("DisplayName") }} as protection_plan_price_name,
{{ adapter.quote("StartDate") }} as starts_at_utc,
{{ adapter.quote("EndDate") }} as ends_at_utc,
case
when {{ adapter.quote("EndDate") }} is null then true else false
end as has_no_end_date,
{{ adapter.quote("CreatedDate") }} as created_at_utc,
cast({{ adapter.quote("CreatedDate") }} as date) as created_date_utc,
{{ adapter.quote("UpdatedDate") }} as updated_at_utc,
cast({{ adapter.quote("UpdatedDate") }} as date) as updated_date_utc,
{{ adapter.quote("_airbyte_extracted_at") }} as dwh_extracted_at_utc
from raw_protection_plan_to_price
)
select
pstp.id_protection_plan_to_price,
pstp.id_protection_plan,
pstp.id_currency,
pt.payment_type,
pcm.price_base_unit,
cm.invoicing_trigger,
pstp.id_user_product_bundle,
pstp.amount_local_curr,
pstp.protection_plan_price_name,
pstp.starts_at_utc,
pstp.ends_at_utc,
pstp.has_no_end_date,
pstp.created_at_utc,
pstp.created_date_utc,
pstp.updated_at_utc,
pstp.updated_date_utc,
pstp.dwh_extracted_at_utc
from stg_core__protection_plan_to_price pstp
left join raw_payment_type pt on pstp.id_payment_type = pt.id
left join raw_price_base_unit pcm on pstp.id_price_base_unit = pcm.id
left join raw_invoicing_trigger cm on pstp.id_invoicing_trigger = cm.id