Merged PR 3038: Adding Protection tables in the scope of New Pricing/Dashboard

# Description

This PR creates 3 new models in staging:
- `stg_core__protection_plan`: main information of a protection plan. Here I denormalise the info coming from Protection, meaning, Protection is not a standalone model.
- `stg_core__protection_plan_to_price`: How much does a protection service cost? It's very similar to the model product_service_to_price.
- `stg_core__protection_plan_cover`: How much does a protection service protects, i.e., covers? I took the liberty of modifying the original name since it was confusing to me. Let me know if you'd like to modify the name.

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #20809

models/staging/core/_core_sources.yml

@@ -245,3 +245,11 @@ sources:
         identifier: InvoicingMethod
       - name: PaymentType
         identifier: PaymentType
+      - name: Protection
+        identifier: Protection
+      - name: ProtectionPlan
+        identifier: ProtectionPlan
+      - name: ProtectionPlanToPrice
+        identifier: ProtectionPlanToPrice
+      - name: ProtectionPlanToCurrency
+        identifier: ProtectionPlanToCurrency

models/staging/core/schema.yml

@@ -705,3 +705,281 @@ models:
           Timestamp of when this data was extracted into DWH.
         tests:
           - not_null
+
+  - name: stg_core__protection_plan
+    description: |
+      Contains all the protection plans, also referred to protection services, operated 
+      by Truvi (Superhog) in the scope of New Pricing.
+    tests:
+      - dbt_expectations.expect_column_pair_values_A_to_be_greater_than_B:
+          column_A: ends_at_utc
+          column_B: starts_at_utc
+          or_equal: True
+
+    columns:
+      - name: id_protection_plan
+        data_type: integer
+        description: |
+          Identifier of the product service to price. Acts as the primary key for this table.
+        tests:
+          - unique
+          - not_null
+      - name: protection_name
+        data_type: string
+        description: |
+          The name given to a protection service.
+      - name: protection_display_name
+        data_type: string
+        description: |
+          The name given to a protection service, usually better fit for visualisation purposes.
+      - name: required_product_services_binary_tier
+        data_type: integer
+        description: |
+          Required product service binary identifier (as powers of 2). It's a geometric progression with a 
+          common ratio of 2. It can be used to uniquely identify which product services are required for a 
+          given protection. The fact that it's a power of 2 is because it identifies if a product service 
+          is applied (true) or not applied (false).
+      - name: starts_at_utc
+        data_type: timestamp
+        description: |
+          Timestamp of when this protection plan starts to be active.
+        tests:
+          - not_null
+      - name: ends_at_utc
+        data_type: timestamp
+        description: |
+          Timestamp of when this protection plan ends to be active.
+          It can be null, thus meaning it's currently active.
+      - name: has_no_end_date
+        data_type: boolean
+        description: |
+          True when ends_at_utc is not set, false otherwise.
+      - name: created_at_utc
+        data_type: timestamp
+        description: |
+          Timestamp of when this protection plan was created.
+      - name: created_date_utc
+        data_type: date
+        description: |
+          Date of when this protection plan was created.
+      - name: updated_at_utc
+        data_type: timestamp
+        description: |
+          Timestamp of when this protection plan was last updated.
+      - name: updated_date_utc
+        data_type: date
+        description: |
+          Date of when this product protection plan was last updated.
+      - name: dwh_extracted_at_utc
+        data_type: timestamp
+        description: |
+          Timestamp of when this data was extracted into DWH.
+        tests:
+          - not_null
+
+  - name: stg_core__protection_plan_to_price
+    description: |
+      Contains the relationship "this protection plan has this price in this currency"
+      in the scope of New Pricing. It accounts for the cost of the service, rather than 
+      the amounts protected.
+      Each protection plan per currency can have different:
+      - Payment types (amount, percentage)
+      - Price computation methods (per booking, per night)
+      - Charging methods (post checkout, pre booking, at waiver payment, etc.) 
+      Additionally, it's possible that a price is modified at user level. This is identifiable
+      by the id_user_product_bundle being set (custom price) vs. not set (default price).
+      Each protection plan and currency line can be active in a given time frame, and this table
+      also contains the history of previous, currently inactive, protection plan prices.
+
+    tests:
+      - dbt_expectations.expect_column_pair_values_A_to_be_greater_than_B:
+          column_A: ends_at_utc
+          column_B: starts_at_utc
+          or_equal: True
+
+    columns:
+      - name: id_protection_plan_to_price
+        data_type: integer
+        description: |
+          Identifier of the protection plan to price. Acts as the primary key for this table.
+        tests:
+          - unique
+          - not_null
+      - name: id_protection_plan
+        data_type: integer
+        description: |
+          Identifier of the protection plan.
+        tests:
+          - not_null
+      - name: id_currency
+        data_type: integer
+        description: |
+          Identifier of the currency.
+        tests:
+          - not_null
+      - name: payment_type
+        data_type: string
+        description: |
+          Type of the payment.
+        tests:
+          - not_null
+          - accepted_values:
+              values:
+                - "AMOUNT"
+                - "PERCENTAGE"
+      - name: price_base_unit
+        data_type: string
+        description: |
+          Represents how the price value should be taken into account, either 
+          if the price is representing the total amount at booking level or 
+          it's a nightly fee.
+        tests:
+          - not_null
+          - accepted_values:
+              values:
+                - "PER BOOKING"
+                - "PER NIGHT"
+      - name: invoicing_trigger
+        data_type: string
+        description: |
+          Represents at which moment in time this service should be invoiced.
+        tests:
+          - not_null
+          - accepted_values:
+              values:
+                - "PRE-BOOKING"
+                - "POST-CHECKOUT"
+                - "AT WAIVER PAYMENT"
+                - "AT DEPOSIT PAYMENT"
+                - "N/A"
+      - name: id_user_product_bundle
+        data_type: integer
+        description: |
+          Identifier of the user product bundle in which this protection plan to price 
+          is applied. It can be null, thus referring to a product bundle per user that 
+          uses a default price.
+      - name: amount_local_curr
+        data_type: decimal
+        description: |
+          Price amount of a given protection plan in the currency stated in id_currency.
+        tests:
+          - not_null
+      - name: protection_plan_price_name
+        data_type: string
+        description: |
+          The name given to a protection plan to price.
+      - name: starts_at_utc
+        data_type: timestamp
+        description: |
+          Timestamp of when this protection plan to price starts to be active.
+        tests:
+          - not_null
+      - name: ends_at_utc
+        data_type: timestamp
+        description: |
+          Timestamp of when this protection plan to price ends to be active.
+          It can be null, thus meaning it's currently active.
+      - name: has_no_end_date
+        data_type: boolean
+        description: |
+          True when ends_at_utc is not set, false otherwise.
+      - name: created_at_utc
+        data_type: timestamp
+        description: |
+          Timestamp of when this protection plan to price was created.
+      - name: created_date_utc
+        data_type: date
+        description: |
+          Date of when this protection plan to price was created.
+      - name: updated_at_utc
+        data_type: timestamp
+        description: |
+          Timestamp of when this protection plan to price was last updated.
+      - name: updated_date_utc
+        data_type: date
+        description: |
+          Date of when this protection plan to price was last updated.
+      - name: dwh_extracted_at_utc
+        data_type: timestamp
+        description: |
+          Timestamp of when this data was extracted into DWH.
+        tests:
+          - not_null
+
+  - name: stg_core__protection_plan_cover
+    description: |
+      Contains the relationship "this protection plan can protect these amounts in this currency"
+      in the scope of New Pricing. It accounts for the cover, i.e., how much we protect, rather than 
+      the price of the protection service. 
+      It's possible that a price is modified at user level. This is identifiable
+      by the id_user_host being set (custom price) vs. not set (default price).
+
+    columns:
+      - name: id_protection_plan_cover
+        data_type: integer
+        description: |
+          Identifier of the protection plan cover. Acts as the primary key for this table.
+        tests:
+          - unique
+          - not_null
+      - name: id_protection_plan
+        data_type: integer
+        description: |
+          Identifier of the protection plan.
+        tests:
+          - not_null
+      - name: id_currency
+        data_type: integer
+        description: |
+          Identifier of the currency.
+        tests:
+          - not_null
+      - name: id_user_host
+        data_type: string
+        description: |
+          Identifier of the user. It can be null, thus referring to a protection 
+          cover that uses a default amount.
+      - name: baseline_protection_cover_local_curr
+        data_type: decimal
+        description: |
+          Baseline cover amount of a given protection plan in the currency stated in id_currency.
+        tests:
+          - not_null
+      - name: lower_protection_cover_local_curr
+        data_type: decimal
+        description: |
+          Lower cover amount of a given protection plan in the currency stated in id_currency.
+        tests:
+          - not_null
+      - name: maximum_protection_cover_local_curr
+        data_type: decimal
+        description: |
+          Maximum cover amount of a given protection plan in the currency stated in id_currency.
+        tests:
+          - not_null
+      - name: protection_plan_cover_name
+        data_type: string
+        description: |
+          The name given to a protection plan cover.
+      - name: created_at_utc
+        data_type: timestamp
+        description: |
+          Timestamp of when this protection plan cover was created.
+      - name: created_date_utc
+        data_type: date
+        description: |
+          Date of when this protection plan cover was created.
+      - name: updated_at_utc
+        data_type: timestamp
+        description: |
+          Timestamp of when this protection plan cover was last updated.
+      - name: updated_date_utc
+        data_type: date
+        description: |
+          Date of when this protection plan cover was last updated.
+      - name: dwh_extracted_at_utc
+        data_type: timestamp
+        description: |
+          Timestamp of when this data was extracted into DWH.
+        tests:
+          - not_null

models/staging/core/stg_core__protection_plan.sql

@@ -0,0 +1,48 @@
+with
+    raw_protection as (
+        select
+            {{ adapter.quote("Id") }} as id,
+
+            {{ adapter.quote("Name") }} as protection_name,
+            {{ adapter.quote("DisplayName") }} as protection_display_name,
+
+            {{ adapter.quote("RequiredProductServices") }}
+            as required_product_services_binary_tier
+
+        from {{ source("core", "Protection") }}
+    ),
+    raw_protection_plan as (
+        select
+            {{ adapter.quote("Id") }} as id_protection_plan,
+            {{ adapter.quote("ProtectionId") }} as id_protection,
+
+            {{ adapter.quote("StartDate") }} as starts_at_utc,
+            {{ adapter.quote("EndDate") }} as ends_at_utc,
+            case
+                when {{ adapter.quote("EndDate") }} is null then true else false
+            end as has_no_end_date,
+
+            {{ adapter.quote("CreatedDate") }} as created_at_utc,
+            cast({{ adapter.quote("CreatedDate") }} as date) as created_date_utc,
+            {{ adapter.quote("UpdatedDate") }} as updated_at_utc,
+            cast({{ adapter.quote("UpdatedDate") }} as date) as updated_date_utc,
+
+            {{ adapter.quote("_airbyte_extracted_at") }} as dwh_extracted_at_utc
+
+        from {{ source("core", "ProtectionPlan") }}
+    )
+select
+    pp.id_protection_plan,
+    p.protection_name,
+    p.protection_display_name,
+    p.required_product_services_binary_tier,
+    pp.starts_at_utc,
+    pp.ends_at_utc,
+    pp.has_no_end_date,
+    pp.created_at_utc,
+    pp.created_date_utc,
+    pp.updated_at_utc,
+    pp.updated_date_utc,
+    pp.dwh_extracted_at_utc
+from raw_protection_plan pp
+left join raw_protection p on pp.id_protection = p.id

models/staging/core/stg_core__protection_plan_cover.sql

@@ -0,0 +1,31 @@
+with
+    raw_protection_plan_cover as (
+        select * from {{ source("core", "ProtectionPlanToCurrency") }}
+    ),
+    stg_core__protection_plan_cover as (
+        select
+            {{ adapter.quote("Id") }} as id_protection_plan_cover,
+            {{ adapter.quote("ProtectionPlanId") }} as id_protection_plan,
+            {{ adapter.quote("CurrencyId") }} as id_currency,
+
+            {{ adapter.quote("SuperhogUserId") }} as id_user_host,
+
+            {{ adapter.quote("DisplayName") }} as protection_plan_cover_name,
+
+            {{ adapter.quote("BaselineProtection") }}
+            as baseline_protection_cover_local_curr,
+            {{ adapter.quote("LowerProtection") }} as lower_protection_cover_local_curr,
+            {{ adapter.quote("MaximumProtection") }}
+            as maximum_protection_cover_local_curr,
+
+            {{ adapter.quote("CreatedDate") }} as created_at_utc,
+            cast({{ adapter.quote("CreatedDate") }} as date) as created_date_utc,
+            {{ adapter.quote("UpdatedDate") }} as updated_at_utc,
+            cast({{ adapter.quote("UpdatedDate") }} as date) as updated_date_utc,
+
+            {{ adapter.quote("_airbyte_extracted_at") }} as dwh_extracted_at_utc
+
+        from raw_protection_plan_cover
+    )
+select *
+from stg_core__protection_plan_cover

models/staging/core/stg_core__protection_plan_to_price.sql

@@ -0,0 +1,73 @@
+with
+    raw_protection_plan_to_price as (
+        select * from {{ source("core", "ProtectionPlanToPrice") }}
+    ),
+    raw_invoicing_trigger as (
+        select
+            {{ adapter.quote("Id") }} as id,
+            upper({{ adapter.quote("FullName") }}) as invoicing_trigger
+        from {{ source("core", "InvoicingMethod") }}
+    ),
+    raw_price_base_unit as (
+        select
+            {{ adapter.quote("Id") }} as id,
+            upper({{ adapter.quote("FullName") }}) as price_base_unit
+        from {{ source("core", "BillingMethod") }}
+    ),
+    raw_payment_type as (
+        select
+            {{ adapter.quote("Id") }} as id,
+            upper({{ adapter.quote("FullName") }}) as payment_type
+        from {{ source("core", "PaymentType") }}
+    ),
+    stg_core__protection_plan_to_price as (
+        select
+            {{ adapter.quote("Id") }} as id_protection_plan_to_price,
+            {{ adapter.quote("ProtectionPlanId") }} as id_protection_plan,
+            {{ adapter.quote("CurrencyId") }} as id_currency,
+            {{ adapter.quote("PaymentTypeId") }} as id_payment_type,
+            {{ adapter.quote("BillingMethodId") }} as id_price_base_unit,
+            {{ adapter.quote("InvoicingMethodId") }} as id_invoicing_trigger,
+
+            {{ adapter.quote("UserProductBundleId") }} as id_user_product_bundle,
+
+            {{ adapter.quote("Amount") }} as amount_local_curr,
+            {{ adapter.quote("DisplayName") }} as protection_plan_price_name,
+
+            {{ adapter.quote("StartDate") }} as starts_at_utc,
+            {{ adapter.quote("EndDate") }} as ends_at_utc,
+            case
+                when {{ adapter.quote("EndDate") }} is null then true else false
+            end as has_no_end_date,
+
+            {{ adapter.quote("CreatedDate") }} as created_at_utc,
+            cast({{ adapter.quote("CreatedDate") }} as date) as created_date_utc,
+            {{ adapter.quote("UpdatedDate") }} as updated_at_utc,
+            cast({{ adapter.quote("UpdatedDate") }} as date) as updated_date_utc,
+
+            {{ adapter.quote("_airbyte_extracted_at") }} as dwh_extracted_at_utc
+
+        from raw_protection_plan_to_price
+    )
+select
+    pstp.id_protection_plan_to_price,
+    pstp.id_protection_plan,
+    pstp.id_currency,
+    pt.payment_type,
+    pcm.price_base_unit,
+    cm.invoicing_trigger,
+    pstp.id_user_product_bundle,
+    pstp.amount_local_curr,
+    pstp.protection_plan_price_name,
+    pstp.starts_at_utc,
+    pstp.ends_at_utc,
+    pstp.has_no_end_date,
+    pstp.created_at_utc,
+    pstp.created_date_utc,
+    pstp.updated_at_utc,
+    pstp.updated_date_utc,
+    pstp.dwh_extracted_at_utc
+from stg_core__protection_plan_to_price pstp
+left join raw_payment_type pt on pstp.id_payment_type = pt.id
+left join raw_price_base_unit pcm on pstp.id_price_base_unit = pcm.id
+left join raw_invoicing_trigger cm on pstp.id_invoicing_trigger = cm.id