# Description
First model for Guest Product Payments. It only contains CIH from Verification Payments (so, the "old" CIH).
Some notes:
* It does not handle taxes computation.
* It converts guest product name to upper case, as the guest product tables are in upper case. I also apply it for payment status, as well as an UNKNOWN status for whenever it's null - we'd need to consider this for the rest of the refactor.
* Computation is placed within a CTE. This is intended since at some point this will include also Guest Product Payments that come from Guest Product related tables.
* Enhanced documentation with respect to Verification Payments V2.
Audit performed:
```
-- THIS GOES INTO AN AUDIT FILE
{% set old_query %}
select
*
from dwh_hybrid.intermediate.int_core__guest_product_payments
{% endset %}
{% set new_query %}
select
id_verification_to_payment,
id_payment,
is_refundable,
created_at_utc,
updated_at_utc,
payment_due_at_utc,
payment_due_date_utc,
payment_paid_at_utc,
payment_paid_date_utc,
payment_reference,
refund_due_at_utc,
refund_due_date_utc,
payment_refunded_at_utc,
payment_refunded_date_utc,
refund_payment_reference,
id_user_host,
id_guest_user as id_user_guest,
id_verification,
id_verification_request,
upper(verification_payment_type) as guest_product_name,
currency,
total_amount_in_txn_currency,
total_amount_in_gbp,
coalesce(upper(payment_status),'UNKNOWN') as payment_status,
notes
from dwh_hybrid.intermediate.int_core__verification_payments_v2
where verification_payment_type = 'CheckInCover'
{% endset %}
{{
audit_helper.compare_and_classify_query_results(
old_query,
new_query,
primary_key_columns=["id_verification_to_payment"],
columns=[
"id_verification_to_payment",
"id_payment",
"is_refundable",
"created_at_utc",
"updated_at_utc",
"payment_due_at_utc",
"payment_due_date_utc",
"payment_paid_at_utc",
"payment_paid_date_utc",
"payment_reference",
"refund_due_at_utc",
"refund_due_date_utc",
"payment_refunded_at_utc",
"payment_refunded_date_utc",
"refund_payment_reference",
"id_user_host",
"id_user_guest",
"id_verification",
"id_verification_request",
"guest_product_name",
"currency",
"total_amount_in_txn_currency",
"total_amount_in_gbp",
"payment_status",
"notes",
],
sample_limit=10000000,
)
}}
```
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models ...
# Description
Fixes the Account Growth issue on the rank/share on revenue metrics.
Issue was that the share needs to be shifted one month for account computation; meaning that the impact score of April relies on the share from March (as there's no April revenue data).
The shift also affected the rank as it's based on the share.
Solution is:
* Re-bring actual revenue share and attribute it to the correct month
* Apply a coalesce giving priority to the revenue share/rank from the correct month. If no data is available (ongoing month), show share/rank of previous month. This is intended to help quantify account importance in the ongoing month - as it's actually the case in the report
Tested locally + run in prod to check the report performance. Small note, this makes revenue share not adding up anymore to 100% as accounts that go live in the same month are excluded. This is exactly the same as for Billable Items.
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #30013
# Description
Remove old models:
- `monthly_aggregated_metrics_history_by_deal`
- `int_monthly_aggregated_metrics_history_by_deal`
# Checklist
- [ ] The edited models and dependants run properly with production data.
- [ ] The edited models are sufficiently documented.
- [ ] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Remove old models
Related work items: #29672
# Description
Updates deprecation dates to 23rd May.
Updates exposure with new Account Growth report
# Checklist
- [X] Project compiles
Related work items: #29374
# Description
Updated exposures with new model for Host Resolutions Payments
Deleted previous models that are not longer being used
# Checklist
- [ ] The edited models and dependants run properly with production data.
- [ ] The edited models are sufficiently documented.
- [ ] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Updated exposure
Related work items: #29373
# Description
Propagates the new account growth model to reporting.
Rounds and a couple of additional fields are handled here for PBI purposes.
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #29374
# Description
New `xero__host_resolutions_payments` for Resolutions Host Payments Report.
I also updated a few descriptions from Superhog to Truvi
# Checklist
- [x] The edited models and dependants run properly with production data.
- [x] The edited models are sufficiently documented.
- [x] The edited models contain PK tests, and I've ran and passed them.
- [x] I have checked for DRY opportunities with other models and docs.
- [x] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Host Resolution model for report
Related work items: #29373
# Description
Creates an intermediate model to compute the impact in revenue (total or rrpr) due to the growth.
This model also categorises the impact, and provides additional attributes for reporting purposes.
This model aims to substitute, in time, the current existing version of `int_monthly_growth_score_by_deal`.
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #29374
# Description
Included new accepted values for Host Resolutions Payments in seed tests
# Checklist
- [x] The edited models and dependants run properly with production data.
- [x] The edited models are sufficiently documented.
- [x] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Fixed tests in seed
Related work items: #29373
# Description
Updated `int_kpis__metric_daily_host_resolutions` to read from `int_xero__host_resolutions_payments` with the new Host Resolutions credit notes.
Now it includes both bank transactions and credit notes for Host Resolutions Payments
# Checklist
- [x] The edited models and dependants run properly with production data.
- [x] The edited models are sufficiently documented.
- [x] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Commit wip
Related work items: #29373
# Description
New model for Resolutions payments
# Checklist
- [x] The edited models and dependants run properly with production data.
- [x] The edited models are sufficiently documented.
- [x] The edited models contain PK tests, and I've ran and passed them.
- [x] I have checked for DRY opportunities with other models and docs.
- [x] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #29373
# Description
Removes exposures of monthly_aggregated_metrics_history_by_deal in Main KPIs
Project compiles.
# Checklist
N/A
Related work items: #29204
# Description
Refactor `xero__net_fees_by_deal`
Changed it so the model reads directly from `xero__credit_note_line_items`, included some fields in this model and added test to schema.
Run the macro to verify both models matched.
There were some decimal approximation differences, verified this by casting to decimal with fewer decimal and they matched 100%.
Next step would be to exclude Resolution Payments from this model, which I will do in the next commit.
# Checklist
- [x] The edited models and dependants run properly with production data.
- [x] The edited models are sufficiently documented.
- [x] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
refactor xero__net_fees_by_deal
Related work items: #29373
# Description
I opted to name the combination of (Platform) Billable Bookings and (API) Billable Verifications as Billable Items. This is to ensure consistent naming.
Changes:
* Renamed `int_kpis_projected__agg_daily_billable_bookings` to `int_kpis_projected__agg_daily_billable_items`. This now has a new CTE named combination_of_billable_items that combines both API and Platform billable items. Renamed any "bookings" field to "items".
* Renamed `int_kpis_projected__agg_monthly_billable_bookings` to `int_kpis_projected__agg_monthly_billable_items`. Renamed any "bookings" field to "items".
* Renamed `int_billable_bookings_growth_score_by_deal` to `int_billable_items_growth_score_by_deal`. This now has a new CTE named `aggregated_monthly_billable_items` that combines the historical information both API and Platform on billable items. Renamed any "bookings" field to "items".
* Changes Schema accordingly. Small note here that the assert dimension completeness stops working since there's dimensions specific to API or Platform, thus I removed it.
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #29374
# Description
Changes:
* 4 new models in the scope of KPIs for Billable Verifications from APIs. I believe it's more correct to say these are Billable Verifications than Billable Bookings since there's some cases in which a Booking can be duplicated and it's billed multiple times. These include:
* A daily metric model - extremely simple. You will notice there's no Billing Country not Listing Segmentation. This is because for ALL API cases this is UNSET, thus, I just remove it.
* An equivalent monthly metric model.
* Two aggregated models per dimension, dimension value: on a daily and a monthly basis.
Important change: the macro that handles the aggregations sets by default Billing Country and Listing Segmentation. I modified a bit the flow so the only required dimension is Global, and these are skipped for APIs models.
This is needed for the changes intended on the Growth score. In there, I'll combine both Platform Billable Bookings with API Billable Verifications.
Notice there's no MTD models. These could be added for sure; but since I'm not creating any metric in Main KPIs or similar, I opted to skip it for now. It can be done later on.
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #29374
# Description
Switches Created Bookings to Billable Bookings in the scope of Projected Bookings. I opted for hard switching (thus removing Projected Created Bookings) altogether to avoid having unused models in production.
This aims to set the ground to include, in the future, APIs Billable Bookings.
Impact in KPIs:
* `int_kpis__agg_daily_created_bookings` now is `int_kpis__agg_daily_billable_bookings`. Aside from the name change, it's reduced to 4 metrics to just one. Schema changes also apply.
Impact in Projected KPIs:
* `int_kpis_projected__agg_daily_created_bookings` now is `int_kpis_projected__agg_daily_billable_bookings`. Fields are also renamed accordingly.
* `int_kpis_projected__agg_monthly_created_bookings` now is `int_kpis_projected__agg_monthly_billable_bookings`. Fields are also renamed accordingly.
* Schema file is also updated accordingly.
Impact on new model for Growth Score:
* `int_created_bookings_growth_score_by_deal` now is `int_billable_bookings_growth_score_by_deal`. Fields are also renamed accordingly.
* Schema is also updated accordingly.
Note that since the end model `int_billable_bookings_growth_score_by_deal` is not used at the moment in reporting, there's no user-facing impact. Also, the rest of modified models are not used for other purposes.
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #29374
# Description
Creates a first unified version of API verifications. This is a very simple model as it aims to capture the minimal essence of Verifications in a unified view. Any very-in-depth-API-specific detail is not available here.
Additionally, I propagated the Check-in date in the Athena model.
This is needed for API Bookings KPIs future models.
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models. **Currently as a view, despite it's 70k records. If this grows we could consider other materialisations**
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #29374
# Description
Removes UNKNOWN in Service Business Type linked to 'BASIC DAMAGE DEPOSIT OR BASIC WAIVER'.
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Set option with basic waiver as deposit management
Related work items: #29649
# Description
After checking the current growth score model... I think I will decommission it for something simpler.
This PR introduces a new growth score purely based on Created Bookings, thus it won't work for API deals. For API deals I will do a similar Total Revenue comparison separately.
This uses the projected bookings at the end of the month, thus this is updated in a daily basis and it's timely.
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #29374
# Description
Small refactor on Growth Score model. It just handles the dimension deals attributes at the end instead of having these in each CTE.
I also forced a partition ordered by deal AFTER the current ordering. This is a side effect because SQL randomly choses a value if the value being used to order was exactly the same (ex: revenue = 0). This was causing issues when auditing the refactor.
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #29374
# Description
Technically, a payout can be submitted while not in a finished incident state. Likely next step from res perspective is just to close the ticket.
This PR just adds two new fields for completion and consistency, and fixes the completion test so it makes sense.
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
# Description
Added new invoices metrics for CIH and S&P API for KPI and Accounting models
# Checklist
- [x] The edited models and dependants run properly with production data.
- [x] The edited models are sufficiently documented.
- [x] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
New API invoices
Related work items: #29276
# Description
Adds monetary value in terms of payouts for further understanding of the potential impact on improving the flagging.
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #29284
# Description
Creates 2 new models in the scope of flagging: how good are we at identifying "at risk" bookings vs. 1) the number of claims generated and 2) the number of submitted payouts?
This only applies for Protected Bookings in New Dash that have been completed (14 days after the check-out) with potential resolutions appearing in Resolutions Center.
The first table `int_flagging_booking_categorisation` contains all the heavy logic to categorise the bookings.
The second view `int_flagging_performance_analysis` computes standard binary classification scores, for the 2 possible ways of tracking.
Tables are already in prod to help you understand while reviewing. You'll see that the figures are still quite low, specially due to small amount of claims/submitted payouts. This makes the true positives being just... 1.
There's heavy test and documentation coverage to ensure there's no mistakes on the computation.
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models. **Materialising as table the first model despite being just 1 record since otherwise tests takes ages**
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #29284
# Description
Adds more robust testing for Total Revenue and Revenue Retained Post-Resolutions in:
* int_mtd_vs_previous_year_metrics
* int_ytd_mtd_main_metrics_overview
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
More robust testing on key metrics
Related work items: #29227
# Description
Now that we finally have customers on CIH API
Nathan noticed an error on the report which we hadn't noticed before, the fees were not included in the reporting model and was previously being taken from the monthly model which did have them. So currently the monthly value is being displayed correctly but not the individual fees for each verification.
# Checklist
- [x] The edited models and dependants run properly with production data.
- [x] The edited models are sufficiently documented.
- [x] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
cih api fix
Related work items: #29225
# Description
As raised by Kayla, Retained metrics are not computed correctly in AM Account Margin report, see screenshot:
Issue lies in the fact that Guest Payments contains more dimensions than the rest of the models. This is generating duplicates in the combination_of_sources CTE, but these are effectively hidden because of the sum() applied of the metrics.
In the case of a Deal having in the same day Guest Payments with different dimensions from those selected, AND on the same day having a value on Invoiced Revenue or Host Resolutions, then the Invoiced Revenue or Host Resolutions would be duplicated.
This PR aims to fix this issue, and also adds a few tests on the base model for AM Account Margin to ensure this is kind of error raises an alert.
I'll be debugging other affected models and increase the robustness test-wise; as well as gathering the full impact.
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #29227
# Description
Adds a macro to compare the contents of two tables. Will simply tell you if the contents are identical or not. Designed to be used in your development flow through the CLI so you can check refactors as you go.
# Description
Small changes to exclude "irrelevant" data. All data is relevant I guess but this is less.
* Exclude churned accounts after 3 months.
* Exclude any data after 24 months.
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #28998
