# Description
Aggregated guest kpis model to reporting
# Checklist
- [x] The edited models and dependants run properly with production data.
- [x] The edited models are sufficiently documented.
- [x] The edited models contain PK tests, and I've ran and passed them.
- [x] I have checked for DRY opportunities with other models and docs.
- [x] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
aggregated guest kpis to reporting
Related work items: #24604
# Description
This query refactors the model `int_core__new_dash_user_overview` to optimize performance.
The output of the model remains unaltered.
The performance gains are obtained by preventing a monster-grade intermediate join with almost 100 million rows + a sequential order that the current `master` model causes. This refactor breaks down the query that caused that into multiple CTEs that pre-aggregate results, avoiding the dimensional explosion.
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged
# Description
PR to fix the kpis_daily_outlier_detector test.
Previously it was using all values for the metric independently of the granularity, now is adding them up by daily value and no other metrics.
# Checklist
- [x] The edited models and dependants run properly with production data.
- [ ] The edited models are sufficiently documented.
- [x] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
fixed outlier_detector
Related work items: #24590
# Description
Aggregated model for guest kpis metrics.
This model aggregates the values for all basic metrics for guest kpis, it has a granularity per metrics of timeframe (YTD, MTD, WTD) and all other dimensions needed (has_payment, has_id_check & main_billing_country_iso_3_per_deal).
I decided not to include the calculated metrics because it adds much more complexity to the model specially when you start considering all possible combinations between dimensions and how this would affect the calculations, in this case conversion_rate, revenue_rate and average_csat_score. I think this could all be managed much simpler inside PBI and since the granularity of the model is not as deep as the daily model, this should help with the loading time for the report.
# Checklist
- [x] The edited models and dependants run properly with production data.
- [x] The edited models are sufficiently documented.
- [x] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #24604
# Description
Creates the models for KPIs for New Dash - Chargeable metrics. In essence, computes 4 metrics:
- Chargeable Services
- Chargeable Amount (in GBP)
- Chargeable Bookings (unique over a time period and dimension, not additive)
- Chargeable Listings (unique over a time period and dimension, not additive)
This is done by creating:
- A Weekly Metric and Monthly Metric model. Here we keep the granularity of id_booking / id_accommodation to be able to compute the uniqueness.
- A Daily, Weekly and Monthly Aggregated models. Same as usual.
- Integrates everything to the existing model for Product New Dash Agg Metrics
- Exposes everything into reporting
NB: I removed on "by_host" in Created Services - I forgot to clear it out.
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #20809
# Description
Added billing_country as dimension to use in the Guest KPIs Report.
Also created a new model that pivots the metrics to leave them as a column and aggregates the values set for different timeframes (YTD, MTD, WTD). For the moment it only aggregates the values for the existing metrics on the existing models, it doesn't compute metrics like rates or averages (avg_csat_score, conversion_rate, etc..). I am not sure if it would be better to compute here in the model or just doing it in PBI, both are viable options not sure if the model is going to end up being too monstrous or it doesn't matter.
@<Oriol Roqué Paniagua> what do you think??
Imagen para ayudar a entender el modelo
# Checklist
- [x] The edited models and dependants run properly with production data.
- [x] The edited models are sufficiently documented.
- [x] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [x] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #24604
# Description
This PR aims to do 2 things:
1. Create the first metric daily model for New Dash - Chargeable Services
* PK at Booking, Date and Service. I added a few more dimensions such as Accommodation and Business Type. The idea is that while Daily Unique Bookings/Accommodations Charged will be close (if not the same) as Charged Services, by having the ID we can compute Monthly/Weekly Unique Bookings/Accommodations Charged in a proper manner. Besides this, we would still compute additive metrics in the future such as the sum of Charged Services and the sum of Service Total Price in GBP.
* `IMPORTANT`: as discussed in the daily, I changed New Pricing models containing "Charged" columns to "Chargeable". This affects the new model for New Dash KPIs, but also `int_core__booking_summary` and `int_core__booking_service_detail`.
2. Small fixes on New Dash - Created Services.
* Mainly, there were some inconsistencies with what was actually written (and really applied) in the schema entry as for what was the PK of the model vs. what was stated in the model. I just re-ordered the columns and re-specified correctly the PK.
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #20809
# Description
First attempt at creating a test.
This test checks that the new values (from yesterday) don't vary too much from the previous days. The user can set the metrics to check on, how much tolerance to use in number of standard deviations allowed to deviate from the average and the number of days to consider in the evaluation.
# Checklist
- [x] The edited models and dependants run properly with production data.
- [ ] The edited models are sufficiently documented.
- [x] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
KPIs outlier detector
Related work items: #24590
# Description
This PR adds a script to build the project documentation and inclues some notes on how to call it and serve the docs.
# Checklist
NA
# Other
NA
Related work items: #23999
# Description
Adds new role `ResolutionApi` in accepted values of `int_core__user_role`
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [NA] I have checked for DRY opportunities with other models and docs.
- [NA] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
# Description
Adds informative fields to New Pricing booking-related tables. These are added into booking_to_service which is a view, and propagated and materialised as table in the `booking_service_detail` and `booking_summary`.
The new fields are:
- Date versions for relevant booking timestamps (Created, Updated, Check-in, Check-out)
- Booking Number of nights
- Ids for the Accommodation and UserProductBundle
- Program Name (aka Product Bundle Name). I dislike the fact that it's named "Product" because you can have Protection Plans in the Program as well... so I opted for a more conventional business-oriented naming, which is Program.
Added a bunch of tests + documentation.
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #20809
# Description
Changes:
* Handles conversion to GBP directly in the models `int_core__product_service_billing_item` and `int_core__protection_plan_billing_item` .
* Adds the newly created field of `is_missing_id_deal` in `int_core__booking_to_service`, to avoid manually computing downstream.
* Computes total price per service in `int_core__booking_service_detail`. Since, technically, the same service can have different billing lines, I also retrieve some min/max charge dates and some booleans to help the identification.
* Computes total price per booking in `int_core__booking_summary`. Since, technically, the same booking can have different services charged in different moments in time, I also retrieve some min/max charge dates. I also computed a very necessary `is_booking_charged` to understand if we're making money or not out of it.
This PR should provide the necessary fields to start computing Revenue for New Dash.
HOWEVER:
1) I still need to handle Guest Payments computation for Waiver/Deposit services. I'll do it in a separated PR.
2) while doing this exercise I noticed that the current data is not good / consistent with what I was expecting in terms of New Pricing documentation. I will check with Dash Squad so it's correctly filled, before reporting fake numbers...
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #20809
# Description
Adds Billing models in intermediate. It's mostly the similar content as for the staging counterparts, but providing useful information such as the currency code and service that this billing line refers to. Same as before, two separate models for Protection Plan and Product Service.
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #20809
# Description
Aims to bring the 2 new billing tables in the scope of New Pricing into DWH.
These correspond to a Product Service table and a Protection Plan table.
Further modelling is expected in intermediate. Main goal would be to enrich the tables of `int_core__booking_service_detail` and `int_core__booking_summary`
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #20809
# Description
This PR has 2 commits:
- The first one handles the removal from the computation any user that has not an id deal properly set. I just created a boolean field in int_core__user_host that identifies if the host has no id_deal. Then apply the new condition in the 2 main usages of New Dash info.
- The second one cleans New Dash KPIs. Since we do not have anymore users without deals, it means that the identification of the host/account is going to be exactly the same if done by id_user_host or id_deal. I hated having id_user_host in KPIs so I've removed it :)
Lastly, this should speed up massively the execution. Not because there's improvements on the code but rather the reduction of data.
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #20809
# Description
Changes:
* Fixes weekly dates for KPIs. Before, joins were not working (but nothing was really used).
* Computes weekly new dash created services and exposes it to reporting
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #20809
# Description
Removed the unused model created for the Guest KPIs reporting.
Added the missing data for the exposure section on the report
# Checklist
- [ ] The edited models and dependants run properly with production data.
- [ ] The edited models are sufficiently documented.
- [ ] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Removed guest metrics aggregated models
Related work items: #23998
# Description
_Describe your motivation and changes here._
# Checklist
- [ ] The edited models and dependants run properly with production data.
- [ ] The edited models are sufficiently documented.
- [ ] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Bugfix kpis__product_new_dash_agg_metrics
# Description
Creates a macro for beautification of categorical values.
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #20809
# Description
This PR handles the computation of KPIs for New Dash, focusing on Created Services.
New dimensions configured in `business_kpis_configuration` and applied in this new models for `NEW_DASH_CREATED_SERVICES`:
* `dim_host`,
* `dim_has_upgraded_service`,
* `dim_new_dash_version`,
* `dim_pricing_service`
New daily metric model `int_kpis__metric_daily_new_dash_created_services`
* Follows a similar pattern as for the rest of daily metric models. The only difference is that is aggregated to `id_booking` to ensure we can handle count distinct of bookings per different time granularities.
* Reads from the new pricing tables `int_core__booking_summary` and `int_core__booking_service_detail`. The main filters applied are selecting only new dash users and only services created after the user move timestamp to new dash.
An additional metric model at monthly level is created `int_kpis__metric_monthly_new_dash_created_services`
These finally go to a dimension aggregated model (`dimension`, `dimension_value`), respectively:
* Daily: `int_kpis__agg_daily_new_dash_created_services`
* Monthly: `int_kpis__agg_monthly_new_dash_created_services`
A final model aims to aggregate the different dimension aggregated metrics for New Dash: `int_kpis__product_new_dash_agg_metrics`
* It computes a `time_granularity` aggregation
* Here I will add additional metrics (such as revenue) once we have them.
A final model reading from the previous is exposed to reporting: `kpis__product_new_dash_agg_metrics`
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #20809
# Description
This fixes the data alert raised in today execution. We have the first user with the role `ScreenAndProtectApi`.
Changes:
* Adds `CancellationApi`, `CheckInHeroApi` and `ScreenAndProtectApi` as accepted roles for `int_core__user_host` computation
* Adds the new `ScreenAndProtectApi` role as accepted values in the `int_core__user_role` model
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #24475
# Description
This table provides the Booking that are under New Pricing. Some of them might also be under New Dashboard.
Changes on existing models:
* Added a few informative fields in the view `int_core__booking_to_service` that enables feeding Booking Service Detail and Booking Summary. These are: `id_verification_request`, `id_deal`, `is_user_in_new_dash`, `new_dash_version`, `user_in_new_dash_since_timestamp_at_utc`.
New models:
* Created a first version of `int_core__booking_summary`. This model mainly retrieves Booking attributes from `int_core__booking_to_service` and aggregates service information (paid services, upgraded services, types of services applied, total price, etc) from the detailed version of this table which is `int_core__booking_service_detail`.
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #20809
# Description
This change removes duplicated verification requests in bookings across different times, previously the model was only considering unique verification requests in a day but counting them twice if they were in different days.
# Checklist
- [x] The edited models and dependants run properly with production data.
- [x] The edited models are sufficiently documented.
- [x] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #23998
# Description
Minor Changes:
* Computation in staging of `is_default_service` column for Basic Screening. This includes `stg_core__product_service`, `stg_core__protection_plan` and the schema entry.
* Propagation of `is_default_service` to intermediate. This includes `int_core__protection_plan_to_price`, `int_core__product_service_to_price` and the schema entry.
* Adapting a small data test to include a new case in `stg_core__booking_view_to_service` schema entry.
Main Changes:
* New fields and re-order existing ones on `int_core__booking_to_service`.
* Changed materialisation of `int_core__booking_to_service` to a view, since the new model `int_core__booking_service_detail` is materialised as table.
New Model:
* First version of `int_core__booking_service_detail`. This model includes the current state of a Service that is applied within a Booking. It includes many Booking attributes, Service attributes as well as some meta data fields regarding the business scope and type, according to the [New Pricing](https://www.notion.so/knowyourguest-superhog/All-Services-Definitions-333fba1fbed54715b6b1c3b5133f0d4f) documentation.
* This is one of the 3 Booking models that I propose to create for New Pricing. You can read about the others [here](https://www.notion.so/knowyourguest-superhog/Services-and-Revenue-modelling-1420446ff9c980118e0cfffa7c41f369), although I'll probably won't do them until New Dash is finalised on tech side.
* This current model should suffice to be able to understand which services are applied within a Booking. Notably you will see that this is not enough to compute revenue, as there's some placeholders in place. This is expected.
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #20809
