# Description
Small PR to fix stuff
# Checklist
- [ ] The edited models and dependants run properly with production data.
- [ ] The edited models are sufficiently documented.
- [ ] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Fixes
Related work items: #27096
# Description
Added booking fees for CIH requests and new model of aggregated monthly fee per user
# Checklist
- [x] The edited models and dependants run properly with production data.
- [x] The edited models are sufficiently documented.
- [x] The edited models contain PK tests, and I've ran and passed them.
- [x] I have checked for DRY opportunities with other models and docs.
- [x] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #26158
# Description
Context: I'm intending to work on Account Managers reporting, that mostly will include reporting at Deal level the Resolutions Payouts as well as the new Retained metrics. While checking the great increase on Resolutions Payouts for October 2024:
I decided to take a quick look into the main players... and surprise surprise we have Guesty:
So Guesty represents 37k over the 73K of October. 50%. Not bad.
The main issue is that we've been aware for months now (since Churn efforts, mostly) that we're not reporting in KPIs those deals that are NOT created in Core. Most notably, API deals which includes... well, Guesty. So creating this kind of in-depth Account Managers improvement without reporting Guesty I think it would be very misleading. Note that the overall figures (Global dimension) are still accurate, though.
What's new:
* A new model named `int_kpis__dimension_deals` that basically retrieves Deals from both Core (as before) and Hubspot. It combines information from both and mostly assumes Hubspot as a better source of information than Core - although we do not have the Main Billing Country there afaik.
* Propagates changes, mostly in the monthly by deal view of Main KPIs. Here I confirm that now Guesty appears, and it only has metrics that come from Xero (APIs Revenue, Total Revenue, Resolutions, etc)
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #25829
# Description
The model `stg_core__superhog_user` contained the fields `created_date_utc` and `updated_date_utc`, which were actually timestamps, not dates. This created issues a while ago in KPIs that were fixed by forcing a date conversion at `int_kpis` level, but the source issue remained unfixed.
This PR just creates the proper fields in staging, meaning:
* `created_at_utc` and `created_date_utc`
* `updated_at_utc` and `updated_date_utc`
And propagates the changes through all dependencies. On KPIs, it also removes the hardcoded date conversion since it's no longer needed. It also adds the different schema entries on which these fields were already defined. Note I didn't update the staging entry since there's no entry for the rest of the fields.
I recommend checking this PR in the commits order.
What I've tested:
* Unified User and User Host have plenty of dependencies that I've checked one by one, at each dependency layer by using dbt docs.
* Run `core__unified_user` and the full execution of +`mtd_aggregated_metrics`
What clearly will fail:
* There's the famous `Users_dashboard` PBI report that shouldn't exist that has a dependency with `core__unified_user`. This report will fail. But it does not exist. I'd like to drop it once and forever
# Checklist
- [X] The edited models and dependants run properly with production data. **See above**
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.
# Other
- [X] Check if a full-refresh is required after this PR is merged. **The staging model is incremental!**
Related work items: #23703
# Description
Rename changes and adding some necessary fields like `id_currency` to some S&P models
# Checklist
- [x] The edited models and dependants run properly with production data.
- [x] The edited models are sufficiently documented.
- [x] The edited models contain PK tests, and I've ran and passed them.
- [x] I have checked for DRY opportunities with other models and docs.
- [x] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Feedback on current process
Related work items: #25360
# Description
Propagates account type to intermediate. Also, adds minimal documentation to `int_core__unified_user`.
There's only 54 test accounts in Unified User and 49 that are considered as User Hosts. So it's not, by all means, an extensive list of test accounts...
There's a mismatch between the `created_date_utc` and `updated_date_utc` that these are timestamps. There's another ticket in the backlog to handle this alongside the dependency, so it's not included here.
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #18944
# Description
Following a communication from Gus, the following fields in Booking table are going to be removed:
* StagingHostBookingId
* AdditionalGuests
This PR aims to remove DWH dependencies.
# Checklist
- [X] The edited models and dependants run properly with production data.
- [NA] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [NA] I have checked for DRY opportunities with other models and docs.
- [NA] I've picked the right materialization for the affected models.
# Other
- [X] Check if a full-refresh is required after this PR is merged. **Yes, Booking table in staging is incremental**
Related work items: #25405
# Description
Adds a table that contain the main information needed for metric computation at verification request level that is part of an A/B test.
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [NA] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #25146
# Description
A very simple, clean table to get the records needed for A/B test tracking on GJ side.
The table already removes weird cases that are somehow usual within A/B test configurations, to ensure that the monitoring is as clean (and less biased) as possible.
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #25145
# Description
This PR aims to properly fix the Accommodation removal of fields that have been dropped today.
# Checklist
- [X] The edited models and dependants run properly with production data.
- [NA] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [NA] I have checked for DRY opportunities with other models and docs.
- [NA] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #25199
# Description
This PR aims to do 2 things:
1. Create the first metric daily model for New Dash - Chargeable Services
* PK at Booking, Date and Service. I added a few more dimensions such as Accommodation and Business Type. The idea is that while Daily Unique Bookings/Accommodations Charged will be close (if not the same) as Charged Services, by having the ID we can compute Monthly/Weekly Unique Bookings/Accommodations Charged in a proper manner. Besides this, we would still compute additive metrics in the future such as the sum of Charged Services and the sum of Service Total Price in GBP.
* `IMPORTANT`: as discussed in the daily, I changed New Pricing models containing "Charged" columns to "Chargeable". This affects the new model for New Dash KPIs, but also `int_core__booking_summary` and `int_core__booking_service_detail`.
2. Small fixes on New Dash - Created Services.
* Mainly, there were some inconsistencies with what was actually written (and really applied) in the schema entry as for what was the PK of the model vs. what was stated in the model. I just re-ordered the columns and re-specified correctly the PK.
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #20809
# Description
Adds new role `ResolutionApi` in accepted values of `int_core__user_role`
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [NA] I have checked for DRY opportunities with other models and docs.
- [NA] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
# Description
Adds informative fields to New Pricing booking-related tables. These are added into booking_to_service which is a view, and propagated and materialised as table in the `booking_service_detail` and `booking_summary`.
The new fields are:
- Date versions for relevant booking timestamps (Created, Updated, Check-in, Check-out)
- Booking Number of nights
- Ids for the Accommodation and UserProductBundle
- Program Name (aka Product Bundle Name). I dislike the fact that it's named "Product" because you can have Protection Plans in the Program as well... so I opted for a more conventional business-oriented naming, which is Program.
Added a bunch of tests + documentation.
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #20809
# Description
Changes:
* Handles conversion to GBP directly in the models `int_core__product_service_billing_item` and `int_core__protection_plan_billing_item` .
* Adds the newly created field of `is_missing_id_deal` in `int_core__booking_to_service`, to avoid manually computing downstream.
* Computes total price per service in `int_core__booking_service_detail`. Since, technically, the same service can have different billing lines, I also retrieve some min/max charge dates and some booleans to help the identification.
* Computes total price per booking in `int_core__booking_summary`. Since, technically, the same booking can have different services charged in different moments in time, I also retrieve some min/max charge dates. I also computed a very necessary `is_booking_charged` to understand if we're making money or not out of it.
This PR should provide the necessary fields to start computing Revenue for New Dash.
HOWEVER:
1) I still need to handle Guest Payments computation for Waiver/Deposit services. I'll do it in a separated PR.
2) while doing this exercise I noticed that the current data is not good / consistent with what I was expecting in terms of New Pricing documentation. I will check with Dash Squad so it's correctly filled, before reporting fake numbers...
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #20809
# Description
Adds Billing models in intermediate. It's mostly the similar content as for the staging counterparts, but providing useful information such as the currency code and service that this billing line refers to. Same as before, two separate models for Protection Plan and Product Service.
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #20809
# Description
This PR has 2 commits:
- The first one handles the removal from the computation any user that has not an id deal properly set. I just created a boolean field in int_core__user_host that identifies if the host has no id_deal. Then apply the new condition in the 2 main usages of New Dash info.
- The second one cleans New Dash KPIs. Since we do not have anymore users without deals, it means that the identification of the host/account is going to be exactly the same if done by id_user_host or id_deal. I hated having id_user_host in KPIs so I've removed it :)
Lastly, this should speed up massively the execution. Not because there's improvements on the code but rather the reduction of data.
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #20809
# Description
This fixes the data alert raised in today execution. We have the first user with the role `ScreenAndProtectApi`.
Changes:
* Adds `CancellationApi`, `CheckInHeroApi` and `ScreenAndProtectApi` as accepted roles for `int_core__user_host` computation
* Adds the new `ScreenAndProtectApi` role as accepted values in the `int_core__user_role` model
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #24475
# Description
This table provides the Booking that are under New Pricing. Some of them might also be under New Dashboard.
Changes on existing models:
* Added a few informative fields in the view `int_core__booking_to_service` that enables feeding Booking Service Detail and Booking Summary. These are: `id_verification_request`, `id_deal`, `is_user_in_new_dash`, `new_dash_version`, `user_in_new_dash_since_timestamp_at_utc`.
New models:
* Created a first version of `int_core__booking_summary`. This model mainly retrieves Booking attributes from `int_core__booking_to_service` and aggregates service information (paid services, upgraded services, types of services applied, total price, etc) from the detailed version of this table which is `int_core__booking_service_detail`.
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #20809
# Description
Minor Changes:
* Computation in staging of `is_default_service` column for Basic Screening. This includes `stg_core__product_service`, `stg_core__protection_plan` and the schema entry.
* Propagation of `is_default_service` to intermediate. This includes `int_core__protection_plan_to_price`, `int_core__product_service_to_price` and the schema entry.
* Adapting a small data test to include a new case in `stg_core__booking_view_to_service` schema entry.
Main Changes:
* New fields and re-order existing ones on `int_core__booking_to_service`.
* Changed materialisation of `int_core__booking_to_service` to a view, since the new model `int_core__booking_service_detail` is materialised as table.
New Model:
* First version of `int_core__booking_service_detail`. This model includes the current state of a Service that is applied within a Booking. It includes many Booking attributes, Service attributes as well as some meta data fields regarding the business scope and type, according to the [New Pricing](https://www.notion.so/knowyourguest-superhog/All-Services-Definitions-333fba1fbed54715b6b1c3b5133f0d4f) documentation.
* This is one of the 3 Booking models that I propose to create for New Pricing. You can read about the others [here](https://www.notion.so/knowyourguest-superhog/Services-and-Revenue-modelling-1420446ff9c980118e0cfffa7c41f369), although I'll probably won't do them until New Dash is finalised on tech side.
* This current model should suffice to be able to understand which services are applied within a Booking. Notably you will see that this is not enough to compute revenue, as there's some placeholders in place. This is expected.
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #20809
# Description
Goal of the PR is start modelising the Services and Revenue for New Dash/New Pricing. Be aware that this is a first model and it's not by all means the definitive nor complete version.
Changes:
* Services name normalisation: It's a bit painful because we usually had two names: 'MyService' and 'My Service'. The initial setup I built was considering the first, but in some new integrated tables it only has the second. So I just standardised everything to be in upper case and with spaces such as 'MY SERVICE'. Might make more sense later on. This affects many existing models.
* Compute a new Service Business Type field in `stg_core__product_service`: Main reason is that effectively Platform scope in New Pricing have business differences in the services types but this does not exist in the backend. I do it at staging level with a dedicated accepted values test to ensure any new update is captured and notified. You can read more about the business logic in this [Notion page](https://www.notion.so/knowyourguest-superhog/All-Services-Definitions-333fba1fbed54715b6b1c3b5133f0d4f).
* Create a new model `int_core__booking_to_service`: this model joins the information of BookingViewToService with BookingView and Booking tables to retrieve a more consolidated version of what we will needed downstream for the future table of Booking Service Detail and the Aggregated version (not included in this PR). This table likely can be enriched in future PRs. At the moment it just contains the bare minimum fields needed for the lines/aggregated versions. (NB: Booking information is needed, for instance, for the services which unit price need to be applied for the number of nights of the booking)
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.
# Other
- [X] Check if a full-refresh is required after this PR is merged. **stg_core__booking_view_to_service needs full refresh**
Related work items: #20809
# Description
Removes Address Validation within DWH.
Note that I keep the 2 reporting fields, nullified. These should be removed after the removal of these fields in PBI.
# Checklist
- [X] The edited models and dependants run properly with production data.
- [NA] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [NA] I have checked for DRY opportunities with other models and docs.
- [NA] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #23976
# Description
Eliminates models that have been switched with new kpis flow.
Also deletes temporary tests and schema entries.
# Checklist
- [X] The edited models and dependants run properly with production data.
- [NA] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [NA] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #23762
# Description
Removes old computation of KPIs of Bookings, Guest Journeys and Guest Payments, after yesterday's switch to new computation.
It cleans 1) models 2) schema entries and 3) temporary tests.
# Checklist
- [X] The edited models and dependants run properly with production data.
- [NA] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them. **(Except for the duplicated Booking to Product Bundle)**
- [X] I have checked for DRY opportunities with other models and docs.
- [NA] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Remove previous KPIs computation for Bookings, Guest Journeys and Guest Payments
Related work items: #23576
# Description
Considers the 2 Claims NewDashVersion and NewDashMoveDate as sources of truth, removing the previous (crazy) logic.
If a user has the claim NewDashVersion, then it's in New Dash. The claim value will also provide the version in which the user appeared (MVP, V2, etc)
If a user has the NewDashMoveDate, it means it has moved from Old Dash. If not, but still has NewDashVersion, it means the user was directly created in New Dash.
The models now provide logic to handle these cases, and it's propagated downstream will ensuring reporting will still work.
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [NOT AT ALL] The edited models contain PK tests, and I've ran and passed them. **MANY ISSUES ON PRODUCTION CURRENTLY**
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Changes claims and logic to consider user is in new dash. Downstream propagation included
Related work items: #23457
# Description
Sets up warning for KPIs models that will be deprecated, which are:
* 13 in core
* 2 in xero
* 5 in cross
I will keep alive the cross models that handle the final aggregations for Main KPIs for the time being, as well as the newly created Churn model that has a dependency on the monthly by deal to be filled into the mtd flow. I think handling exposure logic for Main KPIs could be a separated migration.
In other words, this is already quite a bit to migrate.
# Checklist
**I just checked that dbt compiles correctly**
- [NA] The edited models and dependants run properly with production data.
- [NA] The edited models are sufficiently documented.
- [NA] The edited models contain PK tests, and I've ran and passed them.
- [NA] I have checked for DRY opportunities with other models and docs.
- [NA] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #23268
# Description
Decoms verification payments v1 version, and keeps v2 alive. The PBI of Guest Payments (Biz Overview) was already switched to V2 but the exposures was not reflecting it, so I changed this. Then I dropped all v1 models for verification payments and their entries in schema.
# Checklist
- [X] The edited models and dependants run properly with production data. I did run the upstream dependencies on V2 and I don't see orphans in my dbt docs. However I did not fully run end-to-end all models up to PBI reports since there's other sources involved (Xero, etc).
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [NA] I have checked for DRY opportunities with other models and docs.
- [NA] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #22611
# Description
Moves from intermediate/core to intermediate/cross the following models:
- `int_core__mtd_deal_lifecycle`
- `int_core__mtd_deal_metrics`
to their equivalents:
- `int_mtd_deal_lifecycle`
- `int_mtd_deal_metrics`
This also changes the schema entries, from core to cross, including changing the name of the model in the entry.
This also changes the dependencies, namely in `int_mtd_deal_metrics`, `int_mtd_vs_previous_year_metrics` and `int_monthly_aggregated_metrics_history_by_deal`.
This does NOT aim to alter the logic of the lifecycle in any case; it will be done in a separated PR.
Runs correctly end-to-end. We might need to drop the old models from production manually.
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #22689
# Description
Propagates Protection Plan to price and cover to intermediate. Ideally I wanted to have a single table that contained prices + covers, but it is not straight forward to assume:
* if a modification of the amount covered by a protection plan will create a new protection plan id, or not
* if a modification of the price a protection costs will create a new protection plan id, or not
So I keep it split for the time being. Besides, use cases might be different, ex: I want to see all prices of services (product + protection plan, but no need for cover).
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #20809
