# Description
This PR creates a proper Booking Source in `int_core__bookings` table. This mostly refers to ensure that whenever we do data requests tickets on this regards we have a proper logic in place (as last time we had issues because of this). Logic follows Gus recommendation on applying:
```
case
when upper(coalesce(bsrc.booking_source, 'UNKNOWN')) = 'UNKNOWN'
then b.raw_booking_source
else bsrc.booking_source
end
```
where b. is Booking table and bsrc. is Booking Source table.
Things to note:
* A new `stg_core__booking_source` table is created, full-refreshed from the backend (less than 10 records)
* I added a unique test for `stg_core__booking` that was not in there. I don't document the rest of the model though.
* I remove the previous `id_booking_source` as it wasn't used and the logic in place should be far more accurate.
* I do NOT remove the existing `verification_request_booking_source`. I don't remember what's this but I see this is in use specially on Guest Insights (CSAT) and the legacy Truvi reporting (Bookings, PMS, etc.). Not the scope of this ticket to change it.
* I don't do further propagation, but potentially, this could be extended to New Dash Booking Summary as this source is being shown in the dashboard itself.
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #30415
# Description
First version in staging of stg_core__user_product_bundle_to_guest_product.
This table informs which Guest Products are enabled for a given New Dash bundle (or program).
Be aware that for these to be effective, the bundle needs to be applied and active to an active listing that generates bookings.
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #30165
# Description
Improvements on onboarding data:
* Hubspot services that clients expressed their interest in are now sorted alphabetically. This is needed for:
* A new boolean to compare current active services applied in listings vs. services that captured the interest of the client on onboarding. This can help identify clients that we might need to follow up to ensure all programs are created and applied.
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #30249
# Description
Brings additional info from Hubspot Deals regarding onboarding process
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #30249
# Description
Removes 2 fields that no longer exist in the backend. Needed to do since I need to bring another table to sync_core.
No PBI dependencies found.
Fields are:
* flag_as_problem in SH User
* autonavigate in VerificationSet
# Checklist
- [X] The edited models and dependants run properly with production data.
- [ ] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.
# Other
- [X] Check if a full-refresh is required after this PR is merged.
Related work items: #30229
# Description
Fixes Billable Services categorisation. We were not handling Guest Agreement exclusion. I just changed so instead of including services as billable by default, it's the other way around.
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
# Description
Extracts Integration Type (OSL, PMS, AutoHost, other API stuff) from Hubspot Deals.
This is needed for the categorisation of Deals in the scope of Old Dash to New Dash migration.
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #28987
# Description
Removes incrementality in booking view to service
# Checklist
- [X] The edited models and dependants run properly with production data.
- [ ] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.
# Other
- [X] Check if a full-refresh is required after this PR is merged.
# Description
Brings 3 new properties from HubSpot - Deals:
* Dashboard Type: Old Dash, New Dash or null
* Pricing Structure: this contains several possibilities, such as new pricing v1, v2, old pricing (legacy), api pricing, etc.
* Partnership Services: what services the client has according to Hubspot. Not the best in terms of quality but better than nothing specially in Old Dash. I also handled a small processing since services were differently separated.
This is propagated to intermediate in `int_hubspot__deal`. This is needed to automate the pricing differences for clients in old dash to be moved to new dash.
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented. **Mostly on intermediate, did not bother to update staging besides the new fields.**
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #28627
# Description
After the sync this morning in Core in Airbyte, we have some issues in SH User. This attempts to remove no longer existing fields, namely:
- id_airbnb
- airbnb_url
- platform_comms_recipient
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #28502
# Description
Adds the remaining Guest Products tables to staging
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #28490
# Description
Adds `VerificationRequestGuestProductToPayment` and `VerificationRequestToGuestProduct` to staging.
Note these tables are empty thus it's possible that some tests might need to be adapted in the future.
I also already added the configuration for sync_guest_product sources.
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Guest Products - sync core tables
Related work items: #28490
# Description
First comment time added
# Checklist
- [x] The edited models and dependants run properly with production data.
- [x] The edited models are sufficiently documented.
- [x] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
first comment time added
Related work items: #27812
# Description
New staging model for Resolutions Incidents
It extracts all of the data from the incidents after the ingestion from CosmosDB.
This includes data on the host, the guest, the booking, the listing and all the resolution process until completed if completed.
Though it currently has a lot of problems like missing values, ids not existing in the back end or payments without currencies. Everything is more clarified here https://www.notion.so/truvi/Data-Quality-Report-Resolutions-Incidents-data-1a40446ff9c9807180eaccfeebb53045?pvs=4.
# Checklist
- [x] The edited models and dependants run properly with production data.
- [x] The edited models are sufficiently documented.
- [x] The edited models contain PK tests, and I've ran and passed them.
- [x] I have checked for DRY opportunities with other models and docs.
- [x] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #27811
# Description
Changes:
* Tags services to be billed (invoiced to host) in the staging models of product_service and protection_plan
* Propagates into booking_service_detail master table. This also computes billable dates.
* Propagates into booking_summary master table. This has the final determination of a booking being billable or not, and when the first and last billing should occur.
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #27619
# Description
Ensures `id_role` are lower case. This should avoid issues downstream in dbt tests.
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #26110
# Description
The model `stg_core__superhog_user` contained the fields `created_date_utc` and `updated_date_utc`, which were actually timestamps, not dates. This created issues a while ago in KPIs that were fixed by forcing a date conversion at `int_kpis` level, but the source issue remained unfixed.
This PR just creates the proper fields in staging, meaning:
* `created_at_utc` and `created_date_utc`
* `updated_at_utc` and `updated_date_utc`
And propagates the changes through all dependencies. On KPIs, it also removes the hardcoded date conversion since it's no longer needed. It also adds the different schema entries on which these fields were already defined. Note I didn't update the staging entry since there's no entry for the rest of the fields.
I recommend checking this PR in the commits order.
What I've tested:
* Unified User and User Host have plenty of dependencies that I've checked one by one, at each dependency layer by using dbt docs.
* Run `core__unified_user` and the full execution of +`mtd_aggregated_metrics`
What clearly will fail:
* There's the famous `Users_dashboard` PBI report that shouldn't exist that has a dependency with `core__unified_user`. This report will fail. But it does not exist. I'd like to drop it once and forever
# Checklist
- [X] The edited models and dependants run properly with production data. **See above**
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.
# Other
- [X] Check if a full-refresh is required after this PR is merged. **The staging model is incremental!**
Related work items: #23703
# Description
Propagates `AccountType` to staging by creating the first version of the model `stg_core__account_type`
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #18944
# Description
Staging model for all screen & protect price tables.
There are 4 types of protections for screen & protect and each of them have their own prices
# Checklist
- [x] The edited models and dependants run properly with production data.
- [x] The edited models are sufficiently documented.
- [x] The edited models contain PK tests, and I've ran and passed them.
- [x] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Staging model for all screen & protect price tables
Related work items: #25360
