# Description
This relates to today's data alerts on Growth Score models.
Error has been found in `int_billable_items_growth_score_by_deal`. This only happens on the 1st of the month.
On the 1st day of the month we will have:
* The actual monthly data from KPIs in the previous month.
But also:
* Projected KPIs will still display data for the previous month, in this case though, projected = actuals.
There's a union all that combines actual monthly KPI data for history purposes and the projected data which fails in this case.
Fix is to remove the data from Projected KPIs if the current date of execution (1st of July) equals the first day of the month when computing `end_date` + 1 day. End date being at this moment 30th of June, + 1 day = 1st of July, extracting the first day of the month then 1st July.
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [NA] I have checked for DRY opportunities with other models and docs.
- [NA] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #31665
# Description
Update exposures to include Guest Journey A/B test notebook
Created a new A/B test monitoring model in reporting.
Removed tests of the intermediate model so we don't have duplicated tests since it is basically the same model in reporting
# Checklist
- [x] The edited models and dependants run properly with production data.
- [x] The edited models are sufficiently documented.
- [x] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Update exposures gj ab test
Related work items: #31448
# Description
This PR modifies the test script messaging towards slack. Until now, when tests failed, we simply sent a general message to slack saying something failed. With this PR, we will send one message per test, detailing the table and test, number of bad records, and the compiled test's SQL query that can help us debug.
On testing success, the output will still be a simple success message.
# Checklist
NA
# Other
NA
Related work items: #31476
# Description
Removed tests for non mandatory fields
# Checklist
- [x] The edited models and dependants run properly with production data.
- [x] The edited models are sufficiently documented.
- [x] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Fixed tests from resolutions
Related work items: #31122
# Description
Finished reporting tests for lifecycle
# Checklist
- [x] The edited models and dependants run properly with production data.
- [x] The edited models are sufficiently documented.
- [x] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Finished reporting tests
Related work items: #31340
# Description
Finished updating tests in reporting model
# Checklist
- [x] The edited models and dependants run properly with production data.
- [x] The edited models are sufficiently documented.
- [x] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
finished updating tests in reporting model
Related work items: #31122
# Description
Removed not_null tests for not mandatory fields, guest, host and listing data.
Manual forms are being excluded from the models because of the unreliability of the data
# Checklist
- [x] The edited models and dependants run properly with production data.
- [x] The edited models are sufficiently documented.
- [x] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Removed not_null tests
Related work items: #31122
# Description
Small change to remove new deals without lifecycle state due to joining Truvi late yesterday but showing in Hubspot with effective date early today
# Checklist
- [x] The edited models and dependants run properly with production data.
- [x] The edited models are sufficiently documented.
- [x] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #31340
# Description
Fixes:
* There was a data quality issue in which active listings with bundles were showing more listings than the total active listings. This has been fixed, plus a bunch of tests have been added to make sure this does not happen again.
* There's a field that reaches the 63 character limit. I modified "paid" per "upgraded" which reduces this to intermediate model; however in reporting I need to keep as is to avoid breaking PBI. I did change the name of the field in the schema so tests/docs work well and explicitly mention this in the doc. This will need adaptation but can be done in the future.
* There was tons of dead code tagged as to be deleted that was not deleted, not sure why or when. Anyway this is dropped now. It's not used in PBI except for the fact that columns are dropped in the data model, so we might have a bit of down time to just remove these.
* Few fields were not displayed correctly in dbt docs.
What's new:
* New user categorisation as an alert-based system. This can be enhanced in the future but so far it covers hell of a lot of cases. This also includes data quality alerts (deal not in HS, or account defaults to MVP launch).
* MVP Launch date moved to project yaml since it's a very standard date.
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #30407
# Description
Missed the change of sign for `due_amount_in_local_currency`
# Checklist
- [x] The edited models and dependants run properly with production data.
- [ ] The edited models are sufficiently documented.
- [ ] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
silly me
Related work items: #30204
# Description
We're getting Data Alerts triggered for Guest Journey payments with no id for the guest. This is because we have payments in the DWH for which we still don't have the related Verification Request in the DWH as well.
This PR changes a `LEFT JOIN` with the verification request table into an `INNER JOIN`, effectively making it impossible for a payment to not have user data if the VR is not there.
We increase data latency (the payment won't show up until the related VR is in the DWH) but we win consistency (no orphan payments without their full details).
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
change left join to inner join
# Description
New fields for Invoicing & Crediting that Finance needs for their usage.
Specifically they need to be able to differentiate the amount due because of invoice and not credit notes to know who to pay resolutions claims or not.
Will also add the amounts in local currency.
# Checklist
- [x] The edited models and dependants run properly with production data.
- [x] The edited models are sufficiently documented.
- [x] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
New fields for Invoicing & Crediting
Related work items: #30204
# Description
Including Flex API revenue into KPI models.
Will also update the Main KPIs report
# Checklist
- [x] The edited models and dependants run properly with production data.
- [x] The edited models are sufficiently documented.
- [x] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Add Flex API into KPI models
Related work items: #31041
# Description
Update accounting aggregation seed with the latest changes including the division of waiver payments, confident stay and some new updates from finance.
In the meantime I'll do a full refresh run, that will take quite a while, to check that I am not breaking anything
# Checklist
- [x] The edited models and dependants run properly with production data.
- [x] The edited models are sufficiently documented.
- [x] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Update accounting aggregation
Related work items: #30995
# Description
Remove test `not_null` from `stg_resolutions__incidents` since it is not actually mandatory for the booking to include the guest email information
# Checklist
- [x] The edited models and dependants run properly with production data.
- [x] The edited models are sufficiently documented.
- [x] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Removed test
Related work items: #31043
# Description
This PR creates a proper Booking Source in `int_core__bookings` table. This mostly refers to ensure that whenever we do data requests tickets on this regards we have a proper logic in place (as last time we had issues because of this). Logic follows Gus recommendation on applying:
```
case
when upper(coalesce(bsrc.booking_source, 'UNKNOWN')) = 'UNKNOWN'
then b.raw_booking_source
else bsrc.booking_source
end
```
where b. is Booking table and bsrc. is Booking Source table.
Things to note:
* A new `stg_core__booking_source` table is created, full-refreshed from the backend (less than 10 records)
* I added a unique test for `stg_core__booking` that was not in there. I don't document the rest of the model though.
* I remove the previous `id_booking_source` as it wasn't used and the logic in place should be far more accurate.
* I do NOT remove the existing `verification_request_booking_source`. I don't remember what's this but I see this is in use specially on Guest Insights (CSAT) and the legacy Truvi reporting (Bookings, PMS, etc.). Not the scope of this ticket to change it.
* I don't do further propagation, but potentially, this could be extended to New Dash Booking Summary as this source is being shown in the dashboard itself.
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #30415
# Description
Change in model `int_core__accommodation_to_product_bundle` to remove effective dates and use original start and end date.
Updated downstream models as well
# Checklist
- [x] The edited models and dependants run properly with production data.
- [x] The edited models are sufficiently documented.
- [x] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
data alert fix
Related work items: #30948
