Commit graph

1879 commits

Author SHA1 Message Date
Pablo Martin
efe3f50a37 relax test that didn't make sense, revert wrong solution to issue from this morning 2025-06-12 17:56:24 +02:00
Pablo Martin
2a5c82f62b Merged PR 5455: Bump dbt version to 1.9.8
# Description

This PR changes the `requirements.txt` to allow any patch equal or higher than 1.9.8, but below 1.10.0.

Related work items: #30881
2025-06-12 14:27:45 +00:00
Pablo Martin
9d100192c0 bump dbt version 2025-06-12 16:13:53 +02:00
Joaquin Ossa
20ce96cd29 Merged PR 5449: silly me
# Description

Missed the change of sign for `due_amount_in_local_currency`

# Checklist

- [x] The edited models and dependants run properly with production data.
- [ ] The edited models are sufficiently documented.
- [ ] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

silly me

Related work items: #30204
2025-06-12 10:16:47 +00:00
Joaquin
ae373a4fc9 silly me 2025-06-12 12:07:16 +02:00
Pablo Martin
20477dcc2c Merged PR 5447: Fix GJ Payments consistency with VRs
# Description

We're getting Data Alerts triggered for Guest Journey payments with no id for the guest. This is because we have payments in the DWH for which we still don't have the related Verification Request in the DWH as well.

This PR changes a `LEFT JOIN` with the verification request table into an `INNER JOIN`, effectively making it impossible for a payment to not have user data if the VR is not there.

We increase data latency (the payment won't show up until the related VR is in the DWH) but we win consistency (no orphan payments without their full details).

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

change left join to inner join
2025-06-12 08:06:08 +00:00
Joaquin Ossa
5abbff9560 Merged PR 5441: New fields for Invoicing & Crediting
# Description

New fields for Invoicing & Crediting that Finance needs for their usage.
Specifically they need to be able to differentiate the amount due because of invoice and not credit notes to know who to pay resolutions claims or not.
Will also add the amounts in local currency.

# Checklist

- [x] The edited models and dependants run properly with production data.
- [x] The edited models are sufficiently documented.
- [x] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

New fields for Invoicing & Crediting

Related work items: #30204
2025-06-12 07:23:11 +00:00
Joaquin
6c7e80eb75 Addressed comments 2025-06-12 09:22:22 +02:00
Joaquin Ossa
9c4b12e985 Merged PR 5438: Add Flex API into KPI models
# Description

Including Flex API revenue into KPI models.
Will also update the Main KPIs report

# Checklist

- [x] The edited models and dependants run properly with production data.
- [x] The edited models are sufficiently documented.
- [x] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Add Flex API into KPI models

Related work items: #31041
2025-06-11 15:01:22 +00:00
Joaquin Ossa
fcff46144f Merged PR 5431: Update accounting aggregation
# Description

Update accounting aggregation seed with the latest changes including the division of waiver payments, confident stay and some new updates from finance.

In the meantime I'll do a full refresh run, that will take quite a while, to check that I am not breaking anything

# Checklist

- [x] The edited models and dependants run properly with production data.
- [x] The edited models are sufficiently documented.
- [x] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Update accounting aggregation

Related work items: #30995
2025-06-11 15:01:17 +00:00
Joaquin
80651ce60e New fields for Invoicing & Crediting 2025-06-11 16:57:10 +02:00
Joaquin
940d8b3bcf Updated test 2025-06-11 15:26:44 +02:00
Joaquin Ossa
84e8100706 Merged PR 5439: Removed test
# Description

Remove test `not_null` from `stg_resolutions__incidents` since it is not actually mandatory for the booking to include the guest email information

# Checklist

- [x] The edited models and dependants run properly with production data.
- [x] The edited models are sufficiently documented.
- [x] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Removed test

Related work items: #31043
2025-06-11 13:09:30 +00:00
Joaquin
f482b64528 Removed test 2025-06-11 14:51:20 +02:00
Joaquin
4a4e17f068 Add Flex API into KPI models 2025-06-11 14:44:07 +02:00
Joaquin
3e3b7f049e Removed changes 2025-06-11 14:34:44 +02:00
Joaquin
55699a1647 Addressed comments and all KPI model changes 2025-06-11 12:25:14 +02:00
Joaquin
44b59aafd2 Update accounting aggregation 2025-06-11 09:09:57 +02:00
Oriol Roqué Paniagua
0021827368 Merged PR 5429: Rework Booking Source in int_core__bookings
# Description

This PR creates a proper Booking Source in `int_core__bookings` table. This mostly refers to ensure that whenever we do data requests tickets on this regards we have a proper logic in place (as last time we had issues because of this). Logic follows Gus recommendation on applying:

```
case
  when upper(coalesce(bsrc.booking_source, 'UNKNOWN')) = 'UNKNOWN'
  then b.raw_booking_source
  else bsrc.booking_source
end
```

where b. is Booking table and bsrc. is Booking Source table.

Things to note:
* A new `stg_core__booking_source` table is created, full-refreshed from the backend (less than 10 records)
* I added a unique test for `stg_core__booking` that was not in there. I don't document the rest of the model though.
* I remove the previous `id_booking_source` as it wasn't used and the logic in place should be far more accurate.
* I do NOT remove the existing `verification_request_booking_source`. I don't remember what's this but I see this is in use specially on Guest Insights (CSAT) and the legacy Truvi reporting (Bookings, PMS, etc.). Not the scope of this ticket to change it.
* I don't do further propagation, but potentially, this could be extended to New Dash Booking Summary as this source is being shown in the dashboard itself.

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #30415
2025-06-10 14:38:35 +00:00
Pablo Martin
47c51dd18d missing filter to avoid new vrs appearing in the old cte 2025-06-10 16:35:16 +02:00
Pablo Martin
61ceda5234 Merge branch 'master' of ssh.dev.azure.com:v3/guardhog/Data/data-dwh-dbt-project 2025-06-10 15:08:24 +02:00
Pablo Martin
70cf3ce312 missing cast to make union work 2025-06-10 15:08:18 +02:00
Joaquin Ossa
8b415fc726 Merged PR 5427: data alert fix
# Description

Change in model `int_core__accommodation_to_product_bundle` to remove effective dates and use original start and end date.
Updated downstream models as well

# Checklist

- [x] The edited models and dependants run properly with production data.
- [x] The edited models are sufficiently documented.
- [x] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

data alert fix

Related work items: #30948
2025-06-10 12:57:05 +00:00
Joaquin
5f2bd33eed addressed comments 2025-06-10 14:45:53 +02:00
Joaquin
c5c25af59b addressed comments 2025-06-10 14:42:59 +02:00
Joaquin
1bd2fd9585 updated comments 2025-06-10 14:22:40 +02:00
Joaquin
a543fa1d83 changed names 2025-06-10 14:20:28 +02:00
Joaquin
e85de15d22 data alert fix 2025-06-10 13:58:50 +02:00
Pablo Martin
c8e4c88074 rename model to make version name explicit 2025-06-10 12:43:49 +02:00
Pablo Martin
dd17dff93a Merged PR 5424: Bump int_core__check_in_cover_users version
# Description

This PR bumps the `latest_version` value of `int_core__check_in_cover_users` from `1` to `2`, effectively moving the project to the new logic of CIH users.

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [NA] The edited models contain PK tests, and I've ran and passed them.
- [NA] I have checked for DRY opportunities with other models and docs.
- [NA] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #30165
2025-06-10 10:22:20 +00:00
Pablo Martin
1f8debc41b bump version 2025-06-10 12:21:33 +02:00
Pablo Martin
3518d09970 Merged PR 5423: Update guest_products_feature_flag_activation_timestamp to real value
# Description

Now that the Guest Products have been activated, we need to update the project var `guest_products_feature_flag_activation_timestamp` so that the logic based on it throughout the project starts acting properly.

This PR updates the value of the var with the timestamp that Lawrence from the Guest Squad has shared with us.

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #30165
2025-06-10 10:21:13 +00:00
Pablo Martin
f185f8fe4b update var to actual timestamp 2025-06-10 12:20:08 +02:00
Pablo Martin
58df9fbe87 Merged PR 5422: Change if block to avoid whitespace trimming
# Description

Model `int_core__vr_check_in_cover` is producing invalid SQL because of a typo: a Jinja `if` block gets created with whitespace trimming, so the output gets concatenated to the previous code without whitespace and the resulting SQL is crap.

This PR changes the if block so it doesn't trim whitespace, hence producing valid SQL and the indended query.

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #30165
2025-06-10 10:19:23 +00:00
Pablo Martin
ad518c3083 change if block to avoid whitespace trimming 2025-06-10 11:55:11 +02:00
Oriol Roqué Paniagua
dc0abe6098 Merged PR 5403: Removes test accounts
# Description

This PR directly removes test accounts, and not just flag them, from:
* `int_core__unified_user`, which means is also propagated to `int_core__user_host`.

It does so by creating a new, extremely simple model named: `int_core__user_test_account`

Then, `int_core__user_host` is inner joined with a small explanation comment on:
* `int_core__accommodation`
* `int_core__unique_accommodation_to_user`
* `int_core__bookings`
* `int_core__verification_requests`
* `int_core__payments`

Effectively removing test accounts.

Lastly, any existence and usage of the column `is_test_account` has been removed -> mostly on New Dash-related models.

# Checklist

**As discussed in the daily, I'll be playing around in prod with this PR**

- [ ] The edited models and dependants run properly with production data.
- [ ] The edited models are sufficiently documented.
- [ ] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #27319
2025-06-06 12:17:49 +00:00
uri
08678427ad Bugfix - Host Resolutions Payments not filtering credit notes relevant status 2025-06-05 13:35:34 +02:00
Oriol Roqué Paniagua
9cbb748b0f Merged PR 5389: Adapt Test Account flag with Claim dedicated source
# Description

Applies the new dedicated source of truth of "is this account fake" based on Claims. This removes the previous logic.

In terms of differences, not much because:
1. New logic applied in the backend is pretty much the same as we had with a few improvements
2. This exclusion is only applied for New Dash. I checked the number of test accounts in New Dash and these are 2, the same ones in the new logic and for the old (currently in production). So no real impact.

Impact will come at the moment to start handling exclusions for KPIs, but this will done later on dedicated PRs to assess impacts properly.

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #27319
2025-06-04 12:58:40 +00:00
Joaquin Ossa
7f2a8a98b6 Merged PR 5385: id_deal unique test
# Description

Added unique test for `id_deal` and `id_contact` for `xero_contacts` models.
This should all work after the run

# Checklist

- [x] The edited models and dependants run properly with production data.
- [x] The edited models are sufficiently documented.
- [x] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

id_deal unique test

Related work items: #30809
2025-06-04 12:26:14 +00:00
Joaquin
5fdb5ffcde id_deal unique test 2025-06-04 12:28:52 +02:00
Pablo Martin
8b8b8441d3 Merged PR 5365: New version of CIH users through activity
# Description

This PR adds a new version to `int_core__check_in_cover_users`. The new version respectes the public interface of V1, but changes the strategy to determine which users get included in the list. Instead of relying on their account configuration (v1), v2 simply looks at whether they had any VR created offering CIH in the past 30 days.

This approach changes the semantic of the model, but we deem the new one reasonable.

The motivation behind this change is to ensure this models (1) keeps working after the activation of the Guest Products FF and (2) stays simple. Trying to follow a strategy where we look at both old and new configs was deemed very complex and didn't add much value.

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #30165
2025-06-04 08:55:55 +00:00
Pablo Martin
a401153472 be more precise 2025-06-04 10:54:27 +02:00
Pablo Martin
0c11cfe739 removed unused upstream models 2025-06-04 10:53:24 +02:00
Pablo Martin
159813544f comment on why user and not host 2025-06-04 10:53:01 +02:00
Pablo Martin
54fca806b9 6 months, not one 2025-06-04 10:48:16 +02:00
Oriol Roqué Paniagua
8b8e4143ce Merged PR 5381: Removes multi-user test on New Dash onboarding
# Description

It doesn't look like the multi-user cases per deal are going to be fixed any time soon.
Also, this test was only covering it for newly onboarded deals, while we have more cases for migrated deals that this test was not covering.

# Checklist

- [X] dbt compiles

Related work items: #30479
2025-06-04 08:08:40 +00:00
Joaquin Ossa
536ab9b2a1 Merged PR 5364: Outstanding Payments
# Description

Adds a field that helps categorize whether we owe money to an account or they owe money to Truvi.
Outstanding Resolutions or Outstanding Invoices

The idea is to include this field as a filter in the Invoicing & Crediting report to help Finance and Resolutions in the payment and approval process.

# Checklist

- [x] The edited models and dependants run properly with production data.
- [x] The edited models are sufficiently documented.
- [x] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Outstanding Payments

Related work items: #30204
2025-06-03 12:53:59 +00:00
Joaquin
1b362024f1 addressed comments 2025-06-03 14:17:47 +02:00
Joaquin
d50442e887 New model for deals total due amount 2025-06-03 11:53:40 +02:00
Joaquin
2746977671 commit wip 2025-06-03 10:53:22 +02:00