Commit graph

33 commits

Author SHA1 Message Date
Oriol Roqué Paniagua
321f48ca7d Merged PR 3506: Remove Address Validation
# Description

Removes Address Validation within DWH.
Note that I keep the 2 reporting fields, nullified. These should be removed after the removal of these fields in PBI.

# Checklist

- [X] The edited models and dependants run properly with production data.
- [NA] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [NA] I have checked for DRY opportunities with other models and docs.
- [NA] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #23976
2024-11-14 11:59:36 +00:00
Joaquin Ossa
6f77309b8c addressed comments 2024-11-12 17:10:42 +01:00
Joaquin Ossa
463f5edfc3 new dash new data 2024-11-12 16:33:55 +01:00
Oriol Roqué Paniagua
2f80642f6c Merged PR 3472: Remove unused models and schema entries for deals and accommodations
# Description

Eliminates models that have been switched with new kpis flow.
Also deletes temporary tests and schema entries.

# Checklist

- [X] The edited models and dependants run properly with production data.
- [NA] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [NA] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #23762
2024-11-08 12:05:40 +00:00
Oriol Roqué Paniagua
c21b05f9cf Merged PR 3420: Remove previous KPIs computation for Bookings, Guest Journeys and Guest Payments
# Description

Removes old computation of KPIs of Bookings, Guest Journeys and Guest Payments, after yesterday's switch to new computation.

It cleans 1) models 2) schema entries and 3) temporary tests.

# Checklist

- [X] The edited models and dependants run properly with production data.
- [NA] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them. **(Except for the duplicated Booking to Product Bundle)**
- [X] I have checked for DRY opportunities with other models and docs.
- [NA] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Remove previous KPIs computation for Bookings, Guest Journeys and Guest Payments

Related work items: #23576
2024-11-05 10:12:00 +00:00
Oriol Roqué Paniagua
03797827a4 Merged PR 3396: Changes claims and logic to consider user is in new dash
# Description

Considers the 2 Claims NewDashVersion and NewDashMoveDate as sources of truth, removing the previous (crazy) logic.

If a user has the claim NewDashVersion, then it's in New Dash. The claim value will also provide the version in which the user appeared (MVP, V2, etc)

If a user has the NewDashMoveDate, it means it has moved from Old Dash. If not, but still has NewDashVersion, it means the user was directly created in New Dash.

The models now provide logic to handle these cases, and it's propagated downstream will ensuring reporting will still work.

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [NOT AT ALL] The edited models contain PK tests, and I've ran and passed them. **MANY ISSUES ON PRODUCTION CURRENTLY**
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Changes claims and logic to consider user is in new dash. Downstream propagation included

Related work items: #23457
2024-10-31 15:45:28 +00:00
Oriol Roqué Paniagua
450975301a Merged PR 3373: Setting current KPIs models for deprecation
# Description

Sets up warning for KPIs models that will be deprecated, which are:
* 13 in core
* 2 in xero
* 5 in cross

I will keep alive the cross models that handle the final aggregations for Main KPIs for the time being, as well as the newly created Churn model that has a dependency on the monthly by deal to be filled into the mtd flow. I think handling exposure logic for Main KPIs could be a separated migration.

In other words, this is already quite a bit to migrate.

# Checklist

 **I just checked that dbt compiles correctly**

- [NA] The edited models and dependants run properly with production data.
- [NA] The edited models are sufficiently documented.
- [NA] The edited models contain PK tests, and I've ran and passed them.
- [NA] I have checked for DRY opportunities with other models and docs.
- [NA] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #23268
2024-10-30 08:54:59 +00:00
Oriol Roqué Paniagua
5735251c67 Merged PR 3239: Decommission Verification Payments V1
# Description

Decoms verification payments v1 version, and keeps v2 alive. The PBI of Guest Payments (Biz Overview) was already switched to V2 but the exposures was not reflecting it, so I changed this. Then I dropped all v1 models for verification payments and their entries in schema.

# Checklist

- [X] The edited models and dependants run properly with production data. I did run the upstream dependencies on V2 and I don't see orphans in my dbt docs. However I did not fully run end-to-end all models up to PBI reports since there's other sources involved (Xero, etc).
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [NA] I have checked for DRY opportunities with other models and docs.
- [NA] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #22611
2024-10-18 09:59:29 +00:00
Oriol Roqué Paniagua
004616bb79 Merged PR 3187: Move deal lifecycle related models to cross
# Description

Moves from intermediate/core to intermediate/cross the following models:
- `int_core__mtd_deal_lifecycle`
- `int_core__mtd_deal_metrics`

to their equivalents:
- `int_mtd_deal_lifecycle`
- `int_mtd_deal_metrics`

This also changes the schema entries, from core to cross, including changing the name of the model in the entry.
This also changes the dependencies, namely in `int_mtd_deal_metrics`, `int_mtd_vs_previous_year_metrics` and `int_monthly_aggregated_metrics_history_by_deal`.
This does NOT aim to alter the logic of the lifecycle in any case; it will be done in a separated PR.

Runs correctly end-to-end. We might need to drop the old models from production manually.

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #22689
2024-10-16 11:56:49 +00:00
uri
7302790d42 Add new API role CancellationApi in user_role accepted values 2024-10-09 10:17:34 +02:00
Oriol Roqué Paniagua
ea8972a91b Merged PR 3090: Propagates Protection Plan to intermediate, for price and cover
# Description

Propagates Protection Plan to price and cover to intermediate. Ideally I wanted to have a single table that contained prices + covers, but it is not straight forward to assume:
*  if a modification of the amount covered by a protection plan will create a new protection plan id, or not
* if a modification of the price a protection costs will create a new protection plan id, or not
So I keep it split for the time being. Besides, use cases might be different, ex: I want to see all prices of services (product + protection plan, but no need for cover).

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #20809
2024-10-08 11:44:52 +00:00
Oriol Roqué Paniagua
bad27ecbc3 Merged PR 3087: Adding int_core__product_service_to_price
# Description

Adds int_core__product_service_to_price.

This model is the equivalent of staging but just adding currency code, product service name and a few boolean fields to identify if the product service to price is currently active and if it's the default price or not.
Currently it's a view since it has 16 records, likely it can be transformed into a table in the future.

The idea will be to create a similar table for protection_plan.

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [NA] I've picked the right materialization for the affected models. -> We will see in the future if needs to be adapted

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #20809
2024-10-08 11:39:34 +00:00
uri
911603f2e8 Documenting protection name fields in user_product_bundle 2024-10-07 17:21:52 +02:00
uri
4de5c08a26 Keep latest booking to product bundle state 2024-10-04 10:32:15 +02:00
Oriol Roqué Paniagua
3480d4d787 Merged PR 3047: Override migration date with join date if join date is after migration date
# Description

Quick fix to make lower_limit_migration_date_utc more robust for new dash reporting purposes. I noticed some cases in which users were actually created after the migration date. In these cases, I just keep joined date.

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #22331
2024-10-04 07:16:52 +00:00
Oriol Roqué Paniagua
8e4e967c20 Merged PR 3028: Adding int_core__user_product_bundle_contains_services
# Description

This PR adds a new table named `user_product_bundle_contains_services` in intermediate core.

Mainly, this table serves as a bridge between `user_product_bundle` and `product_services`. A `product_bundle` within `user_product_bundle` can contain 1 or several services, and this was stated in the field `contains_product_services`. The value of this field corresponds to the sum of `product_service_binary_tier` from the services that apply within that bundle. Even though the information is quite concise using this format, it adds extra complexity for analytical purposes, so this new table just duplicates the `user_product_bundle` main attributes as many times as services are contained.

For example:
`id_user_product_bundle` = 383 contains one unique bundle named Basic Program. This bundle has the `chosen_product_services` = 257, which can only be decomposed in power of 2 as of the sum of 256 + 1. This bundle therefore contains 2 services, Basic Screening (`product_service_binary_tier` = 1) and Waiver Pro (`product_service_binary_tier` = 256). Thus, in the new table, we will have 2 records and remove all this logic to something more standard, as seen in this screenshot:

![image (2).png](https://guardhog.visualstudio.com/4148d95f-4b6d-4205-bcff-e9c8e0d2ca65/_apis/git/repositories/54ac356f-aad7-46d2-b62c-e8c5b3bb8ebf/pullRequests/3028/attachments/image%20%282%29.png)

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [ ] The edited models contain PK tests, and I've ran and passed them. *N/A - there's no PK in this table, but the combination of columns that should be unique is tested*
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #20809
2024-10-02 14:54:03 +00:00
Pablo Martín
7d9d9faf71 Merged PR 3024: Activate country test for taxes
# Description

Activates a commented out test for verification payments v2. The test was deactivated because of some master data issues in the backend, but they were fixed after  #22148.

# Checklist

- [NA] The edited models and dependants run properly with production data.
- [NA ] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [NA] I have checked for DRY opportunities with other models and docs.
- [NA] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #20043
2024-10-02 09:09:18 +00:00
Pablo Martin
6124c19f98 activate one test, prepare another but keep it commented 2024-09-27 16:11:34 +02:00
Joaquin Ossa
37d9f5250a Added new role 2024-09-27 09:55:37 +02:00
Joaquin Ossa
9e62786aa9 commit2 2024-09-18 11:26:23 +02:00
Pablo Martin
a20d511b93 missing text 2024-09-17 16:12:38 +02:00
Pablo Martin
049c2d3e31 model and docs+tests 2024-09-17 15:47:45 +02:00
Pablo Martin
f3e9985d0f model and docs 2024-09-17 12:16:52 +02:00
Oriol Roqué Paniagua
7b9ba021c1 Merged PR 2865: Propagate tax exclusive check in hero payments
# Description

This PR propagates tax exclusive check in hero payments for the reporting of Check in Hero.
Additionally, it keeps propagating the amounts with tax inclusiveness in case, at some point, we need them. These are with the new naming convention.

In order not to break anything, the previous amounts are duplicated and aliased in reporting.

Lastly, I spent some time adding some clarifications and documenting payments set to currency and the dependant used for check in hero.

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #20046
2024-09-17 07:25:09 +00:00
Pablo Martin
fda6647637 update schemas 2024-09-16 14:40:01 +02:00
Pablo Martin
6434c5c88d replace custom generic test with standard range between one 2024-09-13 15:21:22 +02:00
Pablo Martin
092b8bd725 clarify refund is not in GBP 2024-09-13 15:21:22 +02:00
Pablo Martin
da3070949a finish columns and schema 2024-09-13 15:20:39 +02:00
Pablo Martin
63599d7a9b add columns 2024-09-13 15:20:39 +02:00
Pablo Martin
2cd115eede remove unnecessary alias, remove unrelated errors in yaml 2024-09-13 15:20:39 +02:00
Pablo Martin
d4b797c741 add alias 2024-09-13 15:20:39 +02:00
Pablo Martin
bbb9558f62 yaml for new version, plus deprecation 2024-09-13 15:20:39 +02:00
Pablo Martin
05d5cc6d10 fix schemas in intermediate 2024-09-12 15:38:50 +02:00
Renamed from models/intermediate/core/schema.yaml (Browse further)