Commit graph

639 commits

Author SHA1 Message Date
uri
3723cf0e57 DRY on stg cosmos db models 2024-09-19 15:17:33 +02:00
uri
1c1f8ea82e First working version of stg_screening__verifications 2024-09-19 14:54:31 +02:00
Oriol Roqué Paniagua
feaedb2a06 Merged PR 2911: Bugfix core__bookings failing dbt test
# Description

Fixes issue

# Checklist

- [ ] The edited models and dependants run properly with production data.
- [ ] The edited models are sufficiently documented.
- [ ] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.
2024-09-19 07:00:15 +00:00
Joaquin Ossa
01eca0dbcc Merged PR 2877: Update model core__verification_payments to include amounts without taxes
# Description

Update model core__verification_payments to include guest payment amounts without taxes

# Checklist

- [x] The edited models and dependants run properly with production data.
- [x] The edited models are sufficiently documented.
- [x] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.
2024-09-18 14:30:23 +00:00
Joaquin Ossa
923bfa7091 Update exposures to pull from version 1 2024-09-18 14:20:48 +02:00
Oriol Roqué Paniagua
94ebe67be1 Merged PR 2898: Remove .sql in the xero__bank_transaction_denom_mart
# Description

Remove .sql in the xero__bank_transaction_denom_mart, effectively fixing the tests that were not running + removing the following warnings:

```
06:15:13  [WARNING]: Did not find matching node for patch with name 'xero__bank_transaction_denom_mart.sql' in the 'models' section of file 'models/reporting/xero/schema.yml'
06:15:14  [WARNING]: Test 'test.dwh_dbt.not_null_xero__bank_transaction_denom_mart.sql_id_line_item.70cdeb323e' (models/reporting/xero/schema.yml) depends on a node named 'xero__bank_transaction_denom_mart.sql' in package '' which was not found
06:15:14  [WARNING]: Test 'test.dwh_dbt.unique_xero__bank_transaction_denom_mart.sql_id_line_item.b85e051d2d' (models/reporting/xero/schema.yml) depends on a node named 'xero__bank_transaction_denom_mart.sql' in package '' which was not found
```

# Checklist

- [X] The edited models and dependants run properly with production data.
- [ ] The edited models are sufficiently documented. *Have not checked*
- [X] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs. *Have not checked*
- [ ] I've picked the right materialization for the affected models. *Have not checked*

# Other

- [ ] Check if a full-refresh is required after this PR is merged.
2024-09-18 12:19:23 +00:00
uri
5620ecbe68 Remove .sql in the xero__bank_transaction_denom_mart 2024-09-18 13:52:55 +02:00
Oriol Roqué Paniagua
985ddb8e4d Merged PR 2887: Check-In Hero - Removing unused fields + schema changes
# Description
This PR closes the ticket #20046

Changes:
* Removes temporary fields marked as legacy in `core__vr_check_in_cover`, after the changes in the 3 PBI Check-In Hero reports.
* Additionally, I modified the schema entry to comply with the current fields available in this model.

# Checklist

- [X] The edited models and dependants run properly with production data. *Checked in local changing the sources to localhost after dropping the fields.*
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs. *N/A*
- [ ] I've picked the right materialization for the affected models. *N/A*

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Remove temporary fields

Related work items: #20046
2024-09-18 11:48:49 +00:00
Joaquin Ossa
40bfa60016 core__verification_payments_v2 2024-09-18 12:41:00 +02:00
Joaquin Ossa
9e62786aa9 commit2 2024-09-18 11:26:23 +02:00
Joaquin Ossa
78e7e37c5e Update model core__verification_payments to include amounts without taxes 2024-09-17 17:22:37 +02:00
Joaquin Ossa
2dd4f66189 update int_core__guest_satisfaction_responses to version 2 of verification_payments 2024-09-17 17:12:46 +02:00
Pablo Martín
4bafeba787 Merged PR 2883: Payaway per month and user
# Description

This PR adds an intermediate tables that holds a combo of all payaway plans with the months (technically, combinations of first day and last day of month) when the plan was the relevant one for invoicing purposes.

The goal is to have this as an intermediate helper for some tax related logic in verification payments.

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #20043
2024-09-17 14:13:01 +00:00
Pablo Martin
a20d511b93 missing text 2024-09-17 16:12:38 +02:00
Pablo Martin
049c2d3e31 model and docs+tests 2024-09-17 15:47:45 +02:00
Pablo Martín
140da0a75d Merged PR 2874: Add Payaway table to intermediate
# Description

Brings the Payaway table from staging to intermediate.

The only significant change is that we change the internal Superhog currency id for the iso4217 code.

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #20043
2024-09-17 12:07:07 +00:00
Pablo Martin
f3e9985d0f model and docs 2024-09-17 12:16:52 +02:00
Pablo Martin
1c04c69e89 fix comments 2024-09-17 11:59:04 +02:00
Pablo Martin
ac87ab4cfd create file 2024-09-17 11:56:21 +02:00
Pablo Martin
624c70872f docs and tests 2024-09-17 11:16:01 +02:00
Pablo Martin
68a9eafb14 typo in file name 2024-09-17 10:36:58 +02:00
Pablo Martin
96d18116ed add staging model 2024-09-17 10:36:43 +02:00
Pablo Martin
68c509050f add payway to sources 2024-09-17 10:32:11 +02:00
Joaquin Ossa
ae1e45a937 Merged PR 2869: Update mtd_aggregated_metrics guest payments to be tax-exclusive
# Description

Update mtd_aggregated_metrics guest payments to be tax-exclusive

# Checklist

- [x] The edited models and dependants run properly with production data.
- [x] The edited models are sufficiently documented.
- [x] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Update mtd_aggregated_metrics guest payments to be tax-exclusive

Related work items: #20045
2024-09-17 07:27:08 +00:00
Oriol Roqué Paniagua
7b9ba021c1 Merged PR 2865: Propagate tax exclusive check in hero payments
# Description

This PR propagates tax exclusive check in hero payments for the reporting of Check in Hero.
Additionally, it keeps propagating the amounts with tax inclusiveness in case, at some point, we need them. These are with the new naming convention.

In order not to break anything, the previous amounts are duplicated and aliased in reporting.

Lastly, I spent some time adding some clarifications and documenting payments set to currency and the dependant used for check in hero.

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #20046
2024-09-17 07:25:09 +00:00
Joaquin Ossa
b21590ab1d Simplified it 2024-09-17 09:05:38 +02:00
Joaquin Ossa
c3e4a07050 Update mtd_aggregated_metrics 2024-09-17 09:05:38 +02:00
Pablo Martin
60ea19f5ef remove dangling comment 2024-09-17 09:05:38 +02:00
Pablo Martin
fcbaae1381 update schemas 2024-09-17 09:05:38 +02:00
Pablo Martin
fd3b67935b remove old booking metrics 2024-09-17 09:05:37 +02:00
Pablo Martin
bbbfaeabdf refactor booking metrics children to use new models 2024-09-17 09:05:37 +02:00
Pablo Martin
4b5c4d1967 create individual models for each booking metric 2024-09-17 09:04:44 +02:00
Joaquin Ossa
50a8a512fe Update mtd_aggregated_metrics guest payments to be tax-exclusive 2024-09-17 08:38:33 +02:00
Joaquin Ossa
3d8ede273e Simplified it 2024-09-16 16:20:36 +02:00
Joaquin Ossa
54af8af30e Update mtd_aggregated_metrics 2024-09-16 16:01:49 +02:00
Joaquin Ossa
2c17df3742 Merged PR 2854: Exposure for e-deposit invoice
# Description

Exposure for e-deposit invoice

# Checklist

- [ ] The edited models and dependants run properly with production data.
- [ ] The edited models are sufficiently documented.
- [ ] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Exposure for e-deposit invoice

Related work items: #20124
2024-09-16 10:25:19 +00:00
Oriol Roqué Paniagua
c1b97e17e6 Merged PR 2852: Fix: ensure priority selection on user migration
# Description

Fixing logic to ensure priority selection of claims when user satisfies multiple claim conditions.
It adds a new parameter that forcefully prioritises the selection of the date value for a certain claim over the others. If the value is repeated among claims, it will select the earliest date.

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #20773
2024-09-16 09:34:44 +00:00
Joaquin Ossa
8fd5061989 Exposure for e-deposit invoice 2024-09-16 11:14:43 +02:00
Oriol Roqué Paniagua
cf1d6e28cc Merged PR 2849: Fix: KYG lite migration with proper date migration handling
# Description

This PR fixes the New Dash migration issue that happened on September 10th 2024. In this migration, users were directly assigned the claim of KygMvp that does not contain a date value. We were using a default hardcode of the first MVP migration, thus in DWH all users have been considered to be migrated late July instead of splitting the first 22 in late July and the ~200 others in September.

The issue lies in the fact that users have configured a ProductBundle and can have Bookings with ProductBundle BEFORE the migration date, which greatly breaks the logic of a migration monitoring.

Changes:
* New migration phase added based on the claim MvpMigratedUser, that Ben created on Friday 13th
* Adaptation of the code in int_core__user_migration to detect if the claim_value (a text field) has a date or not. If so, use that date as long as it's equal or greater than the deployment date, if not use the deployment date. If the claim does not contain a date, use the deployment date (this is the case for the first true 22 migrated users)

I checked that volumes now look correct with this fix.

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #20773
2024-09-16 07:57:41 +00:00
Joaquin Ossa
80a6b225b3 Merged PR 2846: removing duplicates from guesty and removing tests for edeposit_users
# Description

removing duplicates from guesty and removing tests for edeposit_users and edeposit_verification_fees until further notice

# Checklist

- [x] The edited models and dependants run properly with production data.
- [x] The edited models are sufficiently documented.
- [x] The edited models contain PK tests, and I've ran and passed them.
- [x] I have checked for DRY opportunities with other models and docs.
- [x] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

removing duplicates from guesty and removing tests for edeposit_users

Related work items: #20125
2024-09-13 15:16:08 +00:00
Joaquin Ossa
d8501e2df7 kept first updated record instead of last 2024-09-13 17:14:34 +02:00
Joaquin Ossa
32c8ffb655 removef all tests for edeposit_verification_fees 2024-09-13 16:31:47 +02:00
Joaquin Ossa
5551c97db0 removing duplicates from guesty and removing tests for edeposit_users 2024-09-13 16:14:03 +02:00
Pablo Martin
6434c5c88d replace custom generic test with standard range between one 2024-09-13 15:21:22 +02:00
Pablo Martin
4cf74db49a replace hardcode with variable 2024-09-13 15:21:22 +02:00
Pablo Martin
092b8bd725 clarify refund is not in GBP 2024-09-13 15:21:22 +02:00
Pablo Martin
ec8490527d reorder case for readability 2024-09-13 15:20:39 +02:00
Pablo Martin
0312a1ccd8 remove explicit version reference, rely on latest 2024-09-13 15:20:39 +02:00
Pablo Martin
da3070949a finish columns and schema 2024-09-13 15:20:39 +02:00
Pablo Martin
63599d7a9b add columns 2024-09-13 15:20:39 +02:00