Commit graph

944 commits

Author SHA1 Message Date
Oriol Roqué Paniagua
94ebe67be1 Merged PR 2898: Remove .sql in the xero__bank_transaction_denom_mart
# Description

Remove .sql in the xero__bank_transaction_denom_mart, effectively fixing the tests that were not running + removing the following warnings:

```
06:15:13  [WARNING]: Did not find matching node for patch with name 'xero__bank_transaction_denom_mart.sql' in the 'models' section of file 'models/reporting/xero/schema.yml'
06:15:14  [WARNING]: Test 'test.dwh_dbt.not_null_xero__bank_transaction_denom_mart.sql_id_line_item.70cdeb323e' (models/reporting/xero/schema.yml) depends on a node named 'xero__bank_transaction_denom_mart.sql' in package '' which was not found
06:15:14  [WARNING]: Test 'test.dwh_dbt.unique_xero__bank_transaction_denom_mart.sql_id_line_item.b85e051d2d' (models/reporting/xero/schema.yml) depends on a node named 'xero__bank_transaction_denom_mart.sql' in package '' which was not found
```

# Checklist

- [X] The edited models and dependants run properly with production data.
- [ ] The edited models are sufficiently documented. *Have not checked*
- [X] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs. *Have not checked*
- [ ] I've picked the right materialization for the affected models. *Have not checked*

# Other

- [ ] Check if a full-refresh is required after this PR is merged.
2024-09-18 12:19:23 +00:00
uri
5620ecbe68 Remove .sql in the xero__bank_transaction_denom_mart 2024-09-18 13:52:55 +02:00
Oriol Roqué Paniagua
985ddb8e4d Merged PR 2887: Check-In Hero - Removing unused fields + schema changes
# Description
This PR closes the ticket #20046

Changes:
* Removes temporary fields marked as legacy in `core__vr_check_in_cover`, after the changes in the 3 PBI Check-In Hero reports.
* Additionally, I modified the schema entry to comply with the current fields available in this model.

# Checklist

- [X] The edited models and dependants run properly with production data. *Checked in local changing the sources to localhost after dropping the fields.*
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs. *N/A*
- [ ] I've picked the right materialization for the affected models. *N/A*

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Remove temporary fields

Related work items: #20046
2024-09-18 11:48:49 +00:00
Joaquin Ossa
40bfa60016 core__verification_payments_v2 2024-09-18 12:41:00 +02:00
Joaquin Ossa
9e62786aa9 commit2 2024-09-18 11:26:23 +02:00
Pablo Martin
136c297683 add logic 2024-09-17 17:56:56 +02:00
Joaquin Ossa
e854f2a99a commit 2024-09-17 17:22:37 +02:00
Joaquin Ossa
78e7e37c5e Update model core__verification_payments to include amounts without taxes 2024-09-17 17:22:37 +02:00
Joaquin Ossa
17f7b11a79 Merged PR 2885: update int_core__guest_satisfaction_responses to version 2 of verification_pa...
# Description

update int_core__guest_satisfaction_responses to version 2 of verification_payments

# Checklist

- [x] The edited models and dependants run properly with production data.
- [x] The edited models are sufficiently documented.
- [x] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #20044
2024-09-17 15:21:59 +00:00
Joaquin Ossa
2dd4f66189 update int_core__guest_satisfaction_responses to version 2 of verification_payments 2024-09-17 17:12:46 +02:00
Pablo Martín
4bafeba787 Merged PR 2883: Payaway per month and user
# Description

This PR adds an intermediate tables that holds a combo of all payaway plans with the months (technically, combinations of first day and last day of month) when the plan was the relevant one for invoicing purposes.

The goal is to have this as an intermediate helper for some tax related logic in verification payments.

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #20043
2024-09-17 14:13:01 +00:00
Pablo Martin
a20d511b93 missing text 2024-09-17 16:12:38 +02:00
Pablo Martin
049c2d3e31 model and docs+tests 2024-09-17 15:47:45 +02:00
Pablo Martín
140da0a75d Merged PR 2874: Add Payaway table to intermediate
# Description

Brings the Payaway table from staging to intermediate.

The only significant change is that we change the internal Superhog currency id for the iso4217 code.

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #20043
2024-09-17 12:07:07 +00:00
Pablo Martin
f3e9985d0f model and docs 2024-09-17 12:16:52 +02:00
Pablo Martín
07db97b79e Merged PR 2870: Add core.PayAway to staging
# Description

This PR brings the core table `PayAway` into staging, adds documentation and tests.

We are bringing this so we can build right until we reach `int_core__verification_payments_v2`, which needs this info to determine if a waiver was a Superhog-takes-risk type or Host-takes-risk type.

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #20043
2024-09-17 09:59:26 +00:00
Pablo Martin
1c04c69e89 fix comments 2024-09-17 11:59:04 +02:00
Pablo Martin
ac87ab4cfd create file 2024-09-17 11:56:21 +02:00
Pablo Martin
624c70872f docs and tests 2024-09-17 11:16:01 +02:00
Pablo Martin
68a9eafb14 typo in file name 2024-09-17 10:36:58 +02:00
Pablo Martin
96d18116ed add staging model 2024-09-17 10:36:43 +02:00
Pablo Martin
68c509050f add payway to sources 2024-09-17 10:32:11 +02:00
Joaquin Ossa
ae1e45a937 Merged PR 2869: Update mtd_aggregated_metrics guest payments to be tax-exclusive
# Description

Update mtd_aggregated_metrics guest payments to be tax-exclusive

# Checklist

- [x] The edited models and dependants run properly with production data.
- [x] The edited models are sufficiently documented.
- [x] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Update mtd_aggregated_metrics guest payments to be tax-exclusive

Related work items: #20045
2024-09-17 07:27:08 +00:00
Oriol Roqué Paniagua
7b9ba021c1 Merged PR 2865: Propagate tax exclusive check in hero payments
# Description

This PR propagates tax exclusive check in hero payments for the reporting of Check in Hero.
Additionally, it keeps propagating the amounts with tax inclusiveness in case, at some point, we need them. These are with the new naming convention.

In order not to break anything, the previous amounts are duplicated and aliased in reporting.

Lastly, I spent some time adding some clarifications and documenting payments set to currency and the dependant used for check in hero.

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #20046
2024-09-17 07:25:09 +00:00
Joaquin Ossa
cc837ecd7a Merged PR 2861: Update mtd_aggregated_metrics
# Description

Update mtd_aggregated_metrics for guest payments to be tax-exclusive

# Checklist

- [x] The edited models and dependants run properly with production data.
- [x] The edited models are sufficiently documented.
- [x] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Update mtd_aggregated_metrics

Related work items: #20045
2024-09-17 07:09:03 +00:00
Joaquin Ossa
b21590ab1d Simplified it 2024-09-17 09:05:38 +02:00
Joaquin Ossa
c3e4a07050 Update mtd_aggregated_metrics 2024-09-17 09:05:38 +02:00
Pablo Martin
60ea19f5ef remove dangling comment 2024-09-17 09:05:38 +02:00
Pablo Martin
fcbaae1381 update schemas 2024-09-17 09:05:38 +02:00
Pablo Martin
fd3b67935b remove old booking metrics 2024-09-17 09:05:37 +02:00
Pablo Martin
bbbfaeabdf refactor booking metrics children to use new models 2024-09-17 09:05:37 +02:00
Pablo Martin
4b5c4d1967 create individual models for each booking metric 2024-09-17 09:04:44 +02:00
Joaquin Ossa
50a8a512fe Update mtd_aggregated_metrics guest payments to be tax-exclusive 2024-09-17 08:38:33 +02:00
Pablo Martín
02520d87ee Merged PR 2858: Split Booking Metrics
# Description

This is a pure refactor PR for optimization purposes.

The PR splits the model `int_core__mtd_booking_metrics` into four different models, one for each of the metrics that were computed in the old model.

The only model that depended on `int_core__mtd_booking_metrics` was `int_mtd_vs_previous_year_metrics`. Now it depends on the four new models instead.

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #21438
2024-09-16 14:34:42 +00:00
Joaquin Ossa
3d8ede273e Simplified it 2024-09-16 16:20:36 +02:00
Joaquin Ossa
54af8af30e Update mtd_aggregated_metrics 2024-09-16 16:01:49 +02:00
Pablo Martin
aeb4538e50 remove dangling comment 2024-09-16 14:45:09 +02:00
Pablo Martin
fda6647637 update schemas 2024-09-16 14:40:01 +02:00
Pablo Martin
385c49bf2c remove old booking metrics 2024-09-16 14:35:12 +02:00
Pablo Martin
aa06b28a69 refactor booking metrics children to use new models 2024-09-16 13:02:42 +02:00
Pablo Martin
4140f5b917 create individual models for each booking metric 2024-09-16 13:02:09 +02:00
Joaquin Ossa
2c17df3742 Merged PR 2854: Exposure for e-deposit invoice
# Description

Exposure for e-deposit invoice

# Checklist

- [ ] The edited models and dependants run properly with production data.
- [ ] The edited models are sufficiently documented.
- [ ] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Exposure for e-deposit invoice

Related work items: #20124
2024-09-16 10:25:19 +00:00
Oriol Roqué Paniagua
c1b97e17e6 Merged PR 2852: Fix: ensure priority selection on user migration
# Description

Fixing logic to ensure priority selection of claims when user satisfies multiple claim conditions.
It adds a new parameter that forcefully prioritises the selection of the date value for a certain claim over the others. If the value is repeated among claims, it will select the earliest date.

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #20773
2024-09-16 09:34:44 +00:00
Joaquin Ossa
8fd5061989 Exposure for e-deposit invoice 2024-09-16 11:14:43 +02:00
Oriol Roqué Paniagua
cf1d6e28cc Merged PR 2849: Fix: KYG lite migration with proper date migration handling
# Description

This PR fixes the New Dash migration issue that happened on September 10th 2024. In this migration, users were directly assigned the claim of KygMvp that does not contain a date value. We were using a default hardcode of the first MVP migration, thus in DWH all users have been considered to be migrated late July instead of splitting the first 22 in late July and the ~200 others in September.

The issue lies in the fact that users have configured a ProductBundle and can have Bookings with ProductBundle BEFORE the migration date, which greatly breaks the logic of a migration monitoring.

Changes:
* New migration phase added based on the claim MvpMigratedUser, that Ben created on Friday 13th
* Adaptation of the code in int_core__user_migration to detect if the claim_value (a text field) has a date or not. If so, use that date as long as it's equal or greater than the deployment date, if not use the deployment date. If the claim does not contain a date, use the deployment date (this is the case for the first true 22 migrated users)

I checked that volumes now look correct with this fix.

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #20773
2024-09-16 07:57:41 +00:00
Joaquin Ossa
80a6b225b3 Merged PR 2846: removing duplicates from guesty and removing tests for edeposit_users
# Description

removing duplicates from guesty and removing tests for edeposit_users and edeposit_verification_fees until further notice

# Checklist

- [x] The edited models and dependants run properly with production data.
- [x] The edited models are sufficiently documented.
- [x] The edited models contain PK tests, and I've ran and passed them.
- [x] I have checked for DRY opportunities with other models and docs.
- [x] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

removing duplicates from guesty and removing tests for edeposit_users

Related work items: #20125
2024-09-13 15:16:08 +00:00
Joaquin Ossa
d8501e2df7 kept first updated record instead of last 2024-09-13 17:14:34 +02:00
Joaquin Ossa
32c8ffb655 removef all tests for edeposit_verification_fees 2024-09-13 16:31:47 +02:00
Joaquin Ossa
5551c97db0 removing duplicates from guesty and removing tests for edeposit_users 2024-09-13 16:14:03 +02:00
Pablo Martín
a8b4c08328 Merged PR 2771: Compute Taxes in intermediate
# Description

This model joins a couple of models to `int_core__verification_payments` and adds a few fields around taxs (amounts with and without taxes, tax rates, etc).

## __ATTENTION!!!__

This PR uses `dbt`'s [Model Versioning](https://docs.getdbt.com/docs/collaborate/govern/model-versions) and [Deprecation](https://docs.getdbt.com/reference/resource-properties/deprecation_date) features for the first time. Notice the extra, non-usual fluff in the `schema.yml` file, along with the fact that some model files are suffixed with versions.

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
    - _Note that there are a couple of tests that are deactivated (commented out) due to bad data in the backend. Engineering team is working on fixiing that data. I've decided to comment this for now so we can move forward, but I won't be marking the story on this done until the tests are live in production._
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #20043
2024-09-13 13:24:25 +00:00