Commit graph

712 commits

Author SHA1 Message Date
Joaquin Ossa
e9f16342e7 added tests 2024-10-03 12:19:27 +02:00
Oriol Roqué Paniagua
8e4e967c20 Merged PR 3028: Adding int_core__user_product_bundle_contains_services
# Description

This PR adds a new table named `user_product_bundle_contains_services` in intermediate core.

Mainly, this table serves as a bridge between `user_product_bundle` and `product_services`. A `product_bundle` within `user_product_bundle` can contain 1 or several services, and this was stated in the field `contains_product_services`. The value of this field corresponds to the sum of `product_service_binary_tier` from the services that apply within that bundle. Even though the information is quite concise using this format, it adds extra complexity for analytical purposes, so this new table just duplicates the `user_product_bundle` main attributes as many times as services are contained.

For example:
`id_user_product_bundle` = 383 contains one unique bundle named Basic Program. This bundle has the `chosen_product_services` = 257, which can only be decomposed in power of 2 as of the sum of 256 + 1. This bundle therefore contains 2 services, Basic Screening (`product_service_binary_tier` = 1) and Waiver Pro (`product_service_binary_tier` = 256). Thus, in the new table, we will have 2 records and remove all this logic to something more standard, as seen in this screenshot:

![image (2).png](https://guardhog.visualstudio.com/4148d95f-4b6d-4205-bcff-e9c8e0d2ca65/_apis/git/repositories/54ac356f-aad7-46d2-b62c-e8c5b3bb8ebf/pullRequests/3028/attachments/image%20%282%29.png)

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [ ] The edited models contain PK tests, and I've ran and passed them. *N/A - there's no PK in this table, but the combination of columns that should be unique is tested*
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #20809
2024-10-02 14:54:03 +00:00
Joaquin Ossa
28b658a4fe fixed naming 2024-10-02 15:27:47 +02:00
Joaquin Ossa
0a880a138b created macro and fixed naming 2024-10-02 14:50:34 +02:00
Oriol Roqué Paniagua
f2e21496fe Merged PR 3006: Integrating to staging Product Service related tables
# Description
This PR integrates into staging the following Core tables, regarding New Pricing:
- ProductService
- ProductServiceToPrice
- AppliedProductService (IT HAS BEEN REMOVED)

Models are also documented in staging. Tables contain data already but not necessarily records for New Dash, since still we have not had any paid service applied into a booking / V2 has not been launched yet.

New Dash modelisation for V2 will follow once these tables exist in staging, in order to be able to report 1) revenue and 2) services selected.

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models. **N/A, few records so not a big deal. Might need to change in the future.**

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #20809
2024-10-02 09:13:10 +00:00
Pablo Martín
7d9d9faf71 Merged PR 3024: Activate country test for taxes
# Description

Activates a commented out test for verification payments v2. The test was deactivated because of some master data issues in the backend, but they were fixed after  #22148.

# Checklist

- [NA] The edited models and dependants run properly with production data.
- [NA ] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [NA] I have checked for DRY opportunities with other models and docs.
- [NA] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #20043
2024-10-02 09:09:18 +00:00
Joaquin Ossa
aab6155b67 more features into reporting 2024-10-02 10:57:30 +02:00
Joaquin Ossa
bf01360fba Merged PR 3020: new features dates
# Description

Added 2 new columns with a concatenation of month number and full name o short name

# Checklist

- [x] The edited models and dependants run properly with production data.
- [x] The edited models are sufficiently documented.
- [ ] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

new features dates
2024-10-02 08:18:52 +00:00
Joaquin Ossa
c5cc2e2167 new features dates 2024-10-02 10:11:48 +02:00
Joaquin Ossa
f37d419a1a fixed cancellation fee charge 2024-10-02 09:30:33 +02:00
Joaquin Ossa
ae865b8848 Merged PR 2994: Staging Hubspot deals
# Description

Creation on stg_hubspot__deals model

# Checklist

- [x] The edited models and dependants run properly with production data.
- [x] The edited models are sufficiently documented.
- [x] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.
2024-09-27 15:30:53 +00:00
Joaquin Ossa
ee6b613887 Stg Hubspot form_submissions 2024-09-27 17:30:25 +02:00
Pablo Martin
6124c19f98 activate one test, prepare another but keep it commented 2024-09-27 16:11:34 +02:00
Joaquin Ossa
306035d46d Hubspot deals 2024-09-27 12:24:27 +02:00
Joaquin Ossa
20425495f9 staging Hubspot contacts 2024-09-27 10:44:48 +02:00
Joaquin Ossa
37d9f5250a Added new role 2024-09-27 09:55:37 +02:00
Pablo Martin
0cd8f71889 remove variable that doesn't make sense anymore 2024-09-25 16:12:55 +02:00
Pablo Martin
bea2e0d926 fix broken case logic 2024-09-25 16:12:36 +02:00
Oriol Roqué Paniagua
070d067709 Merged PR 2918: Integrates screening API verifications into DWH staging
# Description

This PR integrates screening API verifications into staging.

There's 2 commits:
* The earliest one, it just copy-pastes the strategy followed by edeposit and adapts it to fit screening API case, which is simpler. Note here that the schema entries contain a low number of tests. This is because we only have 7 records in production - and these seem fake anyway :) - so it's complex to extrapolate. Those I could extrapolate (NoFlags/Flagged) I'm taking from the data transformation within the PBI report.
* The last commit, it's just a DRY. It handles the deduplication logic for cosmos db in a macro, and I applied it on both the screening API and the edeposit verifications. Works well from my POV.

Since you guys are far more knowledgeable on APIs scope, I'll let you take a close look in case I missed something.

# Checklist

- [X] The edited models and dependants run properly with production data.
- [ ] The edited models are sufficiently documented. **I guess it can be improved, waiting for your comments here**
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models. **Used default coming from stg_edeposit__verifications**

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #20127
2024-09-25 10:52:50 +00:00
Pablo Martín
260b46171c Merged PR 2888: No taxes for host takes risk waivers
# Description

This PR modifies the V2 of `int_core__verification_payments` to make sure that taxes are not computed for waivers where the host takes the risk (in line with our guest taxes guidelines: https://www.notion.so/knowyourguest-superhog/Guest-Services-Taxes-How-to-calculate-a5ab4c049d61427fafab669dbbffb3a2?pvs=4#323e230365ed496cad6a65b5659895ea).

The PR changes the intermediate column `is_service_subject_to_vat`, making it `false` in the cases where there was no active payaway plan during the month where the payment was generated. This cascades in the main body of the model, because if `is_service_subject_to_vat` is `false`, then the tax amounts get turned into zero.

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

add logic

Related work items: #20043
2024-09-25 10:34:46 +00:00
Joaquin Ossa
838628c7a8 deleted the collate 2024-09-20 15:16:53 +02:00
Joaquin Ossa
5ac1a65e90 Financial rate metrics 2024-09-20 14:53:43 +02:00
uri
3723cf0e57 DRY on stg cosmos db models 2024-09-19 15:17:33 +02:00
uri
1c1f8ea82e First working version of stg_screening__verifications 2024-09-19 14:54:31 +02:00
Oriol Roqué Paniagua
feaedb2a06 Merged PR 2911: Bugfix core__bookings failing dbt test
# Description

Fixes issue

# Checklist

- [ ] The edited models and dependants run properly with production data.
- [ ] The edited models are sufficiently documented.
- [ ] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.
2024-09-19 07:00:15 +00:00
Joaquin Ossa
01eca0dbcc Merged PR 2877: Update model core__verification_payments to include amounts without taxes
# Description

Update model core__verification_payments to include guest payment amounts without taxes

# Checklist

- [x] The edited models and dependants run properly with production data.
- [x] The edited models are sufficiently documented.
- [x] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.
2024-09-18 14:30:23 +00:00
Joaquin Ossa
923bfa7091 Update exposures to pull from version 1 2024-09-18 14:20:48 +02:00
Oriol Roqué Paniagua
94ebe67be1 Merged PR 2898: Remove .sql in the xero__bank_transaction_denom_mart
# Description

Remove .sql in the xero__bank_transaction_denom_mart, effectively fixing the tests that were not running + removing the following warnings:

```
06:15:13  [WARNING]: Did not find matching node for patch with name 'xero__bank_transaction_denom_mart.sql' in the 'models' section of file 'models/reporting/xero/schema.yml'
06:15:14  [WARNING]: Test 'test.dwh_dbt.not_null_xero__bank_transaction_denom_mart.sql_id_line_item.70cdeb323e' (models/reporting/xero/schema.yml) depends on a node named 'xero__bank_transaction_denom_mart.sql' in package '' which was not found
06:15:14  [WARNING]: Test 'test.dwh_dbt.unique_xero__bank_transaction_denom_mart.sql_id_line_item.b85e051d2d' (models/reporting/xero/schema.yml) depends on a node named 'xero__bank_transaction_denom_mart.sql' in package '' which was not found
```

# Checklist

- [X] The edited models and dependants run properly with production data.
- [ ] The edited models are sufficiently documented. *Have not checked*
- [X] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs. *Have not checked*
- [ ] I've picked the right materialization for the affected models. *Have not checked*

# Other

- [ ] Check if a full-refresh is required after this PR is merged.
2024-09-18 12:19:23 +00:00
uri
5620ecbe68 Remove .sql in the xero__bank_transaction_denom_mart 2024-09-18 13:52:55 +02:00
Oriol Roqué Paniagua
985ddb8e4d Merged PR 2887: Check-In Hero - Removing unused fields + schema changes
# Description
This PR closes the ticket #20046

Changes:
* Removes temporary fields marked as legacy in `core__vr_check_in_cover`, after the changes in the 3 PBI Check-In Hero reports.
* Additionally, I modified the schema entry to comply with the current fields available in this model.

# Checklist

- [X] The edited models and dependants run properly with production data. *Checked in local changing the sources to localhost after dropping the fields.*
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs. *N/A*
- [ ] I've picked the right materialization for the affected models. *N/A*

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Remove temporary fields

Related work items: #20046
2024-09-18 11:48:49 +00:00
Joaquin Ossa
40bfa60016 core__verification_payments_v2 2024-09-18 12:41:00 +02:00
Joaquin Ossa
9e62786aa9 commit2 2024-09-18 11:26:23 +02:00
Pablo Martin
136c297683 add logic 2024-09-17 17:56:56 +02:00
Joaquin Ossa
78e7e37c5e Update model core__verification_payments to include amounts without taxes 2024-09-17 17:22:37 +02:00
Joaquin Ossa
2dd4f66189 update int_core__guest_satisfaction_responses to version 2 of verification_payments 2024-09-17 17:12:46 +02:00
Pablo Martín
4bafeba787 Merged PR 2883: Payaway per month and user
# Description

This PR adds an intermediate tables that holds a combo of all payaway plans with the months (technically, combinations of first day and last day of month) when the plan was the relevant one for invoicing purposes.

The goal is to have this as an intermediate helper for some tax related logic in verification payments.

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #20043
2024-09-17 14:13:01 +00:00
Pablo Martin
a20d511b93 missing text 2024-09-17 16:12:38 +02:00
Pablo Martin
049c2d3e31 model and docs+tests 2024-09-17 15:47:45 +02:00
Pablo Martín
140da0a75d Merged PR 2874: Add Payaway table to intermediate
# Description

Brings the Payaway table from staging to intermediate.

The only significant change is that we change the internal Superhog currency id for the iso4217 code.

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #20043
2024-09-17 12:07:07 +00:00
Pablo Martin
f3e9985d0f model and docs 2024-09-17 12:16:52 +02:00
Pablo Martin
1c04c69e89 fix comments 2024-09-17 11:59:04 +02:00
Pablo Martin
ac87ab4cfd create file 2024-09-17 11:56:21 +02:00
Pablo Martin
624c70872f docs and tests 2024-09-17 11:16:01 +02:00
Pablo Martin
68a9eafb14 typo in file name 2024-09-17 10:36:58 +02:00
Pablo Martin
96d18116ed add staging model 2024-09-17 10:36:43 +02:00
Pablo Martin
68c509050f add payway to sources 2024-09-17 10:32:11 +02:00
Joaquin Ossa
ae1e45a937 Merged PR 2869: Update mtd_aggregated_metrics guest payments to be tax-exclusive
# Description

Update mtd_aggregated_metrics guest payments to be tax-exclusive

# Checklist

- [x] The edited models and dependants run properly with production data.
- [x] The edited models are sufficiently documented.
- [x] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Update mtd_aggregated_metrics guest payments to be tax-exclusive

Related work items: #20045
2024-09-17 07:27:08 +00:00
Oriol Roqué Paniagua
7b9ba021c1 Merged PR 2865: Propagate tax exclusive check in hero payments
# Description

This PR propagates tax exclusive check in hero payments for the reporting of Check in Hero.
Additionally, it keeps propagating the amounts with tax inclusiveness in case, at some point, we need them. These are with the new naming convention.

In order not to break anything, the previous amounts are duplicated and aliased in reporting.

Lastly, I spent some time adding some clarifications and documenting payments set to currency and the dependant used for check in hero.

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #20046
2024-09-17 07:25:09 +00:00
Joaquin Ossa
b21590ab1d Simplified it 2024-09-17 09:05:38 +02:00
Joaquin Ossa
c3e4a07050 Update mtd_aggregated_metrics 2024-09-17 09:05:38 +02:00