Commit graph

830 commits

Author SHA1 Message Date
Joaquin Ossa
008f0f9987 Merged PR 3025: more features into reporting
# Description

Included more information in the reporting model to add more details in the report

# Checklist

- [x] The edited models and dependants run properly with production data.
- [x] The edited models are sufficiently documented.
- [x] The edited models contain PK tests, and I've ran and passed them.
- [x] I have checked for DRY opportunities with other models and docs.
- [x] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

more features into reporting
2024-10-02 09:03:42 +00:00
Joaquin Ossa
aab6155b67 more features into reporting 2024-10-02 10:57:30 +02:00
Joaquin Ossa
bf01360fba Merged PR 3020: new features dates
# Description

Added 2 new columns with a concatenation of month number and full name o short name

# Checklist

- [x] The edited models and dependants run properly with production data.
- [x] The edited models are sufficiently documented.
- [ ] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

new features dates
2024-10-02 08:18:52 +00:00
Joaquin Ossa
c5cc2e2167 new features dates 2024-10-02 10:11:48 +02:00
Joaquin Ossa
e001691ed4 Merged PR 3019: fixed cancellation fee charge
# Description

Fixed cancellation fee charge that was being applied to every verification when the threshold was passed, now it will be applied only to those that are actually cancelled

# Checklist

- [x] The edited models and dependants run properly with production data.
- [x] The edited models are sufficiently documented.
- [x] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

fixed cancellation fee charge
2024-10-02 07:56:42 +00:00
Pablo Martín
d4d15dbed4 Merged PR 3015: Add Kosovo to VAT rates
# Description

This PR adds Kosovo to our guest vat rates seed data. It's an exceptional record because it doesn't comply with the ISO 3166 that the model is supposed to follow. But we need to do it because Kosovo is a recognized country in the Superhog backend.

Nila from finance confirmed that the VAT rate should be 0.

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #22148
2024-10-02 07:52:21 +00:00
Joaquin Ossa
f37d419a1a fixed cancellation fee charge 2024-10-02 09:30:33 +02:00
Pablo Martin
dcbfcec627 add note in docs. 2024-10-01 15:52:30 +02:00
Pablo Martin
6f63aae1a1 add kosovo 2024-10-01 15:49:25 +02:00
Joaquin Ossa
ae865b8848 Merged PR 2994: Staging Hubspot deals
# Description

Creation on stg_hubspot__deals model

# Checklist

- [x] The edited models and dependants run properly with production data.
- [x] The edited models are sufficiently documented.
- [x] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.
2024-09-27 15:30:53 +00:00
Pablo Martín
cb9c96930b Merged PR 2996: Activate one test, prepare the other
# Description

There were two tests for `int_core__verification_payments` awaiting activation because the source master data had issues that needed to be fixed.

This PR activates the test for `is_missing_user_country` because the dash team has already fixed the problematic country data.

The other test remains turned off because we still have a master data issue (missing VAT rate for Kosovo), but I've already added a missing where condition, and a reference to the ticket where we are tracking the master data issue.

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

activate one test, prepare another but keep it commented

Related work items: #20043
2024-09-27 14:44:31 +00:00
Pablo Martin
a08941c5dc comment vacuum analyze to run weekend test 2024-09-27 16:29:29 +02:00
Pablo Martin
4ff1375160 tiny commit ever just for foramatting 2024-09-27 16:28:00 +02:00
Pablo Martin
6124c19f98 activate one test, prepare another but keep it commented 2024-09-27 16:11:34 +02:00
Joaquin Ossa
306035d46d Hubspot deals 2024-09-27 12:24:27 +02:00
Joaquin Ossa
20425495f9 staging Hubspot contacts 2024-09-27 10:44:48 +02:00
Joaquin Ossa
c6e14adad5 Merged PR 2988: Added new role
# Description

Added new role to int_core__user_role

# Checklist

- [x] The edited models and dependants run properly with production data.
- [x] The edited models are sufficiently documented.
- [x] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Added new role
2024-09-27 08:01:41 +00:00
Joaquin Ossa
37d9f5250a Added new role 2024-09-27 09:55:37 +02:00
Pablo Martín
a280f79d18 Merged PR 2971: Fix no tax bug
# Description

Fixes a bug introduced by PR: https://guardhog.visualstudio.com/Data/_git/data-pbi-reports/pullrequest/2962

The previous PR made non-waiver services become tax free if the host had an active payaway setting at the payment timestamp, which makes no sense because payaway only relates to waiver and not to other services.

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #20043
2024-09-25 14:24:35 +00:00
Pablo Martin
0cd8f71889 remove variable that doesn't make sense anymore 2024-09-25 16:12:55 +02:00
Pablo Martin
bea2e0d926 fix broken case logic 2024-09-25 16:12:36 +02:00
Oriol Roqué Paniagua
070d067709 Merged PR 2918: Integrates screening API verifications into DWH staging
# Description

This PR integrates screening API verifications into staging.

There's 2 commits:
* The earliest one, it just copy-pastes the strategy followed by edeposit and adapts it to fit screening API case, which is simpler. Note here that the schema entries contain a low number of tests. This is because we only have 7 records in production - and these seem fake anyway :) - so it's complex to extrapolate. Those I could extrapolate (NoFlags/Flagged) I'm taking from the data transformation within the PBI report.
* The last commit, it's just a DRY. It handles the deduplication logic for cosmos db in a macro, and I applied it on both the screening API and the edeposit verifications. Works well from my POV.

Since you guys are far more knowledgeable on APIs scope, I'll let you take a close look in case I missed something.

# Checklist

- [X] The edited models and dependants run properly with production data.
- [ ] The edited models are sufficiently documented. **I guess it can be improved, waiting for your comments here**
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models. **Used default coming from stg_edeposit__verifications**

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #20127
2024-09-25 10:52:50 +00:00
Pablo Martín
260b46171c Merged PR 2888: No taxes for host takes risk waivers
# Description

This PR modifies the V2 of `int_core__verification_payments` to make sure that taxes are not computed for waivers where the host takes the risk (in line with our guest taxes guidelines: https://www.notion.so/knowyourguest-superhog/Guest-Services-Taxes-How-to-calculate-a5ab4c049d61427fafab669dbbffb3a2?pvs=4#323e230365ed496cad6a65b5659895ea).

The PR changes the intermediate column `is_service_subject_to_vat`, making it `false` in the cases where there was no active payaway plan during the month where the payment was generated. This cascades in the main body of the model, because if `is_service_subject_to_vat` is `false`, then the tax amounts get turned into zero.

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

add logic

Related work items: #20043
2024-09-25 10:34:46 +00:00
Pablo Martin
73753092fa yolo activate vacuum analyze 2024-09-20 18:11:59 +02:00
Joaquin Ossa
e401efdd5d Merged PR 2933: Financial rate metrics
# Description

Changed the financial rate metrics to have `number_fomat = "converted_metric_currency_gbp"`

# Checklist

- [x] The edited models and dependants run properly with production data.
- [x] The edited models are sufficiently documented.
- [x] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Financial rate metrics

Related work items: #21564
2024-09-20 14:29:30 +00:00
Joaquin Ossa
838628c7a8 deleted the collate 2024-09-20 15:16:53 +02:00
Joaquin Ossa
5ac1a65e90 Financial rate metrics 2024-09-20 14:53:43 +02:00
Oriol Roqué Paniagua
b3620cc695 Merged PR 2920: Adapt user migration configuration to take into account V2 migrated users
# Description

Prepares setup to automatically detect V2 migrated users from old dash to new dash by using the new claim type KygMoveDate.

# Checklist

- [X] The edited models and dependants run properly with production data. **Works well with current setup but there's no migrated users yet. We'll need to double check once the launch date has happened**
- [ ] The edited models are sufficiently documented. **N/A**
- [ ] The edited models contain PK tests, and I've ran and passed them. **N/A**
- [ ] I have checked for DRY opportunities with other models and docs. **N/A**
- [ ] I've picked the right materialization for the affected models. **N/A**

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #20810
2024-09-20 10:42:12 +00:00
uri
3723cf0e57 DRY on stg cosmos db models 2024-09-19 15:17:33 +02:00
uri
1c1f8ea82e First working version of stg_screening__verifications 2024-09-19 14:54:31 +02:00
Pablo Martin
ea430b1f23 Merge branch 'master' of ssh.dev.azure.com:v3/guardhog/Data/data-dwh-dbt-project 2024-09-19 11:57:18 +02:00
Pablo Martin
cf7a8917a8 add explanatory comment 2024-09-19 11:57:13 +02:00
Oriol Roqué Paniagua
feaedb2a06 Merged PR 2911: Bugfix core__bookings failing dbt test
# Description

Fixes issue

# Checklist

- [ ] The edited models and dependants run properly with production data.
- [ ] The edited models are sufficiently documented.
- [ ] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.
2024-09-19 07:00:15 +00:00
Joaquin Ossa
01eca0dbcc Merged PR 2877: Update model core__verification_payments to include amounts without taxes
# Description

Update model core__verification_payments to include guest payment amounts without taxes

# Checklist

- [x] The edited models and dependants run properly with production data.
- [x] The edited models are sufficiently documented.
- [x] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.
2024-09-18 14:30:23 +00:00
Joaquin Ossa
7ceeabbe7b Revert "commit"
This reverts commit e854f2a99a.
2024-09-18 14:22:33 +02:00
Joaquin Ossa
923bfa7091 Update exposures to pull from version 1 2024-09-18 14:20:48 +02:00
Oriol Roqué Paniagua
94ebe67be1 Merged PR 2898: Remove .sql in the xero__bank_transaction_denom_mart
# Description

Remove .sql in the xero__bank_transaction_denom_mart, effectively fixing the tests that were not running + removing the following warnings:

```
06:15:13  [WARNING]: Did not find matching node for patch with name 'xero__bank_transaction_denom_mart.sql' in the 'models' section of file 'models/reporting/xero/schema.yml'
06:15:14  [WARNING]: Test 'test.dwh_dbt.not_null_xero__bank_transaction_denom_mart.sql_id_line_item.70cdeb323e' (models/reporting/xero/schema.yml) depends on a node named 'xero__bank_transaction_denom_mart.sql' in package '' which was not found
06:15:14  [WARNING]: Test 'test.dwh_dbt.unique_xero__bank_transaction_denom_mart.sql_id_line_item.b85e051d2d' (models/reporting/xero/schema.yml) depends on a node named 'xero__bank_transaction_denom_mart.sql' in package '' which was not found
```

# Checklist

- [X] The edited models and dependants run properly with production data.
- [ ] The edited models are sufficiently documented. *Have not checked*
- [X] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs. *Have not checked*
- [ ] I've picked the right materialization for the affected models. *Have not checked*

# Other

- [ ] Check if a full-refresh is required after this PR is merged.
2024-09-18 12:19:23 +00:00
uri
5620ecbe68 Remove .sql in the xero__bank_transaction_denom_mart 2024-09-18 13:52:55 +02:00
Oriol Roqué Paniagua
985ddb8e4d Merged PR 2887: Check-In Hero - Removing unused fields + schema changes
# Description
This PR closes the ticket #20046

Changes:
* Removes temporary fields marked as legacy in `core__vr_check_in_cover`, after the changes in the 3 PBI Check-In Hero reports.
* Additionally, I modified the schema entry to comply with the current fields available in this model.

# Checklist

- [X] The edited models and dependants run properly with production data. *Checked in local changing the sources to localhost after dropping the fields.*
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs. *N/A*
- [ ] I've picked the right materialization for the affected models. *N/A*

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Remove temporary fields

Related work items: #20046
2024-09-18 11:48:49 +00:00
Joaquin Ossa
40bfa60016 core__verification_payments_v2 2024-09-18 12:41:00 +02:00
Joaquin Ossa
9e62786aa9 commit2 2024-09-18 11:26:23 +02:00
Pablo Martin
136c297683 add logic 2024-09-17 17:56:56 +02:00
Joaquin Ossa
e854f2a99a commit 2024-09-17 17:22:37 +02:00
Joaquin Ossa
78e7e37c5e Update model core__verification_payments to include amounts without taxes 2024-09-17 17:22:37 +02:00
Joaquin Ossa
17f7b11a79 Merged PR 2885: update int_core__guest_satisfaction_responses to version 2 of verification_pa...
# Description

update int_core__guest_satisfaction_responses to version 2 of verification_payments

# Checklist

- [x] The edited models and dependants run properly with production data.
- [x] The edited models are sufficiently documented.
- [x] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #20044
2024-09-17 15:21:59 +00:00
Joaquin Ossa
2dd4f66189 update int_core__guest_satisfaction_responses to version 2 of verification_payments 2024-09-17 17:12:46 +02:00
Pablo Martín
4bafeba787 Merged PR 2883: Payaway per month and user
# Description

This PR adds an intermediate tables that holds a combo of all payaway plans with the months (technically, combinations of first day and last day of month) when the plan was the relevant one for invoicing purposes.

The goal is to have this as an intermediate helper for some tax related logic in verification payments.

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #20043
2024-09-17 14:13:01 +00:00
Pablo Martin
a20d511b93 missing text 2024-09-17 16:12:38 +02:00
Pablo Martin
049c2d3e31 model and docs+tests 2024-09-17 15:47:45 +02:00
Pablo Martín
140da0a75d Merged PR 2874: Add Payaway table to intermediate
# Description

Brings the Payaway table from staging to intermediate.

The only significant change is that we change the internal Superhog currency id for the iso4217 code.

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #20043
2024-09-17 12:07:07 +00:00