Commit graph

565 commits

Author SHA1 Message Date
Joaquin Ossa
798fc7b937 Removed coalesce so it includes NULL for failed validations 2024-08-20 09:49:34 +02:00
Joaquin Ossa
e8b76e130e Created address_validations in intermediate 2024-08-19 11:43:54 +02:00
Joaquin Ossa
52f42b9e9b Revert "Created address_validation in staging"
This reverts commit 4a6fdd5ac7.
2024-08-16 12:57:54 +02:00
Joaquin Ossa
4a6fdd5ac7 Created address_validation in staging 2024-08-16 12:51:13 +02:00
Joaquin Ossa
63d7ebb6d1 Merged PR 2564: New model for verification_to_address_validation
# Description

New model for verification_to_address_validation for check in hero reporting

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #17069
2024-08-16 09:21:31 +00:00
Joaquin Ossa
be9757b886 Modified source and included test for id_address_validation as unique as well 2024-08-16 11:15:40 +02:00
Joaquin Ossa
34ba2a2e4a Merged PR 2563: new address_validation model in staging
# Description

New address_validation model in staging to include in our check in hero reports

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

new address_validation model in staging

Related work items: #17069
2024-08-14 15:37:12 +00:00
Joaquin Ossa
1d4a2d7b34 reorganized columns 2024-08-14 17:36:28 +02:00
Joaquin Ossa
1e818ea306 organized columns 2024-08-14 17:34:04 +02:00
Joaquin Ossa
fa5cfb8c66 removed extra _airbyte_raw_id 2024-08-14 17:31:36 +02:00
Joaquin Ossa
fd44f94b78 new verification_to_address_validation model in staging 2024-08-14 17:30:09 +02:00
Joaquin Ossa
cdd3b6a58f new address_validation model in staging 2024-08-14 17:16:35 +02:00
Pablo Martín
cb3cf24b60 Merged PR 2540: Add UserProductBundle to staging
# Description

Adds Core's `UserProductBundle` table to `staging`.

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #19570
2024-08-12 09:20:35 +00:00
Pablo Martin
2ebf6a711c add tests 2024-08-12 11:20:09 +02:00
Pablo Martin
419f07e77c rename 2024-08-12 11:19:32 +02:00
Pablo Martin
a39f4d24aa add staging model 2024-08-12 11:18:49 +02:00
Pablo Martin
501a063635 add source 2024-08-12 11:16:53 +02:00
Pablo Martin
8c78f880ec change column name, previous was not accurate 2024-08-12 11:08:28 +02:00
Pablo Martin
22ccae5b9a add tests 2024-08-12 11:05:12 +02:00
Pablo Martín
6c77eed984 Merged PR 2539: Claim model to staging
# Description

Brings over the table `Claim` from `sync_core` to `staging`.

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #19570
2024-08-12 09:03:32 +00:00
Pablo Martin
59477a4641 typo 2024-08-12 11:01:09 +02:00
Pablo Martin
51fd0cf3bc add staging model 2024-08-12 11:00:13 +02:00
Pablo Martin
e4b0f54241 add to sources 2024-08-12 10:47:55 +02:00
Pablo Martin
cfe02fb99c add PK test 2024-08-12 10:42:40 +02:00
Pablo Martin
81b900b6d2 add claim to staging 2024-08-12 10:42:29 +02:00
Pablo Martin
0ba3f13055 add claim to sources 2024-08-12 10:37:40 +02:00
Oriol Roqué Paniagua
d323460d1a Merged PR 2534: Deal metrics per customer segmentation
# Description

Modifies `int_core__mtd_deal_metrics` to include the customer segmentation based on listings. `schema.yaml` is also affected including new fields and tests. Hardcoded `int_core__mtd_vs_previous_year_metrics` to avoid propagating this upwards and messing up with the data display.

Overall, follows a similar strategy as we did for Booking and Guest Journey metrics. For reference, [here's the previous PR on GJ](https://guardhog.visualstudio.com/Data/_git/data-dwh-dbt-project/pullrequest/2533).

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #19325
2024-08-09 10:03:35 +00:00
Oriol Roqué Paniagua
6a660aeac8 Merged PR 2533: Guest Journey metrics per customer segmentation
# Description

Modifies `int_core__mtd_guest_journey_metrics` to include the customer segmentation based on listings. `schema.yaml` is also affected including new fields and tests. Hardcoded `int_core__mtd_vs_previous_year_metrics` to avoid propagating this upwards and messing up with the data display.

Overall, follows a similar strategy as we did for Booking metrics.

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #19325
2024-08-09 08:41:14 +00:00
Joaquin Ossa
6116633590 Merged PR 2531: Fixed models for mtd_aggregated_metrics
# Description

Fixed models for mtd_aggregated_metrics

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #19559
2024-08-09 07:11:58 +00:00
Joaquin Ossa
87bee4a95a Fixed models for mtd_aggregated_metrics 2024-08-08 17:06:11 +02:00
Oriol Roqué Paniagua
6caac0bb02 Merged PR 2527: Revert 'Added relative_increment_with_sign_format'
# Description

_Describe your motivation and changes here._

# Checklist

- [ ] The edited models and dependants run properly with production data.
- [ ] The edited models are sufficiently documented.
- [ ] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

# Description

Added relative_increment_with_sign_format for special formatting in PBI

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Added relative_increment_with_sign_format

Reverts !2524

Related work items: #19559
2024-08-08 13:27:33 +00:00
Oriol Roqué Paniagua
661f9b9245 Merged PR 2526: Revert 'Modified mtd_aggregated_metrics to reporting'
# Description

_Describe your motivation and changes here._

# Checklist

- [ ] The edited models and dependants run properly with production data.
- [ ] The edited models are sufficiently documented.
- [ ] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

# Description

Modified mtd_aggregated_metrics to reporting

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Modified mtd_aggregated_metrics to reporting

Reverts !2525

Related work items: #19559
2024-08-08 13:27:06 +00:00
Joaquin Ossa
0777b3210a Merged PR 2525: Modified mtd_aggregated_metrics to reporting
# Description

Modified mtd_aggregated_metrics to reporting

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Modified mtd_aggregated_metrics to reporting

Related work items: #19559
2024-08-08 13:20:31 +00:00
Oriol Roqué Paniagua
b61fe3f70d Merged PR 2522: Don't Repeat Yourself for KPIs - Applied to Bookings
# Description

What's new:
- Creation of `get_kpi_dimensions`: new macro to have a single point of source of configuration for dimensions for the KPIs. It's a way to enforce global variables on-demand. I kind of like this approach and we could do it for Xero models as well :)
- Modification of `int_core__mtd_booking_metrics` and `int_dates_mtd_by_dimension`: removal of duplicated code within the dimension context. Uses Jinja code and applies different configurations depending on the dimension chosen. Still, different metrics are placed in different CTEs. I believe it might be possible to also configure metrics BUT at the cost of over-complexifying the macro logic, so I wouldn't go for it at this stage.

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] **I have checked for DRY opportunities with other models and docs.**
- [X] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #19325
2024-08-08 13:19:57 +00:00
Joaquin Ossa
d11f639562 Modified mtd_aggregated_metrics to reporting 2024-08-08 15:08:11 +02:00
Joaquin Ossa
90a7ad5eea Merged PR 2524: Added relative_increment_with_sign_format
# Description

Added relative_increment_with_sign_format for special formatting in PBI

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Added relative_increment_with_sign_format

Related work items: #19559
2024-08-08 13:02:25 +00:00
Joaquin Ossa
8ac1eb4717 Added relative_increment_with_sign_format 2024-08-08 14:48:12 +02:00
Oriol Roqué Paniagua
afc20f0e20 Merged PR 2519: mtd bookings with 2 dimensions
# Description
This is a first idea of how I'd like to add dimensionality in the KPIs for the mtd models. For the moment, I keep deal_id apart, so I just touch the "mtd" models, that so far only contained "global" metrics.

In this case I include the listing segmentation (0, 1-5, 6-20, etc) in the bookings. To do this, I created 2 new fields: dimension and dimension_values.
I also created a "master" table with `date` - `dimension` - `dimension_value` called `int_dates_mtd_by_dimension`

Important notes:
- I force a hardcode in `int_mtd_vs_previous_year_metrics`. This is to not break production.
- You will notice how repetitive the code is starting to look. My intention with this PR is that we are happy with this approach on the naming, the strategy for joins, etc. If that's ok, next step is going to be doing macros on top. Think of the state of `int_core__mtd_booking_metrics` as the "compiled version" of the macro that should come afterwards.

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #19325
2024-08-08 09:11:01 +00:00
Oriol Roqué Paniagua
7177b41e19 Merged PR 2516: Create host_user and user_role in intermediate
# Description

Adds 2 new tables:
- `int_core__user_role`: contains the relationship of a given user has a role.
- `int_core__user_host`: based on the previous table, it selects the users and main information from those users that are considered as hosts according to the role they have.

Note: I needed to change the test in stg. A user, generally, can have no role, one role, or multiple roles. Thus we cannot propagate this information in the unified_user model.

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #19513
2024-08-07 14:28:34 +00:00
Oriol Roqué Paniagua
ad7d94b604 Merged PR 2511: Adding stg_core__role and stg_core__user_role in staging
# Description

Adds Role and UserRole in staging, in dedicated tables.
Open to discussion if you want me to directly apply this at User table level instead of having them in staging as standalone tables. Otherwise, staying with this PR as is, next step would be to integrate these roles in unified_user and create a dedicated user_host table in intermediate

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #19513
2024-08-07 13:17:31 +00:00
Oriol Roqué Paniagua
94bdc53adf Merged PR 2498: Materialise int_dates_mtd and int_dates_by_deal as table to improve performance
# Description

Materialise int_dates_mtd and int_dates_by_deal as tables. This should improve the run speed as seen in local by quite a bit, and hopefully provide a better starting point for adding new dimensionality on business kpis.

I also documented these 2 models, that were missing :)

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.

# Other

- [] Check if a full-refresh is required after this PR is merged.

Related work items: #19514
2024-08-06 15:03:32 +00:00
Joaquin Ossa
b1bdbc0f39 Merged PR 2459: duplicated_bookings modification
# Description

I modified the model of duplicated_bookings, I saw it had some errors and data that was not being used

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #18875
2024-08-05 09:44:11 +00:00
Joaquin Ossa
7fe2d779e2 Merged PR 2469: Included the exchange rate to GBP in the model
# Description

Included the exchange rate to GBP in the model instead of joining the data inside the PBI report, for both understanding and easier joining

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.

# Other

- [X] Check if a full-refresh is required after this PR is merged.

Included the exchange rate to GBP in the model

Related work items: #18875
2024-08-02 11:10:13 +00:00
Joaquin Ossa
a82ab121f1 Deleted an extra line 2024-08-01 16:18:25 +02:00
Joaquin Ossa
02ddac3167 Added a coalesce for the booking_fee_in_gbp calculation for old cases where there is no preferred currency for the user_guest 2024-08-01 16:15:48 +02:00
Joaquin Ossa
25e657014a Modified filter and table used 2024-08-01 14:32:45 +02:00
Joaquin Ossa
0281e6e72e Included the exchange rate to GBP in the model 2024-08-01 12:58:36 +02:00
Joaquin Ossa
161f4c1645 Merged PR 2467: Added host_booking_fees model to report
# Description

New model for report host_fees

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.

# Other

- [X] Check if a full-refresh is required after this PR is merged.

Related work items: #18875
2024-08-01 10:01:54 +00:00
Joaquin Ossa
a75a4e9763 Added host_booking_fees model to report 2024-07-31 17:35:14 +02:00
Joaquin Ossa
adefe2d3ce Merged PR 2465: Creating model host_booking_fees
# Description

New model to replace in the host_fees report which is now including currencies

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #18875
2024-07-31 15:19:12 +00:00