Commit graph

1461 commits

Author SHA1 Message Date
Joaquin Ossa
ff962608ad Added Address_validation to int_core__vr_check_in_cover 2024-08-20 09:27:49 +02:00
Oriol Roqué Paniagua
e1d04c2e4e Merged PR 2580: Guest Payments metrics per customer segmentation
# Description

Modifies `int_core__mtd_guest_payments_metrics` to include the customer segmentation based on listings. `schema.yaml` is also affected including new fields, tests and apply the proper naming (from guest revenue to guest payments). I also modified a silly naming that was referring to deals to refer to listings/accommodations, my bad.
Added the macro to retrieve the production dimension in `int_core__mtd_vs_previous_year_metrics` to avoid propagating this upwards and messing up with the data display.

Overall, follows a similar strategy as we did for Booking, Guest Journey, Deal and Accommodation metrics. For reference, [here's the previous PR on Accommodations](https://guardhog.visualstudio.com/Data/_git/data-dwh-dbt-project/pullrequest/2575?_a=overview).

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #19325
2024-08-19 11:45:42 +00:00
Oriol Roqué Paniagua
997cb85c6c Merged PR 2577: Adding get_kpi_dimensions_for_production
# Description

Takes into account @<Pablo Martín> 's feedback from the previous PR, slightly modified. This PR separates 1) the dimensions while developing vs. 2) the dimensions once these are available for production. This are within the same file of macro configuration for KPIs, namely `business_kpis_configuration`.

End-goal, all CTEs in `int_mtd_vs_previous_year_metrics` will read from this new macro `get_kpi_dimensions_for_production`, so eventually we won't need any hardcode once we want to add new dimensions. In the meantime, I'll be adding this new line for each PR (still 2 missing :D)

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [ ] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #19325
2024-08-19 09:57:28 +00:00
Joaquin Ossa
e8b76e130e Created address_validations in intermediate 2024-08-19 11:43:54 +02:00
Oriol Roqué Paniagua
89dd8845cc Merged PR 2575: Accommodation metrics per customer segmentation
# Description

Modifies `int_core__mtd_accommodation_metrics` to include the customer segmentation based on listings. `schema.yaml` is also affected including new fields and tests. Hardcoded `int_core__mtd_vs_previous_year_metrics` to avoid propagating this upwards and messing up with the data display.

Overall, follows a similar strategy as we did for Booking, Guest Journey and Deal metrics. For reference, here's [the previous PR on Deal](https://guardhog.visualstudio.com/Data/_git/data-dwh-dbt-project/pullrequest/2534). I noticed that I mixed the schema tests of Deals and Accommodations, this PR should fix both.

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #19325
2024-08-19 09:03:42 +00:00
Joaquin Ossa
52f42b9e9b Revert "Created address_validation in staging"
This reverts commit 4a6fdd5ac7.
2024-08-16 12:57:54 +02:00
Joaquin Ossa
4a6fdd5ac7 Created address_validation in staging 2024-08-16 12:51:13 +02:00
Joaquin Ossa
63d7ebb6d1 Merged PR 2564: New model for verification_to_address_validation
# Description

New model for verification_to_address_validation for check in hero reporting

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #17069
2024-08-16 09:21:31 +00:00
Joaquin Ossa
be9757b886 Modified source and included test for id_address_validation as unique as well 2024-08-16 11:15:40 +02:00
Joaquin Ossa
1d4a2d7b34 reorganized columns 2024-08-14 17:36:28 +02:00
Joaquin Ossa
1e818ea306 organized columns 2024-08-14 17:34:04 +02:00
Joaquin Ossa
fa5cfb8c66 removed extra _airbyte_raw_id 2024-08-14 17:31:36 +02:00
Joaquin Ossa
fd44f94b78 new verification_to_address_validation model in staging 2024-08-14 17:30:09 +02:00
Joaquin Ossa
cdd3b6a58f new address_validation model in staging 2024-08-14 17:16:35 +02:00
Pablo Martin
2ebf6a711c add tests 2024-08-12 11:20:09 +02:00
Pablo Martin
419f07e77c rename 2024-08-12 11:19:32 +02:00
Pablo Martin
a39f4d24aa add staging model 2024-08-12 11:18:49 +02:00
Pablo Martin
501a063635 add source 2024-08-12 11:16:53 +02:00
Pablo Martin
8c78f880ec change column name, previous was not accurate 2024-08-12 11:08:28 +02:00
Pablo Martin
22ccae5b9a add tests 2024-08-12 11:05:12 +02:00
Pablo Martin
59477a4641 typo 2024-08-12 11:01:09 +02:00
Pablo Martin
51fd0cf3bc add staging model 2024-08-12 11:00:13 +02:00
Pablo Martin
e4b0f54241 add to sources 2024-08-12 10:47:55 +02:00
Pablo Martin
cfe02fb99c add PK test 2024-08-12 10:42:40 +02:00
Pablo Martin
81b900b6d2 add claim to staging 2024-08-12 10:42:29 +02:00
Pablo Martin
0ba3f13055 add claim to sources 2024-08-12 10:37:40 +02:00
Oriol Roqué Paniagua
d323460d1a Merged PR 2534: Deal metrics per customer segmentation
# Description

Modifies `int_core__mtd_deal_metrics` to include the customer segmentation based on listings. `schema.yaml` is also affected including new fields and tests. Hardcoded `int_core__mtd_vs_previous_year_metrics` to avoid propagating this upwards and messing up with the data display.

Overall, follows a similar strategy as we did for Booking and Guest Journey metrics. For reference, [here's the previous PR on GJ](https://guardhog.visualstudio.com/Data/_git/data-dwh-dbt-project/pullrequest/2533).

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #19325
2024-08-09 10:03:35 +00:00
Oriol Roqué Paniagua
6a660aeac8 Merged PR 2533: Guest Journey metrics per customer segmentation
# Description

Modifies `int_core__mtd_guest_journey_metrics` to include the customer segmentation based on listings. `schema.yaml` is also affected including new fields and tests. Hardcoded `int_core__mtd_vs_previous_year_metrics` to avoid propagating this upwards and messing up with the data display.

Overall, follows a similar strategy as we did for Booking metrics.

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #19325
2024-08-09 08:41:14 +00:00
Joaquin Ossa
87bee4a95a Fixed models for mtd_aggregated_metrics 2024-08-08 17:06:11 +02:00
Oriol Roqué Paniagua
6caac0bb02 Merged PR 2527: Revert 'Added relative_increment_with_sign_format'
# Description

_Describe your motivation and changes here._

# Checklist

- [ ] The edited models and dependants run properly with production data.
- [ ] The edited models are sufficiently documented.
- [ ] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

# Description

Added relative_increment_with_sign_format for special formatting in PBI

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Added relative_increment_with_sign_format

Reverts !2524

Related work items: #19559
2024-08-08 13:27:33 +00:00
Oriol Roqué Paniagua
661f9b9245 Merged PR 2526: Revert 'Modified mtd_aggregated_metrics to reporting'
# Description

_Describe your motivation and changes here._

# Checklist

- [ ] The edited models and dependants run properly with production data.
- [ ] The edited models are sufficiently documented.
- [ ] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

# Description

Modified mtd_aggregated_metrics to reporting

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Modified mtd_aggregated_metrics to reporting

Reverts !2525

Related work items: #19559
2024-08-08 13:27:06 +00:00
Joaquin Ossa
0777b3210a Merged PR 2525: Modified mtd_aggregated_metrics to reporting
# Description

Modified mtd_aggregated_metrics to reporting

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Modified mtd_aggregated_metrics to reporting

Related work items: #19559
2024-08-08 13:20:31 +00:00
Oriol Roqué Paniagua
b61fe3f70d Merged PR 2522: Don't Repeat Yourself for KPIs - Applied to Bookings
# Description

What's new:
- Creation of `get_kpi_dimensions`: new macro to have a single point of source of configuration for dimensions for the KPIs. It's a way to enforce global variables on-demand. I kind of like this approach and we could do it for Xero models as well :)
- Modification of `int_core__mtd_booking_metrics` and `int_dates_mtd_by_dimension`: removal of duplicated code within the dimension context. Uses Jinja code and applies different configurations depending on the dimension chosen. Still, different metrics are placed in different CTEs. I believe it might be possible to also configure metrics BUT at the cost of over-complexifying the macro logic, so I wouldn't go for it at this stage.

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] **I have checked for DRY opportunities with other models and docs.**
- [X] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #19325
2024-08-08 13:19:57 +00:00
Joaquin Ossa
d11f639562 Modified mtd_aggregated_metrics to reporting 2024-08-08 15:08:11 +02:00
Joaquin Ossa
8ac1eb4717 Added relative_increment_with_sign_format 2024-08-08 14:48:12 +02:00
Oriol Roqué Paniagua
afc20f0e20 Merged PR 2519: mtd bookings with 2 dimensions
# Description
This is a first idea of how I'd like to add dimensionality in the KPIs for the mtd models. For the moment, I keep deal_id apart, so I just touch the "mtd" models, that so far only contained "global" metrics.

In this case I include the listing segmentation (0, 1-5, 6-20, etc) in the bookings. To do this, I created 2 new fields: dimension and dimension_values.
I also created a "master" table with `date` - `dimension` - `dimension_value` called `int_dates_mtd_by_dimension`

Important notes:
- I force a hardcode in `int_mtd_vs_previous_year_metrics`. This is to not break production.
- You will notice how repetitive the code is starting to look. My intention with this PR is that we are happy with this approach on the naming, the strategy for joins, etc. If that's ok, next step is going to be doing macros on top. Think of the state of `int_core__mtd_booking_metrics` as the "compiled version" of the macro that should come afterwards.

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #19325
2024-08-08 09:11:01 +00:00
Oriol Roqué Paniagua
7177b41e19 Merged PR 2516: Create host_user and user_role in intermediate
# Description

Adds 2 new tables:
- `int_core__user_role`: contains the relationship of a given user has a role.
- `int_core__user_host`: based on the previous table, it selects the users and main information from those users that are considered as hosts according to the role they have.

Note: I needed to change the test in stg. A user, generally, can have no role, one role, or multiple roles. Thus we cannot propagate this information in the unified_user model.

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #19513
2024-08-07 14:28:34 +00:00
Oriol Roqué Paniagua
ad7d94b604 Merged PR 2511: Adding stg_core__role and stg_core__user_role in staging
# Description

Adds Role and UserRole in staging, in dedicated tables.
Open to discussion if you want me to directly apply this at User table level instead of having them in staging as standalone tables. Otherwise, staying with this PR as is, next step would be to integrate these roles in unified_user and create a dedicated user_host table in intermediate

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #19513
2024-08-07 13:17:31 +00:00
Oriol Roqué Paniagua
94bdc53adf Merged PR 2498: Materialise int_dates_mtd and int_dates_by_deal as table to improve performance
# Description

Materialise int_dates_mtd and int_dates_by_deal as tables. This should improve the run speed as seen in local by quite a bit, and hopefully provide a better starting point for adding new dimensionality on business kpis.

I also documented these 2 models, that were missing :)

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.

# Other

- [] Check if a full-refresh is required after this PR is merged.

Related work items: #19514
2024-08-06 15:03:32 +00:00
Joaquin Ossa
b1bdbc0f39 Merged PR 2459: duplicated_bookings modification
# Description

I modified the model of duplicated_bookings, I saw it had some errors and data that was not being used

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #18875
2024-08-05 09:44:11 +00:00
Joaquin Ossa
a82ab121f1 Deleted an extra line 2024-08-01 16:18:25 +02:00
Joaquin Ossa
02ddac3167 Added a coalesce for the booking_fee_in_gbp calculation for old cases where there is no preferred currency for the user_guest 2024-08-01 16:15:48 +02:00
Joaquin Ossa
25e657014a Modified filter and table used 2024-08-01 14:32:45 +02:00
Joaquin Ossa
0281e6e72e Included the exchange rate to GBP in the model 2024-08-01 12:58:36 +02:00
Joaquin Ossa
a75a4e9763 Added host_booking_fees model to report 2024-07-31 17:35:14 +02:00
Joaquin Ossa
646ff7276a Added function to get false instead of null in is_duplicate_booking 2024-07-31 16:38:33 +02:00
Joaquin Ossa
78ffefb145 Thanks Uri for the catch, then when id_user_guest is NULL it won't be considered a duplicate. 2024-07-31 16:23:27 +02:00
Joaquin Ossa
6cb06b73cd Modified how stg_core__booking was being called and some typo errors 2024-07-31 15:32:54 +02:00
Joaquin Ossa
00e7554300 Creation of host_booking_fees 2024-07-31 15:16:15 +02:00
Joaquin Ossa
c4fc81d934 Included tests 2024-07-31 12:48:57 +02:00