Commit graph

1106 commits

Author SHA1 Message Date
Joaquin Ossa
b03982b708 Added total fee cost of booking 2024-12-02 08:44:24 +01:00
Joaquin Ossa
126e702c6e Merged PR 3706: model to reporting
# Description

Verifications model to reporting
I created a new folder `cross` for this model in reporting

# Checklist

- [x] The edited models and dependants run properly with production data.
- [x] The edited models are sufficiently documented.
- [x] The edited models contain PK tests, and I've ran and passed them.
- [x] I have checked for DRY opportunities with other models and docs.
- [x] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

model to reporting

Related work items: #24825
2024-11-29 11:48:38 +00:00
Joaquin Ossa
1c9bb5ec31 model to reporting 2024-11-29 11:24:57 +01:00
Joaquin Ossa
23dbf3c97a Merged PR 3705: APIs verifications
# Description

Cross model to unite Edeposit and Guesty verifications

# Checklist

- [x] The edited models and dependants run properly with production data.
- [x] The edited models are sufficiently documented.
- [x] The edited models contain PK tests, and I've ran and passed them.
- [x] I have checked for DRY opportunities with other models and docs.
- [x] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #24825
2024-11-29 10:04:26 +00:00
Joaquin Ossa
edbeac253c Name change 2024-11-29 11:03:59 +01:00
Joaquin Ossa
c6a03d531a Cleaned commit 2024-11-29 10:06:15 +01:00
Joaquin Ossa
4e6690c013 API verifications model 2024-11-29 09:59:35 +01:00
Joaquin Ossa
07038e8c18 added service_business_type to new_dash_created_services models 2024-11-28 16:12:02 +01:00
Joaquin Ossa
c3fedec0f9 Merged PR 3686: Update exposures
# Description

Update exposures on Guest KPIs Report

# Checklist

- [ ] The edited models and dependants run properly with production data.
- [ ] The edited models are sufficiently documented.
- [ ] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Update exposures

Related work items: #24604
2024-11-27 16:43:07 +00:00
Joaquin Ossa
fc2ee14acb Update exposures 2024-11-27 16:29:19 +01:00
Pablo Martín
9dcfbd6fd2 Merged PR 3682: Remove unused accommodation fields
# Description

Dash squad is going to remove four fields from the `Accommodation` table in SQL Server. We are removing them from the project as well to avoid errors. The removal is painless because we weren't really using them for anything important, they were simply brought as part of the `Accommodation` table.

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.

# Other

- [X] Check if a full-refresh is required after this PR is merged.
2024-11-27 14:05:44 +00:00
Pablo Martin
ff2079a0c6 remove fields from docs 2024-11-27 14:47:43 +01:00
Pablo Martin
f460a47833 remove fields from stg 2024-11-27 14:46:41 +01:00
Pablo Martin
878e2e81b6 remove fields from int 2024-11-27 14:46:03 +01:00
Joaquin Ossa
fd7a993085 Merged PR 3678: Changed filters
# Description

Changed filters on the calculation of the aggregated metrics.
There was an issue before with the filter for the yearly metrics and I modified the ones for monthly as well to help with performance.

# Checklist

- [x] The edited models and dependants run properly with production data.
- [x] The edited models are sufficiently documented.
- [x] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Changed filters

Related work items: #24604
2024-11-27 08:58:52 +00:00
Joaquin Ossa
f38696f3a3 Merged PR 3677: fixed test
# Description

Fixed test to detect outliers in kpis daily metric models

# Checklist

- [x] The edited models and dependants run properly with production data.
- [x] The edited models are sufficiently documented.
- [x] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

fixed test

Related work items: #24590
2024-11-27 08:58:46 +00:00
Joaquin Ossa
80f42d5426 Modified mtd filter 2024-11-27 09:52:05 +01:00
Joaquin Ossa
8c52049f86 Changed filters 2024-11-27 09:47:30 +01:00
Joaquin Ossa
613bac4b70 fixed test 2024-11-26 19:06:53 +01:00
Oriol Roqué Paniagua
db2d626ae0 Merged PR 3676: Hotfix - Chargeable Services - Daily Agg
# Description

I missed applying the Chargeable Services in Daily Agg. This fixes it.

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [NA] I have checked for DRY opportunities with other models and docs.
- [NA] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Hotfix

Related work items: #20809
2024-11-26 16:52:48 +00:00
Joaquin Ossa
90356dd9c7 Merged PR 3675: aggregated guest kpis to reporting
# Description

Aggregated guest kpis model to reporting

# Checklist

- [x] The edited models and dependants run properly with production data.
- [x] The edited models are sufficiently documented.
- [x] The edited models contain PK tests, and I've ran and passed them.
- [x] I have checked for DRY opportunities with other models and docs.
- [x] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

aggregated guest kpis to reporting

Related work items: #24604
2024-11-26 16:19:06 +00:00
Pablo Martín
96f061e122 Merged PR 3671: Refactor new dash user overview
# Description

This query refactors the model `int_core__new_dash_user_overview` to optimize performance.

The output of the model remains unaltered.

The performance gains are obtained by preventing a monster-grade intermediate join with almost 100 million rows + a sequential order that the current `master` model causes. This refactor breaks down the query that caused that into multiple CTEs that pre-aggregate results, avoiding the dimensional explosion.

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged
2024-11-26 16:03:22 +00:00
Joaquin Ossa
4360258ac9 aggregated guest kpis to reporting 2024-11-26 16:43:56 +01:00
Joaquin Ossa
0668c0beb4 Merged PR 3672: fixed outlier_detector
# Description

PR to fix the kpis_daily_outlier_detector test.
Previously it was using all values for the metric independently of the granularity, now is adding them up by daily value and no other metrics.

# Checklist

- [x] The edited models and dependants run properly with production data.
- [ ] The edited models are sufficiently documented.
- [x] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

fixed outlier_detector

Related work items: #24590
2024-11-26 15:37:08 +00:00
Joaquin Ossa
dd4dff1479 Merged PR 3666: Aggregated model for guest kpis metrics in intermediate
# Description

Aggregated model for guest kpis metrics.
This model aggregates the values for all basic metrics for guest kpis, it has a granularity per metrics of timeframe (YTD, MTD, WTD) and all other dimensions needed (has_payment, has_id_check & main_billing_country_iso_3_per_deal).
I decided not to include the calculated metrics because it adds much more complexity to the model specially when you start considering all possible combinations between dimensions and how this would affect the calculations, in this case conversion_rate, revenue_rate and average_csat_score. I think this could all be managed much simpler inside PBI and since the granularity of the model is not as deep as the daily model, this should help with the loading time for the report.

# Checklist

- [x] The edited models and dependants run properly with production data.
- [x] The edited models are sufficiently documented.
- [x] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #24604
2024-11-26 15:37:06 +00:00
Joaquin Ossa
0ea07ed6bf fixed outlier_detector 2024-11-26 16:10:41 +01:00
Pablo Martin
511bef5c9f refactor 2024-11-26 16:05:55 +01:00
Joaquin Ossa
c77defab28 modified py_week_current_date calculation 2024-11-26 15:29:14 +01:00
Oriol Roqué Paniagua
42d70f72d5 Merged PR 3667: Creates Chargeable metrics for New Dash KPIs
# Description

Creates the models for KPIs for New Dash - Chargeable metrics. In essence, computes 4 metrics:
- Chargeable Services
- Chargeable Amount (in GBP)
- Chargeable Bookings (unique over a time period and dimension, not additive)
- Chargeable Listings (unique over a time period and dimension, not additive)

This is done by creating:
- A Weekly Metric and Monthly Metric model. Here we keep the granularity of id_booking / id_accommodation to be able to compute the uniqueness.
- A Daily, Weekly and Monthly Aggregated models. Same as usual.
- Integrates everything to the existing model for Product New Dash Agg Metrics
- Exposes everything into reporting

NB: I removed on "by_host" in Created Services - I forgot to clear it out.

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #20809
2024-11-26 14:19:41 +00:00
Joaquin Ossa
2b89af2bd9 modified abreviations 2024-11-26 14:16:53 +01:00
Joaquin Ossa
07ffb14e9c Aggregated model for guest kpi metrics 2024-11-26 14:05:03 +01:00
Joaquin Ossa
b7c5a526f7 Merged PR 3651: Adding billing_country as dimension and new aggregated model
# Description

Added billing_country as dimension to use in the Guest KPIs Report.
Also created a new model that pivots the metrics to leave them as a column and aggregates the values set for different timeframes (YTD, MTD, WTD). For the moment it only aggregates the values for the existing metrics on the existing models, it doesn't compute metrics like rates or averages (avg_csat_score, conversion_rate, etc..). I am not sure if it would be better to compute here in the model or just doing it in PBI, both are viable options not sure if the model is going to end up being too monstrous or it doesn't matter.
@<Oriol Roqué Paniagua> what do you think??

Imagen para ayudar a entender el modelo
![image (2).png](https://guardhog.visualstudio.com/4148d95f-4b6d-4205-bcff-e9c8e0d2ca65/_apis/git/repositories/54ac356f-aad7-46d2-b62c-e8c5b3bb8ebf/pullRequests/3651/attachments/image%20%282%29.png)

# Checklist

- [x] The edited models and dependants run properly with production data.
- [x] The edited models are sufficiently documented.
- [x] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [x] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #24604
2024-11-26 13:01:58 +00:00
Joaquin Ossa
68644630d4 rebase and updated schema 2024-11-26 12:36:54 +01:00
Joaquin Ossa
6b5b7d2d5a fixed model copy paste 2024-11-26 12:36:54 +01:00
Joaquin Ossa
2404905da1 Removed agg model 2024-11-26 12:36:54 +01:00
Joaquin Ossa
b355d8cbc7 Added schema of new model 2024-11-26 12:36:54 +01:00
Joaquin Ossa
de2cb8264b fixed join for billing_countries and created new model for timeframe aggregations 2024-11-26 12:36:54 +01:00
Joaquin Ossa
7717ce3c53 Added billing_country 2024-11-26 12:36:54 +01:00
Joaquin Ossa
961b5312b2 commit 2024-11-26 12:35:28 +01:00
Oriol Roqué Paniagua
2251a45540 Merged PR 3654: New Dash KPIs - Chargeable Services (Revenue)
# Description

This PR aims to do 2 things:

1. Create the first metric daily model for New Dash - Chargeable Services
* PK at Booking, Date and Service. I added a few more dimensions such as Accommodation and Business Type. The idea is that while Daily Unique Bookings/Accommodations Charged will be close (if not the same) as Charged Services, by having the ID we can compute Monthly/Weekly Unique Bookings/Accommodations Charged in a proper manner. Besides this, we would still compute additive metrics in the future such as the sum of Charged Services and the sum of Service Total Price in GBP.
* `IMPORTANT`: as discussed in the daily, I changed New Pricing models containing "Charged" columns to "Chargeable". This affects the new model for New Dash KPIs, but also `int_core__booking_summary` and `int_core__booking_service_detail`.

2. Small fixes on New Dash - Created Services.
* Mainly, there were some inconsistencies with what was actually written (and really applied) in the schema entry as for what was the PK of the model vs. what was stated in the model. I just re-ordered the columns and re-specified correctly the PK.

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #20809
2024-11-26 10:14:37 +00:00
Joaquin Ossa
645958239e Merged PR 3643: KPIs outlier detector
# Description

First attempt at creating a test.
This test checks that the new values (from yesterday) don't vary too much from the previous days. The user can set the metrics to check on, how much tolerance to use in number of standard deviations allowed to deviate from the average and the number of days to consider in the evaluation.

# Checklist

- [x] The edited models and dependants run properly with production data.
- [ ] The edited models are sufficiently documented.
- [x] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

KPIs outlier detector

Related work items: #24590
2024-11-26 10:13:57 +00:00
Pablo Martín
ab0ac1c47e Merged PR 3630: Build docs
# Description

This PR adds a script to build the project documentation and inclues some notes on how to call it and serve the docs.

# Checklist

NA

# Other
NA

Related work items: #23999
2024-11-26 09:58:12 +00:00
Joaquin Ossa
658980e48f Modified name and changed date to group as an argument 2024-11-26 10:33:03 +01:00
Joaquin Ossa
0f5eb5fba2 Addressed comments 2024-11-26 10:16:08 +01:00
Oriol Roqué Paniagua
b108730903 Merged PR 3653: Adds new role ResolutionApi in accepted values of int_core__user_role
# Description

Adds new role `ResolutionApi` in accepted values of `int_core__user_role`

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [NA] I have checked for DRY opportunities with other models and docs.
- [NA] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.
2024-11-26 09:12:09 +00:00
Joaquin Ossa
bede588062 generalized test 2024-11-26 09:27:13 +01:00
Oriol Roqué Paniagua
28114d645e Merged PR 3646: Adding informative fields in booking tables for New Pricing
# Description

Adds informative fields to New Pricing booking-related tables. These are added into booking_to_service which is a view, and propagated and materialised as table in the `booking_service_detail` and `booking_summary`.

The new fields are:
- Date versions for relevant booking timestamps (Created, Updated, Check-in, Check-out)
- Booking Number of nights
- Ids for the Accommodation and UserProductBundle
- Program Name (aka Product Bundle Name). I dislike the fact that it's named "Product" because you can have Protection Plans in the Program as well... so I opted for a more conventional business-oriented naming, which is Program.

Added a bunch of tests + documentation.

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #20809
2024-11-25 15:03:38 +00:00
Joaquin Ossa
aaf512a288 fixed model 2024-11-25 10:11:05 +01:00
Joaquin Ossa
0c5dd298ed added comments 2024-11-22 17:01:34 +01:00
Joaquin Ossa
e44a9c19fc KPIs outlier detector 2024-11-22 16:55:13 +01:00