Commit graph

569 commits

Author SHA1 Message Date
Joaquin Ossa
84f2156c99 commit wip 2025-01-03 11:24:10 +01:00
Oriol Roqué Paniagua
e406bc89ad Merged PR 3926: Propagate to reporting + fix documentation
# Description

* Creates new model monthly_aggregated_metrics_history_by_deal_by_time_window
* Modifies existing intermediate entry on schema to properly fill all fields
* Creates same entry in reporting.

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #25829
2025-01-03 09:30:33 +00:00
Oriol Roqué Paniagua
a61da137fa Merged PR 3924: First version of time window aggregated metrics by deal
# Description

This model aggregates at monthly by deal level different metrics for AM reporting purposes. It also includes revenue retained ratios for client profitability assessment.

There's part of the existing AM report that could be simplified, likely, by using the new macro. This will be explored in a separated PR, if it applies.

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs. ** Checked and there might be possibilities to simplify the code. I'll check ones I finish this line of work**
- [X] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #25829
2025-01-03 08:07:16 +00:00
Oriol Roqué Paniagua
0d7b5ac88a Merged PR 3914: Include Hubspot deals for KPIs
# Description

Context: I'm intending to work on Account Managers reporting, that mostly will include reporting at Deal level the Resolutions Payouts as well as the new Retained metrics. While checking the great increase on Resolutions Payouts for October 2024:

![image (2).png](https://guardhog.visualstudio.com/4148d95f-4b6d-4205-bcff-e9c8e0d2ca65/_apis/git/repositories/54ac356f-aad7-46d2-b62c-e8c5b3bb8ebf/pullRequests/3914/attachments/image%20%282%29.png)

I decided to take a quick look into the main players... and surprise surprise we have Guesty:

![image.png](https://guardhog.visualstudio.com/4148d95f-4b6d-4205-bcff-e9c8e0d2ca65/_apis/git/repositories/54ac356f-aad7-46d2-b62c-e8c5b3bb8ebf/pullRequests/3914/attachments/image.png)

So Guesty represents 37k over the 73K of October. 50%. Not bad.

The main issue is that we've been aware for months now (since Churn efforts, mostly) that we're not reporting in KPIs those deals that are NOT created in Core. Most notably, API deals which includes... well, Guesty. So creating this kind of in-depth Account Managers improvement without reporting Guesty I think it would be very misleading. Note that the overall figures (Global dimension) are still accurate, though.

What's new:
* A new model named `int_kpis__dimension_deals` that basically retrieves Deals from both Core (as before) and Hubspot. It combines information from both and mostly assumes Hubspot as a better source of information than Core - although we do not have the Main Billing Country there afaik.
* Propagates changes, mostly in the monthly by deal view of Main KPIs. Here I confirm that now Guesty appears, and it only has metrics that come from Xero (APIs Revenue, Total Revenue, Resolutions, etc)

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #25829
2024-12-31 15:12:38 +00:00
Oriol Roqué Paniagua
29a005297a Merged PR 3906: Adds 2 new retained income metrics
# Description

It adds 2 new metrics in Main KPIs, both for Global/per Dimension and By Deal. If merged, metrics will NOT appear automatically in the Global/per Dimension nor in the By Deal.

These two new metrics are:
* Income Retained: Total Revenue - Waiver Paid Back to Host
* Income Retained Post Resolutions: Total Revenue - Waiver Paid Back to Host - Host Resolutions Amount Paid

Why these are important:

![image.png](https://guardhog.visualstudio.com/4148d95f-4b6d-4205-bcff-e9c8e0d2ca65/_apis/git/repositories/54ac356f-aad7-46d2-b62c-e8c5b3bb8ebf/pullRequests/3906/attachments/image.png)

Even though we grow considerably in terms of Revenue, the gap of Waivers that we pay back to host is also increasing. Thus the "real" increment is actually lower. However, what I find more interesting is the heavy decrease in Income Retained Post Resolutions. Here's the Year-by-Year comparison:

![image (2).png](https://guardhog.visualstudio.com/4148d95f-4b6d-4205-bcff-e9c8e0d2ca65/_apis/git/repositories/54ac356f-aad7-46d2-b62c-e8c5b3bb8ebf/pullRequests/3906/attachments/image%20%282%29.png)

In November 2024, we're back to the figures of 2023 (-0.4%) and this should be alarming considering we're growing in Total Revenue by 27% and in Retained Income by 41% vs. Nov 23. In terms of business impact, I'd opt for having these 2 metrics computed as these capture a reality that otherwise we keep hidden. Happy to discuss renames and how/if we move forward with this.

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Attached working excel with the extraction
[20241227_retained_revenue_global.xlsx](https://guardhog.visualstudio.com/4148d95f-4b6d-4205-bcff-e9c8e0d2ca65/_apis/git/repositories/54ac356f-aad7-46d2-b62c-e8c5b3bb8ebf/pullRequests/3906/attachments/20241227_retained_revenue_global.xlsx)

Related work items: #25804
2024-12-31 12:10:12 +00:00
Oriol Roqué Paniagua
d26e08bffd Merged PR 3909: Fixes created_date_utc and updated_date_utc in SH User and its dependencies
# Description

The model `stg_core__superhog_user` contained the fields `created_date_utc` and `updated_date_utc`, which were actually timestamps, not dates. This created issues a while ago in KPIs that were fixed by forcing a date conversion at `int_kpis` level, but the source issue remained unfixed.

This PR just creates the proper fields in staging, meaning:
* `created_at_utc` and `created_date_utc`
* `updated_at_utc` and `updated_date_utc`

And propagates the changes through all dependencies. On KPIs, it also removes the hardcoded date conversion since it's no longer needed. It also adds the different schema entries on which these fields were already defined. Note I didn't update the staging entry since there's no entry for the rest of the fields.

I recommend checking this PR in the commits order.

What I've tested:
* Unified User and User Host have plenty of dependencies that I've checked one by one, at each dependency layer by using dbt docs.
* Run `core__unified_user` and the full execution of +`mtd_aggregated_metrics`

What clearly will fail:
* There's the famous `Users_dashboard` PBI report that shouldn't exist that has a dependency with `core__unified_user`. This report will fail. But it does not exist. I'd like to drop it once and forever

# Checklist

- [X] The edited models and dependants run properly with production data. **See above**
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.

# Other

- [X] Check if a full-refresh is required after this PR is merged. **The staging model is incremental!**

Related work items: #23703
2024-12-31 08:04:03 +00:00
Oriol Roqué Paniagua
a51ad3dd69 Merged PR 3901: Booking Service Detail improvements
# Description

Modifies Booking Service Detail table (int_core__booking_service_detail) with the following improvements:
- Considers waiver or deposit service offering as Deposit Management service business type instead of Unknown
- Includes Guest Payments (Waivers outside of Waiver PRO, Deposits) as chargeable services
- Computes total price of Waiver PRO multiplying base unit price per number of nights

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #25228
2024-12-27 10:42:47 +00:00
Joaquin Ossa
3b17fa231e Merged PR 3879: Feedback on current process
# Description

Rename changes and adding some necessary fields like `id_currency` to some S&P models

# Checklist

- [x] The edited models and dependants run properly with production data.
- [x] The edited models are sufficiently documented.
- [x] The edited models contain PK tests, and I've ran and passed them.
- [x] I have checked for DRY opportunities with other models and docs.
- [x] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Feedback on current process

Related work items: #25360
2024-12-23 09:09:13 +00:00
Joaquin Ossa
10f42d1146 Check in Hero users table 2024-12-20 15:36:00 +01:00
Joaquin Ossa
19d883ccd7 Simplify PR 2024-12-20 14:04:42 +01:00
Joaquin Ossa
79b408f3d2 update booking_fee calculation 2024-12-20 14:04:42 +01:00
Joaquin Ossa
3d2477f8f9 Feedback on current process 2024-12-20 14:04:42 +01:00
Oriol Roqué Paniagua
b20b819456 Merged PR 3878: Propagates account type to intermediate
# Description

Propagates account type to intermediate. Also, adds minimal documentation to `int_core__unified_user`.

There's only 54 test accounts in Unified User and 49 that are considered as User Hosts. So it's not, by all means, an extensive list of test accounts...

There's a mismatch between the `created_date_utc` and `updated_date_utc` that these are timestamps. There's another ticket in the backlog to handle this alongside the dependency, so it's not included here.

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #18944
2024-12-19 16:33:35 +00:00
Joaquin Ossa
0cebdcf160 new approach for S&P users data extraction 2024-12-18 11:46:54 +01:00
Joaquin Ossa
24aa9b7021 Helped with readability on test 2024-12-18 10:52:52 +01:00
Joaquin Ossa
c95f551acf s&p update models 2024-12-17 19:09:54 +01:00
Oriol Roqué Paniagua
384665c487 Merged PR 3857: Propagate Amount of Properties in HS deal
# Description

Adds the field amount_of_properties to hubspot deals, in intermediate

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [NA] I have checked for DRY opportunities with other models and docs.
- [NA] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #25351
2024-12-17 09:02:11 +00:00
Joaquin Ossa
3653ad8adf fixed number of nights in edeposit 2024-12-16 13:53:57 +01:00
Joaquin Ossa
4569c5d0da Added the has_user_moved_from_old_dash field 2024-12-13 10:56:56 +01:00
Joaquin Ossa
dbbfe2eb44 Changed description 2024-12-12 11:29:11 +01:00
Joaquin Ossa
e18750d720 Added nbr og nights to avoid calculation in PBI 2024-12-12 11:27:48 +01:00
Joaquin Ossa
85a02f8a1e Merged PR 3826: Check in Hero models
# Description

Check in Hero models for new report, it contains all records of check in hero API with user partner info

# Checklist

- [x] The edited models and dependants run properly with production data.
- [x] The edited models are sufficiently documented.
- [x] The edited models contain PK tests, and I've ran and passed them.
- [x] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Check in Hero models

Related work items: #25403
2024-12-11 16:23:12 +00:00
Joaquin Ossa
34a5b0d86e Adding missing field in S&P models 2024-12-11 16:35:40 +01:00
Joaquin Ossa
11eecfec99 addressed comments 2024-12-11 16:13:34 +01:00
Joaquin Ossa
a564afea3c wip 2024-12-11 15:42:35 +01:00
Joaquin Ossa
cbb93982d0 Check in Hero models 2024-12-11 14:32:39 +01:00
Oriol Roqué Paniagua
c0e6ec2de8 Merged PR 3825: Removing Booking fields soon to be dropped
# Description

Following a communication from Gus, the following fields in Booking table are going to be removed:
* StagingHostBookingId
* AdditionalGuests

This PR aims to remove DWH dependencies.

# Checklist

- [X] The edited models and dependants run properly with production data.
- [NA] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [NA] I have checked for DRY opportunities with other models and docs.
- [NA] I've picked the right materialization for the affected models.

# Other

- [X] Check if a full-refresh is required after this PR is merged. **Yes, Booking table in staging is incremental**

Related work items: #25405
2024-12-11 13:06:28 +00:00
Oriol Roqué Paniagua
06603d01e2 Merged PR 3810: Adds is_end_of_month_or_yesterday for Main KPIs
# Description

Adds `is_end_of_month_or_yesterday` for Main KPIs.
Apparently, the fact that we do not show the ongoing value of the month on some tabs of Main KPIs is the main blocker for Ben C to consistently use Main KPIs, which he actually retrieves from the SH legacy reporting. Since I'm sceptical about the data shown there, I want to remove this blocker.
It will require a small PR on PBI as well.

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [NA] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #25342
2024-12-09 16:13:52 +00:00
Joaquin Ossa
1f683ec651 Final clarifications 2024-12-04 16:33:50 +01:00
Joaquin Ossa
2a19f1d19a Added ProtectionType 2024-12-04 14:36:49 +01:00
Joaquin Ossa
c8ee8bd456 intermediate and reporting models 2024-12-04 11:15:21 +01:00
Oriol Roqué Paniagua
616b949ba4 Merged PR 3769: Adds int_core__ab_test_monitoring_guest_journey
# Description

Adds a table that contain the main information needed for metric computation at verification request level that is part of an A/B test.

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [NA] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #25146
2024-12-04 09:45:09 +00:00
Oriol Roqué Paniagua
37c508fa69 Merged PR 3766: First version of ab_test_tracking_guest_journey
# Description

A very simple, clean table to get the records needed for A/B test tracking on GJ side.

The table already removes weird cases that are somehow usual within A/B test configurations, to ensure that the monitoring is as clean (and less biased) as possible.

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #25145
2024-12-04 07:58:14 +00:00
Oriol Roqué Paniagua
896526fe15 Merged PR 3761: Remove no longer existing fields from Accommodation
# Description

This PR aims to properly fix the Accommodation removal of fields that have been dropped today.

# Checklist

- [X] The edited models and dependants run properly with production data.
- [NA] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [NA] I have checked for DRY opportunities with other models and docs.
- [NA] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #25199
2024-12-03 16:39:05 +00:00
Joaquin Ossa
1af03e2b42 included id_accommodation 2024-12-03 11:58:42 +01:00
Joaquin Ossa
63b3cae062 Removed null fields 2024-12-03 11:35:03 +01:00
Joaquin Ossa
4d0c45e49d updated schemas 2024-12-03 10:46:01 +01:00
Joaquin Ossa
5e17ca93a7 Included level of protection 2024-12-03 10:42:03 +01:00
Joaquin Ossa
d6f30a8e67 Added missing values 2024-12-03 10:37:05 +01:00
Joaquin Ossa
4c7820608a Merged PR 3698: added service_business_type to new_dash_created_services models
# Description

Added service_business_type to new_dash_created_services models.
Points to check on the chargeable_services models:
- I think it also needs to account for `UNKNOWN` values
- Adding the service_business_type to the unique test on the weekly and monthly metric models

```
case
    when icbsd.service_business_type = 'UNKNOWN'
    then 'UNSET'
    else coalesce(icbsd.service_business_type, 'UNSET')
end as service_business_type
```
@<Oriol Roqué Paniagua> Let me know what you think and I can make the changes if needed

# Checklist

- [x] The edited models and dependants run properly with production data.
- [x] The edited models are sufficiently documented.
- [x] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

added service_business_type to new_dash_created_services models

Related work items: #24767
2024-12-02 11:53:48 +00:00
Joaquin Ossa
7e8e403dec Updated all PK in created and chargeable services models 2024-12-02 12:42:47 +01:00
Joaquin Ossa
3015c12e18 updated schema 2024-12-02 11:14:34 +01:00
Joaquin Ossa
62893d77fc fixing the rebase 2024-12-02 11:07:34 +01:00
Joaquin Ossa
54df02cac8 Addressed comments 2024-12-02 11:06:09 +01:00
Joaquin Ossa
b03982b708 Added total fee cost of booking 2024-12-02 08:44:24 +01:00
Joaquin Ossa
edbeac253c Name change 2024-11-29 11:03:59 +01:00
Joaquin Ossa
c6a03d531a Cleaned commit 2024-11-29 10:06:15 +01:00
Joaquin Ossa
4e6690c013 API verifications model 2024-11-29 09:59:35 +01:00
Joaquin Ossa
07038e8c18 added service_business_type to new_dash_created_services models 2024-11-28 16:12:02 +01:00
Pablo Martin
ff2079a0c6 remove fields from docs 2024-11-27 14:47:43 +01:00