Commit graph

1028 commits

Author SHA1 Message Date
Pablo Martin
c3c628aec4 change "tests:" to "data_tests:" 2025-01-13 16:00:35 +01:00
Pablo Martin
b3ba2b5fa0 Merge branch 'master' of ssh.dev.azure.com:v3/guardhog/Data/data-dwh-dbt-project 2025-01-13 15:29:54 +01:00
Pablo Martin
366127489d YOLO 3 2025-01-13 15:29:24 +01:00
Pablo Martin
c545636c64 YOLO LOLO 2025-01-13 15:28:18 +01:00
Pablo Martin
583975f7d5 YOLO 2025-01-13 15:27:52 +01:00
Oriol Roqué Paniagua
33d4f18165 Merged PR 4034: BUGFIX: Ensure id_role are lower case
# Description

Ensures `id_role` are lower case. This should avoid issues downstream in dbt tests.

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #26110
2025-01-13 11:27:29 +00:00
Oriol Roqué Paniagua
3fe9ac140e Merged PR 4031: Host Resolutions KPIs bugfix
# Description

Ensure only Authorised states are selected in bank transactions for Host Resolutions KPIs.

Waiting for final confirmation from Suzannah but I'm pretty sure about this since it's consistent with the P&L (and does not make sense to consider DELETED bank transactions anyway)

For reference, here's the current values in prod for `xero_host_resolution_amount_paid_in_gbp` (top3):
* 2024-12-31	-74318
* 2024-11-30	-87935
* 2024-10-31	-73840

and here the same with this fix. Note that the difference is mostly in November:
* 2024-12-31	-73581
* 2024-11-30	-74803
* 2024-10-31	-72473

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #26247
2025-01-13 09:19:24 +00:00
Oriol Roqué Paniagua
6ace9230e9 Merged PR 4020: Expose new KPIs
# Description

Exposes 2 New KPIs on Revenue Retained. Work was already done, I just removed the temporary filter and adapted the name of the metric.

# Checklist

- [ ] The edited models and dependants run properly with production data.
- [ ] The edited models are sufficiently documented.
- [ ] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Expose new KPIs

Related work items: #25804
2025-01-10 10:07:00 +00:00
Joaquin Ossa
13bb35ec70 PMS data to reporting 2025-01-10 09:51:29 +01:00
Joaquin Ossa
a0ee26e018 pms data for Account Managers Overview 2025-01-10 09:17:29 +01:00
Joaquin Ossa
0c311d02e8 Update name of fields 2025-01-09 16:31:22 +01:00
Joaquin Ossa
bd25b473fb Adding PMS info for Account Margin report 2025-01-09 15:56:58 +01:00
Joaquin Ossa
04a10cf9c5 Merged PR 3968: Updating exposures
# Description

Added the missing link to exposures

# Checklist

- [ ] The edited models and dependants run properly with production data.
- [ ] The edited models are sufficiently documented.
- [ ] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Updating exposures

Related work items: #25827
2025-01-07 15:27:47 +00:00
Oriol Roqué Paniagua
0ec1484846 Merged PR 3967: Adds an "All History" time window to Account Margin
# Description

Quick improvement to be able to report the same setup of metrics by considering all account history up to a certain date.

It adds a new computation flow for this All History window. Note that I needed to update the macro to override this case by using unbounded preceding.

I also took the opportunity to update the exposures of the new report.

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #26085
2025-01-07 15:22:30 +00:00
Joaquin Ossa
9784cb6022 Updating exposures 2025-01-07 16:18:45 +01:00
Joaquin Ossa
55539ab2c9 ordered names to select oldest one created 2025-01-07 11:47:14 +01:00
Oriol Roqué Paniagua
f2abd16a44 Merged PR 3939: Fixes DWH models for new PBI on AM
# Description

Mostly 2 changes:
* Fix metrics with coalesces so values are displayed
* Removes ratios, no longer needed. These are computed in PBI directly, to be able to compute the total figure correctly

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #25829
2025-01-03 16:17:26 +00:00
Joaquin Ossa
36290f19b3 deals consolidation to reporting 2025-01-03 14:59:47 +01:00
Joaquin Ossa
dbf807639e fixing copy paste 2025-01-03 12:00:14 +01:00
Joaquin Ossa
0aeb558a9b fixed booleans 2025-01-03 11:43:58 +01:00
Joaquin Ossa
6516a0fdc6 Updated schema with rebase 2025-01-03 11:29:58 +01:00
Joaquin Ossa
58dc823d27 Added boolean fields and counts per each deal source 2025-01-03 11:26:55 +01:00
Joaquin Ossa
35e7144efd Changed the logic for the model 2025-01-03 11:26:14 +01:00
Joaquin Ossa
84f2156c99 commit wip 2025-01-03 11:24:10 +01:00
Oriol Roqué Paniagua
e406bc89ad Merged PR 3926: Propagate to reporting + fix documentation
# Description

* Creates new model monthly_aggregated_metrics_history_by_deal_by_time_window
* Modifies existing intermediate entry on schema to properly fill all fields
* Creates same entry in reporting.

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #25829
2025-01-03 09:30:33 +00:00
Oriol Roqué Paniagua
564a4f307d Merged PR 3925: Bugfix
# Description

Fixes issues on reporting after name changes

# Checklist

- [ ] The edited models and dependants run properly with production data.
- [ ] The edited models are sufficiently documented.
- [ ] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Bugfix
2025-01-03 08:24:16 +00:00
Oriol Roqué Paniagua
a61da137fa Merged PR 3924: First version of time window aggregated metrics by deal
# Description

This model aggregates at monthly by deal level different metrics for AM reporting purposes. It also includes revenue retained ratios for client profitability assessment.

There's part of the existing AM report that could be simplified, likely, by using the new macro. This will be explored in a separated PR, if it applies.

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs. ** Checked and there might be possibilities to simplify the code. I'll check ones I finish this line of work**
- [X] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #25829
2025-01-03 08:07:16 +00:00
Oriol Roqué Paniagua
0d7b5ac88a Merged PR 3914: Include Hubspot deals for KPIs
# Description

Context: I'm intending to work on Account Managers reporting, that mostly will include reporting at Deal level the Resolutions Payouts as well as the new Retained metrics. While checking the great increase on Resolutions Payouts for October 2024:

![image (2).png](https://guardhog.visualstudio.com/4148d95f-4b6d-4205-bcff-e9c8e0d2ca65/_apis/git/repositories/54ac356f-aad7-46d2-b62c-e8c5b3bb8ebf/pullRequests/3914/attachments/image%20%282%29.png)

I decided to take a quick look into the main players... and surprise surprise we have Guesty:

![image.png](https://guardhog.visualstudio.com/4148d95f-4b6d-4205-bcff-e9c8e0d2ca65/_apis/git/repositories/54ac356f-aad7-46d2-b62c-e8c5b3bb8ebf/pullRequests/3914/attachments/image.png)

So Guesty represents 37k over the 73K of October. 50%. Not bad.

The main issue is that we've been aware for months now (since Churn efforts, mostly) that we're not reporting in KPIs those deals that are NOT created in Core. Most notably, API deals which includes... well, Guesty. So creating this kind of in-depth Account Managers improvement without reporting Guesty I think it would be very misleading. Note that the overall figures (Global dimension) are still accurate, though.

What's new:
* A new model named `int_kpis__dimension_deals` that basically retrieves Deals from both Core (as before) and Hubspot. It combines information from both and mostly assumes Hubspot as a better source of information than Core - although we do not have the Main Billing Country there afaik.
* Propagates changes, mostly in the monthly by deal view of Main KPIs. Here I confirm that now Guesty appears, and it only has metrics that come from Xero (APIs Revenue, Total Revenue, Resolutions, etc)

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #25829
2024-12-31 15:12:38 +00:00
Oriol Roqué Paniagua
29a005297a Merged PR 3906: Adds 2 new retained income metrics
# Description

It adds 2 new metrics in Main KPIs, both for Global/per Dimension and By Deal. If merged, metrics will NOT appear automatically in the Global/per Dimension nor in the By Deal.

These two new metrics are:
* Income Retained: Total Revenue - Waiver Paid Back to Host
* Income Retained Post Resolutions: Total Revenue - Waiver Paid Back to Host - Host Resolutions Amount Paid

Why these are important:

![image.png](https://guardhog.visualstudio.com/4148d95f-4b6d-4205-bcff-e9c8e0d2ca65/_apis/git/repositories/54ac356f-aad7-46d2-b62c-e8c5b3bb8ebf/pullRequests/3906/attachments/image.png)

Even though we grow considerably in terms of Revenue, the gap of Waivers that we pay back to host is also increasing. Thus the "real" increment is actually lower. However, what I find more interesting is the heavy decrease in Income Retained Post Resolutions. Here's the Year-by-Year comparison:

![image (2).png](https://guardhog.visualstudio.com/4148d95f-4b6d-4205-bcff-e9c8e0d2ca65/_apis/git/repositories/54ac356f-aad7-46d2-b62c-e8c5b3bb8ebf/pullRequests/3906/attachments/image%20%282%29.png)

In November 2024, we're back to the figures of 2023 (-0.4%) and this should be alarming considering we're growing in Total Revenue by 27% and in Retained Income by 41% vs. Nov 23. In terms of business impact, I'd opt for having these 2 metrics computed as these capture a reality that otherwise we keep hidden. Happy to discuss renames and how/if we move forward with this.

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Attached working excel with the extraction
[20241227_retained_revenue_global.xlsx](https://guardhog.visualstudio.com/4148d95f-4b6d-4205-bcff-e9c8e0d2ca65/_apis/git/repositories/54ac356f-aad7-46d2-b62c-e8c5b3bb8ebf/pullRequests/3906/attachments/20241227_retained_revenue_global.xlsx)

Related work items: #25804
2024-12-31 12:10:12 +00:00
Oriol Roqué Paniagua
d26e08bffd Merged PR 3909: Fixes created_date_utc and updated_date_utc in SH User and its dependencies
# Description

The model `stg_core__superhog_user` contained the fields `created_date_utc` and `updated_date_utc`, which were actually timestamps, not dates. This created issues a while ago in KPIs that were fixed by forcing a date conversion at `int_kpis` level, but the source issue remained unfixed.

This PR just creates the proper fields in staging, meaning:
* `created_at_utc` and `created_date_utc`
* `updated_at_utc` and `updated_date_utc`

And propagates the changes through all dependencies. On KPIs, it also removes the hardcoded date conversion since it's no longer needed. It also adds the different schema entries on which these fields were already defined. Note I didn't update the staging entry since there's no entry for the rest of the fields.

I recommend checking this PR in the commits order.

What I've tested:
* Unified User and User Host have plenty of dependencies that I've checked one by one, at each dependency layer by using dbt docs.
* Run `core__unified_user` and the full execution of +`mtd_aggregated_metrics`

What clearly will fail:
* There's the famous `Users_dashboard` PBI report that shouldn't exist that has a dependency with `core__unified_user`. This report will fail. But it does not exist. I'd like to drop it once and forever

# Checklist

- [X] The edited models and dependants run properly with production data. **See above**
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.

# Other

- [X] Check if a full-refresh is required after this PR is merged. **The staging model is incremental!**

Related work items: #23703
2024-12-31 08:04:03 +00:00
Oriol Roqué Paniagua
803c5ed3d5 Merged PR 3905: Update exposures
# Description

Update exposures of New Dash PBI after the inclusion of the new tab of Booking Detail

# Checklist

- [ ] The edited models and dependants run properly with production data.
- [ ] The edited models are sufficiently documented.
- [ ] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #25228
2024-12-27 15:49:03 +00:00
Oriol Roqué Paniagua
fda6c06222 Merged PR 3902: Exposes New Dash Booking Summary
# Description

Exposes Booking Summary to reporting, but only for New Dash users.

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #25228
2024-12-27 12:22:19 +00:00
Oriol Roqué Paniagua
a51ad3dd69 Merged PR 3901: Booking Service Detail improvements
# Description

Modifies Booking Service Detail table (int_core__booking_service_detail) with the following improvements:
- Considers waiver or deposit service offering as Deposit Management service business type instead of Unknown
- Includes Guest Payments (Waivers outside of Waiver PRO, Deposits) as chargeable services
- Computes total price of Waiver PRO multiplying base unit price per number of nights

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #25228
2024-12-27 10:42:47 +00:00
Pablo Martin
5fd69b7101 Merge branch 'master' of ssh.dev.azure.com:v3/guardhog/Data/data-dwh-dbt-project 2024-12-23 12:19:55 +01:00
Pablo Martin
b7541b9a2a add grant for billingreader 2024-12-23 12:19:52 +01:00
Joaquin Ossa
3b17fa231e Merged PR 3879: Feedback on current process
# Description

Rename changes and adding some necessary fields like `id_currency` to some S&P models

# Checklist

- [x] The edited models and dependants run properly with production data.
- [x] The edited models are sufficiently documented.
- [x] The edited models contain PK tests, and I've ran and passed them.
- [x] I have checked for DRY opportunities with other models and docs.
- [x] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Feedback on current process

Related work items: #25360
2024-12-23 09:09:13 +00:00
Joaquin Ossa
10f42d1146 Check in Hero users table 2024-12-20 15:36:00 +01:00
Joaquin Ossa
19d883ccd7 Simplify PR 2024-12-20 14:04:42 +01:00
Joaquin Ossa
79b408f3d2 update booking_fee calculation 2024-12-20 14:04:42 +01:00
Joaquin Ossa
3d2477f8f9 Feedback on current process 2024-12-20 14:04:42 +01:00
Joaquin Ossa
2113430e26 Fix naming error 2024-12-20 08:37:31 +01:00
Oriol Roqué Paniagua
b20b819456 Merged PR 3878: Propagates account type to intermediate
# Description

Propagates account type to intermediate. Also, adds minimal documentation to `int_core__unified_user`.

There's only 54 test accounts in Unified User and 49 that are considered as User Hosts. So it's not, by all means, an extensive list of test accounts...

There's a mismatch between the `created_date_utc` and `updated_date_utc` that these are timestamps. There's another ticket in the backlog to handle this alongside the dependency, so it's not included here.

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #18944
2024-12-19 16:33:35 +00:00
Oriol Roqué Paniagua
7b28bbb925 Merged PR 3875: Propagates AccountType to staging
# Description

Propagates `AccountType` to staging by creating the first version of the model `stg_core__account_type`

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #18944
2024-12-19 14:23:47 +00:00
Pablo Martín
14fd52bb36 Merged PR 3871: Add FX rates integration exposure.
# Description

Adds the fx rates integration as an exposure since it's a downstream dependency just like a dashboard would be.

# Checklist

NA

# Other

NA

Related work items: #25608
2024-12-19 11:32:16 +00:00
Joaquin Ossa
6fd9c16e59 Merged PR 3872: Staging model for all screen & protect price tables
# Description

Staging model for all screen & protect price tables.
There are 4 types of protections for screen & protect and each of them have their own prices

# Checklist

- [x] The edited models and dependants run properly with production data.
- [x] The edited models are sufficiently documented.
- [x] The edited models contain PK tests, and I've ran and passed them.
- [x] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Staging model for all screen & protect price tables

Related work items: #25360
2024-12-19 10:24:10 +00:00
Joaquin Ossa
2a87e4860c Added _in_local_currency to fees namings 2024-12-19 11:13:53 +01:00
Joaquin Ossa
5a92337b43 Clarification of long stay 2024-12-18 14:52:46 +01:00
Joaquin Ossa
463226a44b Staging model for all screen & protect price tables 2024-12-18 14:48:36 +01:00
Pablo Martin
4dbecae0ba add exposure 2024-12-18 14:39:23 +01:00
Joaquin Ossa
0cebdcf160 new approach for S&P users data extraction 2024-12-18 11:46:54 +01:00