Commit graph

1568 commits

Author SHA1 Message Date
Pablo Martin
8c7681581c typo 2025-03-18 17:35:07 +01:00
Pablo Martin
32ef6f36a4 create database 2025-03-18 17:33:49 +01:00
Pablo Martin
65e2dea2fc smoke test with env var sourcing included 2025-03-18 17:31:05 +01:00
Pablo Martin
74222a0007 smoke test postgres connection 2025-03-18 17:26:20 +01:00
Pablo Martin
6dea088b8f Merge branch 'master' of ssh.dev.azure.com:v3/guardhog/Data/data-dwh-dbt-project 2025-03-18 17:24:01 +01:00
Pablo Martin
c348430dae prepare env vars on ci script 2025-03-18 17:23:57 +01:00
Oriol Roqué Paniagua
06694faa68 Merged PR 4732: Removes accepted values test for guest product display name
# Description

It's a bit overkill to add an accepted tests in the display name, knowing that it can change. Note that I didn't remove it for the internal name.

This fixes the dbt test error.

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Removes accepted values test for guest product display name

Related work items: #28513
2025-03-18 16:03:49 +00:00
uri
09976bef13 Removes accepted values test for guest product display name 2025-03-18 16:34:00 +01:00
Pablo Martin
a988542679 simple smoke test 2025-03-18 16:16:04 +01:00
Pablo Martin
619c87e874 Merge branch 'master' of ssh.dev.azure.com:v3/guardhog/Data/data-dwh-dbt-project 2025-03-18 15:55:48 +01:00
Pablo Martin
525ed77a87 change pool name again cause i fucked up 2025-03-18 15:55:45 +01:00
Oriol Roqué Paniagua
f799a8d30f Merged PR 4727: Bookings fees are now Old Dashboard Booking Fees in Main KPIs
# Description

Booking fees is widely used with different meanings, for old dash, for new dash, for both, etc. This is painful. First step to align on a proper naming is ensure that what we report in Main KPIs is clearly stated, which in this case, Booking Fees are now called Old Dashboard Booking Fees.

Changes:
* Modify `stg_seed__accounting_aggregations` seed to rename Booking Fees to Old Dashboard Booking Fees. This is for us to clarify. This is only applied for KPIs compute. I also added an empty space that I mistakenly forgot in the past for `financial_l3_aggregation`.
* Modify KPIs source, i.e., `int_kpis__metric_daily_invoiced_revenue`. Here I forcefully modify the name of the field to `xero_old_dashboard_booking_net_fees_in_gbp`.
* Propagate changes of downstream usages of `xero_booking_net_fees_in_gbp` to `xero_old_dashboard_booking_net_fees_in_gbp`. This affects all models, including the reporting model. On this one we still have both names to avoid breaking it. I will need to modify the data glossary in PBI anyway so I'll do this change as well.
* Modify displayed metric name from Booking Fees Revenue to Old Dashboard Booking Fees Revenue.
* Modify schema so it reflects the proper names, descriptions, and tests.
* Ensure outlier and completion tests still work after this change.

I confirm the field `xero_booking_net_fees_in_gbp` does not exist anymore in the rest of DWH after these changes, except for the abovementioned comment on the reporting line.

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #28560
2025-03-18 14:55:32 +00:00
Pablo Martin
35cd4f4fcd change pool 2025-03-18 15:54:46 +01:00
Pablo Martin
c786ccf5c1 remove compile 2025-03-18 13:44:30 +01:00
Pablo Martin
e424ed7581 add ci yml 2025-03-18 13:41:40 +01:00
Oriol Roqué Paniagua
dd9bb8a11b Merged PR 4713: Adds a master table for guest products price plans
# Description

Adds a master table for pricing regarding Guest Products in intermediate.

I also added in staging a missing description for a field.

This is a necessary step to work out the prices on guest products payments, which will come in a separated PR.

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models. **Left it as a view, let's adapt in the future if needed**

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #28513
2025-03-18 10:56:33 +00:00
uri
74a76fe8fc Removes configuration status and adds config id for further joins 2025-03-18 09:20:31 +01:00
uri
91e7fe09b1 Adds end date 2025-03-18 09:09:59 +01:00
uri
cba4b97082 Adds a master table for guest products price plans 2025-03-17 16:35:50 +01:00
Joaquin Ossa
b518c536f2 Merged PR 4698: Added business scope to models
# Description

Added business scope to models `monthly_aggregated_metrics_history_by_deal`

# Checklist

- [x] The edited models and dependants run properly with production data.
- [x] The edited models are sufficiently documented.
- [x] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Added business scope to models

Related work items: #28501
2025-03-14 13:33:10 +00:00
Joaquin
e9cfb6c2bd Addressed comments 2025-03-14 14:20:57 +01:00
Oriol Roqué Paniagua
918ed969d8 Merged PR 4699: Creates a master table for guest products
# Description

Creates a master table for guest products in intermediate. It just gathers information on the name and the latest display name, as well as a couple of additional time fields.

I opted for "latest" rather than "current" because technically Guest Products can be enabled and disabled, and I believe current name would be weird if the product is disabled. Anyway. Next step will be actually creating a master table on guest product configuration, that will handle this disable logic and so on.

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #28513
2025-03-14 13:04:46 +00:00
Joaquin
89b3989d19 Added business scope to models 2025-03-14 11:48:22 +01:00
Joaquin Ossa
72118389d6 Merged PR 4693: Live Dates according to HubSpot
# Description

Made the change as discussed during the daily.
Here we can see an example of the impact with the change (left old lifecycle, right new one)
The biggest impact comes from the Never Booked deals, 110, and Active 34.
![image.png](https://guardhog.visualstudio.com/4148d95f-4b6d-4205-bcff-e9c8e0d2ca65/_apis/git/repositories/54ac356f-aad7-46d2-b62c-e8c5b3bb8ebf/pullRequests/4693/attachments/image.png)

# Checklist

- [x] The edited models and dependants run properly with production data.
- [x] The edited models are sufficiently documented.
- [x] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #28475
2025-03-14 10:20:34 +00:00
Joaquin
c33253b10a addressed comments 2025-03-14 11:19:22 +01:00
Joaquin
525720ea0a Changes in deals lifecycle 2025-03-14 10:39:01 +01:00
Joaquin Ossa
8c97f03487 Merged PR 4697: Churn update
# Description

Added churn details from hubspot (free text detailing churn reason)

# Checklist

- [x] The edited models and dependants run properly with production data.
- [x] The edited models are sufficiently documented.
- [x] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Churn update

Related work items: #28501
2025-03-14 09:33:19 +00:00
Joaquin
e0d18acf8e Churn update 2025-03-14 10:12:54 +01:00
Joaquin
e827c0f9d5 Updated schema 2025-03-13 17:31:15 +01:00
Joaquin
a2b3b7fa6b Updated schema 2025-03-13 17:29:24 +01:00
Joaquin
d6426b0066 change in new deals 2025-03-13 17:19:34 +01:00
Joaquin
1ced79ca0a commit wip 2025-03-13 15:48:15 +01:00
Oriol Roqué Paniagua
bf1e125856 Merged PR 4687: Bugfix - Removal of SH fields that no longer exist
# Description

After the sync this morning in Core in Airbyte, we have some issues in SH User. This attempts to remove no longer existing fields, namely:
- id_airbnb
- airbnb_url
- platform_comms_recipient

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #28502
2025-03-13 12:25:13 +00:00
Oriol Roqué Paniagua
eb26252784 Merged PR 4686: Guest Products - Sync Guest Products
# Description

Adds the remaining Guest Products tables to staging

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #28490
2025-03-13 11:53:04 +00:00
Oriol Roqué Paniagua
7ef460a59e Merged PR 4685: Guest Products - sync core tables
# Description

Adds `VerificationRequestGuestProductToPayment` and `VerificationRequestToGuestProduct` to staging.

Note these tables are empty thus it's possible that some tests might need to be adapted in the future.

I also already added the configuration for sync_guest_product sources.

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Guest Products - sync core tables

Related work items: #28490
2025-03-13 09:27:02 +00:00
Oriol Roqué Paniagua
9ca9c827fe Merged PR 4665: Projected Bookings to the end of the current month
# Description

Computes projected Bookings to the end of the current month per dimension and dimension value

KPIs changes:
* Adds a int_kpis__agg_daily_created_bookings model. This is a standard KPIs model that computes created bookings at a daily level per dimension and dimension value.

Projected KPIs (NEW):
* Adds a new folder named projected within KPIs. This includes a simple Readme.
* Adds a default dimension dates model that ranges from the past 3 complete months to the end of the current month, that is in the future.
* Adds 2 projected models for created bookings: The daily one, that handles the logic of the projection (same month + last 7 days) and the monthly one, that retrieves the end of month projected value as well as some indicators of how good or bad this was for previous months.

How good is the projection at 11th of March (so 10 days in the month)?

`select * from intermediate.int_kpis_projected__agg_monthly_created_bookings order by actual_created_bookings desc`

![image.png](https://guardhog.visualstudio.com/4148d95f-4b6d-4205-bcff-e9c8e0d2ca65/_apis/git/repositories/54ac356f-aad7-46d2-b62c-e8c5b3bb8ebf/pullRequests/4665/attachments/image.png)

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #28251
2025-03-11 14:34:30 +00:00
Joaquin Ossa
a0b2f9b657 Merged PR 4649: Updated exposures
# Description

_Describe your motivation and changes here._

# Checklist

- [x] The edited models and dependants run properly with production data.
- [ ] The edited models are sufficiently documented.
- [ ] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Updated exposures

Related work items: #28190
2025-03-10 10:22:47 +00:00
Joaquin Ossa
459b1b3833 Updated exposures 2025-03-10 10:57:21 +01:00
Joaquin Ossa
cc9b4ade71 Merged PR 4628: Churn deals
# Description

New model for Churn report.
It contains all deals that are churned/cancelled according to HubSpot on a monthly basis, it includes revenue, bookings created (by time windows) and number of listings per deal

# Checklist

- [x] The edited models and dependants run properly with production data.
- [x] The edited models are sufficiently documented.
- [x] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #28190
2025-03-07 10:48:52 +00:00
Joaquin Ossa
f8362c4a71 addressed comments 2025-03-07 11:32:09 +01:00
Joaquin Ossa
2d4cded48c Addressed comments and included changes in reporting 2025-03-07 11:13:03 +01:00
Joaquin Ossa
2461286c9c Updated model 2025-03-07 10:17:11 +01:00
Joaquin Ossa
0536cfb0e2 Fixing formatting 2025-03-06 17:19:15 +01:00
Joaquin Ossa
f8b551a75c Updated existing model with some new fields for churning report 2025-03-06 17:12:32 +01:00
Joaquin Ossa
24d13185eb Added year and month 2025-03-06 15:02:44 +01:00
Joaquin Ossa
1245a88867 New churned deals model 2025-03-06 14:57:23 +01:00
Joaquin Ossa
089a1c37bd commit wip 2025-03-06 11:02:46 +01:00
Joaquin Ossa
9f580867c0 Merged PR 4622: New Dash bug fix
# Description

Filters by `bs.is_user_in_new_dash = true and bs.is_missing_id_deal = false` for services in bookings

# Checklist

- [x] The edited models and dependants run properly with production data.
- [x] The edited models are sufficiently documented.
- [x] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

New Dash bug fix

Related work items: #28194
2025-03-06 08:23:27 +00:00
Joaquin Ossa
d065d4beb6 New Dash bug fix 2025-03-06 08:16:40 +01:00
Oriol Roqué Paniagua
39fc6a7d13 Merged PR 4618: Bugfix - Ensure the PMS is active
# Description

Very silly mistake - affects many reports!

@<Joaquin Ossa> let me know if there's other use cases. I believe that's all of the computations, but you have this more fresh than me.

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #28196
2025-03-05 14:57:57 +00:00