Commit graph

1168 commits

Author SHA1 Message Date
Joaquin Ossa
edd588b3b4 delete b lol 2024-11-12 14:48:09 +01:00
Joaquin Ossa
7e5451604c Guest KPIs reporting 2024-11-12 14:41:01 +01:00
Joaquin Ossa
b2e9a3e0da Merged PR 3476: daily model for guest products
# Description

Replacement PR from the abandondes one for daily metrics for guest KPI model

# Checklist

- [x] The edited models and dependants run properly with production data.
- [x] The edited models are sufficiently documented.
- [x] The edited models contain PK tests, and I've ran and passed them.
- [x] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

daily model for guest products

Related work items: #23371
2024-11-12 07:50:45 +00:00
Oriol Roqué Paniagua
aaba27a0ea Merged PR 3500: Fix missing test
# Description

Fix missing test :)

# Checklist

- [ ] The edited models and dependants run properly with production data.
- [ ] The edited models are sufficiently documented.
- [ ] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #23763
2024-11-11 16:37:58 +00:00
Oriol Roqué Paniagua
9ba0edb82d Merged PR 3482: Adapts date dimension skeleton for Main KPIs
# Description

New model:
* int_kpis__agg_dates_main_kpis - Serves as the skeleton of dates and dimensions for Main KPIs. It's aggregated since it follows a similar aggregation strategy. It's a single model to feed both Main KPIs visualisations. Note boolean fields are real booleans (true/false) while before these were integers (1/0). This also affects downstream models.

Main KPIs flow adaptations to new skeleton model:
* int_monthly_aggregated_metrics_history_by_deal
* int_monthly_churn_metrics - additionally, calls usual KPIs macro instead of old one
* int_mtd_vs_previous_year_metrics

Reporting changes to ensure report is not down:
* mtd_aggregated_metrics - adaptations on booleans (true-1, false-0)

Cleaning:
* get_kpi_dimensions macro is no longer used
* int_dates_by_deal model and schema entry
* int_dates_mtd_by_dimension model and schema entry
* int_dates_mtd model and schema entry

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #23763
2024-11-11 15:57:37 +00:00
Joaquin Ossa
88083fc846 Addressed comments 2024-11-11 16:22:13 +01:00
Joaquin Ossa
eec21fe5da Commit saved model 2024-11-09 15:21:20 +01:00
Joaquin Ossa
98ba74481a daily model for guest products 2024-11-08 15:25:11 +01:00
Oriol Roqué Paniagua
2f80642f6c Merged PR 3472: Remove unused models and schema entries for deals and accommodations
# Description

Eliminates models that have been switched with new kpis flow.
Also deletes temporary tests and schema entries.

# Checklist

- [X] The edited models and dependants run properly with production data.
- [NA] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [NA] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #23762
2024-11-08 12:05:40 +00:00
Oriol Roqué Paniagua
5e1b418570 Merged PR 3469: Switches Listing and Deal metrics
# Description

Switches Listings and Deal metrics to production.
I also modified the dependencies on the old lifecycles to read from the news, so we can remove these in a following PR.

I'm considering re-computing also the dimension - specific models to operate within KPIs folder, but will do this later on in a dedicated PR.

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #23761
2024-11-08 11:11:04 +00:00
Oriol Roqué Paniagua
335ba9a29b Merged PR 3461: Creates models for listing metrics
# Description

Adds listing metrics model. It's a bit similar as the Deal metrics, but here we can aggregate by deal :)

I'm aware that int_kpis__metric_daily_listings takes quite a bit (3 min). To be optimised later on.

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #23567
2024-11-07 16:56:43 +00:00
Joaquin Ossa
dada9c289a Merged PR 3454: Guest payments models for Guest KPIs
# Description

Weekly, monthly and mtd agg for the guest payment models for guest KPIs

# Checklist

- [x] The edited models and dependants run properly with production data.
- [x] The edited models are sufficiently documented.
- [x] The edited models contain PK tests, and I've ran and passed them.
- [x] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #23371
2024-11-07 12:10:14 +00:00
Joaquin Ossa
34ccd894de renamed test entity 2024-11-07 12:56:49 +01:00
Oriol Roqué Paniagua
38c7cc10df Merged PR 3455: Cleans old Xero KPIs models
# Description

Cleans Xero models for the old KPIs flow, temporary tests and schema entries

# Checklist

- [X] The edited models and dependants run properly with production data.
- [NA] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [NA] I have checked for DRY opportunities with other models and docs.
- [NA] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #23760
2024-11-07 11:48:15 +00:00
Joaquin Ossa
11e1804db0 mtd and weekly agg guest payments 2024-11-07 12:15:52 +01:00
Joaquin Ossa
c89c72aace commit agg_monthly_guest_payments 2024-11-07 12:09:20 +01:00
Joaquin Ossa
50b09c56a1 wip agg 2024-11-07 12:06:37 +01:00
Joaquin Ossa
dba28897d6 wip agg 2024-11-07 12:06:37 +01:00
Joaquin Ossa
cf553cfcb1 monthly and weekly metric models 2024-11-07 12:06:37 +01:00
Oriol Roqué Paniagua
8c23f91242 Merged PR 3451: Adds Deal Daily Lifecycle and metrics
# Description

Changes:
* Creates lifecycle_daily_deal, metric_daily_deals and agg_daily_deals. These follow a different strategy due to the nature of the metrics
* Modifies the dimension macro to ensure deal dimension is included in all models except these ones
* Fixes production issue on currently deployed deal lifecycle and metrics

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #23566
2024-11-07 10:49:06 +00:00
Oriol Roqué Paniagua
9ef9a57c03 Merged PR 3443: Switch of Xero metrics New KPIs flow - Invoiced Revenue and Host Resolutions
# Description

Switches Xero metrics to the new KPIs flow. This uses the 2 new entities around Invoiced Revenue and Host Resolutions.

The metrics affected will change computation source in PBI as well (for MTD per category and Monthly by deal views). This directly affects the following metrics:
* Invoiced Operator Revenue
* Invoiced Booking Fees Revenue
* Invoiced Listing Fees Revenue
* Invoiced Verification Fees Revenue
* Invoiced APIs Revenue
* Invoiced Athena Revenue
* Invoiced E-Deposit Revenue
* Damage Host-Waiver Payments
* Host Resolutions Amount Paid
* Host Resolutions Payment Count

Additionally, the following metrics will be indirectly affected since depend partially on Xero:
* Total Revenue
* 4x Total Revenue per (Booking Created, Guest Journey Created, Deals Booked in Month, Listings Booked in Month)
* Waiver Retained

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [NA] I have checked for DRY opportunities with other models and docs. **Same as last time - the way models are called could be improved, but let's switch everything first and then discuss. There's chances these cross models will die anyway**
- [X] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #23759
2024-11-07 07:48:38 +00:00
Joaquin Ossa
dc4f5098a1 Merged PR 3440: New metrics for payments from guest journeys
# Description

Added new metrics to the int_kpis__metric_daily_guest_payments model for all payments associated to bookings that have not been cancelled.
This have been created for the Guest KPIs report

# Checklist

- [x] The edited models and dependants run properly with production data.
- [x] The edited models are sufficiently documented.
- [x] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #23371
2024-11-06 16:28:55 +00:00
Joaquin Ossa
678caa35d7 Removed unnecessary join 2024-11-06 16:27:19 +01:00
Joaquin Ossa
f33bc3a2ca removed new metrics 2024-11-06 16:24:57 +01:00
Joaquin Ossa
b9968c7a72 added new metrics for not cancelled bookings 2024-11-06 12:34:52 +01:00
Joaquin Ossa
f5addc7f4c wip 2024-11-06 12:34:51 +01:00
Joaquin Ossa
9fecfeb91b Merged PR 3439: Weekly aggregated model
# Description

Created new PR to fix schema for test

# Checklist

- [x] The edited models and dependants run properly with production data.
- [x] The edited models are sufficiently documented.
- [x] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #23373
2024-11-06 11:11:07 +00:00
Oriol Roqué Paniagua
fd94c19694 Merged PR 3433: Change lifecycle daily to just handle last available day
# Description

Should improve performance of this daily lc model

# Checklist

- [X] The edited models and dependants run properly with production data.
- [ ] The edited models are sufficiently documented.
- [ ] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.
2024-11-06 11:10:31 +00:00
Joaquin Ossa
41a9b55436 saved 2024-11-06 11:48:38 +01:00
Joaquin Ossa
51fabf9efb Changed test 2024-11-06 11:47:47 +01:00
Joaquin Ossa
8c7b4bcf94 Merged PR 3437: Revert 'Weekly aggregated guest journey metrics'
# Description

_Describe your motivation and changes here._

# Checklist

- [ ] The edited models and dependants run properly with production data.
- [ ] The edited models are sufficiently documented.
- [ ] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

# Description

Here I created a weekly aggregated model for guest journey metrics.
I changed the daily_dimension week start and end to iso so it matches the week date obtained by Postgres for this aggregation

# Checklist

- [x] The edited models and dependants run properly with production data.
- [x] The edited models are sufficiently documented.
- [x] The edited models contain PK tests, and I've ran and passed them.
- [x] I have checked for DRY opportunities with other models and docs.
- [x] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Reverts !3430

Related work items: #23373
2024-11-06 10:27:37 +00:00
Joaquin Ossa
50b87c9d8f Revert "Merged PR 3430: Weekly aggregated guest journey metrics
# Description

Here I created a weekly aggregated model for guest journey metrics.
I changed the daily_dimension week start and end to iso so it matches the week date obtained by Postgres for this aggregation

# Checklist

- [x] The edited models and dependants run properly with production data.
- [x] The edited models are sufficiently documented.
- [x] The edited models contain PK tests, and I've ran and passed them.
- [x] I have checked for DRY opportunities with other models and docs.
- [x] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #23373"
2024-11-06 10:25:04 +00:00
Joaquin Ossa
60204b53e8 Merged PR 3430: Weekly aggregated guest journey metrics
# Description

Here I created a weekly aggregated model for guest journey metrics.
I changed the daily_dimension week start and end to iso so it matches the week date obtained by Postgres for this aggregation

# Checklist

- [x] The edited models and dependants run properly with production data.
- [x] The edited models are sufficiently documented.
- [x] The edited models contain PK tests, and I've ran and passed them.
- [x] I have checked for DRY opportunities with other models and docs.
- [x] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #23373
2024-11-06 10:17:46 +00:00
Joaquin Ossa
4a2c478d8f update schema 2024-11-06 11:11:06 +01:00
Joaquin Ossa
1cd9f88a45 Merged PR 3434: update assert_dimension_completeness test
# Description

Update assert_dimension_completeness test
I tested the models and everything seems to be working correctly
I tried to modify the test as you told me but I couldn't make it work Uri
`I think this could work with fewer repeated lines as well:
select * from discrepancy_{{ metric_column_name }}
{% if not loop.first %}
union all
{% endif %}`

# Checklist

- [x] The edited models and dependants run properly with production data.
- [x] The edited models are sufficiently documented.
- [x] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

update assert_dimension_completeness test
2024-11-06 07:59:44 +00:00
Joaquin
406499a7e0 Merge branch 'models/kpis_test_update' of ssh.dev.azure.com:v3/guardhog/Data/data-dwh-dbt-project into models/kpis_test_update 2024-11-06 08:35:50 +01:00
Joaquin
079eaac870 Update invoice models 2024-11-06 08:35:30 +01:00
Joaquin
ca510f91a7 update assert_dimension_completeness test 2024-11-06 08:29:02 +01:00
Oriol Roqué Paniagua
f5311fa954 Merged PR 3431: Adds Invoicing metrics
# Description

Same old story: includes all metrics coming from Xero.
Adds a daily model, monthly, mtd + monthly agg and mtd agg
A test to compare values new vs. old
AND fixes the issue I mentioned with the timestamp - an issue in the old KPIs.

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [NA] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #23565
2024-11-05 16:57:23 +00:00
Joaquin
0bffa5a862 update assert_dimension_completeness test 2024-11-05 16:45:59 +01:00
Joaquin
7e786649cc Weekly aggregated model 2024-11-05 16:04:41 +01:00
Joaquin
d8a064fd06 commit wip 2024-11-05 15:50:23 +01:00
Joaquin Ossa
4076f016bd Merged PR 3424: aggregated to agg
# Description

Changed names for all aggregated models to agg, updated all references and schemas.
I did a run and test and everything worked correctly

# Checklist

- [x] The edited models and dependants run properly with production data.
- [ ] The edited models are sufficiently documented.
- [x] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

aggregated to agg
2024-11-05 13:46:45 +00:00
Joaquin Ossa
76529fe457 Merged PR 3427: Weekly dimension
# Description

Added week values to the kpis__dimension_dates model for weekly aggregations

# Checklist

- [x] The edited models and dependants run properly with production data.
- [x] The edited models are sufficiently documented.
- [x] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Weekly dimension
2024-11-05 13:46:29 +00:00
Joaquin
c29a47240f Weekly dimension 2024-11-05 13:28:12 +01:00
Joaquin
d76b8ac205 fixed int_monthly_12m_window_contribution_by_deal 2024-11-05 12:53:27 +01:00
Joaquin
abd4a4e8f2 fixed int_monthly_growth_score_by_deal 2024-11-05 12:49:50 +01:00
Joaquin
5bcfd7cb9c Revert cross models 2024-11-05 12:37:31 +01:00
Joaquin
092e37d58c aggregated to agg 2024-11-05 12:17:26 +01:00
Oriol Roqué Paniagua
c21b05f9cf Merged PR 3420: Remove previous KPIs computation for Bookings, Guest Journeys and Guest Payments
# Description

Removes old computation of KPIs of Bookings, Guest Journeys and Guest Payments, after yesterday's switch to new computation.

It cleans 1) models 2) schema entries and 3) temporary tests.

# Checklist

- [X] The edited models and dependants run properly with production data.
- [NA] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them. **(Except for the duplicated Booking to Product Bundle)**
- [X] I have checked for DRY opportunities with other models and docs.
- [NA] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Remove previous KPIs computation for Bookings, Guest Journeys and Guest Payments

Related work items: #23576
2024-11-05 10:12:00 +00:00