# Description
Booking fees is widely used with different meanings, for old dash, for new dash, for both, etc. This is painful. First step to align on a proper naming is ensure that what we report in Main KPIs is clearly stated, which in this case, Booking Fees are now called Old Dashboard Booking Fees.
Changes:
* Modify `stg_seed__accounting_aggregations` seed to rename Booking Fees to Old Dashboard Booking Fees. This is for us to clarify. This is only applied for KPIs compute. I also added an empty space that I mistakenly forgot in the past for `financial_l3_aggregation`.
* Modify KPIs source, i.e., `int_kpis__metric_daily_invoiced_revenue`. Here I forcefully modify the name of the field to `xero_old_dashboard_booking_net_fees_in_gbp`.
* Propagate changes of downstream usages of `xero_booking_net_fees_in_gbp` to `xero_old_dashboard_booking_net_fees_in_gbp`. This affects all models, including the reporting model. On this one we still have both names to avoid breaking it. I will need to modify the data glossary in PBI anyway so I'll do this change as well.
* Modify displayed metric name from Booking Fees Revenue to Old Dashboard Booking Fees Revenue.
* Modify schema so it reflects the proper names, descriptions, and tests.
* Ensure outlier and completion tests still work after this change.
I confirm the field `xero_booking_net_fees_in_gbp` does not exist anymore in the rest of DWH after these changes, except for the abovementioned comment on the reporting line.
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #28560
# Description
Changes:
* Adds Revenue Churn Rate in YTD/MTD Overview. This has several implications, I finally understood how to properly compute a YTD.
The problem is that Revenue Churn Rate is a % of the Total "Churned" Revenue in a 12 m period vs. the Total Revenue in the same 12 m period. This is a bit tricky because it's not really additive, because of the Churn definition. Total Churned Revenue is the Revenue that the churned deals in a month generated on that past 12 months prior to churning.
So - in order to aggregate it properly, we need to do the sum of the Total Churned Revenue and retrieve the Total Revenue on these 12 months, and THEN compute the Churn rate.
This PR mainly retrieves the necessary input from the Churn models and then follows a similar computation as for the rest of YTD/MTD converted metrics.
I'll handle Onboarding MRR in a separated PR as this one is quite dense already.
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #27609, #27805
# Description
Changes:
* On Est. Billable Bookings, keeps previous logic for Old Dash while assuming the first billable service for New Dash
* Adds a Billable Check Out Bookings in Check Out bookings, as defined in the requirements.
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #27619
# Description
Adds new metric - live deals
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #27712
# Description
Changes:
* Fixes listings by properly considering the equal or higher sign. I'm very dumb
* Fixes exclusion of new dash billable bookings by properly applying in both monthly and mtd. I'm very dumb x2
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #27356
# Description
Changes:
* Adapt business_kpis_configuration to include By Business Scope as production dimension.
* Sets in int_mtd_metrics_vs_previous_year the selection of business scope dimension for all KPIs models. This does not affect cross kpis models (churn/mrr). I might need to check later how to adapt Churn to include this dimension, but it's not considered as for this PR. Lastly, Billable Bookings excludes New Dash.
* Adapts condition in int_mtd_metrics_vs_previous_year so MTD values would appear independently of these appearing in the previous year. This is, the model was considering that to show current month MTD values, the dimension needed to exist the year prior. This does not happen with New Dash and I assume we never noticed because in any case, most of our dimensions have quite a long history.
* Adapts int_kpis__agg_dates_main_kpis to include the business scope dimension. By the way it's actually handled, it kind of assumes that a Deal can only be in New or Old Dash (this is correct), but while on New Dash, this deal won't have data for Old Dash (this might not be 100% correct). In any case, the global figure should be ok, and only on the deal + business scope dimensionality this could cause some potential problems. However, this is not being reported anyway at the moment.
* Adapts int_kpis__agg_dates_main_kpis to have a proper variable value for the dimensions, and this is further included in business_kpis_configuration as any other model.
Small changes:
* Adapts Churn metrics to read from dimension_deals rather than core__deals. This should be more accurate anyway.
# Checklist
- [ ] The edited models and dependants run properly with production data.
- [ ] The edited models are sufficiently documented.
- [ ] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Propagates New Dash/Old Dash/APIs split in KPIs as per Business Scope
Related work items: #27356
# Description
Main changes:
* Remove Cancelled Bookings previous logic.
* Speeds up Main KPIs by adding a couple of indexes in `mtd_aggregated_metrics`. Tested in prod, to me it looks like it's quite fast now.
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #24637
# Description
Exposes new metrics:
* Created Bookings (Excl. Cancelled)
* Cancelled Created Bookings
* Check Out Bookings (Excl. Cancelled)
* Cancelled Check Out Bookings
Re-naming of existing metrics:
* Created Bookings -> Total Created Bookings
* Checkout Bookings -> Total Check Out Bookings
Removes exposure of previous Cancelled Bookings. In the monthly by deal model, it's hardcoded still - need to change PBI to remove safely. This will be done later on once we remove the Cancelled Bookings models.
Adapts the existing tests on KPIs to accommodate for the changes by including new metrics. Also, I've set the detector to 5 (from 8) since it's been a while this has not triggered thus might be worth the effort to have more detection capabilities.
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #24637
# Description
Adds new metrics on Revenue for New Dash services:
invoiced_revenue.xero_basic_protection_net_fees_in_gbp,
invoiced_revenue.xero_waiver_pro_net_fees_in_gbp,
invoiced_revenue.xero_id_verification_net_fees_in_gbp,
invoiced_revenue.xero_protection_plus_net_fees_in_gbp,
invoiced_revenue.xero_screening_plus_net_fees_in_gbp,
invoiced_revenue.xero_sex_offenders_check_net_fees_in_gbp,
invoiced_revenue.xero_protection_pro_net_fees_in_gbp
Note that it does not add Basic Screening. This is because so far this is a free service, even though the kpis models already consider such a possibility.
First data appears for December so this will be available starting 1st February
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
New Dash services Invoiced Revenue
Related work items: #26680
# Description
Adds `is_end_of_month_or_yesterday` for Main KPIs.
Apparently, the fact that we do not show the ongoing value of the month on some tabs of Main KPIs is the main blocker for Ben C to consistently use Main KPIs, which he actually retrieves from the SH legacy reporting. Since I'm sceptical about the data shown there, I want to remove this blocker.
It will require a small PR on PBI as well.
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [NA] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #25342
# Description
New model:
* int_kpis__agg_dates_main_kpis - Serves as the skeleton of dates and dimensions for Main KPIs. It's aggregated since it follows a similar aggregation strategy. It's a single model to feed both Main KPIs visualisations. Note boolean fields are real booleans (true/false) while before these were integers (1/0). This also affects downstream models.
Main KPIs flow adaptations to new skeleton model:
* int_monthly_aggregated_metrics_history_by_deal
* int_monthly_churn_metrics - additionally, calls usual KPIs macro instead of old one
* int_mtd_vs_previous_year_metrics
Reporting changes to ensure report is not down:
* mtd_aggregated_metrics - adaptations on booleans (true-1, false-0)
Cleaning:
* get_kpi_dimensions macro is no longer used
* int_dates_by_deal model and schema entry
* int_dates_mtd_by_dimension model and schema entry
* int_dates_mtd model and schema entry
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #23763
# Description
Switches Listings and Deal metrics to production.
I also modified the dependencies on the old lifecycles to read from the news, so we can remove these in a following PR.
I'm considering re-computing also the dimension - specific models to operate within KPIs folder, but will do this later on in a dedicated PR.
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #23761
# Description
Switches Xero metrics to the new KPIs flow. This uses the 2 new entities around Invoiced Revenue and Host Resolutions.
The metrics affected will change computation source in PBI as well (for MTD per category and Monthly by deal views). This directly affects the following metrics:
* Invoiced Operator Revenue
* Invoiced Booking Fees Revenue
* Invoiced Listing Fees Revenue
* Invoiced Verification Fees Revenue
* Invoiced APIs Revenue
* Invoiced Athena Revenue
* Invoiced E-Deposit Revenue
* Damage Host-Waiver Payments
* Host Resolutions Amount Paid
* Host Resolutions Payment Count
Additionally, the following metrics will be indirectly affected since depend partially on Xero:
* Total Revenue
* 4x Total Revenue per (Booking Created, Guest Journey Created, Deals Booked in Month, Listings Booked in Month)
* Waiver Retained
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [NA] I have checked for DRY opportunities with other models and docs. **Same as last time - the way models are called could be improved, but let's switch everything first and then discuss. There's chances these cross models will die anyway**
- [X] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #23759
# Description
Exposes New KPIs of Bookings, Guest Journeys and Guest Payments in the 2 main flows: Monthly+MTD per Global, By Billing Country and By # of Listings Segmentation & Monthly By Deal
I did some checks on Global and By Billing Country and looks good.
I checked some examples for Monthly By Deal and looks good.
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [NO] I have checked for DRY opportunities with other models and docs. **Likely there's ways to create default configs for KPIs saying "hey, this is Main KPIs for a specific view so use this models and these filters". But at this stage since it's a transitional stage I don't want do overthink. Let's tackle this later on if that's ok for you. It means we will live with some repeated expressions**
- [X] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #23455
# Description
Exposes Churn Rates into Main KPIs, specifically Revenue Churn Rate, Bookings Churn Rate and Listings Churn Rate. This is based on the average approach.
Additionally, it adds these 3 metrics in the kpis_additive_metrics_per_dimension_are_consistent test.
Additionally, it removes from int_mtd_vs_previous_year_metrics the computation of the additive Churn Rates. The removal of further unused code will be handled in a separated PR.
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [NA] I have checked for DRY opportunities with other models and docs.
- [NA] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #22691
# Description
Moves from intermediate/core to intermediate/cross the following models:
- `int_core__mtd_deal_lifecycle`
- `int_core__mtd_deal_metrics`
to their equivalents:
- `int_mtd_deal_lifecycle`
- `int_mtd_deal_metrics`
This also changes the schema entries, from core to cross, including changing the name of the model in the entry.
This also changes the dependencies, namely in `int_mtd_deal_metrics`, `int_mtd_vs_previous_year_metrics` and `int_monthly_aggregated_metrics_history_by_deal`.
This does NOT aim to alter the logic of the lifecycle in any case; it will be done in a separated PR.
Runs correctly end-to-end. We might need to drop the old models from production manually.
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #22689
# Description
Main changes:
- Creation of `int_monthly_churn_metrics` model. This follows a similar approach as for mtd models, with jinja loops to aggregate the metrics at different dimensions. This reads from the previous monthly model, thus creating a dependency on Global KPIs with By Deal KPIs.
- Adds the 6 metrics in the main aggregated model of Global KPIs `int_mtd_vs_previous_year_metrics`. It doesn't necessarily mean that the 6 metrics will be made available in the report.
This does NOT display these metrics in the report, and won't be done until deal lifecycle is enriched to consider hubspot offboarding in the state "05-Churning".
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #22691
# Description
Adapts revenue figures in Main KPIs - MTD scope or global view. This includes MTD, Monthly Overview, Global Evolution over Time, Detail by Category. In essence, everything that is not by deal.
The changes are mainly 2:
* Remove the line that deducts the `Waiver Amount Paid Back to Hosts` in all metrics except the `Waiver Net Fees`. This effectively means that the previous `Guest Revenue` = `Guest Payments`, thus I dropped all 3 `Guest Payments` metrics.
* Do a renaming at metric display level, but not in the code. This means that I remove the computation of `guest_revenue_in_gbp` for instance and keep `guest_payments_in_gbp`, and apply the renaming later on, since the modelisation already accounts for defining metric names differently from those of the fields. For the rest of metrics, I revised all metrics name and did changes based on the [whiteboard](https://whiteboard.office.com/me/whiteboards/p/c3BvOmh0dHBzOi8vZ3VhcmRob2ctbXkuc2hhcmVwb2ludC5jb20vcGVyc29uYWwvcGFibG9fbWFydGluX3N1cGVyaG9nX2NvbQ%3d%3d/b!T2D3opQuBECSDnhuFZrUacFu3TxvSvdIsnI4Dxsh2IuaB1AigbciRqkqte61I4wz/01H5SI4J4L7HTPJGUT7JGYKTOSQYYWACXU). I also changed the dedicated data tests in Main KPIs to ensure it's working. I also changed the exclusion logic in reporting based on the name of the metric to not display metrics that depend on the invoicing cycle unless it's 2 months ago or before.
To keep in mind:
* Merging this will automatically display the new figures/naming in production. Might be wise to communicate to stakeholders since some key metrics (namely, Guest Revenue / Total Revenue) will change the meaning.
* We also need to do these changes in the metrics by deal part of the computation. I'd do first the removal of these fields in the PBI report (and take the opportunity to change the Data Catalogue) and then do the PR in DWH to change the logic. Before that though let's check that the names included in this PR are the correct ones :)
# Checklist
- [X] The edited models and dependants run properly with production data.
- [NA] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [NA] I have checked for DRY opportunities with other models and docs.
- [NA] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #22688
# Description
This PR ensures the propagation of the dimensions for KPIs across the key aggregating and exposing models. Additionally, provides these 2 new fields in reporting while **not affecting the current data display**, thus it's safe to work in the PBI report without needing to work in 2 PRs in parallel.
**Changes:**
**1 - Intermediate, `int_mtd_vs_previous_year_metrics`:**
* Removes the temporary filter on `where dimension in ({{ production_dimensions }})`. This will be applied directly to reporting later. This ensures that the new dimension on customer segmentation is fully available only within intermediate.
* Adds `dimension` and `dimension_value` granularity. This includes: 1) adding these fields, 2) joining by these fields with all the source CTEs containing the source models with metrics - which in turn needs the change of the dates model - and 3) joining by these fields in the self-join to compute the incremental vs. previous year.
* Changes on the schema file
**2 - Intermediate, `int_mtd_aggregated_metrics`:**
* Adds `dimension` and `dimension_value` granularity. This includes only adding these fields.
* Changes on the schema file
**3 - Reporting, `mtd_aggregated_metrics`:**
* Adds the filter removed on `int_mtd_vs_previous_year_metrics`. This ensures that only the Global dimension is available for the reporting, thus **no changes from user POV**.
* Adds `dimension` and `dimension_value` granularity. This includes only adding these fields
* Changes on the schema file
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #19325
# Description
Silly change:
* Modifies `int_core__mtd_guest_payments_metrics` to apply the proper key on date, dimension and dimension_value.
* -> *The weird thing is that the previous dbt test I run worked well. Is it possible that the configuration in the schema file prevails on top of the model configuration? I thought it was the other way around...*
Main changes:
* Modifies `int_xero__mtd_invoicing_metrics` to include the customer segmentation based on listings.
* `schema.yaml` is also affected including new fields and tests
* Added the macro to retrieve the production dimension in `int_core__mtd_vs_previous_year_metrics` to avoid propagating this upwards and messing up with the data display.
Overall, follows a similar strategy as we did for Booking, Guest Journey, Deal, Accommodation and Guest Payments metrics. For reference, [here's the previous PR on Guest Payments](https://guardhog.visualstudio.com/Data/_git/data-dwh-dbt-project/pullrequest/2580).
This is the last PR on the source models for KPIs. Will follow: propagation + exposure
# Checklist
- [X] The edited models and dependants run properly with production data.
* **Important note**: this segmentation provides null values for all API-related KPIs. Makes sense, since the 4 deal id we have for APIs do NOT have, or have had, a listing linked to them. I'd say it's not a blocker.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #19325
# Description
Modifies `int_core__mtd_guest_payments_metrics` to include the customer segmentation based on listings. `schema.yaml` is also affected including new fields, tests and apply the proper naming (from guest revenue to guest payments). I also modified a silly naming that was referring to deals to refer to listings/accommodations, my bad.
Added the macro to retrieve the production dimension in `int_core__mtd_vs_previous_year_metrics` to avoid propagating this upwards and messing up with the data display.
Overall, follows a similar strategy as we did for Booking, Guest Journey, Deal and Accommodation metrics. For reference, [here's the previous PR on Accommodations](https://guardhog.visualstudio.com/Data/_git/data-dwh-dbt-project/pullrequest/2575?_a=overview).
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #19325
# Description
Takes into account @<Pablo Martín> 's feedback from the previous PR, slightly modified. This PR separates 1) the dimensions while developing vs. 2) the dimensions once these are available for production. This are within the same file of macro configuration for KPIs, namely `business_kpis_configuration`.
End-goal, all CTEs in `int_mtd_vs_previous_year_metrics` will read from this new macro `get_kpi_dimensions_for_production`, so eventually we won't need any hardcode once we want to add new dimensions. In the meantime, I'll be adding this new line for each PR (still 2 missing :D)
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [ ] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #19325
# Description
Modifies `int_core__mtd_accommodation_metrics` to include the customer segmentation based on listings. `schema.yaml` is also affected including new fields and tests. Hardcoded `int_core__mtd_vs_previous_year_metrics` to avoid propagating this upwards and messing up with the data display.
Overall, follows a similar strategy as we did for Booking, Guest Journey and Deal metrics. For reference, here's [the previous PR on Deal](https://guardhog.visualstudio.com/Data/_git/data-dwh-dbt-project/pullrequest/2534). I noticed that I mixed the schema tests of Deals and Accommodations, this PR should fix both.
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #19325
# Description
Modifies `int_core__mtd_deal_metrics` to include the customer segmentation based on listings. `schema.yaml` is also affected including new fields and tests. Hardcoded `int_core__mtd_vs_previous_year_metrics` to avoid propagating this upwards and messing up with the data display.
Overall, follows a similar strategy as we did for Booking and Guest Journey metrics. For reference, [here's the previous PR on GJ](https://guardhog.visualstudio.com/Data/_git/data-dwh-dbt-project/pullrequest/2533).
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #19325
# Description
Modifies `int_core__mtd_guest_journey_metrics` to include the customer segmentation based on listings. `schema.yaml` is also affected including new fields and tests. Hardcoded `int_core__mtd_vs_previous_year_metrics` to avoid propagating this upwards and messing up with the data display.
Overall, follows a similar strategy as we did for Booking metrics.
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #19325
# Description
This is a first idea of how I'd like to add dimensionality in the KPIs for the mtd models. For the moment, I keep deal_id apart, so I just touch the "mtd" models, that so far only contained "global" metrics.
In this case I include the listing segmentation (0, 1-5, 6-20, etc) in the bookings. To do this, I created 2 new fields: dimension and dimension_values.
I also created a "master" table with `date` - `dimension` - `dimension_value` called `int_dates_mtd_by_dimension`
Important notes:
- I force a hardcode in `int_mtd_vs_previous_year_metrics`. This is to not break production.
- You will notice how repetitive the code is starting to look. My intention with this PR is that we are happy with this approach on the naming, the strategy for joins, etc. If that's ok, next step is going to be doing macros on top. Think of the state of `int_core__mtd_booking_metrics` as the "compiled version" of the macro that should come afterwards.
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #19325
After discussion with Pablo on the fact that Deposits are only with status "Paid" for a given time before they get Cancelled or Refunded, we just believe it's best to remove the Deposits amount from the Guest Payments metric. In any case, this does not represent a Revenue source... This was discovered while doing the data quality assessment for revenue figures ([here](https://www.notion.so/knowyourguest-superhog/Data-quality-assessment-DWH-vs-Finance-revenue-figures-6e3d6b75cdd4463687de899da8aab6fb))
Before, `total_guest_payments_in_gbp` was a standalone metric that computed any payment by the guest with status paid. We were computing revenue based on the `total_guest_income_in_gbp`, which mainly was the sum of waiver payments, deposit fees (not deposit itself!) and check in hero fees.
Mainly what I did is:
- remove the existing `total_guest_payments_in_gbp` in the source models (int_core__xxx_guest_payments_xxx)
- rename the already existing `total_guest_income_in_gbp` to `total_guest_payments_in_gbp`
Related work items: #18787, #18914
Exposes the following metrics:
1. Invoiced Booking Fees
1. Invoiced Listing Fees
1. Invoiced Verification Fees
1. Invoiced Guesty Fees
1. Invoiced E-Deposit Fees
1. Deposit Fees
1. Waiver Amount Paid by Guests
1. Waiver Amount Paid back to Hosts
1. Check-In Hero Amount Paid by Guests
This PR is only for Global metrics, the one by deal id will follow separately.
Keep in mind that merging this PR will make the data appear in the report.
Related work items: #18914
Small refactor to follow up on last week's PR. We removed from the Guest Revenue models the host-takes-waiver aspect, thus these models are now only depending from Core. We just need to migrate it from cross to core.
One small detail as well, since we do not take into account at these models level the host-takes-waiver, technically, I would not call these models revenue but rather Guest Payments. This is why I also took the opportunity to apply this name.
Changes:
- `int_monthly_guest_revenue_by_deal` is now `int_core__monthly_guest_payments_history_by_deal`, and the location has changed from `intermediate.cross` to `intermediate.core`
- `int_mtd_guest_revenue_metrics` is now `int_core__mtd_guest_payments_metrics`, and the location has changed from `intermediate.cross` to `intermediate.core`
- Schema changes, moving these 2 models' documentation with the new naming from Cross to Core
- Provide continuity in following dependants: `int_mtd_vs_previous_year_metrics` and `int_monthly_aggregated_metrics_history_by_deal` now read from the 2 new models respectively. Additionally, the model alias has changed from `guest_revenue` to `guest_payments` to keep consistency.
This PR does not expose new metrics, but should keep the existing ones unaffected.
Related work items: #18914
Adding submetrics of guest revenue:
- deposit fees
- checkin cover fees
- waiver payments
all of this adds up to
- guest income
and the revenue is computed by subtracting waivers paid to hosts
Related work items: #18787
This PR exposes the following metrics to the Main KPIs business overview report, for both Global + By Deal view:
- Total Revenue
- Total Revenue per Booking Created
- Total Revenue per Guest Journey Created
- Total Revenue per Deals Booked in Month (does not apply on the by deal view)
- Total Revenue per Listings Booked in Month
- Invoiced Operator Revenue
- Host Resolution Payment Count
- Host Resolution Amount Paid
Keep in mind Global view will be displaying these metrics once this is merged. I also changed a bit the order of the metric display.
Note that Billable Bookings are not included.
I recommend to review by 1) checking the first commit. This is almost the same as the previous abandoned PR that @<Joaquin Ossa> you already checked on Tuesday. I added a second commit, to be checked later, which basically fixes some stupid issues that if one of the source of revenue is null, then total revenue is null. This is specially critical for the view by deal, since most of them do not have revenue from APIs - thus all total revenue figures were null...
Related work items: #18108, #18109, #18110, #18719
Computing and propagating APIs revenue metrics.
I retrieved the revenues linked to Guesty and e-deposits. The sum of those are considered the total API revenue at this stage.
These 3 metrics are available in upper layers (not exposed yet to the report), just to fix the total revenue computation, which now includes APIs revenue
Related work items: #18719
Propagates billable bookings kpis in intermediate. Does not expose any metric to the report.
Changes:
- Retrieval and computation of previous_year and relative_increment for global view (mtd models)
- Retrieval as is for deal view
Related work items: #18111
This PR aims to propagate the invoicing metrics through the DWH. It does not expose them to users, yet.
This PR effectively computes the following metrics, for both the "global" view (MTD) and the "by deal" view (by_deal):
- Invoiced Operator Revenue
- Host Resolution Count of Payments
- Host Resolution Amount Paid
With these 3 new metrics, we're able to combine them with the existing ones to compute:
- Total Revenue
- Total Revenue per Booking Created
- Total Revenue per Guest Journey Created
- Total Revenue per Deal Booked in Month
- Total Revenue per Listings Booked in Month
You'll also note that I've included standalone metrics for booking fees, listing fees, verification fees and waiver payments. This will not be exposed in this batch 2, but based on the conversation with Finance, will clearly make it for batch 3. I just find it easier to add it now, since it's straight forward.
Main changes:
- `int_mtd_vs_previous_year_metrics` now computes all the above mentioned metrics
- `int_monthly_aggregated_metrics_history_by_deal` now computes all the above mentioned metrics, except Total Revenue per Deal Booked in Month since it does not make sense for the deal view. Additionally, I took the opportunity to include the missing metrics from listings (accommodations). The goal is not necessarily to display them, but at least compute it on our side.
Additional changes:
- In `int_xero__mtd_invoicing_metrics` and `int_xero__monthly_invoicing_history_by_deal`, there's a very silly name change to keep the same convention for fees: from `xero_operator_net_fees` to `xero_operator_net_fees_in_gbp`
- I applied additional changes in `int_monthly_aggregated_metrics_history_by_deal` with the goal to keep the same format as we have in `int_mtd_vs_previous_year_metrics`, this meaning:
1 - explicit alias naming (from `gj` to `guest_journeys`)
2 - keep a similar arrangement of metrics, and clearly separate scopes depending on the metric type
3 - Re-apply autoformatting
Related work items: #18108, #18109, #18110
