This PR creates a new model int_core__mtd_accommodation_segmentation that provides the customer segments based on listing activity:
- 0
- 1-5
- 6-20
- 21-60
- 61+
For end of April 2024, the volume distribution on number of deals and total listings booked is:
For information, I estimate that around 3% of listings with bookings are missed, according to the data displayed in the KPIs for 30th April 2024. This is because we enforce deal-based categorisation (same happens with the deal view, anyway)
Related work items: #19325
After discussion with Pablo on the fact that Deposits are only with status "Paid" for a given time before they get Cancelled or Refunded, we just believe it's best to remove the Deposits amount from the Guest Payments metric. In any case, this does not represent a Revenue source... This was discovered while doing the data quality assessment for revenue figures ([here](https://www.notion.so/knowyourguest-superhog/Data-quality-assessment-DWH-vs-Finance-revenue-figures-6e3d6b75cdd4463687de899da8aab6fb))
Before, `total_guest_payments_in_gbp` was a standalone metric that computed any payment by the guest with status paid. We were computing revenue based on the `total_guest_income_in_gbp`, which mainly was the sum of waiver payments, deposit fees (not deposit itself!) and check in hero fees.
Mainly what I did is:
- remove the existing `total_guest_payments_in_gbp` in the source models (int_core__xxx_guest_payments_xxx)
- rename the already existing `total_guest_income_in_gbp` to `total_guest_payments_in_gbp`
Related work items: #18787, #18914
Small refactor to follow up on last week's PR. We removed from the Guest Revenue models the host-takes-waiver aspect, thus these models are now only depending from Core. We just need to migrate it from cross to core.
One small detail as well, since we do not take into account at these models level the host-takes-waiver, technically, I would not call these models revenue but rather Guest Payments. This is why I also took the opportunity to apply this name.
Changes:
- `int_monthly_guest_revenue_by_deal` is now `int_core__monthly_guest_payments_history_by_deal`, and the location has changed from `intermediate.cross` to `intermediate.core`
- `int_mtd_guest_revenue_metrics` is now `int_core__mtd_guest_payments_metrics`, and the location has changed from `intermediate.cross` to `intermediate.core`
- Schema changes, moving these 2 models' documentation with the new naming from Cross to Core
- Provide continuity in following dependants: `int_mtd_vs_previous_year_metrics` and `int_monthly_aggregated_metrics_history_by_deal` now read from the 2 new models respectively. Additionally, the model alias has changed from `guest_revenue` to `guest_payments` to keep consistency.
This PR does not expose new metrics, but should keep the existing ones unaffected.
Related work items: #18914
This PR computes KPIs for Billable Bookings for both views (Global, by deal id)
It's the 1st version, mainly because numbers are not fully in-line with Jamie's export and I'd like to understand why. It uses booking_charge_events, as suggested by Pablo. In the meantime, I'm debugging the differences based on the invoicing export tool that provides Finance first rough numbers before amendment.
In any case, it can be used for a first rough idea.
Related work items: #18111
Refactor mtd joins to improve performance, as stated in the ticket:
We noticed that some of the new models for MTD purposes (KPIs reporting) take quite a bit of time to run some simple joins.
The main reason is that there's a double join that can be simplified. The current state is:
```
from int_dates_mtd d
inner join
sometable t
on extract(year from t.table_date) = d.year
and extract(month from t.table_date) = d.month
and extract(day from t.table_date) <= d.day
```
and it can be changed to:
```
from int_dates_mtd d
inner join
sometable t
on date_trunc('month', t.table_date)::date = d.first_day_month
and extract(day from t.table_date) <= d.day
```
which is way faster, and keeps the same computation
Related work items: #18330
This PR aims to expose the new metrics to the business KPIs report.
The new metrics exposed are, for the global and the by deal view:
- Guest Revenue
- Guest Revenue per Guest Journey Completed
- Guest Revenue per Guest Journey with Payment
- Guest Payments
- Guest Payments per Guest Journey Completed
- Guest Payments per Guest Journey with Payment
- Guest Journey with Payment
- Guest Journey Payment Rate
Changes:
- Silly change on the naming in the by deal view of `payment_rate_guest_journey` to be consistent with the global view.
- Silly change that I miss some GJ payment metric for the view by deal id.
- Added a new number format called `currency_gbp` - only for monetary metrics, available in the schema files
- Usual procedure to publish metrics: for global metrics, add them in the `int_mtd_aggregated_metrics`. I also changed the order of display.
- **Important**: to avoid displaying revenue figures until Xero invoicing is handled, I created a macro called `is_date_before_previous_month` that is called in the reporting equivalent models: `mtd_aggregated_metrics` in the where section and in the `monthly_aggregated_metrics_history_by_deal` as a case-when.
This should allow to expose all new metrics, and enable the publishing of a new update of the business kpis!
Related work items: #18107
Changing naming to follow convention.
This PR has the following changes:
- the model `int_core__mtd_aggregated_metrics` has been moved to cross and changed the name to `int_mtd_aggregated_metrics`
- the model `int_core__monthly_aggregated_metrics_history_by_deal` has been moved to cross and changed the name to `int_monthly_aggregated_metrics_history_by_deal`
- the reporting models `core__mtd_aggregated_metrics` and `core__monthly_aggregated_metrics_history_by_deal` now source the `int_mtd_aggregated_metrics` and `int_monthly_aggregated_metrics_history_by_deal` to avoid breaking the production dashboard
- the reporting models have been duplicated from core into general with the correct names, i.e., `mtd_aggregated_metrics` and `monthly_aggregated_metrics_history_by_deal`
- Documentation has been moved in intermediate and replicated in reporting, adding comments on the currently in use models that are going to die soon.
This will allow for a transition of the PBI dashboard from one source to another. Exposures file still not touched since technically the report is still sourcing the 'legacy' models. Documentation of the refactor here: https://www.notion.so/knowyourguest-superhog/Refactoring-Business-KPIs-5deb6aadddb34884ae90339402ac16e3
Related work items: #18202
Following yesterday's refactor of booking metrics, this PR provides a refactor of already exposed metrics: listings, deals and guest journeys.
-> Data is consistent with values already exposed.
Changes:
- for `int_core__mtd_listing_metrics`, `int_core__mtd_deal_metrics` and `int_core__mtd_guest_journey_metrics`:
1. remove the computation of the previous year metric value and the relative increment (last part of the query)
2. re-apply the formatting
- for `int_mtd_vs_previous_year_metrics`:
1. Reference listings, deals and GJ models
2. Include the metrics for these types in the `plain_kpi_combination` CTE
3. Add the computation of previous year and relative increment using the macro
- for `int_core__mtd_aggregated_metrics`
1. Remove and "hardcode" sources since all metrics now depend exclusively of `int_mtd_vs_previous_year_metrics`
This PR does not alter the exposed metrics in the production report. It does not aim to change the name of the reporting/intermediate models that expose the information, it will be done in a separated PR.
Documentation: https://www.notion.so/knowyourguest-superhog/Refactoring-Business-KPIs-5deb6aadddb34884ae90339402ac16e3
Related work items: #18202
First step on refactor of kpis:
- Remove relative incremental vs. previous year computation from the source model (`mtd_booking_metrics`, in this case)
- Aggregate the source mtd global metrics models into a single model: `int_mtd_vs_previous_year_metrics` (to enable multi-source weighted metric computation) and compute previous year value and relative increment. Now this logic is encapsulated into a macro `calculate_safe_relative_increment`, easing readability and providing a bit more robustness.
- End-to-end continuity to not break the existing dashboard display in `int_core__mtd_aggregated_metrics`
This is a substep of the global change. All info can be found in the documentation [here](https://www.notion.so/knowyourguest-superhog/Refactoring-Business-KPIs-5deb6aadddb34884ae90339402ac16e3)
Related work items: #18202
New model for guests satisfaction report, I included columns to check what is the guest paying for that might be helpful for analysis as well
Related work items: #16947
Adds the following metrics:
- Guest Journey with Payment
- Guest Journey Payment Rate
by both visions (global and by deal id)
**Important**: it does not expose these metrics to the dashboard, this will be done after we have feedback from Ben R. on the paid GJ without GJ completeness. Missing steps to make them appear is to adapt `int_core__mtd_aggregated_metrics` and `int_core__monthly_aggregated_metrics_history_by_deal` and the respective reporting counterparts.
It adapts:
- `int_core__mtd_guest_journey_metrics`
- `int_core__monthly_guest_journey_history_by_deal`
the approaches are similar in the sense that we join with `int_core__verification_payments` and filter by a PAID status, that has been defined in the `dbt_project.yml` in a similar manner as we did with cancelled bookings. It can happen that the same verification request has multiple payments (see screenshot), which in this case we keep the first date in which the paid payment happens. The volume is quite low anyway.
code for the screenshot:
```
with pre as (
select
id_verification_request,
count(distinct icvp.id_payment) as total_paid_payments
from intermediate.int_core__verification_payments icvp
where icvp.payment_status = 'Paid'
group by 1
)
select
case when total_paid_payments > 2 then 'more than 2'
when total_paid_payments = 2 then '2'
when total_paid_payments = 1 then '1'
end as payment_volume_category,
count(1) as vr_volume
from pre
group by 1
order by 2 desc
```
I also added a missing reference in `schema.yaml` int about `int_core__mtd_guest_journey_metrics`
Related work items: #18105
This PR creates 2 new models:
- `int_core__monthly_aggregated_metrics_history_by_deal`, which just gathers the information of the previously created models that compute the kpis by deal id.
- `core__monthly_aggregated_metrics_history_by_deal`, effectively a copy from intermediate to reporting
It also includes documentation of these 2 models, differences between these and the `mtd_aggregated_metrics` equivalents and references it to exposures. I took the opportunity to update the documentation of the `core__mtd_aggregated_metrics` now that it's a bit more mature.
This should be the last PR for the first draft of 'by deal' metrics.
Related work items: #17689
Adding accommodation metrics by deal id with the model `int_core__monthly_accommodation_history_by_deal`.
With this PR, we have the full set of batch 1 metrics by deal id completed, although separated in different tables. Aggregation will come in a separated PR.
Similarly as the previous PR, this one it's a mix between the logic of `int_core__mtd_accommodation_metrics` and the logic existing for the `int_core__monthly_X_history_by_deal` . It also adds the tests in schema.
Related work items: #17689
Adding the 6 Guest Journey metrics by deal id by creating the model `int_core__monthly_guest_journey_history_by_deal`
The structure for the deal id detail follows yesterday's approach on bookings, namely `int_core__monthly_booking_history_by_deal`, but considering the metric computation of the guest journey, namely `int_core__mtd_guest_journey_metrics`.
It also adds the dbt tests ensuring that date and id_deal are not null and that the combination of both is unique.
Related work items: #17689
This is a first approach to compute some easy metrics for the "deal" based business kpis. At this stage, it contains the information of bookings (created, checkout, cancelled) per deal and month, including both historic months as well as the current one. This do not contain MTD computation because it's overkill to do a MTD at deal level (+ we have 1k deals, so scalability can become a problem in the future)
Models:
- **int_dates_by_deal**: simple model that reads from **int_dates** and just joins it with **unified_users** to retrieve the deals. It will be used as the 'source of truth' for which deals should be considered in a given month, basically, since the first host associated to a deal is created (not necessarily booked)
- **int_core__monthly_booking_history_by_deal**: it contains the history of bookings per deal id in a monthly basis. It should be easy enough to integrate here, in the future and if needed, B2B macro segmentation.
In terms of performance, comparing the model **int_core__monthly_booking_history_by_deal** and **int_core__mtd_booking_metrics** you'll see that I removed the joined with the **int_dates_xxx** in the CTEs. This is because I want to avoid a double join of date & deal that I tried and I stopped after 5 min running. Since this computation is in a monthly basis - no MTD - it's easy enough to just apply the **int_dates_by_deal** on the last part of the query. With this approach, it runs in 7 seconds.
Related work items: #17689
