Adds the following metrics:
- Guest Journey with Payment
- Guest Journey Payment Rate
by both visions (global and by deal id)
**Important**: it does not expose these metrics to the dashboard, this will be done after we have feedback from Ben R. on the paid GJ without GJ completeness. Missing steps to make them appear is to adapt `int_core__mtd_aggregated_metrics` and `int_core__monthly_aggregated_metrics_history_by_deal` and the respective reporting counterparts.
It adapts:
- `int_core__mtd_guest_journey_metrics`
- `int_core__monthly_guest_journey_history_by_deal`
the approaches are similar in the sense that we join with `int_core__verification_payments` and filter by a PAID status, that has been defined in the `dbt_project.yml` in a similar manner as we did with cancelled bookings. It can happen that the same verification request has multiple payments (see screenshot), which in this case we keep the first date in which the paid payment happens. The volume is quite low anyway.
code for the screenshot:
```
with pre as (
select
id_verification_request,
count(distinct icvp.id_payment) as total_paid_payments
from intermediate.int_core__verification_payments icvp
where icvp.payment_status = 'Paid'
group by 1
)
select
case when total_paid_payments > 2 then 'more than 2'
when total_paid_payments = 2 then '2'
when total_paid_payments = 1 then '1'
end as payment_volume_category,
count(1) as vr_volume
from pre
group by 1
order by 2 desc
```
I also added a missing reference in `schema.yaml` int about `int_core__mtd_guest_journey_metrics`
Related work items: #18105
This PR creates 2 new models:
- `int_core__monthly_aggregated_metrics_history_by_deal`, which just gathers the information of the previously created models that compute the kpis by deal id.
- `core__monthly_aggregated_metrics_history_by_deal`, effectively a copy from intermediate to reporting
It also includes documentation of these 2 models, differences between these and the `mtd_aggregated_metrics` equivalents and references it to exposures. I took the opportunity to update the documentation of the `core__mtd_aggregated_metrics` now that it's a bit more mature.
This should be the last PR for the first draft of 'by deal' metrics.
Related work items: #17689
Adding accommodation metrics by deal id with the model `int_core__monthly_accommodation_history_by_deal`.
With this PR, we have the full set of batch 1 metrics by deal id completed, although separated in different tables. Aggregation will come in a separated PR.
Similarly as the previous PR, this one it's a mix between the logic of `int_core__mtd_accommodation_metrics` and the logic existing for the `int_core__monthly_X_history_by_deal` . It also adds the tests in schema.
Related work items: #17689
Adding the 6 Guest Journey metrics by deal id by creating the model `int_core__monthly_guest_journey_history_by_deal`
The structure for the deal id detail follows yesterday's approach on bookings, namely `int_core__monthly_booking_history_by_deal`, but considering the metric computation of the guest journey, namely `int_core__mtd_guest_journey_metrics`.
It also adds the dbt tests ensuring that date and id_deal are not null and that the combination of both is unique.
Related work items: #17689
This is a first approach to compute some easy metrics for the "deal" based business kpis. At this stage, it contains the information of bookings (created, checkout, cancelled) per deal and month, including both historic months as well as the current one. This do not contain MTD computation because it's overkill to do a MTD at deal level (+ we have 1k deals, so scalability can become a problem in the future)
Models:
- **int_dates_by_deal**: simple model that reads from **int_dates** and just joins it with **unified_users** to retrieve the deals. It will be used as the 'source of truth' for which deals should be considered in a given month, basically, since the first host associated to a deal is created (not necessarily booked)
- **int_core__monthly_booking_history_by_deal**: it contains the history of bookings per deal id in a monthly basis. It should be easy enough to integrate here, in the future and if needed, B2B macro segmentation.
In terms of performance, comparing the model **int_core__monthly_booking_history_by_deal** and **int_core__mtd_booking_metrics** you'll see that I removed the joined with the **int_dates_xxx** in the CTEs. This is because I want to avoid a double join of date & deal that I tried and I stopped after 5 min running. Since this computation is in a monthly basis - no MTD - it's easy enough to just apply the **int_dates_by_deal** on the last part of the query. With this approach, it runs in 7 seconds.
Related work items: #17689
The bank transactions table coming from Xero has positive amounts for all transactions by default.
Nevertheless, some transactions are receiving and some are sending.
This PR implements sign for transactions amounts throughout the DWH so that aggregations work properly.
I've also left the transaction sign column in some spots since it might be useful for some aggregation wizardry (ie cancel out receiving transactions in resolutions so that counts are more accurate).
Related work items: #17551
As today it's 1st of July, the logic of selecting all days of the current month for MTD purposes on the business KPIs is ko, since we select up to yesterday.
This PR allows to consider the last day of the previous month as 'current month' only for the first day of the following month, thus ensuring that the most up-to-date data is always displayed in the MTD tab.
Related work items: #17745
Fixing accommodation host by using accommodation to user, after discussion with Ben R.
This improves data quality, even though there's some duplicates removal.
I checked and it effectively removes accommodations that mostly were considered as 'Never Booked', thus not a massive impact is expected for the business kpis. But in any case, let's do things properly :)
Related work items: #17538
This PR closes the first draft of the first batch of business kpis. Host logic has changed to be applied at deal id level.
It's mostly an adapted copy-paste from the accommodation counterpart, specifically:
- `int_core__mtd_deal_lifecycle`: computes the historic deal lifecycle. One line for each deal and MTD date. **Important**: _Not all hosts have a deal set. This will need a data quality report for business teams to fix_
- `int_core__mtd_deal_metrics`: computes the aggregation at MTD date level of the metrics per lifecycle state and activity state
Additionally, this PR changes:
- `int_core__mtd_aggregated_metrics`: it includes the new 3 deal metrics and changes the source of the already existing 3 deal metrics from `mtd_booking_metrics` to the new `mtd_deal_metrics`
- `int_core__mtd_booking_metrics`: removes all code needed to compute the remaining deal metrics, speeding it up considerably.
After this PR, the mtd models run (locally) at the following speed:
- `int_core__mtd_accommodation_lifecycle`: 47 sec
- `int_core__mtd_deal_lifecycle`: 3 sec
- `int_core__mtd_accommodation_metrics`: 5 sec
- `int_core__mtd_deal_metrics`: < 1 sec
- `int_core__mtd_booking_metrics`: 8 sec (quite a reduction)
- `int_core__mtd_guest_journey_metrics`: 5 sec
- `int_core__mtd_aggregated_metrics` and `core__mtd_aggregated_metrics`: < 1 sec
Related work items: #17312
This PR will compute the listing metrics in an aggregated manner to be displayed in the Main KPIs dashboard, specifically:
- New Listings
- First Time Booked Listings
- Churning Listings
- It also adapts the computation for the already existing metrics of Listings Booked in X months
At code level, it contains the following:
- Adds `int_core__mtd_accommodation_metrics`, which computes the aggregation of the lifecycle of listings at date level (unique), being date the corresponding date from `int_dates_mtd`
- Changes `int_core__mtd_aggregated_metrics` to take the accommodation metrics from the new model. Those 3 already existing (Listings booked in X month) now read from the new model as well.
- Changes `int_core__mtd_booking_metrics` to remove unused computation, making it lighter. Specifically, it removes 1) listing related metrics, since now we have a dedicated model and 2) number of guests booked, since it's not used at all.
The resulting values in local are consistent with what is already reported in the staging report.
Related work items: #17312
Adding int_core__accommodation
Includes both:
- Main information of the accommodation, mostly coming from stg_core__accommodation and int_core__country.
- Listing lifecycle computation, based on the created bookings from stg_core__bookings. It's just the current state, no history.
Some considerations:
- I opted to use stg_core__bookings and not int_core__bookings. Main reason is in case at some point we want to add listing-based information to the booking table, it would avoid cyclic references.
- I opted to keep all the logic of 1) accommodation info and 2) lifecycle in the same model. This could be easily split into: lifecycle first that reads uniquely from staging and then the int_core__accommodation that could read from the staging version to retrieve accommodation attributes + the lifecycle one. Up to you
I'd suggest to review first the documentation in schema since it explains the logic applied.
Notion page linked to this task: https://www.notion.so/knowyourguest-superhog/Listing-lifecycle-4dc0311b21ca44f8859969e419872ebd
Related work items: #17312
Adding Country to intermediate, both model + documentation.
At this stage, the model is set as a view but we can discuss what is the best approach
Related work items: #17312
Adding cancelled bookings metric based on the feedback of the tech team. Mainly, the date of a cancelled booking can be considered as the `updated_date_utc` for those bookings with status cancelled, as it's a terminal state and no additional steps should follow.
I also took the opportunity to update:
- The order on the `int_core__mtd_aggregated_metrics`, so it matches the one in the Notion page for the 1st batch, freeing already the space for the order_by numbers for missing metrics
- Make acronyms of alias in the `main_kpi` subquery in `int_core__mtd_booking_metrics` slightly more clear
- Remove empty line at the end of the file in `int_core__mtd_booking_metrics`
Keep in mind that the cancelled bookings metric will be directly available in the dashboard once this PR is approved and DWH re-runs.
Related work items: #17310
