# Description
Changes:
* 4 new models in the scope of KPIs for Billable Verifications from APIs. I believe it's more correct to say these are Billable Verifications than Billable Bookings since there's some cases in which a Booking can be duplicated and it's billed multiple times. These include:
* A daily metric model - extremely simple. You will notice there's no Billing Country not Listing Segmentation. This is because for ALL API cases this is UNSET, thus, I just remove it.
* An equivalent monthly metric model.
* Two aggregated models per dimension, dimension value: on a daily and a monthly basis.
Important change: the macro that handles the aggregations sets by default Billing Country and Listing Segmentation. I modified a bit the flow so the only required dimension is Global, and these are skipped for APIs models.
This is needed for the changes intended on the Growth score. In there, I'll combine both Platform Billable Bookings with API Billable Verifications.
Notice there's no MTD models. These could be added for sure; but since I'm not creating any metric in Main KPIs or similar, I opted to skip it for now. It can be done later on.
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #29374
# Description
Adds a macro to compare the contents of two tables. Will simply tell you if the contents are identical or not. Designed to be used in your development flow through the CLI so you can check refactors as you go.
# Description
To be able to compute By Deal KPIs similarly as we do for Main KPIs we need to propagate the "By Deal" dimension first in intermediate. This does not impact reporting since there's a macro that already cuts the dimensions to be displayed, but I modified the name so it's clearer.
Changes:
* In `int_mtd_vs_previous_year_metrics`, I added a new `dimension_list`. This is applied to all initial CTEs that filter by dimension. Note that I added here the `by_deal` dimension.
* Modified the name of `get_kpi_dimensions_for_production` to `get_main_kpis_dimensions_for_production`. Now it's more explicit that this is the configuration for Main KPIs reporting. This affects both `business_kpis_configuration` and it's usage in `mtd_aggregated_metrics`.
* Modified the tests in `int_mtd_vs_previous_year_metrics` and `int_mtd_aggregated_metrics` to include the new dimension `by_deal`.
* It seems by adding this now autoformatting works again on this model! I'll tag all cases that are just because of autoformatting.
MD5 trick has been applied in `reporting.mtd_aggregated_metrics`.
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #28998
# Description
Creates a single model in KPIs, aggregated by dimension/dimension value, in a monthly basis; named: `int_kpis__agg_monthly_onboarding_mrr`.
Attention! This substitutes both current models, namely `int_monthly_onboarding_mrr_per_deal` and `int_mtd_agg_onboarding_mrr_revenue`. However, both models are currently used to retrieve data for Main KPIs. I just combined both into one, so it will simplify also the gathering of data later on.
Note that this model is special in the sense that the onboarding mrr per deal is computed for global, listing segmentation and billing country dimensions; while the total onboarding mrr is only done for global and listing segmentation (as it's based on the number of listings segmentation for the total compute).
This has been tested with dbt audit helper and md5. This has been a bit more complex since it's comparing 1 new model vs 2 existing models. For reference, this is the md5 comparison used:
```
SELECT md5(array_agg(md5((t1.*)::varchar))::varchar)
FROM (
SELECT
mrr.date,
mrr.dimension,
mrr.dimension_value,
mrr.expected_mrr_per_deal,
total.number_of_new_deals,
total.expected_mrr as total_expected_mrr
FROM intermediate.int_monthly_onboarding_mrr_per_deal mrr
left join intermediate.int_mtd_agg_onboarding_mrr_revenue total
on mrr.date = total.date
and mrr.dimension = total.dimension
and mrr.dimension_value = total.dimension_value
ORDER BY date, dimension, dimension_value
) AS t1
union all
SELECT md5(array_agg(md5((t2.*)::varchar))::varchar)
FROM (
SELECT *
FROM intermediate.int_kpis__agg_monthly_onboarding_mrr
ORDER BY date, dimension, dimension_value
) AS t2
```
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #27120, #28949
# Description
Changes:
* Adapt business_kpis_configuration to include By Business Scope as production dimension.
* Sets in int_mtd_metrics_vs_previous_year the selection of business scope dimension for all KPIs models. This does not affect cross kpis models (churn/mrr). I might need to check later how to adapt Churn to include this dimension, but it's not considered as for this PR. Lastly, Billable Bookings excludes New Dash.
* Adapts condition in int_mtd_metrics_vs_previous_year so MTD values would appear independently of these appearing in the previous year. This is, the model was considering that to show current month MTD values, the dimension needed to exist the year prior. This does not happen with New Dash and I assume we never noticed because in any case, most of our dimensions have quite a long history.
* Adapts int_kpis__agg_dates_main_kpis to include the business scope dimension. By the way it's actually handled, it kind of assumes that a Deal can only be in New or Old Dash (this is correct), but while on New Dash, this deal won't have data for Old Dash (this might not be 100% correct). In any case, the global figure should be ok, and only on the deal + business scope dimensionality this could cause some potential problems. However, this is not being reported anyway at the moment.
* Adapts int_kpis__agg_dates_main_kpis to have a proper variable value for the dimensions, and this is further included in business_kpis_configuration as any other model.
Small changes:
* Adapts Churn metrics to read from dimension_deals rather than core__deals. This should be more accurate anyway.
# Checklist
- [ ] The edited models and dependants run properly with production data.
- [ ] The edited models are sufficiently documented.
- [ ] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Propagates New Dash/Old Dash/APIs split in KPIs as per Business Scope
Related work items: #27356
# Description
Changes:
* Adds Business Scope split on Billable Bookings + propagates towards Agg/Metric Monthly/MTD models
This is a temporary modification until the ticket on Billable Bookings for New Dash is handled.
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #27356
# Description
Changes:
* Propagates business scope, based on deal, for Deal and Listing metrics. This already handles the daily metric and the daily aggregation.
* Modifies lifecycle_daily_deal to depend on dimension_deals and compute API segmentation.
* Creates new metric: Live Deals, that includes New, Active and Reactivated. This will be needed for YTD/MTD overview.
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #27356
# Description
Changes:
* Business scope is now propagated in Host Resolutions and Invoiced Revenue Monthly/MTD Metric and Aggregated models
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #27356
# Description
Changes:
* Switches dash_source to business_scope in all kpis models, schema and macro configuration.
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #27356
# Description
Changes:
* Adds Dash Source in Monthly and MTD Metric models for Verification Request dependant KPIs
* Propagates to Aggregated Monthly and MTD models for VR dependant KPIs
* Eliminates unused Weekly Guest Payment models.
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #27356
# Description
Adds new category Dash Source for Check-Out bookings KPIs
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #27356
# Description
Quick improvement to be able to report the same setup of metrics by considering all account history up to a certain date.
It adds a new computation flow for this All History window. Note that I needed to update the macro to override this case by using unbounded preceding.
I also took the opportunity to update the exposures of the new report.
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #26085
# Description
This model aggregates at monthly by deal level different metrics for AM reporting purposes. It also includes revenue retained ratios for client profitability assessment.
There's part of the existing AM report that could be simplified, likely, by using the new macro. This will be explored in a separated PR, if it applies.
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs. ** Checked and there might be possibilities to simplify the code. I'll check ones I finish this line of work**
- [X] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #25829
# Description
Creates the models for KPIs for New Dash - Chargeable metrics. In essence, computes 4 metrics:
- Chargeable Services
- Chargeable Amount (in GBP)
- Chargeable Bookings (unique over a time period and dimension, not additive)
- Chargeable Listings (unique over a time period and dimension, not additive)
This is done by creating:
- A Weekly Metric and Monthly Metric model. Here we keep the granularity of id_booking / id_accommodation to be able to compute the uniqueness.
- A Daily, Weekly and Monthly Aggregated models. Same as usual.
- Integrates everything to the existing model for Product New Dash Agg Metrics
- Exposes everything into reporting
NB: I removed on "by_host" in Created Services - I forgot to clear it out.
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #20809
# Description
This PR has 2 commits:
- The first one handles the removal from the computation any user that has not an id deal properly set. I just created a boolean field in int_core__user_host that identifies if the host has no id_deal. Then apply the new condition in the 2 main usages of New Dash info.
- The second one cleans New Dash KPIs. Since we do not have anymore users without deals, it means that the identification of the host/account is going to be exactly the same if done by id_user_host or id_deal. I hated having id_user_host in KPIs so I've removed it :)
Lastly, this should speed up massively the execution. Not because there's improvements on the code but rather the reduction of data.
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #20809
# Description
Creates a macro for beautification of categorical values.
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #20809
# Description
This PR handles the computation of KPIs for New Dash, focusing on Created Services.
New dimensions configured in `business_kpis_configuration` and applied in this new models for `NEW_DASH_CREATED_SERVICES`:
* `dim_host`,
* `dim_has_upgraded_service`,
* `dim_new_dash_version`,
* `dim_pricing_service`
New daily metric model `int_kpis__metric_daily_new_dash_created_services`
* Follows a similar pattern as for the rest of daily metric models. The only difference is that is aggregated to `id_booking` to ensure we can handle count distinct of bookings per different time granularities.
* Reads from the new pricing tables `int_core__booking_summary` and `int_core__booking_service_detail`. The main filters applied are selecting only new dash users and only services created after the user move timestamp to new dash.
An additional metric model at monthly level is created `int_kpis__metric_monthly_new_dash_created_services`
These finally go to a dimension aggregated model (`dimension`, `dimension_value`), respectively:
* Daily: `int_kpis__agg_daily_new_dash_created_services`
* Monthly: `int_kpis__agg_monthly_new_dash_created_services`
A final model aims to aggregate the different dimension aggregated metrics for New Dash: `int_kpis__product_new_dash_agg_metrics`
* It computes a `time_granularity` aggregation
* Here I will add additional metrics (such as revenue) once we have them.
A final model reading from the previous is exposed to reporting: `kpis__product_new_dash_agg_metrics`
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #20809
# Description
New model:
* int_kpis__agg_dates_main_kpis - Serves as the skeleton of dates and dimensions for Main KPIs. It's aggregated since it follows a similar aggregation strategy. It's a single model to feed both Main KPIs visualisations. Note boolean fields are real booleans (true/false) while before these were integers (1/0). This also affects downstream models.
Main KPIs flow adaptations to new skeleton model:
* int_monthly_aggregated_metrics_history_by_deal
* int_monthly_churn_metrics - additionally, calls usual KPIs macro instead of old one
* int_mtd_vs_previous_year_metrics
Reporting changes to ensure report is not down:
* mtd_aggregated_metrics - adaptations on booleans (true-1, false-0)
Cleaning:
* get_kpi_dimensions macro is no longer used
* int_dates_by_deal model and schema entry
* int_dates_mtd_by_dimension model and schema entry
* int_dates_mtd model and schema entry
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #23763
# Description
Changes:
* Creates lifecycle_daily_deal, metric_daily_deals and agg_daily_deals. These follow a different strategy due to the nature of the metrics
* Modifies the dimension macro to ensure deal dimension is included in all models except these ones
* Fixes production issue on currently deployed deal lifecycle and metrics
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #23566
# Description
Considers the 2 Claims NewDashVersion and NewDashMoveDate as sources of truth, removing the previous (crazy) logic.
If a user has the claim NewDashVersion, then it's in New Dash. The claim value will also provide the version in which the user appeared (MVP, V2, etc)
If a user has the NewDashMoveDate, it means it has moved from Old Dash. If not, but still has NewDashVersion, it means the user was directly created in New Dash.
The models now provide logic to handle these cases, and it's propagated downstream will ensuring reporting will still work.
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [NOT AT ALL] The edited models contain PK tests, and I've ran and passed them. **MANY ISSUES ON PRODUCTION CURRENTLY**
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Changes claims and logic to consider user is in new dash. Downstream propagation included
Related work items: #23457
