# Description
Changes:
* 4 new models in the scope of KPIs for Billable Verifications from APIs. I believe it's more correct to say these are Billable Verifications than Billable Bookings since there's some cases in which a Booking can be duplicated and it's billed multiple times. These include:
* A daily metric model - extremely simple. You will notice there's no Billing Country not Listing Segmentation. This is because for ALL API cases this is UNSET, thus, I just remove it.
* An equivalent monthly metric model.
* Two aggregated models per dimension, dimension value: on a daily and a monthly basis.
Important change: the macro that handles the aggregations sets by default Billing Country and Listing Segmentation. I modified a bit the flow so the only required dimension is Global, and these are skipped for APIs models.
This is needed for the changes intended on the Growth score. In there, I'll combine both Platform Billable Bookings with API Billable Verifications.
Notice there's no MTD models. These could be added for sure; but since I'm not creating any metric in Main KPIs or similar, I opted to skip it for now. It can be done later on.
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #29374
# Description
Switches Created Bookings to Billable Bookings in the scope of Projected Bookings. I opted for hard switching (thus removing Projected Created Bookings) altogether to avoid having unused models in production.
This aims to set the ground to include, in the future, APIs Billable Bookings.
Impact in KPIs:
* `int_kpis__agg_daily_created_bookings` now is `int_kpis__agg_daily_billable_bookings`. Aside from the name change, it's reduced to 4 metrics to just one. Schema changes also apply.
Impact in Projected KPIs:
* `int_kpis_projected__agg_daily_created_bookings` now is `int_kpis_projected__agg_daily_billable_bookings`. Fields are also renamed accordingly.
* `int_kpis_projected__agg_monthly_created_bookings` now is `int_kpis_projected__agg_monthly_billable_bookings`. Fields are also renamed accordingly.
* Schema file is also updated accordingly.
Impact on new model for Growth Score:
* `int_created_bookings_growth_score_by_deal` now is `int_billable_bookings_growth_score_by_deal`. Fields are also renamed accordingly.
* Schema is also updated accordingly.
Note that since the end model `int_billable_bookings_growth_score_by_deal` is not used at the moment in reporting, there's no user-facing impact. Also, the rest of modified models are not used for other purposes.
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #29374
# Description
As raised by Kayla, Retained metrics are not computed correctly in AM Account Margin report, see screenshot:
Issue lies in the fact that Guest Payments contains more dimensions than the rest of the models. This is generating duplicates in the combination_of_sources CTE, but these are effectively hidden because of the sum() applied of the metrics.
In the case of a Deal having in the same day Guest Payments with different dimensions from those selected, AND on the same day having a value on Invoiced Revenue or Host Resolutions, then the Invoiced Revenue or Host Resolutions would be duplicated.
This PR aims to fix this issue, and also adds a few tests on the base model for AM Account Margin to ensure this is kind of error raises an alert.
I'll be debugging other affected models and increase the robustness test-wise; as well as gathering the full impact.
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #29227
# Description
Creates a single model in KPIs, aggregated by dimension/dimension value, in a monthly basis; named: `int_kpis__agg_monthly_onboarding_mrr`.
Attention! This substitutes both current models, namely `int_monthly_onboarding_mrr_per_deal` and `int_mtd_agg_onboarding_mrr_revenue`. However, both models are currently used to retrieve data for Main KPIs. I just combined both into one, so it will simplify also the gathering of data later on.
Note that this model is special in the sense that the onboarding mrr per deal is computed for global, listing segmentation and billing country dimensions; while the total onboarding mrr is only done for global and listing segmentation (as it's based on the number of listings segmentation for the total compute).
This has been tested with dbt audit helper and md5. This has been a bit more complex since it's comparing 1 new model vs 2 existing models. For reference, this is the md5 comparison used:
```
SELECT md5(array_agg(md5((t1.*)::varchar))::varchar)
FROM (
SELECT
mrr.date,
mrr.dimension,
mrr.dimension_value,
mrr.expected_mrr_per_deal,
total.number_of_new_deals,
total.expected_mrr as total_expected_mrr
FROM intermediate.int_monthly_onboarding_mrr_per_deal mrr
left join intermediate.int_mtd_agg_onboarding_mrr_revenue total
on mrr.date = total.date
and mrr.dimension = total.dimension
and mrr.dimension_value = total.dimension_value
ORDER BY date, dimension, dimension_value
) AS t1
union all
SELECT md5(array_agg(md5((t2.*)::varchar))::varchar)
FROM (
SELECT *
FROM intermediate.int_kpis__agg_monthly_onboarding_mrr
ORDER BY date, dimension, dimension_value
) AS t2
```
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #27120, #28949
# Description
Creates a new model in intermediate/kpis called `int_kpis__agg_monthly_churn_contribution`. This follows the convention (aggregated = by dimension and dimension value) and monthly (1 dimension value per dimension per date where date is unique in month).
This single model aims to substitute, in the future, the 2 churn models simultaneously:
* `int_monthly_12m_window_contribution_by_deal`
* `int_monthly_churn_metrics`
Main changes:
* There's no longer a dependency with `int_monthly_aggregated_metrics_history_by_deal`. Rather, it gets the metrics and dimensions from wherever needed and nothing else.
* I also took the opportunity to clean any code that was not needed after combining both models, resulting in dropping a whole CTE.
* Updated schema description for clarification purposes.
The output of the new model is identical to the 2nd model, `int_monthly_churn_metrics`, confirmed with the md5 trick. Model runs in 1.45 seconds in my laptop so no performance issue (it's technically faster than running the 2 current models which is around 6.3 seconds but whatever).
Note that this has NO effect yet on production. The switch will be handled in a separated PR.
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #28948
# Description
Daily model for Total and Retained Revenue metrics.
I opted for keeping both Total Revenue alongside of Revenue Retained and Revenue Retained Post-Resolutions within the same model.
Please, verify that you're happy with the naming of `total_and_retained_revenue`, before continuing with the usual monthly/mtd metric/aggregated models.
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #28946
# Description
Added current deal lifecycle to new dash overview
# Checklist
- [x] The edited models and dependants run properly with production data.
- [x] The edited models are sufficiently documented.
- [x] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Added current deal lifecycle to new dash overview
Related work items: #28873
# Description
Main Changes:
* Computes the total booking fees in `int_kpis__metric_daily_invoiced_revenue`
* Propagates within KPIs, including schema and tests
* Propagates within cross in both "by deal" and "by dimension/global"
* Propagates to main kpis tests
Small fix:
* In `int_mtd_vs_previous_year_metrics`, manual formatting because sqlfmt is broken on this model...
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #28560
# Description
Booking fees is widely used with different meanings, for old dash, for new dash, for both, etc. This is painful. First step to align on a proper naming is ensure that what we report in Main KPIs is clearly stated, which in this case, Booking Fees are now called Old Dashboard Booking Fees.
Changes:
* Modify `stg_seed__accounting_aggregations` seed to rename Booking Fees to Old Dashboard Booking Fees. This is for us to clarify. This is only applied for KPIs compute. I also added an empty space that I mistakenly forgot in the past for `financial_l3_aggregation`.
* Modify KPIs source, i.e., `int_kpis__metric_daily_invoiced_revenue`. Here I forcefully modify the name of the field to `xero_old_dashboard_booking_net_fees_in_gbp`.
* Propagate changes of downstream usages of `xero_booking_net_fees_in_gbp` to `xero_old_dashboard_booking_net_fees_in_gbp`. This affects all models, including the reporting model. On this one we still have both names to avoid breaking it. I will need to modify the data glossary in PBI anyway so I'll do this change as well.
* Modify displayed metric name from Booking Fees Revenue to Old Dashboard Booking Fees Revenue.
* Modify schema so it reflects the proper names, descriptions, and tests.
* Ensure outlier and completion tests still work after this change.
I confirm the field `xero_booking_net_fees_in_gbp` does not exist anymore in the rest of DWH after these changes, except for the abovementioned comment on the reporting line.
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #28560
