# Description
Removes old onboarding mrr models, that are no longer used.
dbt compiles correctly and no deprecated warning is showing anymore.
This finishes the refactor
Related work items: #28949
# Description
Puts in production the new KPIs model for Onboarding MRR.
The change has been validated with MD5 trick.
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #28949
# Description
Creates a single model in KPIs, aggregated by dimension/dimension value, in a monthly basis; named: `int_kpis__agg_monthly_onboarding_mrr`.
Attention! This substitutes both current models, namely `int_monthly_onboarding_mrr_per_deal` and `int_mtd_agg_onboarding_mrr_revenue`. However, both models are currently used to retrieve data for Main KPIs. I just combined both into one, so it will simplify also the gathering of data later on.
Note that this model is special in the sense that the onboarding mrr per deal is computed for global, listing segmentation and billing country dimensions; while the total onboarding mrr is only done for global and listing segmentation (as it's based on the number of listings segmentation for the total compute).
This has been tested with dbt audit helper and md5. This has been a bit more complex since it's comparing 1 new model vs 2 existing models. For reference, this is the md5 comparison used:
```
SELECT md5(array_agg(md5((t1.*)::varchar))::varchar)
FROM (
SELECT
mrr.date,
mrr.dimension,
mrr.dimension_value,
mrr.expected_mrr_per_deal,
total.number_of_new_deals,
total.expected_mrr as total_expected_mrr
FROM intermediate.int_monthly_onboarding_mrr_per_deal mrr
left join intermediate.int_mtd_agg_onboarding_mrr_revenue total
on mrr.date = total.date
and mrr.dimension = total.dimension
and mrr.dimension_value = total.dimension_value
ORDER BY date, dimension, dimension_value
) AS t1
union all
SELECT md5(array_agg(md5((t2.*)::varchar))::varchar)
FROM (
SELECT *
FROM intermediate.int_kpis__agg_monthly_onboarding_mrr
ORDER BY date, dimension, dimension_value
) AS t2
```
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #27120, #28949
# Description
Tags as deprecated the following models:
* `int_monthly_onboarding_mrr_per_deal`
* `int_mtd_agg_onboarding_mrr_revenue`
Project compiles.
Related work items: #28949
# Description
Removes the models:
* int_monthly_12m_window_contribution_by_deal
* int_monthly_churn_metrics
as well as their entries in the schema files.
Project compiles and KPIs run works.
This closes stage 3 of the refactor.
# Checklist
- [ ] The edited models and dependants run properly with production data.
- [ ] The edited models are sufficiently documented.
- [ ] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Remove previous Churn models
Related work items: #28948
# Description
Small but important change.
Churn metrics will be propagated in production by sourcing intermediate/kpis rather than the cross models.
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #28948
# Description
Creates a new model in intermediate/kpis called `int_kpis__agg_monthly_churn_contribution`. This follows the convention (aggregated = by dimension and dimension value) and monthly (1 dimension value per dimension per date where date is unique in month).
This single model aims to substitute, in the future, the 2 churn models simultaneously:
* `int_monthly_12m_window_contribution_by_deal`
* `int_monthly_churn_metrics`
Main changes:
* There's no longer a dependency with `int_monthly_aggregated_metrics_history_by_deal`. Rather, it gets the metrics and dimensions from wherever needed and nothing else.
* I also took the opportunity to clean any code that was not needed after combining both models, resulting in dropping a whole CTE.
* Updated schema description for clarification purposes.
The output of the new model is identical to the 2nd model, `int_monthly_churn_metrics`, confirmed with the md5 trick. Model runs in 1.45 seconds in my laptop so no performance issue (it's technically faster than running the 2 current models which is around 6.3 seconds but whatever).
Note that this has NO effect yet on production. The switch will be handled in a separated PR.
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #28948
# Description
Extracts Integration Type (OSL, PMS, AutoHost, other API stuff) from Hubspot Deals.
This is needed for the categorisation of Deals in the scope of Old Dash to New Dash migration.
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #28987
# Description
According to our previous conversation, modified the values for active listings and how they will be shown on the report's funnel.
_This changes will crash the report for now, I was thinking on adding a **work in progress** warning to both Overview and User Details tab in the meantime. After this the report should be soon ready_
Now we are going to have:
- **total_listings**
- **total_active_listings**
- **total_active_listings_with_active_product_bundle**
- **total_active_listings_with_active_product_bundle_with_paid_service**
Also fixed an aggregation mishap I had with the created bookings aggregated models
# Checklist
- [x] The edited models and dependants run properly with production data.
- [x] The edited models are sufficiently documented.
- [x] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Update to listing count in new dash overview
Related work items: #28640
# Description
Churn models being tagged as deprecated for next week, for information.
# Checklist
- [ ] The edited models and dependants run properly with production data.
- [ ] The edited models are sufficiently documented.
- [ ] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Deprecation of Churn Models
Related work items: #28948
# Description
Decouples Churn/Account Margin model of `int_monthly_aggregated_metrics_history_by_deal_by_time_window` from the model `int_monthly_aggregated_metrics_history_by_deal`.
Changes:
* Removes dependency and reads from KPIs models.
* Removes unused deal stage variable
* Jinja template for time windows with dynamic selection to DRY
This has been tested with dbt audit tools and the md5 for pure refactoring, so output is exactly the same.
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #28947
# Description
Starting 2nd stage of the refactor so Account Managers stops depending on int_monthly_aggregated_metrics_history_by_deal.
This has been tested with the audit and the differences only happen with the rank of cases that are equal (ex: there's 2 deals with 0 revenue, then who is the 1st and the 2nd is random).
There are other possibilities of refactoring here (including hubspot deals info in int_kpis__dimension_deals, for instance) but I feel like this is a separated line of work if we want to.
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #28947
# Description
Changes:
* If metric is null don't compute test.
* Use 1 year data to compare against.
* For debugging purposes, compute also the amount of days used in the comparison.
The metric is not null is a necessary feature to avoid raising errors while there's no Host Resolutions in month. Keep in mind the test will keep failing and this is EXPECTED because of a massive increase in New Listings:
# Checklist
- [X] The edited models and dependants run properly with production data.
- [ ] The edited models are sufficiently documented.
- [ ] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
If metric is null don't compute test. 1y data to compare against.
# Description
Adds audit helper.
Removes computation of Total and Retained Revenue from cross models.
I've tested audit helper and the trick with the hashes, as discussed in the demo.
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them. **Except for today's outlier detector!**
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #28946
# Description
New kpis model for created bookings in new dash.
I also correctly updated the joins in `int_kpis__product_new_dash_agg_metrics`
# Checklist
- [x] The edited models and dependants run properly with production data.
- [x] The edited models are sufficiently documented.
- [x] The edited models contain PK tests, and I've ran and passed them.
- [x] I have checked for DRY opportunities with other models and docs.
- [x] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
# Description
Adds 4 models for Total and Retained Revenue:
* 2 Metric models on Monthly and MTD
* 2 Aggregated models on Monthly and MTD
This follows the standard for KPIs. First manual checks vs data look ok, although I'll run the proper test analysis with the dbt audit tool in the next PR when merging into the current production tables.
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #28946
# Description
Fixes Blank stuff
# Checklist
- [ ] The edited models and dependants run properly with production data.
- [ ] The edited models are sufficiently documented.
- [ ] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Fix Business Targets
Related work items: #28958
# Description
Adding total_listings_with_active_product_bundle_with_paid_service which weirdly enough it was already on the schemas but not in the model.
This is the same value that's going to be displayed on the new offered services tab
# Checklist
- [x] The edited models and dependants run properly with production data.
- [x] The edited models are sufficiently documented.
- [x] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Adding total_listings_with_active_product_bundle_with_paid_service
Related work items: #28640
# Description
Daily model for Total and Retained Revenue metrics.
I opted for keeping both Total Revenue alongside of Revenue Retained and Revenue Retained Post-Resolutions within the same model.
Please, verify that you're happy with the naming of `total_and_retained_revenue`, before continuing with the usual monthly/mtd metric/aggregated models.
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #28946
# Description
Removes incrementality in booking view to service
# Checklist
- [X] The edited models and dependants run properly with production data.
- [ ] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.
# Other
- [X] Check if a full-refresh is required after this PR is merged.
# Description
Removing not_null test for `booking_source`
# Checklist
- [x] The edited models and dependants run properly with production data.
- [x] The edited models are sufficiently documented.
- [x] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
removed test of not_null
# Description
Fixed weekly and monthly granularity where previously I wasn't filtering `end_of_week` or `end_of_month` dates for these granularities
# Checklist
- [x] The edited models and dependants run properly with production data.
- [x] The edited models are sufficiently documented.
- [x] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Fixed weekly and monthly granularity
Related work items: #28640
# Description
Added current deal lifecycle to new dash overview
# Checklist
- [x] The edited models and dependants run properly with production data.
- [x] The edited models are sufficiently documented.
- [x] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Added current deal lifecycle to new dash overview
Related work items: #28873
