Commit graph

179 commits

Author SHA1 Message Date
Oriol Roqué Paniagua
dc8eee7128 Merged PR 4935: KPIs Refactor Stage 4 - Onboarding MRR model in KPIs
# Description

Creates a single model in KPIs, aggregated by dimension/dimension value, in a monthly basis; named: `int_kpis__agg_monthly_onboarding_mrr`.

Attention! This substitutes both current models, namely `int_monthly_onboarding_mrr_per_deal` and `int_mtd_agg_onboarding_mrr_revenue`. However, both models are currently used to retrieve data for Main KPIs. I just combined both into one, so it will simplify also the gathering of data later on.

Note that this model is special in the sense that the onboarding mrr per deal is computed for global, listing segmentation and billing country dimensions; while the total onboarding mrr is only done for global and listing segmentation (as it's based on the number of listings segmentation for the total compute).

This has been tested with dbt audit helper and md5. This has been a bit more complex since it's comparing 1 new model vs 2 existing models. For reference, this is the md5 comparison used:

```
SELECT md5(array_agg(md5((t1.*)::varchar))::varchar)
  FROM (
        SELECT
        	mrr.date,
        	mrr.dimension,
        	mrr.dimension_value,
        	mrr.expected_mrr_per_deal,
        	total.number_of_new_deals,
        	total.expected_mrr as total_expected_mrr
          FROM intermediate.int_monthly_onboarding_mrr_per_deal mrr
          left join intermediate.int_mtd_agg_onboarding_mrr_revenue total
          on mrr.date = total.date
          and mrr.dimension = total.dimension
          and mrr.dimension_value = total.dimension_value
         ORDER BY date, dimension, dimension_value
       ) AS t1
union all
SELECT md5(array_agg(md5((t2.*)::varchar))::varchar)
  FROM (
        SELECT *
          FROM intermediate.int_kpis__agg_monthly_onboarding_mrr
         ORDER BY date, dimension, dimension_value
       ) AS t2
```

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #27120, #28949
2025-04-07 12:27:28 +00:00
Oriol Roqué Paniagua
e8a2fb1ae5 Merged PR 4926: KPIs Refactor Stage 3 - Creates dedicated Churn model in KPIs
# Description

Creates a new model in intermediate/kpis called `int_kpis__agg_monthly_churn_contribution`. This follows the convention (aggregated = by dimension and dimension value) and monthly (1 dimension value per dimension per date where date is unique in month).

This single model aims to substitute, in the future, the 2 churn models simultaneously:
* `int_monthly_12m_window_contribution_by_deal`
* `int_monthly_churn_metrics`

Main changes:
* There's no longer a dependency with `int_monthly_aggregated_metrics_history_by_deal`. Rather, it gets the metrics and dimensions from wherever needed and nothing else.
* I also took the opportunity to clean any code that was not needed after combining both models, resulting in dropping a whole CTE.
* Updated schema description for clarification purposes.

The output of the new model is identical to the 2nd model, `int_monthly_churn_metrics`, confirmed with the md5 trick. Model runs in 1.45 seconds in my laptop so no performance issue (it's technically faster than running the 2 current models which is around 6.3 seconds but whatever).

Note that this has NO effect yet on production. The switch will be handled in a separated PR.

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #28948
2025-04-04 14:02:32 +00:00
Joaquin
82d6870aba Fixed weekly model 2025-04-03 17:44:09 +02:00
Joaquin
82829db3d2 Update to listing count in new dash overview 2025-04-03 16:00:43 +02:00
Joaquin
4c011d7063 Changed loop 2025-04-02 12:55:04 +02:00
Joaquin
e8da311463 Readded schema 2025-04-02 12:07:39 +02:00
Joaquin
ab179577b1 removed macro and kept all logic inside the model 2025-04-02 12:05:01 +02:00
Joaquin
78e2474a87 Update reporting model 2025-04-02 12:05:01 +02:00
Joaquin
60dfae5cf4 Addressed comments 2025-04-02 12:04:26 +02:00
Joaquin
264ce4e203 Fixed model aggregations 2025-04-02 12:04:26 +02:00
Joaquin
2a797ce0e8 New dash bookings agg models 2025-04-02 12:04:26 +02:00
Joaquin
fb2b2def52 New created bookings with services model 2025-04-02 12:04:26 +02:00
uri
547069ede0 Adds Monthly/MTD and Aggregated/Metric models for Total and Retained Revenue 2025-04-01 17:41:56 +02:00
Oriol Roqué Paniagua
adcb8d2231 Merged PR 4874: Daily model for Total and Retained Revenue
# Description

Daily model for Total and Retained Revenue metrics.

I opted for keeping both Total Revenue alongside of Revenue Retained and Revenue Retained Post-Resolutions within the same model.

Please, verify that you're happy with the naming of `total_and_retained_revenue`, before continuing with the usual monthly/mtd metric/aggregated models.

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #28946
2025-04-01 13:44:30 +00:00
Joaquin
454d7648f6 Fixed weekly and monthly granularity 2025-03-28 11:53:32 +01:00
Joaquin Ossa
e4fb1af349 Merged PR 4833: Added current deal lifecycle to new dash overview
# Description

Added current deal lifecycle to new dash overview

# Checklist

- [x] The edited models and dependants run properly with production data.
- [x] The edited models are sufficiently documented.
- [x] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Added current deal lifecycle to new dash overview

Related work items: #28873
2025-03-28 08:20:58 +00:00
Joaquin
b138e0c626 Added boolean 2025-03-28 09:02:12 +01:00
Joaquin
7696585a6b third fix 2025-03-27 16:41:11 +01:00
Joaquin
684c233ff9 second fix 2025-03-27 16:30:22 +01:00
Joaquin
d353e852cd Fixing model 2025-03-27 14:38:53 +01:00
Joaquin
c902ae2dff Commit wip test 2025-03-27 14:27:38 +01:00
Joaquin
6287b57171 Addressed comments 2025-03-27 12:31:52 +01:00
Joaquin
83dd419e19 Aggregated models 2025-03-27 11:40:08 +01:00
Joaquin
9d99fd6088 commit wip 2025-03-27 11:37:15 +01:00
Joaquin
6dcc32b907 Commit wip 2025-03-27 11:37:15 +01:00
Joaquin
d0bc64b868 commit wip 2025-03-27 11:37:15 +01:00
Joaquin
2e984aa61b fixed tests 2025-03-27 11:37:15 +01:00
Joaquin
18988eadde Committing for testing 2025-03-27 11:37:15 +01:00
Joaquin
c60bfc6ccb Addressed comments 2025-03-27 11:24:17 +01:00
Joaquin
3f08229256 New Dash Listings with services 2025-03-26 17:01:23 +01:00
Joaquin
87517f6a92 Listings offered services 2025-03-26 10:25:14 +01:00
Joaquin
525c12dc83 Commit wip 2025-03-26 10:25:14 +01:00
Joaquin
20a8958679 Commit wip 2025-03-25 17:11:02 +01:00
Joaquin
2c5301c158 commit wip 2025-03-25 12:26:11 +01:00
Joaquin
721f3e4e9e Addressed comments 2025-03-25 12:26:11 +01:00
Joaquin
1ebd1c5ccb Created new kpi daily model for new dash users 2025-03-25 12:26:11 +01:00
Pablo Martín
36e6a35d12 Merged PR 4753: Add sqlfluff check
# Description

Adds another check to CI that validates that compiled SQL is valid.

Related work items: #28628
2025-03-21 10:10:44 +00:00
Pablo Martin
6a38a4ec48 modify ugly name 2025-03-20 17:03:13 +01:00
Oriol Roqué Paniagua
098ab51439 Merged PR 4757: Creates a Total Booking Fees metric (Old dash + New dash)
# Description

Main Changes:
* Computes the total booking fees in `int_kpis__metric_daily_invoiced_revenue`
* Propagates within KPIs, including schema and tests
* Propagates within cross in both "by deal" and "by dimension/global"
* Propagates to main kpis tests

Small fix:
* In `int_mtd_vs_previous_year_metrics`, manual formatting because sqlfmt is broken on this model...

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #28560
2025-03-20 13:45:44 +00:00
Oriol Roqué Paniagua
f799a8d30f Merged PR 4727: Bookings fees are now Old Dashboard Booking Fees in Main KPIs
# Description

Booking fees is widely used with different meanings, for old dash, for new dash, for both, etc. This is painful. First step to align on a proper naming is ensure that what we report in Main KPIs is clearly stated, which in this case, Booking Fees are now called Old Dashboard Booking Fees.

Changes:
* Modify `stg_seed__accounting_aggregations` seed to rename Booking Fees to Old Dashboard Booking Fees. This is for us to clarify. This is only applied for KPIs compute. I also added an empty space that I mistakenly forgot in the past for `financial_l3_aggregation`.
* Modify KPIs source, i.e., `int_kpis__metric_daily_invoiced_revenue`. Here I forcefully modify the name of the field to `xero_old_dashboard_booking_net_fees_in_gbp`.
* Propagate changes of downstream usages of `xero_booking_net_fees_in_gbp` to `xero_old_dashboard_booking_net_fees_in_gbp`. This affects all models, including the reporting model. On this one we still have both names to avoid breaking it. I will need to modify the data glossary in PBI anyway so I'll do this change as well.
* Modify displayed metric name from Booking Fees Revenue to Old Dashboard Booking Fees Revenue.
* Modify schema so it reflects the proper names, descriptions, and tests.
* Ensure outlier and completion tests still work after this change.

I confirm the field `xero_booking_net_fees_in_gbp` does not exist anymore in the rest of DWH after these changes, except for the abovementioned comment on the reporting line.

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #28560
2025-03-18 14:55:32 +00:00
Joaquin
e9cfb6c2bd Addressed comments 2025-03-14 14:20:57 +01:00
Joaquin
c33253b10a addressed comments 2025-03-14 11:19:22 +01:00
Joaquin
525720ea0a Changes in deals lifecycle 2025-03-14 10:39:01 +01:00
Joaquin
e827c0f9d5 Updated schema 2025-03-13 17:31:15 +01:00
Joaquin
a2b3b7fa6b Updated schema 2025-03-13 17:29:24 +01:00
Joaquin
d6426b0066 change in new deals 2025-03-13 17:19:34 +01:00
Joaquin
1ced79ca0a commit wip 2025-03-13 15:48:15 +01:00
Oriol Roqué Paniagua
9ca9c827fe Merged PR 4665: Projected Bookings to the end of the current month
# Description

Computes projected Bookings to the end of the current month per dimension and dimension value

KPIs changes:
* Adds a int_kpis__agg_daily_created_bookings model. This is a standard KPIs model that computes created bookings at a daily level per dimension and dimension value.

Projected KPIs (NEW):
* Adds a new folder named projected within KPIs. This includes a simple Readme.
* Adds a default dimension dates model that ranges from the past 3 complete months to the end of the current month, that is in the future.
* Adds 2 projected models for created bookings: The daily one, that handles the logic of the projection (same month + last 7 days) and the monthly one, that retrieves the end of month projected value as well as some indicators of how good or bad this was for previous months.

How good is the projection at 11th of March (so 10 days in the month)?

`select * from intermediate.int_kpis_projected__agg_monthly_created_bookings order by actual_created_bookings desc`

![image.png](https://guardhog.visualstudio.com/4148d95f-4b6d-4205-bcff-e9c8e0d2ca65/_apis/git/repositories/54ac356f-aad7-46d2-b62c-e8c5b3bb8ebf/pullRequests/4665/attachments/image.png)

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #28251
2025-03-11 14:34:30 +00:00
Oriol Roqué Paniagua
1a4a9fac37 Merged PR 4578: Propagates the month attribution from invoicing data
# Description

Following the changes of the month attribution on invoicing data depending if the document is posted on Xero or in Hyperline, this PR ensures that the 2 affected areas capture this change.

This means:
* Xero - Sales Monthly Trends
* KPIs - Invoiced Revenue

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #28158
2025-03-04 14:13:53 +00:00
Joaquin Ossa
da0b0dad67 Updated all service_business_type tests 2025-02-28 11:14:40 +01:00