Commit graph

1248 commits

Author SHA1 Message Date
Joaquin Ossa
ee1bd1976a Merged PR 4086: Changed to cube instead of rollup
# Description

Changed to cube instead of rollup to get all `by_billing_country` having global segmentation.
Updated macro so instead of `By # of Listings Booked in 12 Months` is `By # of Listings`

# Checklist

- [x] The edited models and dependants run properly with production data.
- [x] The edited models are sufficiently documented.
- [x] The edited models contain PK tests, and I've ran and passed them.
- [x] I have checked for DRY opportunities with other models and docs.
- [x] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

changed to cube instead of rollup

Related work items: #26223
2025-01-16 13:45:06 +00:00
Joaquin Ossa
f3201b3c77 changed to cube instead of rollup 2025-01-16 14:21:00 +01:00
Joaquin Ossa
e98b60f335 Merged PR 4050: Onboarding MRR Model
# Description

Creates onboarding MRR Model
It contains the expected MRR by country and by number of listings per deal (according to HubSpot data) per date. The date is the last day of the month where it considers data from the previous 12 months up to date.

# Checklist

- [x] The edited models and dependants run properly with production data.
- [x] The edited models are sufficiently documented.
- [x] The edited models contain PK tests, and I've ran and passed them.
- [x] I have checked for DRY opportunities with other models and docs.
- [x] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #26223
2025-01-16 11:09:58 +00:00
Joaquin Ossa
095bfb8ab1 removed UNSET billing_countries 2025-01-16 11:35:39 +01:00
Oriol Roqué Paniagua
6aafd605b2 Merged PR 4077: Propagate account code and account name for Xero - reporting models
# Description

Propagate account code and account name for Xero reporting models, specifically:
* xero__credit_note_line_items
* xero__invoice_line_items
* xero__sales_denom_mart

These are already available in intermediate models. This is independent from KPIs, but I feel like we should propagate this to reporting since are useful fields for accounting purposes. Happy to discuss if this needs to be reflected in PBI.

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [NA] I have checked for DRY opportunities with other models and docs.
- [NA] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #26531
2025-01-16 10:04:41 +00:00
Joaquin Ossa
5bf2fba4cf Found the issue that was excluding some deals 2025-01-15 16:35:02 +01:00
Joaquin Ossa
261c6d3c49 Updated logic of model 2025-01-15 12:03:25 +01:00
Joaquin Ossa
b063454409 commit wip 2025-01-14 17:37:58 +01:00
Joaquin Ossa
1db8a050ce creates onboarding mrr model 2025-01-14 17:37:58 +01:00
Joaquin Ossa
77a70d21a4 commit wip schema 2025-01-14 17:37:58 +01:00
Joaquin Ossa
d0406e975a commit wip to rebase 2025-01-14 17:37:58 +01:00
Joaquin Ossa
92c0574374 commit uri's query 2025-01-14 17:37:58 +01:00
Joaquin Ossa
ae4f656d54 Merged PR 4060: New fields to int_kpis__dimension_deals
# Description

As suggestedin the previous PR, I'm adding new fields from HubSpot to the `int_kpis__dimension_deals`.
This includes starting date and month, cancellation date and month and segmentation by number of properties

# Checklist

- [x] The edited models and dependants run properly with production data.
- [x] The edited models are sufficiently documented.
- [x] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #26223
2025-01-14 16:35:27 +00:00
Joaquin Ossa
32251fa5e9 addressed comments 2025-01-14 17:25:59 +01:00
Joaquin Ossa
42cea78b2c fixed test value 2025-01-14 16:51:49 +01:00
Joaquin Ossa
1de5a6d33e new field to int_kpis__dimension_deals 2025-01-14 16:48:49 +01:00
Joaquin Ossa
4654aad54c wip 2025-01-14 16:46:07 +01:00
Pablo Martín
c91a27e976 Merged PR 4039: Bulk update tests keys
# Description

After our upgrade to 1.9 https://guardhog.visualstudio.com/Data/_git/data-dwh-dbt-project/pullrequest/3970, dbt has two type of tests: `unit_tests` and `data_tests`.

Before, we only had `tests` in our yml files, and this will get deprecated at some point. Now is the easiest time to rename all of those into `data_tests` so that we are already compliant with the new naming.

This PR:
- bulk updates all `tests` keys in YAML files to `data_tests`.
- updates the YAML schema we are pulling from github to valid our dbt yml files so that it follows the latest version. This is necessary to avoid VSCode throwing error-highlights for `data-tests` entries.

# Checklist

NA

# Other

NA

Related work items: #26305
2025-01-13 15:01:37 +00:00
Pablo Martin
c3c628aec4 change "tests:" to "data_tests:" 2025-01-13 16:00:35 +01:00
Pablo Martin
db1edf7a1c update schema definitions for yaml files 2025-01-13 15:57:34 +01:00
Pablo Martin
b3ba2b5fa0 Merge branch 'master' of ssh.dev.azure.com:v3/guardhog/Data/data-dwh-dbt-project 2025-01-13 15:29:54 +01:00
Pablo Martin
366127489d YOLO 3 2025-01-13 15:29:24 +01:00
Pablo Martin
c545636c64 YOLO LOLO 2025-01-13 15:28:18 +01:00
Pablo Martin
583975f7d5 YOLO 2025-01-13 15:27:52 +01:00
Pablo Martín
6e631e79b0 Merged PR 3970: bump versions to 1.9
# Description

This PR modifies the `requirements.txt` file to upgrade the versions of `dbt-core` and `dbt-postgres`.

# Checklist

NA

# Other

NA

Related work items: #20120
2025-01-13 11:36:54 +00:00
Oriol Roqué Paniagua
33d4f18165 Merged PR 4034: BUGFIX: Ensure id_role are lower case
# Description

Ensures `id_role` are lower case. This should avoid issues downstream in dbt tests.

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #26110
2025-01-13 11:27:29 +00:00
Oriol Roqué Paniagua
3fe9ac140e Merged PR 4031: Host Resolutions KPIs bugfix
# Description

Ensure only Authorised states are selected in bank transactions for Host Resolutions KPIs.

Waiting for final confirmation from Suzannah but I'm pretty sure about this since it's consistent with the P&L (and does not make sense to consider DELETED bank transactions anyway)

For reference, here's the current values in prod for `xero_host_resolution_amount_paid_in_gbp` (top3):
* 2024-12-31	-74318
* 2024-11-30	-87935
* 2024-10-31	-73840

and here the same with this fix. Note that the difference is mostly in November:
* 2024-12-31	-73581
* 2024-11-30	-74803
* 2024-10-31	-72473

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #26247
2025-01-13 09:19:24 +00:00
Oriol Roqué Paniagua
6ace9230e9 Merged PR 4020: Expose new KPIs
# Description

Exposes 2 New KPIs on Revenue Retained. Work was already done, I just removed the temporary filter and adapted the name of the metric.

# Checklist

- [ ] The edited models and dependants run properly with production data.
- [ ] The edited models are sufficiently documented.
- [ ] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Expose new KPIs

Related work items: #25804
2025-01-10 10:07:00 +00:00
Joaquin Ossa
1217c92337 Merged PR 4018: PMS data to reporting
# Description

PMS data to reporting

# Checklist

- [x] The edited models and dependants run properly with production data.
- [x] The edited models are sufficiently documented.
- [x] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

PMS data to reporting

Related work items: #26082
2025-01-10 08:57:07 +00:00
Joaquin Ossa
13bb35ec70 PMS data to reporting 2025-01-10 09:51:29 +01:00
Joaquin Ossa
f8a1742dd7 Merged PR 4017: PMS data for Account Managers Overview
# Description

PMS data added to `int_monthly_growth_score_by_deal` for Account Managers Overview

# Checklist

- [x] The edited models and dependants run properly with production data.
- [x] The edited models are sufficiently documented.
- [x] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

pms data for Account Managers Overview

Related work items: #26082
2025-01-10 08:45:19 +00:00
Joaquin Ossa
a0ee26e018 pms data for Account Managers Overview 2025-01-10 09:17:29 +01:00
Joaquin Ossa
580b194be7 Merged PR 4009: Adding PMS info for Account Margin report
# Description

Adding PMS info for Account Margin report.
I added a boolean with if they have an active PMS integration or not and the name or names of the PMSs.
I see a lot of changes made in `int_monthly_aggregated_metrics_history_by_deal_by_time_window`, probably because of the formatting when saving.

# Checklist

- [x] The edited models and dependants run properly with production data.
- [x] The edited models are sufficiently documented.
- [x] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Adding PMS info for Account Margin report
2025-01-10 07:39:06 +00:00
Joaquin Ossa
0c311d02e8 Update name of fields 2025-01-09 16:31:22 +01:00
Joaquin Ossa
bd25b473fb Adding PMS info for Account Margin report 2025-01-09 15:56:58 +01:00
Pablo Martin
198a71dee8 bump versions 2025-01-07 17:56:39 +01:00
Joaquin Ossa
04a10cf9c5 Merged PR 3968: Updating exposures
# Description

Added the missing link to exposures

# Checklist

- [ ] The edited models and dependants run properly with production data.
- [ ] The edited models are sufficiently documented.
- [ ] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Updating exposures

Related work items: #25827
2025-01-07 15:27:47 +00:00
Oriol Roqué Paniagua
0ec1484846 Merged PR 3967: Adds an "All History" time window to Account Margin
# Description

Quick improvement to be able to report the same setup of metrics by considering all account history up to a certain date.

It adds a new computation flow for this All History window. Note that I needed to update the macro to override this case by using unbounded preceding.

I also took the opportunity to update the exposures of the new report.

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #26085
2025-01-07 15:22:30 +00:00
Joaquin Ossa
9784cb6022 Updating exposures 2025-01-07 16:18:45 +01:00
Joaquin Ossa
f5e417c7db Merged PR 3957: ordered names to select oldest one created
# Description

Ordered the deals so the selected name to show is the oldest one by creation, except Xero that doesn't contain `creation_date` so instead I used `updated_date`.
I also partly added the entry in exposure, I only have the link to the power BI missing which I'll add once it is uploaded.

# Checklist

- [x] The edited models and dependants run properly with production data.
- [x] The edited models are sufficiently documented.
- [x] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

ordered names to select oldest one created

Related work items: #25827
2025-01-07 12:55:00 +00:00
Joaquin Ossa
55539ab2c9 ordered names to select oldest one created 2025-01-07 11:47:14 +01:00
Oriol Roqué Paniagua
f2abd16a44 Merged PR 3939: Fixes DWH models for new PBI on AM
# Description

Mostly 2 changes:
* Fix metrics with coalesces so values are displayed
* Removes ratios, no longer needed. These are computed in PBI directly, to be able to compute the total figure correctly

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #25829
2025-01-03 16:17:26 +00:00
Joaquin Ossa
6d261e281e Merged PR 3937: deals consolidation to reporting
# Description

Deals consolidation to reporting

# Checklist

- [x] The edited models and dependants run properly with production data.
- [x] The edited models are sufficiently documented.
- [x] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

deals consolidation to reporting

Related work items: #25827
2025-01-03 14:49:38 +00:00
Joaquin Ossa
36290f19b3 deals consolidation to reporting 2025-01-03 14:59:47 +01:00
Joaquin Ossa
c81af72602 Merged PR 3922: Id_deal consolidation model
# Description

Intermediate model that joins all `id_deal` sources into one table.
I am using the `company_name` from core, the `deal_name` from hubspot and `contact_name` from xero as the `deal_name` for all.
There are many ids that are clearly tests or deleted

# Checklist

- [x] The edited models and dependants run properly with production data.
- [x] The edited models are sufficiently documented.
- [x] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #25827
2025-01-03 13:30:48 +00:00
Joaquin Ossa
dbf807639e fixing copy paste 2025-01-03 12:00:14 +01:00
Joaquin Ossa
0aeb558a9b fixed booleans 2025-01-03 11:43:58 +01:00
Joaquin Ossa
6516a0fdc6 Updated schema with rebase 2025-01-03 11:29:58 +01:00
Joaquin Ossa
58dc823d27 Added boolean fields and counts per each deal source 2025-01-03 11:26:55 +01:00
Joaquin Ossa
35e7144efd Changed the logic for the model 2025-01-03 11:26:14 +01:00