Commit graph

1915 commits

Author SHA1 Message Date
Pablo Martin
229ab901d1 restrict list to models, tests and exposures 2025-03-21 15:02:31 +01:00
Pablo Martin
5d92c43215 add ls step 2025-03-21 14:38:48 +01:00
Pablo Martin
40f28a9411 remove master trigger 2025-03-21 14:25:36 +01:00
Pablo Martin
e0fbf093b8 undo stupid change 2025-03-21 12:33:06 +01:00
Pablo Martin
395c064365 edit one model to test state detection 2025-03-21 12:33:06 +01:00
Pablo Martin
bc7ea8d001 make dbt docs a bit more efficient, hopefully 2025-03-21 12:33:06 +01:00
Pablo Martin
14b927b8d4 typo 2025-03-21 12:33:06 +01:00
Pablo Martin
d26c70bba4 add steps in ci 2025-03-21 12:33:06 +01:00
Oriol Roqué Paniagua
8924778a52 Merged PR 4770: Same freshness in MTD and YTD/target for Resolutions Payouts
# Description

Ensures Resolutions Payouts (in amount paid) is timely

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Same freshness in MTD and YTD/target for Resolutions Payouts

Related work items: #28560
2025-03-21 11:10:26 +00:00
Pablo Martín
36e6a35d12 Merged PR 4753: Add sqlfluff check
# Description

Adds another check to CI that validates that compiled SQL is valid.

Related work items: #28628
2025-03-21 10:10:44 +00:00
Oriol Roqué Paniagua
98c40d21d2 Merged PR 4769: Refactors display exclusion in main metrics
# Description

I noticed this morning that after the name changes some exclusions have been messed up.

I think is about time we handle this properly. Following what we did for `ytd_mtd_aggregated_main_metrics_overview`, I apply the same logic: the metric display configuration is handled for each metric in the configuration rather than by some crazy-complex-name-logic.

I also took the opportunity to make Host Resolutions timely after a discussion with Chloe yesterday. Finance handles resolutions payments twice a week, and the "delay" in time is mostly coming from Resolutions accepting to pay someone vs. Finance handling the payment. In any case we're talking about a few days maximum difference, which I believe it's 1) process-related freshness, rather than data-related and 2) much better to have visibility in a timely manner rather than waiting for the 20th on next month.

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs. **Logic is common among YTD and MTD models. At some moment this can be improved**
- [X] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #28560
2025-03-21 09:40:20 +00:00
Oriol Roqué Paniagua
240d6ec59e Merged PR 4763: Align metrics within KPIs report
# Description

There's misalignment in the different areas in the business, but even within our own report...

**I'm not saying this is perfect** - but at least it forces a common Data Glossary within Main KPIs. I'd suggest later on reviewing naming - same as we need to do for revenue anyway -, but for the meantime, at least have consistency on our side.

Changes:
* Est. Billable Bookings -> Billable Bookings. We have other metrics stated as Estimated in the Data Glossary.
* Waiver and Resolutions payments to host are now called Payouts. This is not perfect but at least is clear we're paying out, so it's a cost.
* Host Resolutions Payment Rate is now explicitly mentioning how it's being computed to avoid confusion with the Payout Rate, that does not appear here yet, but appears in the YTD.

Additional Changes:
* Tests are also aligned with new names.
* Re-order booking display, so Cancelled Bookings (inclusion and exclusion) are shown below. Having them the first is weird.

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #28560
2025-03-20 17:00:36 +00:00
Pablo Martin
6a38a4ec48 modify ugly name 2025-03-20 17:03:13 +01:00
Oriol Roqué Paniagua
6b9434355a Merged PR 4759: Adds Booking Fee per Billable Booking
# Description

Changes:
* Creates Booking Fees in YTD/MTD
* Creates Booking Fees per Billable Booking in YTD/MTD. This is exposed forward.
* Removes code for Onboarding MRR in YTD/MTD.

Additional fixes:
* Small mistake - Revenue Churn is now considered as NEGATIVE

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #28560
2025-03-20 16:01:27 +00:00
Pablo Martin
61007cff09 exclude sources as well 2025-03-20 16:55:23 +01:00
Pablo Martin
a56a96d1c4 exclude test generated sql files 2025-03-20 16:49:31 +01:00
Oriol Roqué Paniagua
098ab51439 Merged PR 4757: Creates a Total Booking Fees metric (Old dash + New dash)
# Description

Main Changes:
* Computes the total booking fees in `int_kpis__metric_daily_invoiced_revenue`
* Propagates within KPIs, including schema and tests
* Propagates within cross in both "by deal" and "by dimension/global"
* Propagates to main kpis tests

Small fix:
* In `int_mtd_vs_previous_year_metrics`, manual formatting because sqlfmt is broken on this model...

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #28560
2025-03-20 13:45:44 +00:00
Oriol Roqué Paniagua
93b31854f7 Merged PR 4754: Updates targets FY25 & FY26
# Description

Main changes:
* Modifies target seeds to include FY2025 and FY2026. This includes now absolute values for equivalent rate metrics. This also includes Booking Fees per Billable Booking, despite this metric is still not computed. This will be done in a separated PR.
* Removes Onboarding MRR as this requirement is no longer needed, and there has not been any target computed for this metric. This is modified to include Revenue Churn as an absolute figure.

Small note. Targets for FY 2025 rely on the actuals from Finance. Up to January 2025 included, these should match with our YTD values, but it's not the case due to small differences in Billable Bookings / Deals; as well as accrued revenue. Accrued revenue is very troublesome specially on Guesty side, since this affects Total Revenue, and this affects also Resolutions Payout Rate, and so on...

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #28560
2025-03-20 13:10:22 +00:00
Pablo Martin
fa914e8e26 try with more cores 2025-03-20 13:17:05 +01:00
Pablo Martin
00ea2ff146 parallelize with xargs 2025-03-20 12:51:04 +01:00
Pablo Martin
2dd13e84d7 add steps in ci pipeline 2025-03-20 12:39:43 +01:00
Pablo Martin
478e953700 add sqlfluff dep and script and config 2025-03-20 12:37:47 +01:00
Pablo Martín
a4965ed9b2 Merged PR 4752: Move pipeline YAML
# Description

Starts a new `ci` folder and places the CI pipeline YAML definition file there.

Related work items: #28628
2025-03-20 11:30:12 +00:00
Pablo Martin
48528033d9 move file there 2025-03-20 12:24:40 +01:00
Pablo Martin
daea8d229b start ci folder 2025-03-20 12:24:32 +01:00
Joaquin Ossa
5d28f787a6 Merged PR 4743: Filtered test accounts
# Description

Filtered test accounts according to `Account_type` in Cosmos
Unfortunately this doesn't actually filter all test accounts, there are some that are correctly classified and I will have to ask specifically to either remove or classify correctly

# Checklist

- [x] The edited models and dependants run properly with production data.
- [x] The edited models are sufficiently documented.
- [x] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Filtered test accounts

Related work items: #28239
2025-03-20 08:36:49 +00:00
Joaquin
d06f58b76e Removed test users from staging 2025-03-20 09:30:33 +01:00
Joaquin Ossa
9e59fa4933 Merged PR 4723: Adding id_deal
# Description

Added `id_deal` to edeposit models for Invoice reporting

# Checklist

- [x] The edited models and dependants run properly with production data.
- [x] The edited models are sufficiently documented.
- [x] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.

# Other

- [x] Check if a full-refresh is required after this PR is merged.

Adding id_deal

Related work items: #28239
2025-03-19 15:50:37 +00:00
Joaquin
a85033922c Filtered test accounts 2025-03-19 16:50:19 +01:00
Pablo Martín
c44a099751 Merged PR 4735: Test PR for CI stuff
You can ignore me, I'm not important.
2025-03-19 14:34:26 +00:00
Pablo Martin
918d618f75 adjust naming in deletion step 2025-03-19 15:26:50 +01:00
Pablo Martin
d334e95738 typo 2025-03-19 15:23:59 +01:00
Pablo Martin
841b64dc85 unfuck variables 2025-03-19 15:19:27 +01:00
Pablo Martin
1e19efc89f make temporal db name selection a bit smarter 2025-03-19 15:17:33 +01:00
Joaquin
bcdb39849d Addressed comments 2025-03-19 12:06:05 +01:00
Joaquin Ossa
cef363847a Merged PR 4736: adding id_deal to api models
# Description

Adding `id_deal` to API models in Check in Cover and Screen & Protect

# Checklist

- [x] The edited models and dependants run properly with production data.
- [x] The edited models are sufficiently documented.
- [x] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #28239
2025-03-19 10:33:49 +00:00
Pablo Martin
821c7d2850 remove pr stuff because it's fake news 2025-03-19 09:26:03 +01:00
Joaquin
5d3f2f6908 adding id_deal to api models 2025-03-19 09:23:23 +01:00
Pablo Martin
4b33114d67 modify pipeline 2025-03-19 09:19:30 +01:00
Oriol Roqué Paniagua
5928a198b6 Merged PR 4734: Bugfix - Ensures accounts cannot be duplicated in Xero sales monthly trends
# Description

This fixes the duplication error when the same deal has more than one contact. Impact was not massive, it just affected because of the Home to Host account and this has happened during the past week. There were other cases that contained duplicates but these were not invoiced so it didn't affect the total sum.

Changes:
* Contacts now has a boolean field indicating if a deal exists.
* Contacts now has a boolean field indicating if that contact has a deal informed and it's the last updated one. This is opinionated - it could have been done with creation, for instance.
* Sales monthly trends forces the last updated contact to be true.
* Improved robustness by adding tests.

Note that modifying the logic in intermediate on `int_xero__sales_monthly_trends` by changing `id_deal` per `id_contact` would not have worked - we need to do this compute by deal to ensure that any invoicing is reported MoM and YTD.

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #28576
2025-03-19 08:19:20 +00:00
Pablo Martin
7cc893cad4 silly change just to trigger PR 2025-03-19 09:14:26 +01:00
Pablo Martin
c00d02685d add dbt compile step 2025-03-18 18:12:53 +01:00
Pablo Martin
a1ed622bee build docs 2025-03-18 18:02:28 +01:00
Pablo Martin
43132c41b5 typo 2025-03-18 17:51:51 +01:00
Pablo Martin
074ac0e2ce install dbt thingies in CI 2025-03-18 17:48:49 +01:00
Pablo Martin
f4769c2e55 delete database 2025-03-18 17:38:55 +01:00
Pablo Martin
8c7681581c typo 2025-03-18 17:35:07 +01:00
Pablo Martin
32ef6f36a4 create database 2025-03-18 17:33:49 +01:00
Pablo Martin
65e2dea2fc smoke test with env var sourcing included 2025-03-18 17:31:05 +01:00
Pablo Martin
74222a0007 smoke test postgres connection 2025-03-18 17:26:20 +01:00