# Description
This PR fixes the New Dash migration issue that happened on September 10th 2024. In this migration, users were directly assigned the claim of KygMvp that does not contain a date value. We were using a default hardcode of the first MVP migration, thus in DWH all users have been considered to be migrated late July instead of splitting the first 22 in late July and the ~200 others in September.
The issue lies in the fact that users have configured a ProductBundle and can have Bookings with ProductBundle BEFORE the migration date, which greatly breaks the logic of a migration monitoring.
Changes:
* New migration phase added based on the claim MvpMigratedUser, that Ben created on Friday 13th
* Adaptation of the code in int_core__user_migration to detect if the claim_value (a text field) has a date or not. If so, use that date as long as it's equal or greater than the deployment date, if not use the deployment date. If the claim does not contain a date, use the deployment date (this is the case for the first true 22 migrated users)
I checked that volumes now look correct with this fix.
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #20773
# Description
removing duplicates from guesty and removing tests for edeposit_users and edeposit_verification_fees until further notice
# Checklist
- [x] The edited models and dependants run properly with production data.
- [x] The edited models are sufficiently documented.
- [x] The edited models contain PK tests, and I've ran and passed them.
- [x] I have checked for DRY opportunities with other models and docs.
- [x] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
removing duplicates from guesty and removing tests for edeposit_users
Related work items: #20125
# Description
This model joins a couple of models to `int_core__verification_payments` and adds a few fields around taxs (amounts with and without taxes, tax rates, etc).
## __ATTENTION!!!__
This PR uses `dbt`'s [Model Versioning](https://docs.getdbt.com/docs/collaborate/govern/model-versions) and [Deprecation](https://docs.getdbt.com/reference/resource-properties/deprecation_date) features for the first time. Notice the extra, non-usual fluff in the `schema.yml` file, along with the fact that some model files are suffixed with versions.
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- _Note that there are a couple of tests that are deactivated (commented out) due to bad data in the backend. Engineering team is working on fixiing that data. I've decided to comment this for now so we can move forward, but I won't be marking the story on this done until the tests are live in production._
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #20043
# Description
This PR:
- Adds some `vscode` settings that make VSCode autoformat yaml files on save, like we do with `.sql` files. This should help with formatting consistency.
- I also made some notes about this on the readme.
- Finally, I made a couple of tiny unrelated changes on the readme since I was there.
Related work items: #21344
# Description
Exposes the 2 new deal fields to reporting to make it accessible for PBI
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Exposing Deal info to reporting
Related work items: #18911, #19083
# Description
Changes (only in intermediate):
* Applies sqlfmt in KPIs source models (for some of them it was already applied). Specifically, the 3 Core models ONLY contains formatting changes
* Adds `main_deal_name` and `main_billing_country_iso_3_per_deal` in `int_monthly_aggregated_metrics_history_by_deal`
* Adds the 2 new fields in the schema entry of `int_monthly_aggregated_metrics_history_by_deal`, including the dbt test not null in the deal name.
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #18911, #19083
# Description
Changes:
* Adding `main_deal_name` and `main_billing_country_iso_3_per_deal` in `int_dates_by_deal` model.
* Documents the 2 new fields. Also, ensures `main_deal_name` is not null
* Removes `id_deal not null` condition since it's enforced on the inner join with `int_core__deal`
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #18911, #19083
# Description
Fixed model that wasn't mapping correctly because I am dumb
# Checklist
- [x] The edited models and dependants run properly with production data.
- [x] The edited models are sufficiently documented.
- [x] The edited models contain PK tests, and I've ran and passed them.
- [x] I have checked for DRY opportunities with other models and docs.
- [x] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Fixed model that wasn't mapping correctly
Related work items: #20125
# Description
Created a mapping for some users that have a different id in Cosmos so we can join them using data that Ray shared with me, with this we can get all currency information for each user
# Checklist
- [x] The edited models and dependants run properly with production data.
- [x] The edited models are sufficiently documented.
- [x] The edited models contain PK tests, and I've ran and passed them.
- [x] I have checked for DRY opportunities with other models and docs.
- [x] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #20125
# Description
This PR is a pure refactor (as in, doesn't change the output of the model at all).
The only purpose of the changes in this PR is to improve the performance on the model. The strategy to achieve this has been to inline the basic CTEs of the model (inline=replace references to the CTE with direct `ref` inside the model).
This works because it breaks an optimization fence.
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #20835
# Description
Adds a new dbt test that will fail if the maximum date of a given column is different from yesterday. It uses `current_date`.
Additionally, I changed the test `kpis_global_metrics_outlier_detection` on the name of the parameter `detector_strength` to `detector_tolerance`, as a higher value of the parameter indicates that will be less likely to raise an alert.
Verified in local that tests passes if the execution is normal. Verified in local that the tests fails if manually deleting the latest date in the table.
# Checklist - Does not apply
- [ ] The edited models and dependants run properly with production data.
- [ ] The edited models are sufficiently documented.
- [ ] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #20824
# Description
Updated link to use for dbt Power User usage with sqlfmt
# Checklist (N/A)
- [ ] The edited models and dependants run properly with production data.
- [ ] The edited models are sufficiently documented.
- [ ] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Specifying sqlfmt instructions to follow for dbt Power User
Related work items: #20860
# Description
This PR aims to add a new field in the master table of deals, `int_core__deal`, to incorporate the name of the account/client in a new field called `main_deal_name`.
Since a deal can have several accounts and there's no deal name existing in the backend, I just use `first_name`, `last_name` and `company_name` as the sources for the potential names. With some character removal via regex, I take the name that gets repeated the most within certain obvious exceptions (i.e., no "Bookings", "LLC", etc).
This is opinionated and could be done differently, happy to hear your thoughts.
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #18911
