# Description
This PR aims to remove custom pricing/protection fields for New Pricing.
After discussing with Clay, custom pricing/protected amounts are not allowed in New Pricing. This user-specific changes might be applied in the form of discounts, but these are not defined/implemented yet.
There were some fields created in the New Pricing tables that were considering the custom possibility, and Gus confirm that these will come as Null. So I prefer to delete them now, have clean code, and if in X months dev team wants to remove these, we should be good.
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [NA] I have checked for DRY opportunities with other models and docs.
- [NA] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #20809
# Description
Propagates Protection Plan to price and cover to intermediate. Ideally I wanted to have a single table that contained prices + covers, but it is not straight forward to assume:
* if a modification of the amount covered by a protection plan will create a new protection plan id, or not
* if a modification of the price a protection costs will create a new protection plan id, or not
So I keep it split for the time being. Besides, use cases might be different, ex: I want to see all prices of services (product + protection plan, but no need for cover).
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #20809
# Description
Adds int_core__product_service_to_price.
This model is the equivalent of staging but just adding currency code, product service name and a few boolean fields to identify if the product service to price is currently active and if it's the default price or not.
Currently it's a view since it has 16 records, likely it can be transformed into a table in the future.
The idea will be to create a similar table for protection_plan.
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [NA] I've picked the right materialization for the affected models. -> We will see in the future if needs to be adapted
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #20809
# Description
This PR fixes today's issue on dbt test, since we had a duplicated booking in this table.
Technically, according to Daga, a booking can only be overridden with a new program ONLY if the first program was the default BasicScreening, and it's a new bundle containing a Paid Service. Once there's a product bundle applied to a booking that contains a paid service, it cannot be overridden.
Thus in essence, by just keeping the latest update date it will only take into account the potential switch to a paid service, that is what really matters, and cannot change afterwards.
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #22346
# Description
Creation on stg_hubspot__form_submissions model
# Checklist
- [x] The edited models and dependants run properly with production data.
- [x] The edited models are sufficiently documented.
- [x] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Stg Hubspot form_submissions
# Description
Creation on stg_hubspot__contacts model
# Checklist
- [x] The edited models and dependants run properly with production data.
- [x] The edited models are sufficiently documented.
- [x] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
staging Hubspot contacts
# Description
Quick fix to make lower_limit_migration_date_utc more robust for new dash reporting purposes. I noticed some cases in which users were actually created after the migration date. In these cases, I just keep joined date.
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #22331
# Description
This PR creates 3 new models in staging:
- `stg_core__protection_plan`: main information of a protection plan. Here I denormalise the info coming from Protection, meaning, Protection is not a standalone model.
- `stg_core__protection_plan_to_price`: How much does a protection service cost? It's very similar to the model product_service_to_price.
- `stg_core__protection_plan_cover`: How much does a protection service protects, i.e., covers? I took the liberty of modifying the original name since it was confusing to me. Let me know if you'd like to modify the name.
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #20809
# Description
This PR adds a new table named `user_product_bundle_contains_services` in intermediate core.
Mainly, this table serves as a bridge between `user_product_bundle` and `product_services`. A `product_bundle` within `user_product_bundle` can contain 1 or several services, and this was stated in the field `contains_product_services`. The value of this field corresponds to the sum of `product_service_binary_tier` from the services that apply within that bundle. Even though the information is quite concise using this format, it adds extra complexity for analytical purposes, so this new table just duplicates the `user_product_bundle` main attributes as many times as services are contained.
For example:
`id_user_product_bundle` = 383 contains one unique bundle named Basic Program. This bundle has the `chosen_product_services` = 257, which can only be decomposed in power of 2 as of the sum of 256 + 1. This bundle therefore contains 2 services, Basic Screening (`product_service_binary_tier` = 1) and Waiver Pro (`product_service_binary_tier` = 256). Thus, in the new table, we will have 2 records and remove all this logic to something more standard, as seen in this screenshot:
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [ ] The edited models contain PK tests, and I've ran and passed them. *N/A - there's no PK in this table, but the combination of columns that should be unique is tested*
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #20809
# Description
This PR integrates into staging the following Core tables, regarding New Pricing:
- ProductService
- ProductServiceToPrice
- AppliedProductService (IT HAS BEEN REMOVED)
Models are also documented in staging. Tables contain data already but not necessarily records for New Dash, since still we have not had any paid service applied into a booking / V2 has not been launched yet.
New Dash modelisation for V2 will follow once these tables exist in staging, in order to be able to report 1) revenue and 2) services selected.
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models. **N/A, few records so not a big deal. Might need to change in the future.**
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #20809
# Description
Activates a commented out test for verification payments v2. The test was deactivated because of some master data issues in the backend, but they were fixed after #22148.
# Checklist
- [NA] The edited models and dependants run properly with production data.
- [NA ] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [NA] I have checked for DRY opportunities with other models and docs.
- [NA] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #20043
# Description
Added 2 new columns with a concatenation of month number and full name o short name
# Checklist
- [x] The edited models and dependants run properly with production data.
- [x] The edited models are sufficiently documented.
- [ ] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
new features dates
# Description
Creation on stg_hubspot__deals model
# Checklist
- [x] The edited models and dependants run properly with production data.
- [x] The edited models are sufficiently documented.
- [x] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
# Description
This PR integrates screening API verifications into staging.
There's 2 commits:
* The earliest one, it just copy-pastes the strategy followed by edeposit and adapts it to fit screening API case, which is simpler. Note here that the schema entries contain a low number of tests. This is because we only have 7 records in production - and these seem fake anyway :) - so it's complex to extrapolate. Those I could extrapolate (NoFlags/Flagged) I'm taking from the data transformation within the PBI report.
* The last commit, it's just a DRY. It handles the deduplication logic for cosmos db in a macro, and I applied it on both the screening API and the edeposit verifications. Works well from my POV.
Since you guys are far more knowledgeable on APIs scope, I'll let you take a close look in case I missed something.
# Checklist
- [X] The edited models and dependants run properly with production data.
- [ ] The edited models are sufficiently documented. **I guess it can be improved, waiting for your comments here**
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models. **Used default coming from stg_edeposit__verifications**
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #20127
