# Description
This PR provides a suggested script to run `dbt test` in production, leave logs and send slack messages upon success/failure. It also updates the readme details on how to deploy so that deployers know how to set it.
# Checklist
- __NA__ ~~[ ] The edited models and dependants run properly with production data.~~
- __NA__ ~~[ ] The edited models are sufficiently documented.~~
- __NA__ ~~[ ] The edited models contain PK tests, and I've ran and passed them.~~
- __NA__ ~~[ ] I have checked for DRY opportunities with other models and docs.~~
- __NA__ ~~[ ] I've picked the right materialization for the affected models.~~
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #16959
# Description
e-deposit verifications data to intermediate
This holds data for each user of the amount of revenue generated each month of creation and check out month
# Checklist
- [X] The edited models and dependants run properly with production data.
- [x] The edited models are sufficiently documented.
- [x] The edited models contain PK tests, and I've ran and passed them.
- [x] I have checked for DRY opportunities with other models and docs.
- [x] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #20125
# Description
Same PR as before, just adds a new commit that fixes my silly issue in prod. I owe some drinks :D
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #19570
# Description
fixed testing for core__check_in_cover_listings and removed null in accepted values for int_core__bookings in verification_request_booking_source
# Checklist
- [x] The edited models and dependants run properly with production data.
- [x] The edited models are sufficiently documented.
- [x] The edited models contain PK tests, and I've ran and passed them.
- [x] I have checked for DRY opportunities with other models and docs.
- [x] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #20229
# Description
_Describe your motivation and changes here._
# Checklist
- [ ] The edited models and dependants run properly with production data.
- [ ] The edited models are sufficiently documented.
- [ ] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
# Description
Automates the new dash query through DWH. It's a first step towards enabling a very simple PBI reporting on new dash performance. Data is aggregated at user level with the main performance indicators that we're currently reporting. I added a few more informative fields from user side.
Changes:
* in intermediate, creation of `int_core__new_dash_user_overview`. It mainly uses the models in previous PRs to avoid crazy computations, since most of the logic has already been implemented. The only thing is that it's "hardcoded" (with variable) the categorisation of what is NOT a paid service.
* in reporting, creation of `core__new_dash_user_overview`. It just exposes the info from intermediate to reporting.
* added schema entries for intermediate/core and reporting/core, it's the same description for both cases.
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Reverts !2657
Related work items: #19570
# Description
Automates the new dash query through DWH. It's a first step towards enabling a very simple PBI reporting on new dash performance. Data is aggregated at user level with the main performance indicators that we're currently reporting. I added a few more informative fields from user side.
Changes:
* in intermediate, creation of `int_core__new_dash_user_overview`. It mainly uses the models in previous PRs to avoid crazy computations, since most of the logic has already been implemented. The only thing is that it's "hardcoded" (with variable) the categorisation of what is NOT a paid service.
* in reporting, creation of `core__new_dash_user_overview`. It just exposes the info from intermediate to reporting.
* added schema entries for intermediate/core and reporting/core, it's the same description for both cases.
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #19570
# Description
Adds accommodation to product bundle in intermediate. The source table is currently empty, because there's no product bundle different to Basic Screening applied into a listing yet.
Thus, it's possible that this modelisation needs to change in the future since not having data forces us to trust the business logic without a proper technical documentation.
# Checklist
- [ ] The edited models and dependants run properly with production data. **N/A no data**
- [X] The edited models are sufficiently documented.
- [ ] The edited models contain PK tests, and I've ran and passed them. **N/A no data**
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models. **N/A no data - but view should make the trick for the time being**
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #19570
# Description
Fixed bookings test to include null for verification_request_booking_source
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #20231
# Description
Creates a new view with the bookings that come from New Dash, linked to the Product Bundles. Extensive documentation added, please check it out and if something is not clear I'll modify it.
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models. To be discussed. There's not huge data still so view looks ok to me. I guess, maybe, in the future, we will have a flag or similar in int_core__bookings to detect if a bookings comes from New Dash/New Pricing or not, but it's still early to decide. In any case this model is needed for immediate New Dash MVP reporting so I propose to keep it like this as a view and afterwards we can decide if it makes more sense to materialise it differently.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #19570
# Description
Fixes int_dates_by_deal tests
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #20318, #20319
# Description
Adds three more columns to the staging model of `verifications` for edeposit: `ListingId`, `NightlyFee` and `CompanyName`.
# Checklist
- [X] The edited models and dependants run properly with production data.
- ~~[ ] The edited models are sufficiently documented.~~ __No, but I promise they will be.__
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #20123
# Description
Adds accommodation to product bundle table from sync_core to staging. Note that this table still has no data because so far no listing has an associated product bundle.
Small change: removed in schema pending confirmation comments from Lou now that we've got an answer confirming it.
# Checklist
- [ ] The edited models and dependants run properly with production data: **N/A - there's no data :(**
- [X] The edited models are sufficiently documented.
- [ ] The edited models contain PK tests, and I've ran and passed them. **N/A - there's no data :(**
- [X] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models. **N/A - there's no data :(**
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #19570
# Description
This PR fixes a bug in the deployment script. The bug makes the alerts not work as expected, because the bash piping will trigger alerts if the log writing of commands fail, instead of if commands fail, which is what is expected.
This PR modifies the way logging works. Logging remains the same, but alerts now will work as expected.
Related work items: #18183
# Description
This PR modifies the `docker compose` file that we recommend for local development, adding to it Postgres server parameters. This parameters are expected to provide a better experience for developers in terms of performance.
# Checklist
- NA ~~[ ] The edited models and dependants run properly with production data.~~
- NA ~~[ ] The edited models are sufficiently documented.~~
- NA ~~[ ] The edited models contain PK tests, and I've ran and passed them.~~
- NA ~~[ ] I have checked for DRY opportunities with other models and docs.~~
- NA ~~[ ] I've picked the right materialization for the affected models.~~
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #20193
# Description
Working version of User Product Bundle in intermediate. I tried to be quite explicit in the documentation of the model and the choices made (both in the code itself and in the schema). There's some opinionated choices so feel free to challenge them.
There's a small change on the user_migration model, in which I didn't properly set a field into a date.
Note that there's some schema comments pending from Lou's validation. Up to you if we prefer to wait until resolved or we move forward - to me, it's not blocking.
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #19570
# Description
Adds a missing field, `version`.
# Checklist
- [X] The edited models and dependants run properly with production data.
- [ ] The edited models are sufficiently documented. -> __NO, but I promise they will be eventually__
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #20123
# Description
Creates exposure entries for the accounting reports that are already in production (Invoicing and Crediting and Resolutions Host Payments).
# Checklist
- __NA__ ~~[ ] The edited models and dependants run properly with production data.~~
- [X] The edited models are sufficiently documented.
- __NA__ ~~[ ] The edited models contain PK tests, and I've ran and passed them.~~
- __NA__ ~~[ ] I have checked for DRY opportunities with other models and docs.~~
- __NA__ ~~[ ] I've picked the right materialization for the affected models.~~
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #17550
# Description
First step towards reporting New Dash is to be able within DWH to know which hosts have been migrated.
In order to do so, and anticipating that there's going to be new phases in the future, I've created a `int_core__user_migration` model that reads from a configuration macro `get_new_dash_migration_phases_config` that will allow semi-automatic user retrieval in the future. This avoids nasty hardcoding within the model itself.
The information of whether a user is migrated, in which phase and when the phase was deployed is available at user level in the `int_core__user_host` table.
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models. **-> I selected a view for this model since I don't believe we should materialse this data other than the user host table**
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #19570
# Description
Sets the parameter to display the KPIs by number of listings to prod. I will move forward without the review as we need a simultaneous deployment. The combination of changes were reviewed yesterday in local.
# Checklist
- [ ] The edited models and dependants run properly with production data.
- [ ] The edited models are sufficiently documented.
- [ ] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #19325
# Description
This PR turns the JSON documents of the verification container in sync into a table in staging. It also performs the necessary deduplication of records by ID and timestamp along the way.
I'm intentionally skipping proper documentation to unblock a colleague while I fetch the necessary knowledge to populate the schema docs properly. I pinky promise I won't forget and I'll come back and fix it.
# Checklist
- [X] The edited models and dependants run properly with production data.
- [ ] ~~The edited models are sufficiently documented.~~ __Nope, read above.__
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #20123
# Description
Changes:
* Separate 1) the internal naming of dimensions available within DWH vs. 2) the display of the dimensions in the reporting. Mainly it changes the "by_number_of_listings" to display "By # of Listings Booked in 12 Months". I edited the production macro since to me it's linked to when things are available for display.
* Add preceding zeros on the segmentation so it's ordered correctly. Before, the segment 21-60 was displayed before the 6-20.
* Also added some capital letters to the schema config of the reporting model :)
I attach a screenshot of how it looks in PBI in my local development branch to exemplify why this is "Beautification". Be aware that merging this also puts in production the dimensions.
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #19325
# Description
This PR ensures the propagation of the dimensions for KPIs across the key aggregating and exposing models. Additionally, provides these 2 new fields in reporting while **not affecting the current data display**, thus it's safe to work in the PBI report without needing to work in 2 PRs in parallel.
**Changes:**
**1 - Intermediate, `int_mtd_vs_previous_year_metrics`:**
* Removes the temporary filter on `where dimension in ({{ production_dimensions }})`. This will be applied directly to reporting later. This ensures that the new dimension on customer segmentation is fully available only within intermediate.
* Adds `dimension` and `dimension_value` granularity. This includes: 1) adding these fields, 2) joining by these fields with all the source CTEs containing the source models with metrics - which in turn needs the change of the dates model - and 3) joining by these fields in the self-join to compute the incremental vs. previous year.
* Changes on the schema file
**2 - Intermediate, `int_mtd_aggregated_metrics`:**
* Adds `dimension` and `dimension_value` granularity. This includes only adding these fields.
* Changes on the schema file
**3 - Reporting, `mtd_aggregated_metrics`:**
* Adds the filter removed on `int_mtd_vs_previous_year_metrics`. This ensures that only the Global dimension is available for the reporting, thus **no changes from user POV**.
* Adds `dimension` and `dimension_value` granularity. This includes only adding these fields
* Changes on the schema file
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #19325
# Description
Finished adding address_validation to reporting for check-in hero
# Checklist
- [x] The edited models and dependants run properly with production data.
- [x] The edited models are sufficiently documented.
- [x] The edited models contain PK tests, and I've ran and passed them.
- [x] I have checked for DRY opportunities with other models and docs.
- [x] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
