# Description
Main changes:
* Includes 4 new fields to take into account 12 month created bookings. Specifically:
`deal_created_bookings_12_months_window`
`global_created_bookings_12_months_window`
`deal_contribution_share_to_global_created_bookings`
`deal_contribution_rank_to_global_created_bookings`
This also renames a CTE, that was previously stating it was revenue. Same for inline comments. Also includes documentation of this fields.
* Score range modification: Now, growth scores are multiplied by 100 and weighted score by 1000. This makes it easier to display and understand (Growth cannot be less than -100, threshold value is now -1, 0 and 1).
I checked that the content already in production has not change (ex: we still have the same 15 top losers for September).
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [NA] I have checked for DRY opportunities with other models and docs.
- [NA] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #22635
# Description
Main changes:
- Creation of `int_monthly_churn_metrics` model. This follows a similar approach as for mtd models, with jinja loops to aggregate the metrics at different dimensions. This reads from the previous monthly model, thus creating a dependency on Global KPIs with By Deal KPIs.
- Adds the 6 metrics in the main aggregated model of Global KPIs `int_mtd_vs_previous_year_metrics`. It doesn't necessarily mean that the 6 metrics will be made available in the report.
This does NOT display these metrics in the report, and won't be done until deal lifecycle is enriched to consider hubspot offboarding in the state "05-Churning".
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #22691
# Description
This PR creates a new model that depends on int_monthly_aggregated_metrics_history_by_deal. The idea is that this is used for Churn computation (Booking Churn, Revenue Churn, Listing Churn) later on.
The idea is relatively simple. Measure how much a Deal has been contributing to a Global amount (sum of metric for all deals) over the preceding period of 12 months. You will notice that there's 2 computations, the "additive" and the "average" one. This is because we still need to align with Matt/Suzannah on which approach makes more sense, but we need data for it. I'm not sure the namings are good though so happy to see your suggestions.
You will also notice that there's no filter by deal_lifecycle_state = '06-Churning'. This will be done in a separated model, whenever we attribute this model to the mtd computation. The reason is simple - this model stays at deal level, thus meaning we can do the dimension aggregation and even a lifecycle aggregation if needed, depending on the needs.
Be aware that this effectively means that MTD KPIs models will depend on the "monthly by deal" models. This has some cons in terms of dependency management but cannot be overcome since we the metric total revenue depends on many subsets. In essence, I don't see another way of doing it unless doing a massive KPIs refactor. I prefer to wait until the Product KPIs discussions are finished and then we see how we approach it.
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #22691
# Description
Creating exposures entry for top_losers report
# Checklist
- [NA] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [NA] The edited models contain PK tests, and I've ran and passed them.
- [NA] I have checked for DRY opportunities with other models and docs.
- [NA] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #22635
# Description
Copies intermediate to reporting for growth score by deal. Schema is copy-paste from intermediate changing the model's name.
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [NA] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #22635
# Description
This PR brings Hubspot deal pipelines and stages into staging.
It all comes from one same source table: stages have been normalized from the nested JSON array they come in through Airbyte.
This is a necessary step to make Deal data more consumable in `int_`.
I also had to slightly expand an existing macro to allow specifying table aliases besides column names. This is probably a pattern we might want to extend to other macros.
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #22772
# Description
Creates a model to identify deal growth based on YoY performance of Created Bookings, YoY performance of Listings Booked in Month and one month shifted YoY performance of Revenue.
Also added weighted score to account for revenue size.
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs. **Probably something can be done here, sorry I've not checked.**
- [X] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #22635
# Description
Main changes:
* Guest revenue is now guest payments. PBI uses Guest revenue, so alias is changed at reporting level, while it uses guest_payments_in_gbp field.
* Removal of Waiver Amount Paid back to Host to Guest revenue and Total revenue.
# Checklist
- [X] The edited models and dependants run properly with production data.
- [NA] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [NA] I have checked for DRY opportunities with other models and docs.
- [NA] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #22688
# Description
Adapts revenue figures in Main KPIs - MTD scope or global view. This includes MTD, Monthly Overview, Global Evolution over Time, Detail by Category. In essence, everything that is not by deal.
The changes are mainly 2:
* Remove the line that deducts the `Waiver Amount Paid Back to Hosts` in all metrics except the `Waiver Net Fees`. This effectively means that the previous `Guest Revenue` = `Guest Payments`, thus I dropped all 3 `Guest Payments` metrics.
* Do a renaming at metric display level, but not in the code. This means that I remove the computation of `guest_revenue_in_gbp` for instance and keep `guest_payments_in_gbp`, and apply the renaming later on, since the modelisation already accounts for defining metric names differently from those of the fields. For the rest of metrics, I revised all metrics name and did changes based on the [whiteboard](https://whiteboard.office.com/me/whiteboards/p/c3BvOmh0dHBzOi8vZ3VhcmRob2ctbXkuc2hhcmVwb2ludC5jb20vcGVyc29uYWwvcGFibG9fbWFydGluX3N1cGVyaG9nX2NvbQ%3d%3d/b!T2D3opQuBECSDnhuFZrUacFu3TxvSvdIsnI4Dxsh2IuaB1AigbciRqkqte61I4wz/01H5SI4J4L7HTPJGUT7JGYKTOSQYYWACXU). I also changed the dedicated data tests in Main KPIs to ensure it's working. I also changed the exclusion logic in reporting based on the name of the metric to not display metrics that depend on the invoicing cycle unless it's 2 months ago or before.
To keep in mind:
* Merging this will automatically display the new figures/naming in production. Might be wise to communicate to stakeholders since some key metrics (namely, Guest Revenue / Total Revenue) will change the meaning.
* We also need to do these changes in the metrics by deal part of the computation. I'd do first the removal of these fields in the PBI report (and take the opportunity to change the Data Catalogue) and then do the PR in DWH to change the logic. Before that though let's check that the names included in this PR are the correct ones :)
# Checklist
- [X] The edited models and dependants run properly with production data.
- [NA] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [NA] I have checked for DRY opportunities with other models and docs.
- [NA] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #22688
# Description
Add a new seed with some data coming from a manual file as part of #22703.
I'm adding it here to make it available for some Python notebook errands. Might be useful in the future as well.
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [NA] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #22703
# Description
The exposure for this report is wrong. It reads from a differnt model than what it states. This PR points to the actual model.
# Checklist
- [NA] The edited models and dependants run properly with production data.
- [NA] The edited models are sufficiently documented.
- [NA] The edited models contain PK tests, and I've ran and passed them.
- [NA] I have checked for DRY opportunities with other models and docs.
- [NA] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #22186
# Description
PR to fix dbt test on user_role, by just adding the new API role "CancellationApi"
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [NA] I have checked for DRY opportunities with other models and docs.
- [NA] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #22627
# Description
This PR artificially splits Athena and e-deposit domains into two different lines of folders in staging, intermediate and reporting. I say artificially because the `verifications` data still comes from the same sync table for both lineages.
This is in preparation for the upcoming database migration in which the source databases will be effectively split.
The reporting models in the `reporting/edeposit` folder are still untouched because they have PBI dependencies. Once the PBIs are managed (after this PR), we will make a different PR to clean them up and remove deprecated models and docs here.
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #22186
# Description
This PR aims to remove custom pricing/protection fields for New Pricing.
After discussing with Clay, custom pricing/protected amounts are not allowed in New Pricing. This user-specific changes might be applied in the form of discounts, but these are not defined/implemented yet.
There were some fields created in the New Pricing tables that were considering the custom possibility, and Gus confirm that these will come as Null. So I prefer to delete them now, have clean code, and if in X months dev team wants to remove these, we should be good.
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [NA] I have checked for DRY opportunities with other models and docs.
- [NA] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #20809
# Description
Propagates Protection Plan to price and cover to intermediate. Ideally I wanted to have a single table that contained prices + covers, but it is not straight forward to assume:
* if a modification of the amount covered by a protection plan will create a new protection plan id, or not
* if a modification of the price a protection costs will create a new protection plan id, or not
So I keep it split for the time being. Besides, use cases might be different, ex: I want to see all prices of services (product + protection plan, but no need for cover).
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #20809
# Description
Adds int_core__product_service_to_price.
This model is the equivalent of staging but just adding currency code, product service name and a few boolean fields to identify if the product service to price is currently active and if it's the default price or not.
Currently it's a view since it has 16 records, likely it can be transformed into a table in the future.
The idea will be to create a similar table for protection_plan.
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [NA] I've picked the right materialization for the affected models. -> We will see in the future if needs to be adapted
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #20809
# Description
Small PR to provide the Protection Name into User Product Bundle
There's some additional automatic formatting changes
# Checklist
- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
Related work items: #20809
# Description
Deactivates the accepted values test that we had in the edeposit verifications staging table.
The table is more complex that we initially thought (multiple values can appear comma separated) and we don't really care enough to increase to make it worth it to increase the complexity of the test, so we are removing it.
# Checklist
- [NA] The edited models and dependants run properly with production data.
- [NA] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [NA] I have checked for DRY opportunities with other models and docs.
- [NA] I've picked the right materialization for the affected models.
# Other
- [ ] Check if a full-refresh is required after this PR is merged.
# Description
This PR changes the documentation on how we set up our local Postgres instance to leverage Foreign Data Wrappers to have a more convenient development workflow in dbt.
@<Joaquin Ossa> and @<Oriol Roqué Paniagua> : for this review, I want to ask you to:
- Take a look at the entire `dev-env/local_dwh.md` file with a critical eye and make suggestions, spot mistakes, look for gaps, etc.
- And also follow and use (so we know the instructions are complete) the section `dev-env/local_dwh.md#Filling `dwh` with copied data`. This bit is new and we haven't discussed, but I hope the instructions are self-descriptive. Please, try to follow them and validate that you can run things as described before approving this PR.
