Merged PR 5463: New Dash user categorisation + revamp model

# Description

Fixes:
* There was a data quality issue in which active listings with bundles were showing more listings than the total active listings. This has been fixed, plus a bunch of tests have been added to make sure this does not happen again.
* There's a field that reaches the 63 character limit. I modified "paid" per "upgraded" which reduces this to intermediate model; however in reporting I need to keep as is to avoid breaking PBI. I did change the name of the field in the schema so tests/docs work well and explicitly mention this in the doc. This will need adaptation but can be done in the future.
* There was tons of dead code tagged as to be deleted that was not deleted, not sure why or when. Anyway this is dropped now. It's not used in PBI except for the fact that columns are dropped in the data model, so we might have a bit of down time to just remove these.
* Few fields were not displayed correctly in dbt docs.

What's new:
* New user categorisation as an alert-based system. This can be enhanced in the future but so far it covers hell of a lot of cases. This also includes data quality alerts (deal not in HS, or account defaults to MVP launch).
* MVP Launch date moved to project yaml since it's a very standard date.

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #30407
This commit is contained in:
Oriol Roqué Paniagua 2025-06-13 14:05:30 +00:00
parent 11bdc7d4b3
commit a8927b7a44
7 changed files with 321 additions and 154 deletions

View file

@ -1810,6 +1810,14 @@ models:
these users can be considered in New Dash. If the user
has moved from Old Dash, it will be the new_dash_move_at_utc.
If not, it will correspond to the joined_at_utc.
- name: has_new_dash_default_migration_date
data_type: boolean
description: |
Flag to determine if the user migration date corresponds to
the default migration date. True if the user has the default
migration date, false if the user has a custom migration date
or is not in New Dash.
If true it might represent a data quality issue.
- name: int_core__new_dash_users
description: |
@ -1862,6 +1870,12 @@ models:
these users can be considered in New Dash. If the user
has moved from Old Dash, it will be the new_dash_move_at_utc.
If not, it will correspond to the joined_at_utc.
- name: is_default_migration_date
data_type: boolean
description: |
Flag to determine if the user migration date corresponds to
the default migration date. It might represent a data quality
issue.
- name: int_core__user_product_bundle
description: |
@ -2316,6 +2330,13 @@ models:
data_tests:
- not_null
- name: has_new_dash_default_migration_date
data_type: boolean
description: |
Flag to determine if the user migration date corresponds to
the default migration date. It might represent a data quality
issue.
- name: deal_lifecycle_state
data_type: character varying
description: |
@ -2402,7 +2423,7 @@ models:
description: |
Count of active listings that currently have an active product bundle.
- name: total_active_listings_with_active_product_bundle_with_paid_service
- name: total_active_listings_with_active_upgraded_product_bundle
data_type: integer
description: |
Count of active listings that currently have an active paid service
@ -2419,41 +2440,18 @@ models:
Integer-based flag version of
total_active_listings_with_active_product_bundle.
- name: has_active_listings_with_active_paid_service_applied
- name: has_active_listings_with_active_upgraded_service_applied
data_type: integer
description: |
Integer-based flag version of
total_active_listings_with_active_product_bundle_with_paid_service.
total_active_listings_with_active_upgraded_product_bundle.
# To be deleted from here
- name: total_listings_with_product_bundle_with_paid_service
data_type: integer
description: |
Count of listings that have, or have had, a paid service
product bundle activated.
- name: total_listings_with_active_product_bundle_with_paid_service
data_type: integer
description: |
Count of listings that currently have an active paid service
product bundle.
- name: has_listings_with_paid_service_applied
data_type: integer
description: |
Integer-based flag version of total_listings_with_product_bundle_with_paid_service.
- name: has_listings_with_active_paid_service_applied
data_type: integer
description: |
Integer-based flag version of total_listings_with_active_product_bundle_with_paid_service.
# To be deleted until here
- name: total_bookings_with_product_bundle
data_type: integer
description: |
Count of bookings that have a product bundle associated.
- name: total_bookings_with_product_bundle_with_paid_service
- name: total_bookings_with_upgraded_product_bundle
data_type: integer
description: |
Count of bookings that have a product bundle associated that contain
@ -2464,10 +2462,44 @@ models:
description: |
Integer-based flag version of total_bookings_with_product_bundle.
- name: has_bookings_with_product_bundle_with_paid_service
- name: has_bookings_with_upgraded_product_bundle
data_type: integer
description: |
Integer-based flag version of total_bookings_with_product_bundle_with_paid_service.
Integer-based flag version of total_bookings_with_upgraded_product_bundle.
- name: user_categorisation
data_type: string
description: |
Alert-based categorisation of the user based on the activity or lack of it.
The possible values are:
- "00 - No Alert"
- "01 - No Listings"
- "02 - No Active Listings"
- "03 - No Bookings - No Upgraded Program in Listings"
- "04 - No Bookings - Has Upgraded Program in Listings"
- "05 - Only Basic Screening Bookings - No Upgraded Program in Listings"
- "06 - Only Basic Screening Bookings - Has Upgraded Program in Listings"
- "07 - Has Upgraded Bookings - No Upgraded Program in Listings"
- "08 - Has Upgraded Bookings - Not all Listings have Upgraded Program Applied"
- "98 - Has Churned"
- "99 - Has Data Quality Issues"
It cannot be null.
data_tests:
- not_null
- accepted_values:
values:
- "00 - No Alert"
- "01 - No Listings"
- "02 - No Active Listings"
- "03 - No Bookings - No Upgraded Program in Listings"
- "04 - No Bookings - Has Upgraded Program in Listings"
- "05 - Only Basic Screening Bookings - No Upgraded Program in Listings"
- "06 - Only Basic Screening Bookings - Has Upgraded Program in Listings"
- "07 - Has Upgraded Bookings - No Upgraded Program in Listings"
- "08 - Has Upgraded Bookings - Not all Listings have Upgraded Program Applied"
- "98 - Has Churned"
- "99 - Has Data Quality Issues"
- name: int_core__invoicing_price_plans_per_month
description: |