counterweight/data-dwh-dbt-project
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Merged PR 3396: Changes claims and logic to consider user is in new dash

Browse source 
# Description

Considers the 2 Claims NewDashVersion and NewDashMoveDate as sources of truth, removing the previous (crazy) logic.

If a user has the claim NewDashVersion, then it's in New Dash. The claim value will also provide the version in which the user appeared (MVP, V2, etc)

If a user has the NewDashMoveDate, it means it has moved from Old Dash. If not, but still has NewDashVersion, it means the user was directly created in New Dash.

The models now provide logic to handle these cases, and it's propagated downstream will ensuring reporting will still work.

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [NOT AT ALL] The edited models contain PK tests, and I've ran and passed them. **MANY ISSUES ON PRODUCTION CURRENTLY**
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Changes claims and logic to consider user is in new dash. Downstream propagation included

Related work items: #23457
This commit is contained in:
Oriol Roqué Paniagua 2024-10-31 15:45:28 +00:00
parent bf2310301a
commit 03797827a4
7 changed files with 145 additions and 195 deletions
Download patch file Download diff file Expand all files Collapse all files

122
models/intermediate/core/schema.yml
View file

@ -2378,30 +2378,53 @@ models:
description: |
Date of the last time the information of the Host was updated
in our systems.
- name: is_user_migrated
- name: is_user_in_new_dash
data_type: boolean
description: |
Flag to determine if this user host has been migrated according
to the logic implemented in user_migration table.
- name: migration_phase
Flag to determine if this user host is in New Dash or not.
- name: has_user_moved_from_old_dash
data_type: boolean
description: |
Flag to determine if this user host is in New Dash and has
been moved from the old dash.
- name: new_dash_version
data_type: string
description: |
The name of the phase this user was first migrated.
- name: lower_limit_migration_date_utc
For users that are in New Dash, specifies the New Dash Version
in which these users were moved or joined.
- name: new_dash_move_date_utc
data_type: date
description: |
The date that the deployment of the migration happened.
It does not necessarily mean that this user was migrated in
this date. This user could have not been migrated before
this date.
For users that are in New Dash and have been moved from
Old Dash, specifies the date in which users switched.
- name: new_dash_move_at_utc
data_type: timestamp
description: |
For users that are in New Dash and have been moved from
Old Dash, specifies the timestamp in which users switched.
This is an estimate since we don't know for sure the exact
timestamp. Currently defaulting to 00:00:00h of given move
date.
- name: user_in_new_dash_since_date_utc
data_type: date
description: |
For users that are in New Dash, the effective date since
these users can be considered in New Dash. It the user
has moved from Old Dash, it will be the new_dash_move_date_utc.
If not, it will correspond to the joined_date_utc.
- name: user_in_new_dash_since_timestamp_at_utc
data_type: timestamp
description: |
For users that are in New Dash, the effective date since
these users can be considered in New Dash. It the user
has moved from Old Dash, it will be the new_dash_move_at_utc.
If not, it will correspond to the joined_at_utc.
- name: int_core__user_migration
- name: int_core__new_dash_users
description: |
This table provides information of the host users that have been migrated.
At this stage, the main objective is to account for the user migration within
the scope of New Dashboard migration.
It uses the migration configuration settled in the macro:
- user_migration_configuration -> get_new_dash_migration_phases_config
This table provides information of the host users that are in New Dash.
Users can be in New Dash because 1) have moved (migrated) from Old Dash
or 2) have been directly created within New Dash.
columns:
- name: id_user_host
@ -2410,32 +2433,44 @@ models:
tests:
- not_null
- unique
- name: migration_phase
- name: new_dash_version
data_type: string
description: |
The name of the phase this user was first migrated.
The name of the New Dash version this user appeared firstly.
tests:
- not_null
- name: lower_limit_migration_date_utc
data_type: date
description: |
The date that the deployment of the migration happened.
It does not necessarily mean that this user was migrated in
this date. This user could have not been migrated before
this date.
tests:
- not_null
- name: claim_enforcement_priority
data_type: integer
description: |
Informative field. It just states which claim enforcement priority
has been used for the computation.
- name: is_overridden_with_joined_date
- name: has_user_moved_from_old_dash
data_type: boolean
description: |
Informative field. Boolean to indicate if the expected migration date
based in the logic is finally overridden with the user joined date
in case the user was created after the expected migration date.
Flag to determine if this user host is in New Dash and has
been moved from the old dash.
- name: new_dash_move_date_utc
data_type: date
description: |
For users that are in New Dash and have been moved from
Old Dash, specifies the date in which users switched.
- name: new_dash_move_at_utc
data_type: timestamp
description: |
For users that are in New Dash and have been moved from
Old Dash, specifies the timestamp in which users switched.
This is an estimate since we don't know for sure the exact
timestamp. Currently defaulting to 00:00:00h of given move
date.
- name: user_in_new_dash_since_date_utc
data_type: date
description: |
For users that are in New Dash, the effective date since
these users can be considered in New Dash. It the user
has moved from Old Dash, it will be the new_dash_move_date_utc.
If not, it will correspond to the joined_date_utc.
- name: user_in_new_dash_since_timestamp_at_utc
data_type: timestamp
description: |
For users that are in New Dash, the effective date since
these users can be considered in New Dash. It the user
has moved from Old Dash, it will be the new_dash_move_at_utc.
If not, it will correspond to the joined_at_utc.
- name: int_core__address_validations
description:
@ -2525,13 +2560,6 @@ models:
Thus, the relationship in this table only shows, from user point of view, what
Product Bundles she/he can apply into a Listing.
tests:
- dbt_utils.unique_combination_of_columns:
combination_of_columns:
- id_user_host
- id_product_bundle
- original_starts_at_utc
columns:
- name: id_user_product_bundle
data_type: bigint
@ -2552,9 +2580,7 @@ models:
data_type: int
description: |
The identifier of the Product Bundle associated to the user. The same Product
Bundle can be applied into many users.
tests:
- not_null
Bundle can be applied into many users. Can be null if the bundle is custom.
- name: id_protection_plan
data_type: int
@ -3368,9 +3394,7 @@ models:
- name: id_product_bundle
data_type: bigint
description: |
The identifier of the product bundle.
tests:
- not_null
The identifier of the product bundle. Can be null if it's a custom bundle.
- name: id_product_service
data_type: bigint