Merged PR 2849: Fix: KYG lite migration with proper date migration handling

# Description

This PR fixes the New Dash migration issue that happened on September 10th 2024. In this migration, users were directly assigned the claim of KygMvp that does not contain a date value. We were using a default hardcode of the first MVP migration, thus in DWH all users have been considered to be migrated late July instead of splitting the first 22 in late July and the ~200 others in September.

The issue lies in the fact that users have configured a ProductBundle and can have Bookings with ProductBundle BEFORE the migration date, which greatly breaks the logic of a migration monitoring.

Changes:
* New migration phase added based on the claim MvpMigratedUser, that Ben created on Friday 13th
* Adaptation of the code in int_core__user_migration to detect if the claim_value (a text field) has a date or not. If so, use that date as long as it's equal or greater than the deployment date, if not use the deployment date. If the claim does not contain a date, use the deployment date (this is the case for the first true 22 migrated users)

I checked that volumes now look correct with this fix.

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #20773
This commit is contained in:
Oriol Roqué Paniagua 2024-09-16 07:57:41 +00:00
parent 80a6b225b3
commit cf1d6e28cc
2 changed files with 31 additions and 9 deletions

View file

@ -1,5 +1,6 @@
{% set migration_phases = get_new_dash_migration_phases_config() %}
{{ config(materialized="table") }}
with
stg_core__claim as (select * from {{ ref("stg_core__claim") }}),
@ -15,8 +16,24 @@ with
end as migration_phase,
case
{% for phase in migration_phases %}
when upper(claim_type) = '{{ phase.claim_type }}'
then date('{{ phase.deployment_date }}')
-- When the claim value text contains a Date, use the greatest
-- value of the claim value itself or the default deployment date.
-- This is to ensure that any user migration happens at least
-- after the default deployment date
when
upper(claim_type) = '{{ phase.claim_type }}'
and claim_value ~ '^\d{4}-\d{2}-\d{2}$'
then
greatest(
date(claim_value),
date('{{ phase.default_deployment_date }}')
)
-- If the claim value text does not contain a Date, use the
-- default deployment date
when
upper(claim_type) = '{{ phase.claim_type }}'
and not claim_value ~ '^\d{4}-\d{2}-\d{2}$'
then date('{{ phase.default_deployment_date }}')
{% endfor %}
else null
end as lower_limit_migration_date_utc
@ -28,7 +45,8 @@ with
{% endfor %}
)
-- GET ONLY THE FIRST TIME THE USER WAS MIGRATED
-- Get only the first time the user was migrated, among different claim types that
-- satisfy the user migration possibility
select id_user_host, migration_phase, lower_limit_migration_date_utc
from
(