Merged PR 3909: Fixes created_date_utc and updated_date_utc in SH User and its dependencies
# Description The model `stg_core__superhog_user` contained the fields `created_date_utc` and `updated_date_utc`, which were actually timestamps, not dates. This created issues a while ago in KPIs that were fixed by forcing a date conversion at `int_kpis` level, but the source issue remained unfixed. This PR just creates the proper fields in staging, meaning: * `created_at_utc` and `created_date_utc` * `updated_at_utc` and `updated_date_utc` And propagates the changes through all dependencies. On KPIs, it also removes the hardcoded date conversion since it's no longer needed. It also adds the different schema entries on which these fields were already defined. Note I didn't update the staging entry since there's no entry for the rest of the fields. I recommend checking this PR in the commits order. What I've tested: * Unified User and User Host have plenty of dependencies that I've checked one by one, at each dependency layer by using dbt docs. * Run `core__unified_user` and the full execution of +`mtd_aggregated_metrics` What clearly will fail: * There's the famous `Users_dashboard` PBI report that shouldn't exist that has a dependency with `core__unified_user`. This report will fail. But it does not exist. I'd like to drop it once and forever # Checklist - [X] The edited models and dependants run properly with production data. **See above** - [X] The edited models are sufficiently documented. - [X] The edited models contain PK tests, and I've ran and passed them. - [X] I have checked for DRY opportunities with other models and docs. - [X] I've picked the right materialization for the affected models. # Other - [X] Check if a full-refresh is required after this PR is merged. **The staging model is incremental!** Related work items: #23703
This commit is contained in:
parent
803c5ed3d5
commit
d26e08bffd
8 changed files with 42 additions and 17 deletions
|
|
@ -38,7 +38,9 @@ select
|
|||
su.avatar,
|
||||
su.id_airbnb,
|
||||
su.airbnb_url,
|
||||
su.created_at_utc,
|
||||
su.created_date_utc,
|
||||
su.updated_at_utc,
|
||||
su.updated_date_utc,
|
||||
su.verified_at_utc,
|
||||
su.verified_date_utc,
|
||||
|
|
|
|||
|
|
@ -48,7 +48,9 @@ select
|
|||
d.main_billing_country_iso_3_per_deal,
|
||||
uu.joined_at_utc,
|
||||
uu.joined_date_utc,
|
||||
uu.created_at_utc,
|
||||
uu.created_date_utc,
|
||||
uu.updated_at_utc,
|
||||
uu.updated_date_utc,
|
||||
case when uu.id_deal is null then true else false end as is_missing_id_deal,
|
||||
case
|
||||
|
|
|
|||
|
|
@ -276,12 +276,18 @@ models:
|
|||
- name: airbnb_url
|
||||
data_type: character varying
|
||||
description: The Airbnb URL.
|
||||
- name: created_at_utc
|
||||
data_type: timestamp
|
||||
description: The timestamp when the user was created.
|
||||
- name: created_date_utc
|
||||
data_type: date
|
||||
description: The date when the user was created.
|
||||
- name: updated_at_utc
|
||||
data_type: timestamp
|
||||
description: The timestamp when the user was created. The field name needs to be fixed.
|
||||
description: The timestamp when the user was updated.
|
||||
- name: updated_date_utc
|
||||
data_type: timestamp
|
||||
description: The timestamp when the user was updated. The field name needs to be fixed.
|
||||
data_type: date
|
||||
description: The date when the user was updated.
|
||||
- name: verified_at_utc
|
||||
data_type: timestamp
|
||||
description: The timestamp when the user was verified.
|
||||
|
|
@ -1965,15 +1971,18 @@ models:
|
|||
data_type: date
|
||||
description: |
|
||||
Date of when the Host user joined Superhog.
|
||||
- name: created_at_utc
|
||||
data_type: timestamp
|
||||
description: The timestamp when the user was created.
|
||||
- name: created_date_utc
|
||||
data_type: date
|
||||
description: |
|
||||
Date of when the Host user was created in our systems.
|
||||
description: The date when the user was created.
|
||||
- name: updated_at_utc
|
||||
data_type: timestamp
|
||||
description: The timestamp when the user was updated.
|
||||
- name: updated_date_utc
|
||||
data_type: date
|
||||
description: |
|
||||
Date of the last time the information of the Host was updated
|
||||
in our systems.
|
||||
description: The date when the user was updated.
|
||||
- name: is_missing_id_deal
|
||||
data_type: boolean
|
||||
description: |
|
||||
|
|
|
|||
|
|
@ -27,7 +27,7 @@ with
|
|||
from {{ ref("int_kpis__dimension_dates") }} as ikdd
|
||||
left join
|
||||
{{ ref("int_core__user_host") }} as icuh
|
||||
on ikdd.date >= date(icuh.created_date_utc)
|
||||
on ikdd.date >= icuh.created_date_utc
|
||||
left join {{ ref("int_core__deal") }} as icd on icuh.id_deal = icd.id_deal
|
||||
left join
|
||||
{{ ref("int_kpis__dimension_daily_accommodation") }} as icmas
|
||||
|
|
|
|||
|
|
@ -41,12 +41,12 @@ with
|
|||
select
|
||||
d.date,
|
||||
h.id_deal,
|
||||
min(date(h.created_date_utc)) as creation_date_utc,
|
||||
min(h.created_date_utc) as creation_date_utc,
|
||||
min(b.created_date_utc) as first_time_booked_date_utc,
|
||||
max(b.created_date_utc) as last_time_booked_date_utc,
|
||||
max(b.previous_booked_date) as second_to_last_time_booked_date_utc
|
||||
from int_kpis__dimension_dates d
|
||||
inner join int_core__user_host h on d.date >= date(h.created_date_utc)
|
||||
inner join int_core__user_host h on d.date >= h.created_date_utc
|
||||
left join
|
||||
booked_days_per_deal b
|
||||
on h.id_deal = b.id_deal
|
||||
|
|
|
|||
|
|
@ -26,7 +26,9 @@ select
|
|||
avatar as avatar,
|
||||
id_airbnb as id_airbnb,
|
||||
airbnb_url as airbnb_url,
|
||||
created_at_utc as created_at_utc,
|
||||
created_date_utc as created_date_utc,
|
||||
updated_at_utc as updated_at_utc,
|
||||
updated_date_utc as updated_date_utc,
|
||||
verified_at_utc as verified_at_utc,
|
||||
verified_date_utc as verified_date_utc,
|
||||
|
|
|
|||
|
|
@ -736,13 +736,21 @@ models:
|
|||
data_type: character varying
|
||||
description: ""
|
||||
|
||||
- name: created_at_utc
|
||||
data_type: timestamp
|
||||
description: The timestamp when the user was created.
|
||||
|
||||
- name: created_date_utc
|
||||
data_type: timestamp without time zone
|
||||
description: The date time when the user data was created.
|
||||
data_type: date
|
||||
description: The date when the user was created.
|
||||
|
||||
- name: updated_at_utc
|
||||
data_type: timestamp
|
||||
description: The timestamp when the user was updated.
|
||||
|
||||
- name: updated_date_utc
|
||||
data_type: timestamp without time zone
|
||||
description: The date when the user data was created.
|
||||
data_type: date
|
||||
description: The date when the user was updated.
|
||||
|
||||
- name: verified_at_utc
|
||||
data_type: timestamp without time zone
|
||||
|
|
|
|||
|
|
@ -13,8 +13,10 @@ with
|
|||
{{ adapter.quote("Avatar") }} as avatar,
|
||||
{{ adapter.quote("AirbnbId") }} as id_airbnb,
|
||||
{{ adapter.quote("AirbnbUrl") }} as airbnb_url,
|
||||
{{ adapter.quote("CreatedDate") }} as created_date_utc,
|
||||
{{ adapter.quote("UpdatedDate") }} as updated_date_utc,
|
||||
{{ adapter.quote("CreatedDate") }} as created_at_utc,
|
||||
date({{ adapter.quote("CreatedDate") }}) as created_date_utc,
|
||||
{{ adapter.quote("UpdatedDate") }} as updated_at_utc,
|
||||
date({{ adapter.quote("UpdatedDate") }}) as updated_date_utc,
|
||||
{{ adapter.quote("VerifiedDate") }} as verified_at_utc,
|
||||
cast({{ adapter.quote("VerifiedDate") }} as date) as verified_date_utc,
|
||||
{{ adapter.quote("FlagAsProblem") }} as flag_as_problem,
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue