Merged PR 3909: Fixes created_date_utc and updated_date_utc in SH User and its dependencies

# Description

The model `stg_core__superhog_user` contained the fields `created_date_utc` and `updated_date_utc`, which were actually timestamps, not dates. This created issues a while ago in KPIs that were fixed by forcing a date conversion at `int_kpis` level, but the source issue remained unfixed.

This PR just creates the proper fields in staging, meaning:
* `created_at_utc` and `created_date_utc`
* `updated_at_utc` and `updated_date_utc`

And propagates the changes through all dependencies. On KPIs, it also removes the hardcoded date conversion since it's no longer needed. It also adds the different schema entries on which these fields were already defined. Note I didn't update the staging entry since there's no entry for the rest of the fields.

I recommend checking this PR in the commits order.

What I've tested:
* Unified User and User Host have plenty of dependencies that I've checked one by one, at each dependency layer by using dbt docs.
* Run `core__unified_user` and the full execution of +`mtd_aggregated_metrics`

What clearly will fail:
* There's the famous `Users_dashboard` PBI report that shouldn't exist that has a dependency with `core__unified_user`. This report will fail. But it does not exist. I'd like to drop it once and forever

# Checklist

- [X] The edited models and dependants run properly with production data. **See above**
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.

# Other

- [X] Check if a full-refresh is required after this PR is merged. **The staging model is incremental!**

Related work items: #23703
This commit is contained in:
Oriol Roqué Paniagua 2024-12-31 08:04:03 +00:00
parent 803c5ed3d5
commit d26e08bffd
8 changed files with 42 additions and 17 deletions

View file

@ -38,7 +38,9 @@ select
su.avatar,
su.id_airbnb,
su.airbnb_url,
su.created_at_utc,
su.created_date_utc,
su.updated_at_utc,
su.updated_date_utc,
su.verified_at_utc,
su.verified_date_utc,

View file

@ -48,7 +48,9 @@ select
d.main_billing_country_iso_3_per_deal,
uu.joined_at_utc,
uu.joined_date_utc,
uu.created_at_utc,
uu.created_date_utc,
uu.updated_at_utc,
uu.updated_date_utc,
case when uu.id_deal is null then true else false end as is_missing_id_deal,
case

View file

@ -276,12 +276,18 @@ models:
- name: airbnb_url
data_type: character varying
description: The Airbnb URL.
- name: created_at_utc
data_type: timestamp
description: The timestamp when the user was created.
- name: created_date_utc
data_type: date
description: The date when the user was created.
- name: updated_at_utc
data_type: timestamp
description: The timestamp when the user was created. The field name needs to be fixed.
description: The timestamp when the user was updated.
- name: updated_date_utc
data_type: timestamp
description: The timestamp when the user was updated. The field name needs to be fixed.
data_type: date
description: The date when the user was updated.
- name: verified_at_utc
data_type: timestamp
description: The timestamp when the user was verified.
@ -1965,15 +1971,18 @@ models:
data_type: date
description: |
Date of when the Host user joined Superhog.
- name: created_at_utc
data_type: timestamp
description: The timestamp when the user was created.
- name: created_date_utc
data_type: date
description: |
Date of when the Host user was created in our systems.
description: The date when the user was created.
- name: updated_at_utc
data_type: timestamp
description: The timestamp when the user was updated.
- name: updated_date_utc
data_type: date
description: |
Date of the last time the information of the Host was updated
in our systems.
description: The date when the user was updated.
- name: is_missing_id_deal
data_type: boolean
description: |

View file

@ -27,7 +27,7 @@ with
from {{ ref("int_kpis__dimension_dates") }} as ikdd
left join
{{ ref("int_core__user_host") }} as icuh
on ikdd.date >= date(icuh.created_date_utc)
on ikdd.date >= icuh.created_date_utc
left join {{ ref("int_core__deal") }} as icd on icuh.id_deal = icd.id_deal
left join
{{ ref("int_kpis__dimension_daily_accommodation") }} as icmas

View file

@ -41,12 +41,12 @@ with
select
d.date,
h.id_deal,
min(date(h.created_date_utc)) as creation_date_utc,
min(h.created_date_utc) as creation_date_utc,
min(b.created_date_utc) as first_time_booked_date_utc,
max(b.created_date_utc) as last_time_booked_date_utc,
max(b.previous_booked_date) as second_to_last_time_booked_date_utc
from int_kpis__dimension_dates d
inner join int_core__user_host h on d.date >= date(h.created_date_utc)
inner join int_core__user_host h on d.date >= h.created_date_utc
left join
booked_days_per_deal b
on h.id_deal = b.id_deal

View file

@ -26,7 +26,9 @@ select
avatar as avatar,
id_airbnb as id_airbnb,
airbnb_url as airbnb_url,
created_at_utc as created_at_utc,
created_date_utc as created_date_utc,
updated_at_utc as updated_at_utc,
updated_date_utc as updated_date_utc,
verified_at_utc as verified_at_utc,
verified_date_utc as verified_date_utc,

View file

@ -736,13 +736,21 @@ models:
data_type: character varying
description: ""
- name: created_at_utc
data_type: timestamp
description: The timestamp when the user was created.
- name: created_date_utc
data_type: timestamp without time zone
description: The date time when the user data was created.
data_type: date
description: The date when the user was created.
- name: updated_at_utc
data_type: timestamp
description: The timestamp when the user was updated.
- name: updated_date_utc
data_type: timestamp without time zone
description: The date when the user data was created.
data_type: date
description: The date when the user was updated.
- name: verified_at_utc
data_type: timestamp without time zone

View file

@ -13,8 +13,10 @@ with
{{ adapter.quote("Avatar") }} as avatar,
{{ adapter.quote("AirbnbId") }} as id_airbnb,
{{ adapter.quote("AirbnbUrl") }} as airbnb_url,
{{ adapter.quote("CreatedDate") }} as created_date_utc,
{{ adapter.quote("UpdatedDate") }} as updated_date_utc,
{{ adapter.quote("CreatedDate") }} as created_at_utc,
date({{ adapter.quote("CreatedDate") }}) as created_date_utc,
{{ adapter.quote("UpdatedDate") }} as updated_at_utc,
date({{ adapter.quote("UpdatedDate") }}) as updated_date_utc,
{{ adapter.quote("VerifiedDate") }} as verified_at_utc,
cast({{ adapter.quote("VerifiedDate") }} as date) as verified_date_utc,
{{ adapter.quote("FlagAsProblem") }} as flag_as_problem,