Merged PR 3909: Fixes created_date_utc and updated_date_utc in SH User and its dependencies

# Description

The model `stg_core__superhog_user` contained the fields `created_date_utc` and `updated_date_utc`, which were actually timestamps, not dates. This created issues a while ago in KPIs that were fixed by forcing a date conversion at `int_kpis` level, but the source issue remained unfixed.

This PR just creates the proper fields in staging, meaning:
* `created_at_utc` and `created_date_utc`
* `updated_at_utc` and `updated_date_utc`

And propagates the changes through all dependencies. On KPIs, it also removes the hardcoded date conversion since it's no longer needed. It also adds the different schema entries on which these fields were already defined. Note I didn't update the staging entry since there's no entry for the rest of the fields.

I recommend checking this PR in the commits order.

What I've tested:
* Unified User and User Host have plenty of dependencies that I've checked one by one, at each dependency layer by using dbt docs.
* Run `core__unified_user` and the full execution of +`mtd_aggregated_metrics`

What clearly will fail:
* There's the famous `Users_dashboard` PBI report that shouldn't exist that has a dependency with `core__unified_user`. This report will fail. But it does not exist. I'd like to drop it once and forever

# Checklist

- [X] The edited models and dependants run properly with production data. **See above**
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.

# Other

- [X] Check if a full-refresh is required after this PR is merged. **The staging model is incremental!**

Related work items: #23703
This commit is contained in:
Oriol Roqué Paniagua 2024-12-31 08:04:03 +00:00
parent 803c5ed3d5
commit d26e08bffd
8 changed files with 42 additions and 17 deletions

View file

@ -38,7 +38,9 @@ select
su.avatar, su.avatar,
su.id_airbnb, su.id_airbnb,
su.airbnb_url, su.airbnb_url,
su.created_at_utc,
su.created_date_utc, su.created_date_utc,
su.updated_at_utc,
su.updated_date_utc, su.updated_date_utc,
su.verified_at_utc, su.verified_at_utc,
su.verified_date_utc, su.verified_date_utc,

View file

@ -48,7 +48,9 @@ select
d.main_billing_country_iso_3_per_deal, d.main_billing_country_iso_3_per_deal,
uu.joined_at_utc, uu.joined_at_utc,
uu.joined_date_utc, uu.joined_date_utc,
uu.created_at_utc,
uu.created_date_utc, uu.created_date_utc,
uu.updated_at_utc,
uu.updated_date_utc, uu.updated_date_utc,
case when uu.id_deal is null then true else false end as is_missing_id_deal, case when uu.id_deal is null then true else false end as is_missing_id_deal,
case case

View file

@ -276,12 +276,18 @@ models:
- name: airbnb_url - name: airbnb_url
data_type: character varying data_type: character varying
description: The Airbnb URL. description: The Airbnb URL.
- name: created_at_utc
data_type: timestamp
description: The timestamp when the user was created.
- name: created_date_utc - name: created_date_utc
data_type: date
description: The date when the user was created.
- name: updated_at_utc
data_type: timestamp data_type: timestamp
description: The timestamp when the user was created. The field name needs to be fixed. description: The timestamp when the user was updated.
- name: updated_date_utc - name: updated_date_utc
data_type: timestamp data_type: date
description: The timestamp when the user was updated. The field name needs to be fixed. description: The date when the user was updated.
- name: verified_at_utc - name: verified_at_utc
data_type: timestamp data_type: timestamp
description: The timestamp when the user was verified. description: The timestamp when the user was verified.
@ -1965,15 +1971,18 @@ models:
data_type: date data_type: date
description: | description: |
Date of when the Host user joined Superhog. Date of when the Host user joined Superhog.
- name: created_at_utc
data_type: timestamp
description: The timestamp when the user was created.
- name: created_date_utc - name: created_date_utc
data_type: date data_type: date
description: | description: The date when the user was created.
Date of when the Host user was created in our systems. - name: updated_at_utc
data_type: timestamp
description: The timestamp when the user was updated.
- name: updated_date_utc - name: updated_date_utc
data_type: date data_type: date
description: | description: The date when the user was updated.
Date of the last time the information of the Host was updated
in our systems.
- name: is_missing_id_deal - name: is_missing_id_deal
data_type: boolean data_type: boolean
description: | description: |

View file

@ -27,7 +27,7 @@ with
from {{ ref("int_kpis__dimension_dates") }} as ikdd from {{ ref("int_kpis__dimension_dates") }} as ikdd
left join left join
{{ ref("int_core__user_host") }} as icuh {{ ref("int_core__user_host") }} as icuh
on ikdd.date >= date(icuh.created_date_utc) on ikdd.date >= icuh.created_date_utc
left join {{ ref("int_core__deal") }} as icd on icuh.id_deal = icd.id_deal left join {{ ref("int_core__deal") }} as icd on icuh.id_deal = icd.id_deal
left join left join
{{ ref("int_kpis__dimension_daily_accommodation") }} as icmas {{ ref("int_kpis__dimension_daily_accommodation") }} as icmas

View file

@ -41,12 +41,12 @@ with
select select
d.date, d.date,
h.id_deal, h.id_deal,
min(date(h.created_date_utc)) as creation_date_utc, min(h.created_date_utc) as creation_date_utc,
min(b.created_date_utc) as first_time_booked_date_utc, min(b.created_date_utc) as first_time_booked_date_utc,
max(b.created_date_utc) as last_time_booked_date_utc, max(b.created_date_utc) as last_time_booked_date_utc,
max(b.previous_booked_date) as second_to_last_time_booked_date_utc max(b.previous_booked_date) as second_to_last_time_booked_date_utc
from int_kpis__dimension_dates d from int_kpis__dimension_dates d
inner join int_core__user_host h on d.date >= date(h.created_date_utc) inner join int_core__user_host h on d.date >= h.created_date_utc
left join left join
booked_days_per_deal b booked_days_per_deal b
on h.id_deal = b.id_deal on h.id_deal = b.id_deal

View file

@ -26,7 +26,9 @@ select
avatar as avatar, avatar as avatar,
id_airbnb as id_airbnb, id_airbnb as id_airbnb,
airbnb_url as airbnb_url, airbnb_url as airbnb_url,
created_at_utc as created_at_utc,
created_date_utc as created_date_utc, created_date_utc as created_date_utc,
updated_at_utc as updated_at_utc,
updated_date_utc as updated_date_utc, updated_date_utc as updated_date_utc,
verified_at_utc as verified_at_utc, verified_at_utc as verified_at_utc,
verified_date_utc as verified_date_utc, verified_date_utc as verified_date_utc,

View file

@ -736,13 +736,21 @@ models:
data_type: character varying data_type: character varying
description: "" description: ""
- name: created_at_utc
data_type: timestamp
description: The timestamp when the user was created.
- name: created_date_utc - name: created_date_utc
data_type: timestamp without time zone data_type: date
description: The date time when the user data was created. description: The date when the user was created.
- name: updated_at_utc
data_type: timestamp
description: The timestamp when the user was updated.
- name: updated_date_utc - name: updated_date_utc
data_type: timestamp without time zone data_type: date
description: The date when the user data was created. description: The date when the user was updated.
- name: verified_at_utc - name: verified_at_utc
data_type: timestamp without time zone data_type: timestamp without time zone

View file

@ -13,8 +13,10 @@ with
{{ adapter.quote("Avatar") }} as avatar, {{ adapter.quote("Avatar") }} as avatar,
{{ adapter.quote("AirbnbId") }} as id_airbnb, {{ adapter.quote("AirbnbId") }} as id_airbnb,
{{ adapter.quote("AirbnbUrl") }} as airbnb_url, {{ adapter.quote("AirbnbUrl") }} as airbnb_url,
{{ adapter.quote("CreatedDate") }} as created_date_utc, {{ adapter.quote("CreatedDate") }} as created_at_utc,
{{ adapter.quote("UpdatedDate") }} as updated_date_utc, date({{ adapter.quote("CreatedDate") }}) as created_date_utc,
{{ adapter.quote("UpdatedDate") }} as updated_at_utc,
date({{ adapter.quote("UpdatedDate") }}) as updated_date_utc,
{{ adapter.quote("VerifiedDate") }} as verified_at_utc, {{ adapter.quote("VerifiedDate") }} as verified_at_utc,
cast({{ adapter.quote("VerifiedDate") }} as date) as verified_date_utc, cast({{ adapter.quote("VerifiedDate") }} as date) as verified_date_utc,
{{ adapter.quote("FlagAsProblem") }} as flag_as_problem, {{ adapter.quote("FlagAsProblem") }} as flag_as_problem,