counterweight/data-dwh-dbt-project
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Merged PR 3625: Excludes new dash users without id deal

Browse source 
# Description

This PR has 2 commits:
- The first one handles the removal from the computation any user that has not an id deal properly set. I just created a boolean field in int_core__user_host that identifies if the host has no id_deal. Then apply the new condition in the 2 main usages of New Dash info.
- The second one cleans New Dash KPIs. Since we do not have anymore users without deals, it means that the identification of the host/account is going to be exactly the same if done by id_user_host or id_deal. I hated having id_user_host in KPIs so I've removed it :)

Lastly, this should speed up massively the execution. Not because there's improvements on the code but rather the reduction of data.

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #20809
This commit is contained in:
Oriol Roqué Paniagua 2024-11-21 16:30:47 +00:00
parent 2e7c85d11b
commit b0bed479c7
9 changed files with 16 additions and 39 deletions
Download patch file Download diff file Expand all files Collapse all files

4
macros/business_kpis_configuration.sql
View file

@ -65,9 +65,6 @@ Please note that strings should be encoded with " ' your_value_here ' ",
{% macro dim_has_id_check() %}
{{ return({"dimension": "'by_has_id_check'", "dimension_value": "has_id_check"}) }}
{% endmacro %}
{% macro dim_host() %}
{{ return({"dimension": "'by_host'", "dimension_value": "id_user_host"}) }}
{% endmacro %}
{% macro dim_has_upgraded_service() %}
{{
return(
@ -143,7 +140,6 @@ Provides a general assignement for the Dimensions available for each KPI
{% if entity_name == "NEW_DASH_CREATED_SERVICES" %}
{% set additional_dimensions = additional_dimensions + [
dim_host(),
dim_has_upgraded_service(),
dim_new_dash_version(),
dim_pricing_service(),

1
models/intermediate/core/int_core__user_host.sql
View file

@ -49,6 +49,7 @@ select
uu.joined_date_utc,
uu.created_date_utc,
uu.updated_date_utc,
case when uu.id_deal is null then true else false end as is_missing_id_deal,
case
when ndu.id_user_host is not null then true else false
end as is_user_in_new_dash,

3
models/intermediate/core/int_core__user_product_bundle.sql
View file

@ -44,7 +44,7 @@ from stg_core__user_product_bundle upb
/*
There are many not migrated users that have product bundles.
Since users are not migrated, these bundles cannot be active.
In order to avoid potential upstream problems, it is enforced
In order to avoid potential downstream problems, it is enforced
that product bundles need to be from users that have been
already migrated.
*/
@ -52,4 +52,5 @@ inner join
int_core__user_host uh
on upb.id_user = uh.id_user_host
and uh.is_user_in_new_dash = true
and uh.is_missing_id_deal = false
left join stg_core__protection_plan pp on upb.id_protection_plan = pp.id_protection_plan

5
models/intermediate/core/schema.yml
View file

@ -1850,6 +1850,11 @@ models:
description: |
Date of the last time the information of the Host was updated
in our systems.
- name: is_missing_id_deal
data_type: boolean
description: |
Flag to identify if a user is missing the id_deal (true) or
not (false).
- name: is_user_in_new_dash
data_type: boolean
description: |

4
models/intermediate/kpis/int_kpis__metric_daily_new_dash_created_services.sql
View file

@ -5,7 +5,6 @@ select
date(icbsd.service_detail_created_at_utc) as date,
coalesce(icbsd.service_name) as service_name,
-- Dimensions --
coalesce(icbs.id_user_host, 'UNSET') as id_user_host,
coalesce(icbs.id_deal, 'UNSET') as id_deal,
case when icbsd.is_upgraded_service then 'YES' else 'NO' end as is_upgraded_service,
coalesce(icbs.new_dash_version, 'UNSET') as new_dash_version,
@ -28,6 +27,7 @@ left join
and date(icbsd.service_detail_created_at_utc) = icmas.date
where
icbs.is_user_in_new_dash = true
and icbs.is_missing_id_deal = false
and icbsd.service_detail_created_at_utc
>= icbs.user_in_new_dash_since_timestamp_at_utc
group by 1, 2, 3, 4, 5, 6, 7, 8, 9
group by 1, 2, 3, 4, 5, 6, 7, 8

4
models/intermediate/kpis/int_kpis__metric_monthly_new_dash_created_services.sql
View file

@ -4,7 +4,6 @@
unique_key=[
"end_date",
"service_name",
"id_user_host",
"new_dash_version",
"active_accommodations_per_deal_segmentation",
],
@ -16,7 +15,6 @@ select
d.first_day_month as start_date,
d.date as end_date,
cs.service_name,
cs.id_user_host,
cs.active_accommodations_per_deal_segmentation,
-- Dimensions --
cs.new_dash_version,
@ -31,4 +29,4 @@ left join
{{ ref("int_kpis__metric_daily_new_dash_created_services") }} cs
on date_trunc('month', cs.date)::date = d.first_day_month
where d.is_end_of_month = true and cs.id_deal is not null
group by 1, 2, 3, 4, 5, 6, 7, 8, 9
group by 1, 2, 3, 4, 5, 6, 7, 8

4
models/intermediate/kpis/int_kpis__metric_weekly_new_dash_created_services.sql
View file

@ -4,7 +4,6 @@
unique_key=[
"end_date",
"service_name",
"id_user_host",
"new_dash_version",
"active_accommodations_per_deal_segmentation",
],
@ -16,7 +15,6 @@ select
d.first_day_week as start_date,
d.date as end_date,
cs.service_name,
cs.id_user_host,
cs.active_accommodations_per_deal_segmentation,
-- Dimensions --
cs.new_dash_version,
@ -31,4 +29,4 @@ left join
{{ ref("int_kpis__metric_daily_new_dash_created_services") }} cs
on date_trunc('week', cs.date)::date = d.first_day_week
where d.is_end_of_week = true and cs.id_deal is not null
group by 1, 2, 3, 4, 5, 6, 7, 8, 9
group by 1, 2, 3, 4, 5, 6, 7, 8

29
models/intermediate/kpis/schema.yml
View file

@ -5324,12 +5324,6 @@ models:
tests:
- not_null
- name: id_user_host
data_type: string
description: Unique identifier of the user that acts as Host.
tests:
- not_null
- name: id_deal
data_type: string
description: Unique identifier of an account.
@ -5397,7 +5391,7 @@ models:
The unique key corresponds to:
- end_date,
- service_name,
- id_user_host,
- id_deal,
- active_accommodations_per_deal_segmentation.
tests:
@ -5405,7 +5399,7 @@ models:
combination_of_columns:
- end_date
- service_name
- id_user_host
- id_deal
- active_accommodations_per_deal_segmentation
columns:
@ -5429,12 +5423,6 @@ models:
tests:
- not_null
- name: id_user_host
data_type: string
description: Unique identifier of the user that acts as Host.
tests:
- not_null
- name: id_deal
data_type: string
description: Unique identifier of an account.
@ -5510,7 +5498,7 @@ models:
The unique key corresponds to:
- end_date,
- service_name,
- id_user_host,
- id_deal,
- active_accommodations_per_deal_segmentation.
tests:
@ -5518,7 +5506,7 @@ models:
combination_of_columns:
- end_date
- service_name
- id_user_host
- id_deal
- active_accommodations_per_deal_segmentation
columns:
@ -5542,12 +5530,6 @@ models:
tests:
- not_null
- name: id_user_host
data_type: string
description: Unique identifier of the user that acts as Host.
tests:
- not_null
- name: id_deal
data_type: string
description: Unique identifier of an account.
@ -5655,7 +5637,6 @@ models:
- by_new_dash_version
- by_has_upgraded_service
- by_service
- by_host
- name: dimension_value
data_type: string
@ -5778,7 +5759,6 @@ models:
- by_new_dash_version
- by_has_upgraded_service
- by_service
- by_host
- name: dimension_value
data_type: string
@ -5844,7 +5824,6 @@ models:
- by_new_dash_version
- by_has_upgraded_service
- by_service
- by_host
- name: dimension_value
data_type: string

1
models/reporting/kpis/schema.yml
View file

@ -250,7 +250,6 @@ models:
- "By New Dash Version"
- "By Has Upgraded Service"
- "By Service"
- "By Host"
- name: dimension_value
data_type: string