counterweight/data-dwh-dbt-project
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Merged PR 5242: Adds new model in intermediate for new dash deal onboarding

Browse source 
# Description

Adds new model in intermediate for new dash deal onboarding.
Adds additional fields in new dash tables for services completion at program level.

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [ ] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #30249
This commit is contained in:
Oriol Roqué Paniagua 2025-05-16 17:09:30 +00:00
parent 43a20a3c22
commit 5455bd2c01
6 changed files with 606 additions and 3 deletions
Download patch file Download diff file Expand all files Collapse all files

4
models/intermediate/core/int_core__accommodation_to_product_bundle.sql
View file

@ -30,6 +30,10 @@ select
else greatest(date(atpb.ends_at_utc), upb.effective_end_date_utc)
end as effective_end_date_utc,
atpb.has_no_end_date,
upb.product_bundle_services,
upb.is_custom_bundle,
upb.has_upgraded_services,
upb.has_billable_services,
atpb.created_at_utc,
atpb.created_date_utc,
atpb.updated_at_utc,

32
models/intermediate/core/int_core__user_product_bundle.sql
View file

@ -4,8 +4,27 @@ with
select * from {{ ref("stg_core__user_product_bundle") }}
),
stg_core__protection_plan as (select * from {{ ref("stg_core__protection_plan") }}),
int_core__user_host as (select * from {{ ref("int_core__user_host") }})
int_core__user_host as (select * from {{ ref("int_core__user_host") }}),
int_core__user_product_bundle_contains_services as (
select * from {{ ref("int_core__user_product_bundle_contains_services") }}
),
product_bundle_services_agg as (
select
id_user_product_bundle,
case
when sum(cast(not is_default_service as integer)) > 0
then true
else false
end as has_upgraded_services,
case
when sum(cast(is_billable_service as integer)) > 0 then true else false
end as has_billable_services,
string_agg(
distinct service_name, '|' order by service_name asc
) as product_bundle_services
from int_core__user_product_bundle_contains_services
group by 1
)
select
upb.id_user_product_bundle,
upb.id_user as id_user_host,
@ -17,6 +36,7 @@ select
pp.protection_display_name,
upb.display_on_front_end,
upb.chosen_product_services,
pbsa.product_bundle_services,
upb.starts_at_utc as original_starts_at_utc,
upb.ends_at_utc as original_ends_at_utc,
/*
@ -39,6 +59,11 @@ select
upb.created_date_utc,
upb.updated_at_utc,
upb.updated_date_utc,
case
when upb.id_product_bundle is null then true else false
end as is_custom_bundle,
pbsa.has_upgraded_services,
pbsa.has_billable_services,
upb.dwh_extracted_at
from stg_core__user_product_bundle upb
/*
@ -55,3 +80,6 @@ inner join
and uh.is_missing_id_deal = false
and uh.is_test_account = false
left join stg_core__protection_plan pp on upb.id_protection_plan = pp.id_protection_plan
left join
product_bundle_services_agg pbsa
on upb.id_user_product_bundle = pbsa.id_user_product_bundle

2
models/intermediate/core/int_core__user_product_bundle_contains_services.sql
View file

@ -27,6 +27,7 @@ select
pb.product_bundle_name,
ps.service_business_type,
ps.is_default_service,
ps.is_billable_service,
ps.product_service_display_name as service_name
from stg_core__user_product_bundle pb
inner join int_core__user_host uh on pb.id_user = uh.id_user_host
@ -44,6 +45,7 @@ select
pb.product_bundle_name,
'PROTECTION' as service_business_type,
pp.is_default_service,
pp.is_billable_service,
pp.protection_display_name as service_name
from stg_core__user_product_bundle pb
inner join int_core__user_host uh on pb.id_user = uh.id_user_host

53
models/intermediate/core/schema.yml
View file

@ -1927,6 +1927,11 @@ models:
a chosen_product_services = 257 means it has the services 1 + 256, which are the
Basic Screening and the Waiver Pro.
- name: product_bundle_services
data_type: string
description: |
List of services that are included in this bundle, separated by "|", ordered alphabetically.
- name: original_starts_at_utc
data_type: timestamp
description: |
@ -1987,6 +1992,23 @@ models:
description: |
Date of when this User has Product Bundle was last updated in the Backend.
- name: is_custom_bundle
data_type: boolean
description: |
True if the bundle or program is custom, false if it's a default one.
- name: has_billable_services
data_type: boolean
description: |
True if the bundle or program contains at least one billable service, false
otherwise.
- name: has_upgraded_services
data_type: boolean
description: |
True if the bundle or program contains at least one service different than
the default one, false otherwise.
- name: dwh_extracted_at
data_type: timestamp
description: |
@ -2141,7 +2163,7 @@ models:
data_tests:
- not_null
- name: product_bundle_name
- name: user_product_bundle_name
data_type: string
description: |
The name of the Product Bundle.
@ -2208,6 +2230,29 @@ models:
description: |
Date of when this Accommodation to Product Bundle record was last updated in the Backend.
- name: product_bundle_services
data_type: string
description: |
List of services that are included in the bundle applied to the listing,
separated by "|", ordered alphabetically.
- name: is_custom_bundle
data_type: boolean
description: |
True if the bundle or program applied to the listing is custom, false if it's a default one.
- name: has_billable_services
data_type: boolean
description: |
True if the bundle or program applied to the listing contains at least one billable service,
false otherwise.
- name: has_upgraded_services
data_type: boolean
description: |
True if the bundle or program applied to the listing contains at least one service different
than the default one, false otherwise.
- name: dwh_extracted_at
data_type: timestamp with timezone
description: |
@ -2842,6 +2887,12 @@ models:
data_tests:
- not_null
- name: is_billable_service
data_type: boolean
description: |
Flag that determines if the service is billable to the
host (True) or not (False).
- name: int_core__product_service_to_price
description: |
This model provides the information related to the prices of the different

282
models/intermediate/cross/int_new_dash_deal_onboarding.sql Normal file
View file

@ -0,0 +1,282 @@
{{ config(materialized="table") }}
with
int_core__user_host as (select * from {{ ref("int_core__user_host") }}),
int_hubspot__deal as (select * from {{ ref("int_hubspot__deal") }}),
int_core__accommodation as (select * from {{ ref("int_core__accommodation") }}),
int_core__booking_summary as (select * from {{ ref("int_core__booking_summary") }}),
int_core__user_product_bundle_contains_services as (
select * from {{ ref("int_core__user_product_bundle_contains_services") }}
),
int_core__user_product_bundle as (
select * from {{ ref("int_core__user_product_bundle") }}
),
int_core__accommodation_to_product_bundle as (
select * from {{ ref("int_core__accommodation_to_product_bundle") }}
),
int_xero__sales_denom_mart as (
select * from {{ ref("int_xero__sales_denom_mart") }}
),
new_dash_new_business_accounts as (
select
ihd.id_deal,
ihd.deal_name,
ihd.onboarding_owner,
ihd.account_manager,
ihd.live_date_utc,
ihd.contract_signed_date_utc,
ihd.cancellation_date_utc,
ihd.expressed_service_interest,
string_agg(icuh.company_name, ', ') as platform_company_name,
min(
icuh.user_in_new_dash_since_timestamp_at_utc
) as backend_account_creation_utc,
count(distinct icuh.id_user_host) as count_platform_accounts
from int_core__user_host icuh
-- Deal needs to exist in HS
inner join int_hubspot__deal ihd on icuh.id_deal = ihd.id_deal
where
-- Filters to replicate New Dash Overview
icuh.is_user_in_new_dash = true
and icuh.is_missing_id_deal = false
and icuh.is_test_account = false
-- Filter to only select new business (i.e., not migrated from Old Dash)
and icuh.has_user_moved_from_old_dash = false
group by 1, 2, 3, 4, 5, 6, 7, 8
),
program_creation_per_deal as (
select
id_deal,
min(icupb.created_at_utc) as first_program_created_at_utc,
min(
case
when icupb.has_upgraded_services then icupb.created_at_utc else null
end
) as first_upgraded_program_created_at_utc,
count(
distinct icupb.id_user_product_bundle
) as count_programs_at_deal_level,
count(
distinct case
when icupb.has_upgraded_services
then icupb.id_user_product_bundle
else null
end
) as count_upgraded_programs_at_deal_level,
string_agg(
distinct service_name, '|' order by service_name asc
) as services_in_programs_at_deal_level
from int_core__user_product_bundle icupb
inner join int_core__user_host icuh on icupb.id_user_host = icuh.id_user_host
left join
int_core__user_product_bundle_contains_services cs
on icupb.id_user_product_bundle = cs.id_user_product_bundle
where icuh.id_deal is not null
group by 1
),
listing_creation_per_deal as (
select
id_deal,
min(ica.created_at_utc) as first_listing_created_at_utc,
count(distinct ica.id_accommodation) as count_listings,
count(
distinct
case when ica.is_active = true then ica.id_accommodation else null end
) as count_active_listings
from int_core__accommodation ica
inner join int_core__user_host icuh on ica.id_user_host = icuh.id_user_host
where
-- Filters to replicate New Dash Overview
icuh.is_user_in_new_dash = true
and icuh.is_missing_id_deal = false
and icuh.is_test_account = false
-- Filter to only select new business (i.e., not migrated from Old Dash)
and icuh.has_user_moved_from_old_dash = false
group by 1
),
upgraded_program_applied_to_listing_per_deal as (
select
id_deal,
min(
atpb.created_at_utc
) as first_upgraded_program_applied_to_listing_at_utc,
count(
distinct atpb.id_user_product_bundle
) as count_upgraded_programs_at_listing_level,
count(
distinct case
when atpb.has_no_end_date and ica.is_active
then atpb.id_user_product_bundle
else null
end
) as count_active_upgraded_programs_at_active_listing_level,
count(
distinct atpb.id_accommodation
) as count_listings_with_upgraded_programs,
count(
distinct case
when atpb.has_no_end_date and ica.is_active
then atpb.id_accommodation
else null
end
) as count_active_listings_with_active_upgraded_programs,
string_agg(
distinct service_name, '|' order by service_name asc
) as services_in_programs_applied_to_listings,
string_agg(
distinct case
when atpb.has_no_end_date and ica.is_active
then service_name
else null
end,
'|'
order by
case
when atpb.has_no_end_date and ica.is_active
then service_name
else null
end asc
) as active_services_in_programs_applied_to_listings
from int_core__accommodation_to_product_bundle atpb
inner join int_core__user_host icuh on atpb.id_user_host = icuh.id_user_host
left join
int_core__user_product_bundle_contains_services cs
on atpb.id_user_product_bundle = cs.id_user_product_bundle
left join
int_core__accommodation ica on atpb.id_accommodation = ica.id_accommodation
where
icuh.id_deal is not null
-- Force that programs in listings have upgraded services
and atpb.has_upgraded_services = true
group by 1
),
booking_creation_per_deal as (
select
id_deal,
min(bs.booking_created_at_utc) as first_booking_created_at_utc,
min(
case
when bs.has_paid_services then bs.booking_created_at_utc else null
end
) as first_booking_with_paid_services_created_at_utc,
count(distinct bs.id_booking) as count_bookings,
count(
distinct case when bs.has_paid_services then bs.id_booking else null end
) as count_bookings_with_paid_service
from int_core__booking_summary bs
where id_deal is not null
group by 1
),
invoice_per_deal as (
select
id_deal,
min(document_issued_at_utc) at time zone 'utc' as first_invoice_at_utc
from int_xero__sales_denom_mart
where
-- Select only invoices
upper(document_class) = 'INVOICE'
and upper(document_status) in ('PAID', 'AUTHORISED')
and id_deal is not null
group by 1
),
combination_of_sources as (
select
ndnba.id_deal,
ndnba.deal_name,
ndnba.onboarding_owner,
ndnba.account_manager,
ndnba.platform_company_name,
ndnba.count_platform_accounts,
-- Activity --
coalesce(l.count_listings, 0) as count_listings,
coalesce(l.count_active_listings, 0) as count_active_listings,
coalesce(
upl.count_listings_with_upgraded_programs, 0
) as count_listings_with_upgraded_programs,
coalesce(
upl.count_active_listings_with_active_upgraded_programs, 0
) as count_active_listings_with_active_upgraded_programs,
coalesce(b.count_bookings, 0) as count_bookings,
coalesce(
b.count_bookings_with_paid_service, 0
) as count_bookings_with_paid_service,
-- Program funnel --
coalesce(p.count_programs_at_deal_level, 0) as count_programs_at_deal_level,
coalesce(
p.count_upgraded_programs_at_deal_level, 0
) as count_upgraded_programs_at_deal_level,
coalesce(
upl.count_upgraded_programs_at_listing_level, 0
) as count_upgraded_programs_at_listing_level,
coalesce(
upl.count_active_upgraded_programs_at_active_listing_level, 0
) as count_active_upgraded_programs_at_active_listing_level,
-- HubSpot main dates --
ndnba.contract_signed_date_utc,
ndnba.live_date_utc,
ndnba.cancellation_date_utc,
-- First time timestamps --
ndnba.backend_account_creation_utc,
l.first_listing_created_at_utc,
b.first_booking_created_at_utc,
p.first_program_created_at_utc,
p.first_upgraded_program_created_at_utc,
upl.first_upgraded_program_applied_to_listing_at_utc,
b.first_booking_with_paid_services_created_at_utc,
inv.first_invoice_at_utc,
-- Services --
ndnba.expressed_service_interest,
p.services_in_programs_at_deal_level,
upl.services_in_programs_applied_to_listings,
upl.active_services_in_programs_applied_to_listings,
-- Additional for filtering --
case
when ndnba.cancellation_date_utc is not null then true else false
end as has_churned
from new_dash_new_business_accounts ndnba
left join listing_creation_per_deal l on ndnba.id_deal = l.id_deal
left join program_creation_per_deal p on ndnba.id_deal = p.id_deal
left join
upgraded_program_applied_to_listing_per_deal upl
on ndnba.id_deal = upl.id_deal
left join booking_creation_per_deal b on ndnba.id_deal = b.id_deal
left join invoice_per_deal inv on ndnba.id_deal = inv.id_deal
)
select
*,
-- Basic Alerts --
case
when first_listing_created_at_utc is null and count_listings = 0
then false
else true
end as has_listings,
case when count_active_listings = 0 then false else true end as has_active_listings,
case
when first_booking_created_at_utc is null and count_bookings = 0
then false
else true
end as has_bookings,
case
when first_invoice_at_utc is null then false else true
end as has_been_invoiced,
-- Advanced Alerts --
case
when count_bookings > 0 and count_bookings_with_paid_service = 0
then true
else false
end as are_all_bookings_free,
case
when
count_bookings_with_paid_service > 0
and count_active_upgraded_programs_at_active_listing_level = 0
then true
else false
end as is_account_no_longer_generating_paid_bookings,
case
when
active_services_in_programs_applied_to_listings
<> services_in_programs_applied_to_listings
then true
else false
end as has_account_changed_services_applied_in_listings
from combination_of_sources

236
models/intermediate/cross/schema.yml
View file

@ -3695,3 +3695,239 @@ models:
description: |
Flag indicating if the growth score is overridden to -1 due to
cancellation in the same month.
- name: int_new_dash_deal_onboarding
description: |
A dedicated model to track the onboarding stages of new accounts (deals)
in New Dash.
This excludes any deal that has been migrated from Old Dash, so it just
contains "new business".
columns:
- name: id_deal
data_type: text
description: |
Unique identifier of an account.
data_tests:
- not_null
- unique
- name: deal_name
data_type: text
description: |
Name of the deal according to HubSpot.
- name: onboarding_owner
data_type: text
description: |
Name of the person that is in charge of onboarding this account.
- name: account_manager
data_type: text
description: |
Account manager in charge of the account.
- name: platform_company_name
data_type: text
description: |
Name of the company in Truvi's backend.
- name: count_platform_accounts
data_type: integer
description: |
Amount of Backend accounts (users) linked to this Deal.
- name: count_programs_at_deal_level
data_type: integer
description: |
Total amount of programs that this account has. These might not
necessarily be applied to a Listing.
- name: count_listings
data_type: integer
description: |
Total count of Listings from this account.
- name: count_active_listings
data_type: integer
description: |
Count of Listings that are currently active, meaning, that have not
been deactivated.
- name: count_listings_with_upgraded_programs
data_type: integer
description: |
Count of Listings that have had a program applied that contains at least
one service different to Basic Screening.
- name: count_active_listings_with_active_upgraded_programs
data_type: integer
description: |
Count of Listings that are currently active and that currently have an
active upgraded program, meaning, that contains at least one service
different to Basic Screening.
- name: count_bookings
data_type: integer
description: |
Total count of Bookings generated from this account.
- name: count_bookings_with_paid_service
data_type: integer
description: |
Count of Bookings that have at least one paid service.
- name: count_upgraded_programs_at_deal_level
data_type: integer
description: |
Count of programs that this account has that contain, at least, one
service different to Basic Screening. These might not necessarily
be applied to a Listing.
- name: count_upgraded_programs_at_listing_level
data_type: integer
description: |
Count of programs that contain at least one service different to
Basic Screening that have been applied to a Listing.
- name: count_active_upgraded_programs_at_active_listing_level
data_type: integer
description: |
Count of programs that contain at least one service different to
Basic Screening that are currently active and applied to an
active Listing.
- name: contract_signed_date_utc
data_type: date
description: |
Date in which the contract was signed according to HubSpot.
- name: live_date_utc
data_type: date
description: |
Date in which the Deal went live according to HubSpot.
- name: cancellation_date_utc
data_type: date
description: |
Date in which the Deal was cancelled according to HubSpot.
- name: backend_account_creation_utc
data_type: timestamp without timezone
description: |
Timestamp in which the account was created in the backend.
- name: first_listing_created_at_utc
data_type: timestamp without timezone
description: |
Timestamp in which the first listing was created in the backend.
- name: first_booking_created_at_utc
data_type: timestamp without timezone
description: |
Timestamp in which the first booking was created in the backend.
- name: first_program_created_at_utc
data_type: timestamp without timezone
description: |
Timestamp in which the first program was created at account level
in the backend.
- name: first_upgraded_program_created_at_utc
data_type: timestamp without timezone
description: |
Timestamp in which the first program was applied to a listing
for this account, in the backend.
- name: first_upgraded_program_applied_to_listing_at_utc
data_type: timestamp without timezone
description: |
Timestamp in which the first upgraded program was applied to a listing
for this account, in the backend.
- name: first_booking_with_paid_services_created_at_utc
data_type: timestamp without timezone
description: |
Timestamp in which the first booking that contained paid services
was created for this account, in the backend.
- name: first_invoice_at_utc
data_type: timestamp without timezone
description: |
Timestamp in which the first invoice happened for this account in Xero.
- name: expressed_service_interest
data_type: text
description: |
List of services that during onboarding generated interest
to the client.
- name: services_in_programs_at_deal_level
data_type: text
description: |
List of all distinct services that appear in programs at
deal level, separated by "|" and ordered alphabetically.
- name: services_in_programs_applied_to_listings
data_type: text
description: |
List of all distinct services that are applied in all listings
for this account, separated by "|" and ordered alphabetically.
- name: has_churned
data_type: boolean
description: |
True if the account has a cancellation date in HubSpot,
False otherwise.
- name: has_listings
data_type: boolean
description: |
True if the account has at least one listing appearing in
the backend, False otherwise.
- name: has_active_listings
data_type: boolean
description: |
True if the account has at least one listing that is currently
active in the backend, False otherwise.
- name: has_bookings
data_type: boolean
description: |
True if the account has at least one booking appearing in
the backend, False otherwise.
- name: has_been_invoiced
data_type: boolean
description: |
True if the account has at least one invoice appearing in
Xero, False otherwise.
- name: are_all_bookings_free
data_type: boolean
description: |
True if the account has bookings but all of them are free,
meaning, there's not a single service being paid.
- name: is_account_no_longer_generating_paid_bookings
data_type: boolean
description: |
True if the account has had in the past paid bookings but
at the moment there's not a single listing that contains
an active upgraded program.
Encouraged to be used alongside a filter to determine if
the account has churned or not.
- name: has_account_changed_services_applied_in_listings
data_type: boolean
description: |
True if the active services in programs applied to listings
is different than the services that were historically applied.
By nature, this always means that at least one service was
being applied before that is no longer being applied. This can
indicate a real decrease in services applied that could be linked to
a potential revenue loss (decrease in booking fees),
although it's also possible that the account has changed from a
certain low-level tier to a higher-level one (ex: from Basic
Protection to Protection Pro).