counterweight/data-dwh-dbt-project
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

change "tests:" to "data_tests:"

Browse source
This commit is contained in:
Pablo Martin 2025-01-13 16:00:35 +01:00
parent db1edf7a1c
commit c3c628aec4
29 changed files with 1565 additions and 1565 deletions
Download patch file Download diff file Expand all files Collapse all files

34
models/staging/athena/schema.yml
View file

@ -12,21 +12,21 @@ models:
- name: id_verification
data_type: character varying
description: "Unique id for the specific transaction."
tests:
data_tests:
- unique
- not_null
- name: id_booking
data_type: text
description: ""
tests:
data_tests:
- not_null:
where: created_at_utc > '2024-04-01T00:00:00Z' and verification_status != 'InsufficientInformation'
- name: id_user_partner
data_type: text
description: The unique ID of the partner calling the API.
tests:
data_tests:
- not_null
- name: id_accommodation
@ -38,7 +38,7 @@ models:
description: |
Indicates whether the verification is for V1 (Athena) or V2
(e-deposit).
tests:
data_tests:
- not_null
- accepted_values:
values:
@ -51,7 +51,7 @@ models:
of the user. Note that the Athena/e-deposit user also has a configured
nightly fee. It's unclear at this point which one has priority for
billing.
tests:
data_tests:
- dbt_expectations.expect_column_values_to_be_between:
min_value: 0
max_value: 100
@ -61,7 +61,7 @@ models:
data_type: text
description: |
This field shows the outcome of the Verification itself.
tests:
data_tests:
- not_null
- accepted_values:
values:
@ -85,7 +85,7 @@ models:
description: |
Null if the phone number shows no issues, otherwise it details the
problems attached to the given phone number.
tests:
data_tests:
- accepted_values:
values:
- "Phone number not reachable"
@ -95,7 +95,7 @@ models:
- name: watch_list
data_type: text
description: ""
tests:
data_tests:
- accepted_values:
values:
- "Match"
@ -119,7 +119,7 @@ models:
Indicates if the booking has been cancelled or not. At the source,
null and false values have the same meaning, so here we turn nulls
into false to keep thing simple.
tests:
data_tests:
- not_null
- accepted_values:
values:
@ -129,7 +129,7 @@ models:
- name: cancelled_at_utc
data_type: timestamp without time zone
description: If the booking was cancelled, when did we learn about it.
tests:
data_tests:
- not_null:
where: is_cancelled = true
@ -207,7 +207,7 @@ models:
description: |
Timestamp of the last time the record was modified before screening
happened. Only relevant for V1 records.
tests:
data_tests:
- not_null:
where: version = 'V1'
@ -216,7 +216,7 @@ models:
description: |
Date of the last time the record was modified before screening
happened. Only relevant for V1 records.
tests:
data_tests:
- not_null:
where: version = 'V1'
@ -225,7 +225,7 @@ models:
description: |
Timestamp of the last edit of the record, as set by the
Athena/e-deposit application.
tests:
data_tests:
- not_null
- name: updated_date_utc
@ -233,7 +233,7 @@ models:
description: |
Date of the last edit of the record, as set by the
Athena/e-deposit application.
tests:
data_tests:
- not_null
- name: athena_creation_at_utc
@ -252,18 +252,18 @@ models:
data_type: timestamp without time zone
description: |
The internal application timestamp of when this record was created.
tests:
data_tests:
- not_null
- name: created_date_utc
data_type: timestamp without time zone
description: |
The internal application date of when this record was created.
tests:
data_tests:
- not_null
- name: cosmos_db_timestamp_utc
data_type: timestamp with time zone
description: The internal Cosmos DB timestamp of the last record update.
tests:
data_tests:
- not_null

30
models/staging/check_in_hero/schema.yml
View file

@ -9,26 +9,26 @@ models:
- name: id_record
data_type: text
description: Unique identifier for the record.
tests:
data_tests:
- unique
- not_null
- name: id_user_partner
data_type: text
description: Identifier for the partner user associated with the check-in.
tests:
data_tests:
- not_null
- name: id_reservation
data_type: text
description: Unique identifier for the reservation associated with the check-in.
tests:
data_tests:
- not_null
- name: id_accommodation
data_type: text
description: Unique identifier for the accommodation associated with the reservation.
tests:
data_tests:
- not_null
- name: guest_last_name
@ -54,7 +54,7 @@ models:
- name: reservation_total_amount_in_local_currency
data_type: numeric
description: Total cost amount in guest currency for the reservation.
tests:
data_tests:
- dbt_expectations.expect_column_values_to_be_between:
min_value: 0
strictly: true
@ -63,19 +63,19 @@ models:
data_type: text
description: |
Currency ISO code used by guest for the reservation.
tests:
data_tests:
- not_null
- name: channel
data_type: text
description: Booking channel or source.
tests:
data_tests:
- not_null
- name: checkin_date_utc
data_type: date
description: Check-in date for the booking.
tests:
data_tests:
- not_null
- name: checkin_time_utc
@ -85,7 +85,7 @@ models:
- name: checkout_date_utc
data_type: date
description: Check-out date for the booking.
tests:
data_tests:
- not_null
- name: listing_name
@ -119,35 +119,35 @@ models:
- name: updated_at_utc
data_type: timestamp without time zone
description: Timestamp of the last update to the record.
tests:
data_tests:
- not_null
- name: updated_date_utc
data_type: date
description: Date of the last update to the record.
tests:
data_tests:
- not_null
- name: cosmos_creation_at_utc
data_type: timestamp without time zone
description: Timestamp when the record was created in Cosmos DB.
tests:
data_tests:
- not_null
- name: cosmos_creation_date_utc
data_type: date
description: Date when the record was created in Cosmos DB.
tests:
data_tests:
- not_null
- name: created_date_utc
data_type: date
description: Date when the record was created.
tests:
data_tests:
- not_null
- name: cosmos_db_timestamp_utc
data_type: timestamp with time zone
description: Internal Cosmos DB timestamp of the last record update.
tests:
data_tests:
- not_null

20
models/staging/core/_core_sources.yml
View file

@ -11,7 +11,7 @@ sources:
data_type: character varying
quote: True
description: "UUID for the user."
tests:
data_tests:
- unique
- not_null
- dbt_expectations.expect_column_values_to_match_regex:
@ -20,14 +20,14 @@ sources:
data_type: bigint
quote: True
description: "WIP. Is this an increment unique ID?"
tests:
data_tests:
- unique
- not_null
- name: Email
data_type: character varying
quote: True
description: "Email for this user."
tests:
data_tests:
- unique
- not_null
- name: Title
@ -39,20 +39,20 @@ sources:
quote: True
description: "The Hubspot Deal Id that this user account belongs to. Multiple users can all be part of one Deal."
# This test can't be enforced yet due to some bad test data in the production database. Once that's deal with, this should be activated.
#tests:
#data_tests:
# - dbt_expectations.expect_column_values_to_match_regex:
# regex: "^[0-9]{10,11}$"
- name: Deleted
data_type: boolean
quote: True
description: "WIP. I'm guessing this signals that the user wanted to be deleted?"
tests:
data_tests:
- not_null
- name: JoinDate
data_type: timestamp without time zone
quote: True
description: "Timestamp on which the user was created."
tests:
data_tests:
- not_null
- name: LastName
data_type: character varying
@ -62,7 +62,7 @@ sources:
data_type: character varying
quote: True
description: ""
tests:
data_tests:
- not_null
- name: FirstName
data_type: character varying
@ -103,7 +103,7 @@ sources:
A user can be created by another user. If this user has been created by another user, this is the UUID of the creator.
Else, this is null.
tests:
data_tests:
- dbt_expectations.expect_column_values_to_match_regex:
regex: "^[0-9A-Fa-f]{8}-[0-9A-Fa-f]{4}-4[0-9A-Fa-f]{3}-[89ABab][0-9A-Fa-f]{3}-[0-9A-Fa-f]{12}$"
row_condition: '"CreatedUserId" is not null'
@ -151,7 +151,7 @@ sources:
data_type: boolean
quote: True
description: ""
tests:
data_tests:
- not_null
- name: _airbyte_raw_id
data_type: character varying
@ -159,7 +159,7 @@ sources:
- name: _airbyte_extracted_at
data_type: timestamp with time zone
description: "{{ doc('_airbyte_extracted_at_desc') }}"
tests:
data_tests:
- not_null
- dbt_expectations.expect_row_values_to_have_recent_data:
datepart: day

322
models/staging/core/schema.yml
View file

File diff suppressed because it is too large Load diff

34
models/staging/edeposit/schema.yml
View file

@ -12,21 +12,21 @@ models:
- name: id_verification
data_type: character varying
description: "Unique id for the specific transaction."
tests:
data_tests:
- unique
- not_null
- name: id_booking
data_type: text
description: ""
tests:
data_tests:
- not_null:
where: created_at_utc > '2024-04-01T00:00:00Z' and verification_status != 'InsufficientInformation'
- name: id_user_partner
data_type: text
description: The unique ID of the partner calling the API.
tests:
data_tests:
- not_null
- name: id_accommodation
@ -38,7 +38,7 @@ models:
description: |
Indicates whether the verification is for V1 (Athena) or V2
(e-deposit).
tests:
data_tests:
- not_null
- accepted_values:
values:
@ -51,7 +51,7 @@ models:
of the user. Note that the Athena/e-deposit user also has a configured
nightly fee. It's unclear at this point which one has priority for
billing.
tests:
data_tests:
- dbt_expectations.expect_column_values_to_be_between:
min_value: 0
max_value: 100
@ -61,7 +61,7 @@ models:
data_type: text
description: |
This field shows the outcome of the Verification itself.
tests:
data_tests:
- not_null
- accepted_values:
values:
@ -85,7 +85,7 @@ models:
description: |
Null if the phone number shows no issues, otherwise it details the
problems attached to the given phone number.
tests:
data_tests:
- accepted_values:
values:
- "Phone number not reachable"
@ -95,7 +95,7 @@ models:
- name: watch_list
data_type: text
description: ""
tests:
data_tests:
- accepted_values:
values:
- "Match"
@ -119,7 +119,7 @@ models:
Indicates if the booking has been cancelled or not. At the source,
null and false values have the same meaning, so here we turn nulls
into false to keep thing simple.
tests:
data_tests:
- not_null
- accepted_values:
values:
@ -129,7 +129,7 @@ models:
- name: cancelled_at_utc
data_type: timestamp without time zone
description: If the booking was cancelled, when did we learn about it.
tests:
data_tests:
- not_null:
where: is_cancelled = true
@ -207,7 +207,7 @@ models:
description: |
Timestamp of the last time the record was modified before screening
happened. Only relevant for V1 records.
tests:
data_tests:
- not_null:
where: version = 'V1'
@ -216,7 +216,7 @@ models:
description: |
Date of the last time the record was modified before screening
happened. Only relevant for V1 records.
tests:
data_tests:
- not_null:
where: version = 'V1'
@ -225,7 +225,7 @@ models:
description: |
Timestamp of the last edit of the record, as set by the
Athena/e-deposit application.
tests:
data_tests:
- not_null
- name: updated_date_utc
@ -233,7 +233,7 @@ models:
description: |
Date of the last edit of the record, as set by the
Athena/e-deposit application.
tests:
data_tests:
- not_null
- name: athena_creation_at_utc
@ -252,18 +252,18 @@ models:
data_type: timestamp without time zone
description: |
The internal application timestamp of when this record was created.
tests:
data_tests:
- not_null
- name: created_date_utc
data_type: timestamp without time zone
description: |
The internal application date of when this record was created.
tests:
data_tests:
- not_null
- name: cosmos_db_timestamp_utc
data_type: timestamp with time zone
description: The internal Cosmos DB timestamp of the last record update.
tests:
data_tests:
- not_null

54
models/staging/hubspot/schema.yml
View file

@ -7,7 +7,7 @@ models:
- name: id_contact
data_type: character varying
description: "Unique id for each contact information."
tests:
data_tests:
- unique
- not_null
@ -22,37 +22,37 @@ models:
- name: properties
data_type: jsonb
description: "Json with all contact information for this record."
tests:
data_tests:
- not_null
- name: created_at_utc
data_type: timestamp with time zone
description: "Timestamp of when this record was created."
tests:
data_tests:
- not_null
- name: created_date_utc
data_type: timestamp without time zone
description: "Date of when this record was created."
tests:
data_tests:
- not_null
- name: updated_at_utc
data_type: timestamp with time zone
description: "Timestamp of when this record was last updated."
tests:
data_tests:
- not_null
- name: updated_date_utc
data_type: timestamp without time zone
description: "Date of when this record was last updated."
tests:
data_tests:
- not_null
- name: dwh_extracted_at_utc
data_type: timestamp with time zone
description: "Timestamp of when data was extracted to DWH."
tests:
data_tests:
- not_null
- name: stg_hubspot__form_submissions
@ -61,14 +61,14 @@ models:
- name: id_form
data_type: character varying
description: "Unique id for each form submission."
tests:
data_tests:
- not_null
- unique
- name: values
data_type: jsonb
description: "Json with value information for each form"
tests:
data_tests:
- not_null
- name: page_url
@ -78,31 +78,31 @@ models:
- name: submitted_at_utc
data_type: timestamp with time zone
description: "Timestamp of when this record was created."
tests:
data_tests:
- not_null
- name: submitted_date_utc
data_type: timestamp without time zone
description: "Date of when this record was created."
tests:
data_tests:
- not_null
- name: updated_at_utc
data_type: timestamp with time zone
description: "Timestamp of when this record was last updated."
tests:
data_tests:
- not_null
- name: updated_date_utc
data_type: timestamp without time zone
description: "Date of when this record was last updated."
tests:
data_tests:
- not_null
- name: dwh_extracted_at_utc
data_type: timestamp with time zone
description: "Timestamp of when data was extracted to DWH."
tests:
data_tests:
- not_null
- name: stg_hubspot__deal_pipelines
@ -115,33 +115,33 @@ models:
- name: id_deal_pipeline
data_type: character varying
description: "Unique id for each pipeline."
tests:
data_tests:
- not_null
- unique
- name: deal_pipeline_name
data_type: character varying
description: Name for the pipeline.
tests:
data_tests:
- not_null
- unique
- name: is_active
data_type: boolean
description: Flag indicating if the pipeline is currently active.
tests:
data_tests:
- not_null
- name: created_at_utc
data_type: timestamp
description: When was this record created.
tests:
data_tests:
- not_null
- name: updated_at_utc
data_type: timestamp
description: When was this record last updated.
tests:
data_tests:
- not_null
- name: dwh_extracted_at_utc
@ -151,7 +151,7 @@ models:
- name: stg_hubspot__deal_pipeline_stages
description: |
The different stages of deal pipelines.
tests:
data_tests:
- dbt_utils.unique_combination_of_columns:
combination_of_columns:
- id_deal_pipeline
@ -160,13 +160,13 @@ models:
- name: id_deal_pipeline
data_type: character varying
description: ID of the deal pipeline this stage belongs to.
tests:
data_tests:
- not_null
- name: id_stage
data_type: character varying
description: Unique ID for this stage.
tests:
data_tests:
- not_null
- unique
@ -175,25 +175,25 @@ models:
description: |
The name of this stage. There might be name collisions across
pipelines.
tests:
data_tests:
- not_null
- name: is_active
data_type: boolean
description: Flag indicating wheter the stage is currently active.
tests:
data_tests:
- not_null
- name: created_at_utc
data_type: timestamp
description: When was this record created.
tests:
data_tests:
- not_null
- name: updated_at_utc
data_type: timestamp
description: When was this record last updated.
tests:
data_tests:
- not_null
- name: dwh_extracted_at_utc
@ -212,7 +212,7 @@ models:
- name: id_deal
data_type: character varying
description: Unique ID for this deal.
tests:
data_tests:
- unique
- not_null

38
models/staging/screen_and_protect/schema.yml
View file

@ -10,20 +10,20 @@ models:
- name: id_verification
data_type: text
description: Unique identifier for the verification request.
tests:
data_tests:
- unique
- not_null
- name: id_booking
data_type: text
description: Unique identifier for the booking associated with the verification.
tests:
data_tests:
- not_null
- name: id_user_partner
data_type: text
description: Identifier for the partner user initiating the verification.
tests:
data_tests:
- not_null
- name: id_accommodation
@ -33,7 +33,7 @@ models:
- name: protection_type
data_type: text
description: Specific protection type for the request.
tests:
data_tests:
- not_null
- accepted_values:
values:
@ -48,7 +48,7 @@ models:
description: Field used for protection type "STANDALONE PROTECTION" to
indicate the starting amount of protection.
In local currency.
tests:
data_tests:
- dbt_expectations.expect_column_values_to_be_between:
min_value: 0
max_value: 100000
@ -59,7 +59,7 @@ models:
description: Field used for protection type "SCREEN & PROTECT",
"BASIC PROTECTION" or "DAMAGE WAIVER" to show the basic amount of protection.
In local currency.
tests:
data_tests:
- dbt_expectations.expect_column_values_to_be_between:
min_value: 0
max_value: 100000
@ -70,7 +70,7 @@ models:
description: Field used for protection type "SCREEN & PROTECT",
or "STANDALONE PROTECTION" to show the basic amount of protection.
In local currency.
tests:
data_tests:
- dbt_expectations.expect_column_values_to_be_between:
min_value: 0
max_value: 50000000
@ -83,7 +83,7 @@ models:
- name: verification_status
data_type: text
description: Outcome of the verification process.
tests:
data_tests:
- not_null
- accepted_values:
values:
@ -106,7 +106,7 @@ models:
- name: watch_list
data_type: text
description: Indicates if the guest was flagged on a watchlist.
tests:
data_tests:
- accepted_values:
values:
- "MATCH"
@ -127,13 +127,13 @@ models:
- name: is_cancelled
data_type: boolean
description: Indicates if the booking was canceled.
tests:
data_tests:
- not_null
- name: cancelled_at_utc
data_type: timestamp without time zone
description: Timestamp when the booking was canceled.
tests:
data_tests:
- not_null:
where: is_cancelled = true
@ -196,50 +196,50 @@ models:
- name: status_updated_at_utc
data_type: timestamp without time zone
description: Timestamp of the last status update for the verification.
tests:
data_tests:
- not_null
- name: status_updated_date_utc
data_type: date
description: Date of the last status update for the verification.
tests:
data_tests:
- not_null
- name: updated_at_utc
data_type: timestamp without time zone
description: Timestamp of the last update to the record.
tests:
data_tests:
- not_null
- name: updated_date_utc
data_type: date
description: Date of the last update to the record.
tests:
data_tests:
- not_null
- name: creation_at_utc
data_type: timestamp without time zone
description: |
Timestamp of when the reservation was created.
tests:
data_tests:
- not_null
- name: creation_date_utc
data_type: date
description: |
Date of when the reservation was created.
tests:
data_tests:
- not_null
- name: cosmos_created_date_utc
data_type: date
description: |
Date of when the verification request was created in Cosmos DB.
tests:
data_tests:
- not_null
- name: cosmos_db_timestamp_utc
data_type: timestamp with time zone
description: Internal Cosmos DB timestamp of the last record update.
tests:
data_tests:
- not_null

22
models/staging/screening/schema.yml
View file

@ -9,20 +9,20 @@ models:
- name: id_verification
data_type: character varying
description: Unique id for the specific transaction.
tests:
data_tests:
- unique
- not_null
- name: id_seon
data_type: text
description: The identifier in Seon.
tests:
data_tests:
- not_null
- name: id_user_partner
data_type: text
description: The unique ID of the partner calling the API.
tests:
data_tests:
- not_null
- name: id_watch_list
@ -33,7 +33,7 @@ models:
data_type: text
description: |
noFlags if the email shows no issues, Flagged otherwise.
tests:
data_tests:
- accepted_values:
values:
- "Flagged"
@ -43,7 +43,7 @@ models:
data_type: text
description: |
noFlags if the phone number shows no issues, Flagged otherwise.
tests:
data_tests:
- accepted_values:
values:
- "Flagged"
@ -53,7 +53,7 @@ models:
data_type: text
description: |
noFlags if not in the watch list, Flagged otherwise.
tests:
data_tests:
- accepted_values:
values:
- "Flagged"
@ -98,7 +98,7 @@ models:
description: |
Timestamp of the last edit of the record, as set by
Screening API.
tests:
data_tests:
- not_null
- name: updated_date_utc
@ -106,25 +106,25 @@ models:
description: |
Date of the last edit of the record, as set by
Screening API.
tests:
data_tests:
- not_null
- name: created_at_utc
data_type: timestamp without time zone
description: |
The internal application timestamp of when this record was created.
tests:
data_tests:
- not_null
- name: created_date_utc
data_type: timestamp without time zone
description: |
The internal application date of when this record was created.
tests:
data_tests:
- not_null
- name: cosmos_db_timestamp_utc
data_type: timestamp with time zone
description: The internal Cosmos DB timestamp of the last record update.
tests:
data_tests:
- not_null

108
models/staging/stripe/_stripe_sources.yml
View file

@ -6,8 +6,8 @@ version: 2
# It reads two different schemas (stripe_uk and stripe_us) that have exactly the
# same table structure. To save work, we use YAML anchors (see the funny syntax)
# under sources > stripe_uk > tables and sources > stripe_us > tables.
# Basically, if you change anything in the stripe_uk section, changes will
# happen in both schemas.
# Basically, if you change anything in the stripe_uk section, changes will
# happen in both schemas.
sources:
- name: stripe_uk
@ -25,7 +25,7 @@ sources:
- name: id
data_type: character varying
description: "{{ doc('generic_id_desc') }}"
tests:
data_tests:
- unique
- not_null
- dbt_expectations.expect_column_values_to_match_regex:
@ -33,17 +33,17 @@ sources:
- name: fee
data_type: bigint
description: "Fees (in cents) paid for this transaction. Represented as a positive integer when assessed."
tests:
data_tests:
- not_null
- name: net
data_type: bigint
description: "Net impact to a Stripe balance (in cents). A positive value represents incrementing a Stripe balance, and a negative value decrementing a Stripe balance. You can calculate the net impact of a transaction on a balance by amount - fee"
tests:
data_tests:
- not_null
- name: type
data_type: character varying
description: "Transaction type: adjustment, advance, advance_funding, anticipation_repayment, application_fee, application_fee_refund, charge, climate_order_purchase, climate_order_refund, connect_collection_transfer, contribution, issuing_authorization_hold, issuing_authorization_release, issuing_dispute, issuing_transaction, obligation_outbound, obligation_reversal_inbound, payment, payment_failure_refund, payment_network_reserve_hold, payment_network_reserve_release, payment_refund, payment_reversal, payment_unreconciled, payout, payout_cancel, payout_failure, network_cost, refund, refund_failure, reserve_transaction, reserved_funds, stripe_fee, stripe_fx_fee, tax_fee, topup, topup_reversal, transfer, transfer_cancel, transfer_failure, or transfer_refund. Learn more about balance transaction types and what they represent. To classify transactions for accounting purposes, consider reporting_category instead."
tests:
data_tests:
- not_null
- dbt_expectations.expect_column_values_to_be_in_set:
value_set:
@ -90,17 +90,17 @@ sources:
"transfer_failure",
"transfer_refund",
"balance_transfer_outbound",
"balance_transfer_inbound"
"balance_transfer_inbound",
]
- name: amount
data_type: bigint
description: "Gross amount of this transaction (in cents). A positive value represents funds charged to another party, and a negative value represents funds sent to another party."
tests:
data_tests:
- not_null
- name: object
data_type: character varying
description: "Silly column. The value is always `balance_transaction`."
tests:
data_tests:
- dbt_expectations.expect_column_values_to_match_regex:
regex: "^balance_transaction$"
- name: source
@ -109,19 +109,19 @@ sources:
- name: status
data_type: character varying
description: "The transaction’s net funds status in the Stripe balance, which are either available or pending."
tests:
data_tests:
- not_null
- dbt_expectations.expect_column_values_to_be_in_set:
value_set: ["available", "pending"]
- name: created
data_type: bigint
description: "{{ doc('stripe_created_desc') }}"
tests:
data_tests:
- not_null
- name: currency
data_type: character varying
description: "{{ doc('raw_stripe_currency_desc') }}"
tests:
data_tests:
- not_null
- dbt_expectations.expect_column_value_lengths_to_equal:
value: 3
@ -134,7 +134,7 @@ sources:
- name: available_on
data_type: bigint
description: "The date that the transaction’s net funds become available in the Stripe balance."
tests:
data_tests:
- not_null
- name: exchange_rate
data_type: numeric
@ -145,7 +145,7 @@ sources:
- name: reporting_category
data_type: character varying
description: "Read more at https://stripe.com/docs/reports/reporting-categories."
tests:
data_tests:
- not_null
- dbt_expectations.expect_column_values_to_be_in_set:
value_set:
@ -160,7 +160,7 @@ sources:
"other_adjustment",
"refund_failure",
"dispute_reversal",
"payout_reversal"
"payout_reversal",
]
- name: _airbyte_raw_id
data_type: character varying
@ -168,7 +168,7 @@ sources:
- name: _airbyte_extracted_at
data_type: timestamp with time zone
description: "{{ doc('_airbyte_extracted_at_desc') }}"
tests:
data_tests:
- not_null
- dbt_expectations.expect_row_values_to_have_recent_data:
datepart: day
@ -186,7 +186,7 @@ sources:
- name: id
data_type: character varying
description: "{{ doc('generic_id_desc') }}"
tests:
data_tests:
- unique
- not_null
- dbt_expectations.expect_column_values_to_match_regex:
@ -197,7 +197,7 @@ sources:
- name: paid
data_type: boolean
description: "true if the charge succeeded, or was successfully authorized for later capture."
tests:
data_tests:
- not_null
- name: order
data_type: character varying
@ -205,14 +205,14 @@ sources:
- name: amount
data_type: bigint
description: "Amount intended to be collected by this payment. A positive integer representing how much to charge in the smallest currency unit (e.g., 100 cents to charge $1.00 or 100 to charge ¥100, a zero-decimal currency). The minimum amount is $0.50 US or equivalent in charge currency. The amount value supports up to eight digits (e.g., a value of 99999999 for a USD charge of $999,999.99)."
tests:
data_tests:
- not_null
- dbt_expectations.expect_column_values_to_be_between:
min_value: 0
- name: object
data_type: character varying
description: "Silly column. The value is always `charge`."
tests:
data_tests:
- dbt_expectations.expect_column_values_to_match_regex:
regex: "^charge$"
- name: review
@ -224,14 +224,14 @@ sources:
- name: status
data_type: character varying
description: "The status of the payment is either succeeded, pending, or failed."
tests:
data_tests:
- not_null
- dbt_expectations.expect_column_values_to_be_in_set:
value_set: ["succeeded", "pending", "failed"]
- name: created
data_type: bigint
description: "{{ doc('stripe_created_desc') }}"
tests:
data_tests:
- not_null
- name: dispute
data_type: character varying
@ -248,17 +248,17 @@ sources:
- name: updated
data_type: bigint
description: "{{ doc('stripe_created_desc') }}"
tests:
data_tests:
- not_null
- name: captured
data_type: boolean
description: "If the charge was created without capturing, this Boolean represents whether it is still uncaptured or has since been captured."
tests:
data_tests:
- not_null
- name: currency
data_type: character varying
description: "{{ doc('raw_stripe_currency_desc') }}"
tests:
data_tests:
- not_null
- dbt_expectations.expect_column_value_lengths_to_equal:
value: 3
@ -277,7 +277,7 @@ sources:
- name: refunded
data_type: boolean
description: "Whether the charge has been fully refunded. If the charge is only partially refunded, this attribute will still be false."
tests:
data_tests:
- not_null
- name: shipping
data_type: jsonb
@ -315,14 +315,14 @@ sources:
- name: payment_intent
data_type: character varying
description: "ID of the PaymentIntent associated with this charge, if one exists."
tests:
data_tests:
- not_null
- dbt_expectations.expect_column_values_to_match_regex:
regex: "^pi_.{24}$"
- name: payment_method
data_type: character varying
description: "ID of the payment method used in this charge."
tests:
data_tests:
- not_null
- dbt_expectations.expect_column_values_to_match_regex:
regex: "^pm_.{24}$"
@ -335,7 +335,7 @@ sources:
- name: amount_captured
data_type: bigint
description: "Amount in cents captured (can be less than the amount attribute on the charge if a partial capture was made)."
tests:
data_tests:
- not_null
- dbt_expectations.expect_column_values_to_be_between:
min_value: 0
@ -343,7 +343,7 @@ sources:
- name: amount_refunded
data_type: bigint
description: "Amount in cents refunded (can be less than the amount attribute on the charge if a partial refund was issued)."
tests:
data_tests:
- not_null
- dbt_expectations.expect_column_values_to_be_between:
min_value: 0
@ -363,7 +363,7 @@ sources:
- name: balance_transaction
data_type: character varying
description: "ID of the balance transaction that describes the impact of this charge on your account balance (not including refunds or disputes)."
tests:
data_tests:
- dbt_expectations.expect_column_values_to_match_regex:
regex: "^txn_.{24}$"
row_condition: "balance_transaction is not null"
@ -394,7 +394,7 @@ sources:
- name: _airbyte_extracted_at
data_type: timestamp with time zone
description: "{{ doc('_airbyte_extracted_at_desc') }}"
tests:
data_tests:
- not_null
- dbt_expectations.expect_row_values_to_have_recent_data:
datepart: day
@ -414,7 +414,7 @@ sources:
- name: id
data_type: character varying
description: "{{ doc('generic_id_desc') }}"
tests:
data_tests:
- unique
- not_null
- dbt_expectations.expect_column_values_to_match_regex:
@ -422,14 +422,14 @@ sources:
- name: amount
data_type: bigint
description: "Amount intended to be collected by this PaymentIntent. A positive integer representing how much to charge in the smallest currency unit (e.g., 100 cents to charge $1.00 or 100 to charge ¥100, a zero-decimal currency). The minimum amount is $0.50 US or equivalent in charge currency. The amount value supports up to eight digits (e.g., a value of 99999999 for a USD charge of $999,999.99)."
tests:
data_tests:
- not_null
- dbt_expectations.expect_column_values_to_be_between:
min_value: 0
- name: object
data_type: character varying
description: "Silly column. The value is always `payment_intent`."
tests:
data_tests:
- dbt_expectations.expect_column_values_to_match_regex:
regex: "^payment_intent$"
- name: review
@ -441,7 +441,7 @@ sources:
- name: status
data_type: character varying
description: "Status of this PaymentIntent, one of requires_payment_method, requires_confirmation, requires_action, processing, requires_capture, canceled, or succeeded. Read more about each PaymentIntent status."
tests:
data_tests:
- not_null
- dbt_expectations.expect_column_values_to_be_in_set:
value_set:
@ -461,7 +461,7 @@ sources:
- name: created
data_type: bigint
description: "{{ doc('stripe_created_desc') }}"
tests:
data_tests:
- not_null
- name: invoice
data_type: character varying
@ -469,12 +469,12 @@ sources:
- name: updated
data_type: bigint
description: "{{ doc('stripe_created_desc')}}"
tests:
data_tests:
- not_null
- name: currency
data_type: character varying
description: "{{ doc('raw_stripe_currency_desc') }}"
tests:
data_tests:
- not_null
- dbt_expectations.expect_column_value_lengths_to_equal:
value: 3
@ -492,7 +492,7 @@ sources:
- name: metadata
data_type: jsonb
description: "{{ doc('stripe_metadata_desc') }}"
tests:
data_tests:
- not_null
- name: shipping
data_type: jsonb
@ -526,7 +526,7 @@ sources:
- name: latest_charge
data_type: character varying
description: "The latest charge created by this PaymentIntent."
tests:
data_tests:
- dbt_expectations.expect_column_values_to_match_regex:
regex: "^(ch|py)_.{24}$"
row_condition: "latest_charge is not null"
@ -551,7 +551,7 @@ sources:
- name: amount_received
data_type: bigint
description: "Amount that this PaymentIntent collects."
tests:
data_tests:
- not_null
- dbt_expectations.expect_column_values_to_be_between:
min_value: 0
@ -559,7 +559,7 @@ sources:
- name: amount_capturable
data_type: bigint
description: "Amount that can be captured from this PaymentIntent."
tests:
data_tests:
- not_null
- dbt_expectations.expect_column_values_to_be_between:
min_value: 0
@ -611,7 +611,7 @@ sources:
- name: _airbyte_extracted_at
data_type: timestamp with time zone
description: "{{ doc('_airbyte_extracted_at_desc') }}"
tests:
data_tests:
- not_null
- dbt_expectations.expect_row_values_to_have_recent_data:
datepart: day
@ -628,7 +628,7 @@ sources:
- name: id
data_type: character varying
description: "{{ doc('generic_id_desc') }}"
tests:
data_tests:
- unique
- not_null
- dbt_expectations.expect_column_values_to_match_regex:
@ -636,21 +636,21 @@ sources:
- name: amount
data_type: bigint
description: "Amount, in cents."
tests:
data_tests:
- not_null
- dbt_expectations.expect_column_values_to_be_between:
min_value: 0
- name: charge
data_type: character varying
description: "ID of the charge that’s refunded."
tests:
data_tests:
- not_null
- dbt_expectations.expect_column_values_to_match_regex:
regex: "^(ch|py)_.{24}$"
- name: object
data_type: character varying
description: "Silly column. The value is always `refund`."
tests:
data_tests:
- dbt_expectations.expect_column_values_to_match_regex:
regex: "^refund$"
- name: reason
@ -659,7 +659,7 @@ sources:
- name: status
data_type: character varying
description: "Status of the refund. This can be pending, requires_action, succeeded, failed, or canceled. Learn more about failed refunds."
tests:
data_tests:
- not_null
- dbt_expectations.expect_column_values_to_be_in_set:
value_set:
@ -673,7 +673,7 @@ sources:
- name: created
data_type: bigint
description: "{{ doc('stripe_created_desc') }}"
tests:
data_tests:
- not_null
- name: updated
data_type: bigint
@ -681,7 +681,7 @@ sources:
- name: currency
data_type: character varying
description: "{{ doc('raw_stripe_currency_desc') }}"
tests:
data_tests:
- not_null
- dbt_expectations.expect_column_value_lengths_to_equal:
value: 3
@ -691,7 +691,7 @@ sources:
- name: payment_intent
data_type: character varying
description: "ID of the PaymentIntent that’s refunded."
tests:
data_tests:
- not_null
- dbt_expectations.expect_column_values_to_match_regex:
regex: "^pi_.{24}$"
@ -704,7 +704,7 @@ sources:
- name: balance_transaction
data_type: character varying
description: "Balance transacion that describes the impact on your account balance."
tests:
data_tests:
- dbt_expectations.expect_column_values_to_match_regex:
regex: "^txn_.{24}$"
row_condition: "balance_transaction is not null"
@ -720,7 +720,7 @@ sources:
- name: _airbyte_extracted_at
data_type: timestamp with time zone
description: "{{ doc('_airbyte_extracted_at_desc') }}"
tests:
data_tests:
- not_null
- dbt_expectations.expect_row_values_to_have_recent_data:
datepart: day

8
models/staging/xedotcom/schema.yml
View file

@ -13,18 +13,18 @@ models:
description: A unique ID for the record, derived from concatenating the
currencies and date. Currency order is relevant (EURUSD != USDEUR).
data_type: text
tests:
data_tests:
- not_null
- unique
- name: from_currency
description: The source currency, represented as an ISO 4217 code.
data_type: character
tests:
data_tests:
- not_null
- name: to_currency
description: The target currency, represented as an ISO 4217 code.
data_type: character
tests:
data_tests:
- not_null
- name: rate
description: >-
@ -38,7 +38,7 @@ models:
The rate can be smaller than one, but can't be negative.
data_type: numeric
tests:
data_tests:
- not_negative_or_zero
- not_null
- name: rate_date_utc

82
models/staging/xero/schema.yml
View file

@ -10,7 +10,7 @@ models:
- name: id_contact
data_type: character varying
description: Xero's unique identifier for the contact.
tests:
data_tests:
- not_null
- unique
@ -21,14 +21,14 @@ models:
- name: contact_name
data_type: character varying
description: The name for the customer.
tests:
data_tests:
- not_null
- unique
- name: is_customer
data_type: boolean
description: Flag that shows if the contact is a Superhog customer.
tests:
data_tests:
- not_null
- accepted_values:
values:
@ -38,7 +38,7 @@ models:
- name: is_supplier
data_type: boolean
description: Flag that shows if the contact is a Superhog supplier.
tests:
data_tests:
- not_null
- accepted_values:
values:
@ -199,7 +199,7 @@ models:
- name: id_invoice
data_type: character varying
description: Xero's unique identifier for the invoice.
tests:
data_tests:
- not_null
- unique
@ -216,7 +216,7 @@ models:
description: |
This field indicates whether the invoice is from Superhog towards a customer
(value is "ACCREC") or from a supplier towards Superhog (value is "ACCPAY").
tests:
data_tests:
- not_null
- accepted_values:
values:
@ -226,31 +226,31 @@ models:
- name: total_amount_local_curr
data_type: numeric
description: The total amount to be paid, in the currency the invoice is denominated in.
tests:
data_tests:
- not_null
- name: total_amount_wo_tax_local_curr
data_type: numeric
description: The total amount to be paid minus taxes, in the currency the invoice is denominated in.
tests:
data_tests:
- not_null
- name: total_tax_local_curr
data_type: numeric
description: The total tax, in the currency the invoice is denominated in.
tests:
data_tests:
- not_null
- name: total_due_local_curr
data_type: numeric
description: The total amount outstanding right now, in the currency the invoice is denominated in.
tests:
data_tests:
- not_null
- name: total_paid_local_curr
data_type: numeric
description: The total amount that has already been paid, in the currency the invoice is denominated in.
tests:
data_tests:
- not_null
- name: invoice_status
@ -261,7 +261,7 @@ models:
Can be one of: PAID, VOIDED, DRAFT, DELETED, AUTHORISED, SUBMITTED.
You can read more here: https://developer.xero.com/documentation/api/accounting/types#invoices
tests:
data_tests:
- not_null
- accepted_values:
values:
@ -278,7 +278,7 @@ models:
The contact related to this invoice.
The customer if it's an Accounts Receivables one, the supplier if it's an Accounts Payables one.
tests:
data_tests:
- not_null
- name: invoice_due_date_utc
@ -297,7 +297,7 @@ models:
- name: line_items
data_type: jsonb
description: An array with all the line items and their details.
tests:
data_tests:
- not_null
- name: reference
@ -321,7 +321,7 @@ models:
- name: invoice_currency_iso_4217
data_type: character varying
description: The ISO 4217 code for the currency which this invoice uses as unit of account.
tests:
data_tests:
- not_null
- length_between:
min_length: 3
@ -337,7 +337,7 @@ models:
For example, if the invoice is 135 ZAR, and this rate is 0.0167, the invoice value in GBP is 2.25
(135ZAR * 0.0167GBP/ZAR = 2.25GBP).
tests:
data_tests:
- not_null
- not_negative
@ -399,7 +399,7 @@ models:
Indicates whether the amounts included in line items are tax
inclusive (Inclusive), tax exclusive (Exclusive) or simply,
there are no taxes on the invoice (NoTax).
tests:
data_tests:
- accepted_values:
values:
- Inclusive
@ -428,7 +428,7 @@ models:
- name: id_credit_note
data_type: character varying
description: Xero's unique identifier for the credit note.
tests:
data_tests:
- not_null
- unique
@ -451,7 +451,7 @@ models:
- name: credit_note_type
data_type: character varying
description: The type of the credit note (e.g., ACCRECCREDIT or ACCPAYCREDIT).
tests:
data_tests:
- not_null
- accepted_values:
values:
@ -468,32 +468,32 @@ models:
The implied exchange rate between the credit note currency and GBP.
This is the XXX to GBP rate, indicating how many GBP one unit of XXX is worth.
tests:
data_tests:
- not_null
- not_negative
- name: total_amount_local_curr
data_type: numeric
description: The total amount of the credit note in the local currency.
tests:
data_tests:
- not_null
- name: subtotal_local_curr
data_type: numeric
description: The subtotal of the credit note in the local currency, before tax.
tests:
data_tests:
- not_null
- name: total_tax_local_curr
data_type: numeric
description: The total tax amount of the credit note in the local currency.
tests:
data_tests:
- not_null
- name: remaining_credit_local_curr
data_type: numeric
description: The remaining credit amount in the local currency.
tests:
data_tests:
- not_null
- name: applied_amount
@ -508,7 +508,7 @@ models:
Can be one of: PAID, VOIDED, DRAFT, DELETED, AUTHORISED, SUBMITTED.
You can read more here: https://developer.xero.com/documentation/api/accounting/types#invoices
tests:
data_tests:
- not_null
- accepted_values:
values:
@ -522,7 +522,7 @@ models:
- name: full_contact_details
data_type: jsonb
description: The full contact details related to the credit note.
tests:
data_tests:
- not_null
- name: credit_note_due_date
@ -532,7 +532,7 @@ models:
- name: line_items
data_type: jsonb
description: An array of line items associated with the credit note.
tests:
data_tests:
- not_null
- name: date_string
@ -573,7 +573,7 @@ models:
Indicates whether the amounts included in line items are tax
inclusive (Inclusive), tax exclusive (Exclusive) or simply,
there are no taxes on the credit note (NoTax).
tests:
data_tests:
- accepted_values:
values:
- Inclusive
@ -597,7 +597,7 @@ models:
- name: id_bank_transaction
data_type: character varying
description: Xero's unique identifier for the transaction.
tests:
data_tests:
- not_null
- unique
@ -633,7 +633,7 @@ models:
negative, amounts we receive are positive.
You can read more here: https://developer.xero.com/documentation/api/accounting/types#bank-transactions
tests:
data_tests:
- not_null
- accepted_values:
values:
@ -653,7 +653,7 @@ models:
receiving transactions show as 1. This helps in converting the
transaction amounts in the right sign, since Xero brings all amounts
as positive by default.
tests:
data_tests:
- not_null
- accepted_values:
values:
@ -668,7 +668,7 @@ models:
Can be one of: AUTHORISED, DELETED.
You can read more here: https://developer.xero.com/documentation/api/accounting/types#bank-transaction-status-codes
tests:
data_tests:
- not_null
- accepted_values:
values:
@ -688,7 +688,7 @@ models:
description: |
Total of bank transaction tax inclusive, in the currency the
transaction is denominated in.
tests:
data_tests:
- not_null
- name: total_amount_wo_tax_local_curr
@ -706,7 +706,7 @@ models:
- name: transaction_currency_iso_4217
data_type: character varying
description: The ISO 4217 code for the currency of the transaction.
tests:
data_tests:
- not_null
- length_between:
min_length: 3
@ -729,7 +729,7 @@ models:
For example, if the transaction is 135 ZAR, and this rate is 0.0167,
the transaction value in GBP is 2.25 (135ZAR * 0.0167GBP/ZAR =
2.25GBP).
tests:
data_tests:
- not_negative
- name: line_amount_tax_inclusiveness
@ -738,7 +738,7 @@ models:
Indicates whether the amounts included in line items are tax
inclusive (Inclusive), tax exclusive (Exclusive) or simply,
there are no taxes on the invoice (NoTax).
tests:
data_tests:
- accepted_values:
values:
- Inclusive
@ -755,19 +755,19 @@ models:
- name: line_items
data_type: jsonb
description: An array with all the line items and their details.
tests:
data_tests:
- not_null
- name: bank_account
data_type: jsonb
description: An array with the details of the related bank account.
tests:
data_tests:
- not_null
- name: is_reconciled
data_type: boolean
description: A flag indicating if the transaction has been reconciled.
tests:
data_tests:
- not_null
- name: has_attachments
@ -801,11 +801,11 @@ models:
- name: id_account
data_type: character varying
description: Xero's unique identifier for the account.
tests:
data_tests:
- not_null
- unique
- name: account_code
data_type: character varying
description: Human readable account code.
tests:
data_tests:
- unique