Commit graph

211 commits

Author SHA1 Message Date
Pablo Martin
475116a32b rename field in docs 2024-09-04 09:12:17 +02:00
Pablo Martin
e750e288c5 fix typo in schema 2024-09-03 18:19:17 +02:00
Pablo Martin
9f80f0a916 shit date casts left 2024-09-03 18:03:19 +02:00
Pablo Martin
37856ec606 shift renames left, remove deprecated fields from int 2024-09-03 17:55:24 +02:00
Pablo Martin
c35c5cb033 schema and tests 2024-09-03 17:55:24 +02:00
Pablo Martin
9a6490e7fd minor change in model 2024-09-03 17:55:24 +02:00
Oriol Roqué Paniagua
6d59e21310 Merged PR 2725: Force id user field to lower in staging
# Description

Forces lower case to all id_users in staging. Removes hardcoded lower case in intermediate. Adapts readme to contemplate the lowering of id users.

I propose to merge, run in prod and run tests in prod as a proper evaluation method.

BTW, I only find one id_user_host that was in capital letters, so that's why probably we didn't care that much about this. Still, I prefer have things clean from the start!

```
select *
from staging.stg_core__booking scb
left join intermediate.int_core__unified_user icuu
on lower(scb.id_user_host) = lower(icuu.id_user)
where scb.id_user_host <> icuu.id_user
```

# Checklist

- [ ] The edited models and dependants run properly with production data. **All models run in stg, did not check all the dependants**
- [ ] The edited models are sufficiently documented. **Have not checked**
- [ ] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #20776
2024-09-03 14:36:21 +00:00
Joaquin Ossa
0cb03e0808 Merged PR 2707: E-Deposit users to staging
# Description

E-Deposit users to staging to have currencies for PBI report

# Checklist

- [x] The edited models and dependants run properly with production data.
- [x] The edited models are sufficiently documented.
- [x] The edited models contain PK tests, and I've ran and passed them.
- [x] I have checked for DRY opportunities with other models and docs.
- [x] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #20125
2024-09-03 12:53:41 +00:00
Joaquin Ossa
322b122925 Added missing tests 2024-09-03 10:21:23 +02:00
Joaquin Ossa
fa6f7f8ff8 Filtering out test users so all tests work correctly 2024-09-03 10:15:20 +02:00
Joaquin Ossa
6dedbc04d7 Added more tests but still waiting confirmation on tests users from Ray and Ana 2024-09-02 16:55:56 +02:00
Joaquin Ossa
129b00e29b Added more description and tests to schema 2024-09-02 12:44:05 +02:00
Joaquin Ossa
ee4d213274 edeposit_users to staging 2024-09-02 11:33:25 +02:00
Joaquin Ossa
b6a752fd74 edeposit_users to staging 2024-09-02 11:33:21 +02:00
Joaquin Ossa
42510bbb4d Just committing to save change and create a new branch for basic to push basic changes 2024-08-30 10:33:43 +02:00
Joaquin Ossa
6adc424963 addressed Pablo's comments, removed the repetitive casts, added some nut_null tests and fixed some of the names and descriptions discrepancies 2024-08-29 14:25:00 +02:00
Pablo Martín
3c73bcf965 Merged PR 2646: Add missing edeposit columns
# Description

Adds three more columns to the staging model of `verifications` for edeposit: `ListingId`, `NightlyFee` and `CompanyName`.

# Checklist

- [X] The edited models and dependants run properly with production data.
- ~~[ ] The edited models are sufficiently documented.~~ __No, but I promise they will be.__
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #20123
2024-08-27 09:22:52 +00:00
Oriol Roqué Paniagua
cb3d1e628b Merged PR 2647: Adding accommodation_to_product_bundle to stg
# Description

Adds accommodation to product bundle table from sync_core to staging. Note that this table still has no data because so far no listing has an associated product bundle.

Small change: removed in schema pending confirmation comments from Lou now that we've got an answer confirming it.

# Checklist

- [ ] The edited models and dependants run properly with production data: **N/A - there's no data :(**
- [X] The edited models are sufficiently documented.
- [ ] The edited models contain PK tests, and I've ran and passed them. **N/A - there's no data :(**
- [X] I have checked for DRY opportunities with other models and docs.
- [ ] I've picked the right materialization for the affected models. **N/A - there's no data :(**

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #19570
2024-08-27 08:57:23 +00:00
Pablo Martin
4cc456c3b0 add missing columns to model 2024-08-27 10:33:18 +02:00
Pablo Martin
c0f4a59aeb add version field to staging table 2024-08-26 09:30:10 +02:00
Pablo Martin
4214c6b1c5 schema, small fixes on model 2024-08-21 17:48:06 +02:00
Pablo Martin
40168f74b3 reorder fields 2024-08-21 15:13:19 +02:00
Pablo Martin
34a2279387 some casts 2024-08-21 15:12:05 +02:00
Pablo Martin
64e5d78578 progress in staging, ordered fields 2024-08-21 12:56:45 +02:00
Pablo Martin
2331db2803 add source file 2024-08-21 12:46:05 +02:00
Joaquin Ossa
63d7ebb6d1 Merged PR 2564: New model for verification_to_address_validation
# Description

New model for verification_to_address_validation for check in hero reporting

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #17069
2024-08-16 09:21:31 +00:00
Joaquin Ossa
be9757b886 Modified source and included test for id_address_validation as unique as well 2024-08-16 11:15:40 +02:00
Joaquin Ossa
1d4a2d7b34 reorganized columns 2024-08-14 17:36:28 +02:00
Joaquin Ossa
1e818ea306 organized columns 2024-08-14 17:34:04 +02:00
Joaquin Ossa
fa5cfb8c66 removed extra _airbyte_raw_id 2024-08-14 17:31:36 +02:00
Joaquin Ossa
fd44f94b78 new verification_to_address_validation model in staging 2024-08-14 17:30:09 +02:00
Joaquin Ossa
cdd3b6a58f new address_validation model in staging 2024-08-14 17:16:35 +02:00
Pablo Martin
2ebf6a711c add tests 2024-08-12 11:20:09 +02:00
Pablo Martin
419f07e77c rename 2024-08-12 11:19:32 +02:00
Pablo Martin
a39f4d24aa add staging model 2024-08-12 11:18:49 +02:00
Pablo Martin
501a063635 add source 2024-08-12 11:16:53 +02:00
Pablo Martin
8c78f880ec change column name, previous was not accurate 2024-08-12 11:08:28 +02:00
Pablo Martin
22ccae5b9a add tests 2024-08-12 11:05:12 +02:00
Pablo Martin
59477a4641 typo 2024-08-12 11:01:09 +02:00
Pablo Martin
51fd0cf3bc add staging model 2024-08-12 11:00:13 +02:00
Pablo Martin
e4b0f54241 add to sources 2024-08-12 10:47:55 +02:00
Pablo Martin
cfe02fb99c add PK test 2024-08-12 10:42:40 +02:00
Pablo Martin
81b900b6d2 add claim to staging 2024-08-12 10:42:29 +02:00
Pablo Martin
0ba3f13055 add claim to sources 2024-08-12 10:37:40 +02:00
Oriol Roqué Paniagua
7177b41e19 Merged PR 2516: Create host_user and user_role in intermediate
# Description

Adds 2 new tables:
- `int_core__user_role`: contains the relationship of a given user has a role.
- `int_core__user_host`: based on the previous table, it selects the users and main information from those users that are considered as hosts according to the role they have.

Note: I needed to change the test in stg. A user, generally, can have no role, one role, or multiple roles. Thus we cannot propagate this information in the unified_user model.

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #19513
2024-08-07 14:28:34 +00:00
Oriol Roqué Paniagua
ad7d94b604 Merged PR 2511: Adding stg_core__role and stg_core__user_role in staging
# Description

Adds Role and UserRole in staging, in dedicated tables.
Open to discussion if you want me to directly apply this at User table level instead of having them in staging as standalone tables. Otherwise, staying with this PR as is, next step would be to integrate these roles in unified_user and create a dedicated user_host table in intermediate

# Checklist

- [X] The edited models and dependants run properly with production data.
- [X] The edited models are sufficiently documented.
- [X] The edited models contain PK tests, and I've ran and passed them.
- [X] I have checked for DRY opportunities with other models and docs.
- [X] I've picked the right materialization for the affected models.

# Other

- [ ] Check if a full-refresh is required after this PR is merged.

Related work items: #19513
2024-08-07 13:17:31 +00:00
Pablo Martin
43a7ead6d3 missing field in stg 2024-07-08 16:29:35 +02:00
Pablo Martin
64377da570 PK test 2024-07-01 16:54:58 +02:00
Pablo Martin
83f67a06bf model 2024-07-01 16:53:54 +02:00
Pablo Martin
2f133c1424 add source 2024-07-01 16:50:12 +02:00