stuff
This commit is contained in:
Pablo Martin
parent
09c7d86a91
commit
2e396f4407
1 changed files with 140 additions and 46 deletions
186
log.md
186
log.md
|
|
@ -1916,44 +1916,36 @@ Stuff I want to read:;
|
||||||
# 2024-02-01
|
# 2024-02-01
|
||||||
|
|
||||||
- [X] Invoicing
|
- [X] Invoicing
|
||||||
- [ ] Ryan
|
- [X] Ryan
|
||||||
- [ ] Leo
|
- [X] Leo
|
||||||
- [ ] Invoicing backlog
|
- [X] Invoicing headsup to Stakeholders
|
||||||
|
- [X] Invoicing backlog
|
||||||
- [ ] Wireguard
|
- [ ] Wireguard
|
||||||
|
- [ ] Create interface for all traffic
|
||||||
|
- [ ] Document
|
||||||
|
- [ ]
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
# Now
|
# 2024-02-02
|
||||||
|
|
||||||
|
- [X] Document SH invoicing as Data Product
|
||||||
|
- [X] Document Stripe reports as Data Product
|
||||||
|
- [X] DOcument Stripe as a data source
|
||||||
|
- [X] Try out Stripe integration with Airbyte
|
||||||
|
- [ ] Set up dbt runner machine in dev
|
||||||
|
- [ ] Document infra
|
||||||
|
|
||||||
|
|
||||||
Wireguard VPN
|
# 2024-02-05
|
||||||
- Delete everything
|
|
||||||
- Set up firewall listening on 52420
|
|
||||||
- Create point to point
|
|
||||||
- Set it up
|
|
||||||
- Hit dummy HTTP from Windows
|
|
||||||
- Hit dummy HTTP from Linux
|
|
||||||
- Hit coredns from Windows
|
|
||||||
- Hit coredns from Linux
|
|
||||||
- Unsuccessfully try to jump anywhere else
|
|
||||||
- Extend to point to site
|
|
||||||
- Change server config
|
|
||||||
- Try to reach airbyte web from windows
|
|
||||||
- Try to reach airbyte web from linux
|
|
||||||
- Try to reach airbyte ssh from windows
|
|
||||||
- Try to reach airbyte ssh from linux
|
|
||||||
- Try to reach DWH from windows
|
|
||||||
- Try to reach DWH from linux
|
|
||||||
- Try to reach Core from windows
|
|
||||||
- Try to reach Core from linux
|
|
||||||
|
|
||||||
|
|
||||||
Architecture
|
- Data Catalogue
|
||||||
- [ ] Document dev ramblings
|
- [X] Make big announcement + Quick Presentation Video
|
||||||
|
- [X] Add Stripe reporting as data product in data catalogue
|
||||||
|
- [X] Add invoicing tool as Data Product
|
||||||
|
|
||||||
Invoicing Reformation
|
Invoicing Reformation
|
||||||
|
- [X] Finish stage descriptions
|
||||||
- [ ] Finish stage descriptions
|
|
||||||
- [X] Drop existing knowledge on inputs
|
- [X] Drop existing knowledge on inputs
|
||||||
- [X] Drop details on Stripe exports
|
- [X] Drop details on Stripe exports
|
||||||
- [X] Get access to Stripe from Louise
|
- [X] Get access to Stripe from Louise
|
||||||
|
|
@ -1967,7 +1959,7 @@ Invoicing Reformation
|
||||||
- [X] Backlog discussion
|
- [X] Backlog discussion
|
||||||
- [X] Quick coffee with Amanda
|
- [X] Quick coffee with Amanda
|
||||||
- [X] Phase 1 summary in Notion + Email
|
- [X] Phase 1 summary in Notion + Email
|
||||||
- [ ] January processing
|
- [X] January processing
|
||||||
- New improvements
|
- New improvements
|
||||||
- Automated Stripe Transaction Exports
|
- Automated Stripe Transaction Exports
|
||||||
- Exchange Rate Formalization
|
- Exchange Rate Formalization
|
||||||
|
|
@ -1985,27 +1977,129 @@ Invoicing Reformation
|
||||||
- Pablo runs the exports
|
- Pablo runs the exports
|
||||||
- Everything gets shared with Finance team
|
- Everything gets shared with Finance team
|
||||||
|
|
||||||
- Data Catalogue
|
- [X] Equipment
|
||||||
- [X] Make big announcement + Quick Presentation Video
|
|
||||||
- [ ] Add CosmosDB to data catalogue
|
|
||||||
- [ ] Add Stripe reporting as data product in data catalogue
|
|
||||||
- [ ] Add invoicing tool as Data Product
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
- [ ] Ask IT for <data-team@superhog.com> mail
|
|
||||||
- [ ] Place this in Backlog: <https://superhogteam.slack.com/archives/C06A52YUDM1/p1702557374922989>
|
|
||||||
- [ ] Add story to backlog about rough DealId compliance monitoring through quick and dirty weekly extract
|
|
||||||
- [ ] Story about checking for accounts with no deal id
|
|
||||||
- [ ] Story about report to spot non-matching pricing details across Dashboard Superhog.
|
|
||||||
- [ ] Story about having a currency exchange rate database in DWH
|
|
||||||
|
|
||||||
- [ ] Equipment
|
|
||||||
- Bookcase for private office <https://www.ikea.com/es/es/p/billy-libreria-blanco-00263850/> x 1
|
- Bookcase for private office <https://www.ikea.com/es/es/p/billy-libreria-blanco-00263850/> x 1
|
||||||
- Organizers for private office <https://www.ikea.com/es/es/p/tjena-organizador-escritorio-blanco-60395452/> x 4
|
- Organizers for private office <https://www.ikea.com/es/es/p/tjena-organizador-escritorio-blanco-60395452/> x 4
|
||||||
- Organizers for private office <https://www.ikea.com/es/es/p/vattenkar-estante-escritorio-blanco-20541568/> x 4
|
- Organizers for private office <https://www.ikea.com/es/es/p/vattenkar-estante-escritorio-blanco-20541568/> x 4
|
||||||
- Post-its, pens, pencils, a couple boxes of white A4, scissors, tape, etc
|
- Post-its, pens, pencils, a couple boxes of white A4, scissors, tape, etc
|
||||||
|
|
||||||
|
|
||||||
|
- [X] Bitcoin competencies test
|
||||||
|
- [X] Organize week
|
||||||
|
- [X] Marcel afternoon
|
||||||
|
- [X] Schedule with candidate
|
||||||
|
- [X] sh-invoicing issue
|
||||||
|
- [X] dbt runner
|
||||||
|
|
||||||
|
## Permissions for dbt user in DWH
|
||||||
|
|
||||||
|
- It needs
|
||||||
|
- To be able to create schemas and operate on them
|
||||||
|
- To be able to ONLY READ from the sync schema
|
||||||
|
- Nothing else
|
||||||
|
|
||||||
|
|
||||||
|
Strategy:
|
||||||
|
- Create a `modeler` role.
|
||||||
|
- The modeler can:
|
||||||
|
- Read anything in the database
|
||||||
|
- Do anything in the `staging`, `intermediate` and `reporting` schemas.
|
||||||
|
|
||||||
|
|
||||||
|
```sql
|
||||||
|
|
||||||
|
CREATE ROLE modeler WITH NOLOGIN;
|
||||||
|
ALTER SCHEMA staging OWNER TO modeler;
|
||||||
|
ALTER SCHEMA intermediate OWNER TO modeler;
|
||||||
|
ALTER SCHEMA reporting OWNER TO modeler;
|
||||||
|
|
||||||
|
CREATE ROLE dbt_user WITH PASSWORD <password-here>;
|
||||||
|
GRANT ROLE modeler to dbt_user;
|
||||||
|
|
||||||
|
|
||||||
|
```
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
Testing
|
||||||
|
```sql
|
||||||
|
-- Reading from sync schema should work
|
||||||
|
-- Create on sync schema should not work
|
||||||
|
-- Inserting on sync schema should not work
|
||||||
|
|
||||||
|
-- Creating schema should work
|
||||||
|
-- Creating new table in new schema should work
|
||||||
|
-- Insert data in new table in news chema should work
|
||||||
|
-- Reading from that new table should work
|
||||||
|
-- Creating a view in new schema should work
|
||||||
|
|
||||||
|
-- Creating new table in schema `staging` should work
|
||||||
|
-- Inserting data in new table in newschema should work
|
||||||
|
-- Reading from the new table in `staging` should work
|
||||||
|
-- Creating a view in `staging` should work
|
||||||
|
|
||||||
|
-- Doing anything on `public schema should not work`
|
||||||
|
|
||||||
|
```
|
||||||
|
|
||||||
|
## Product weekly
|
||||||
|
|
||||||
|
- Finance
|
||||||
|
- They are using things smoothly, success
|
||||||
|
- Also collecting small details
|
||||||
|
- Any updates from other teams on this?
|
||||||
|
- Infra
|
||||||
|
- Working on it
|
||||||
|
- I want to go to production this week
|
||||||
|
|
||||||
|
|
||||||
|
- Louise
|
||||||
|
- Stripe US account almost ready
|
||||||
|
- Is that going to take waiver payments? Are we synced with finance? What's the rollout plan?
|
||||||
|
- Yes, all payments in US dollar will go through there from Monday onwards.
|
||||||
|
- I will receive access to the Stripe US account.
|
||||||
|
- Who's doing this?
|
||||||
|
- Louise, but she will hand things over to Amanda.
|
||||||
|
|
||||||
|
## 2024-02-06
|
||||||
|
|
||||||
|
- [X] Add silly shell script to git repo + instructions on how to deploy
|
||||||
|
|
||||||
|
# Now
|
||||||
|
|
||||||
|
- [ ] document infra
|
||||||
|
- Continue
|
||||||
|
|
||||||
|
- [ ] Plan holidays
|
||||||
|
|
||||||
|
|
||||||
|
Architecture
|
||||||
|
- [ ] Set up dbt runner properly
|
||||||
|
- [ ] Document deployment plan
|
||||||
|
- [ ] Learn how the hell does the Azure cli work with the templates
|
||||||
|
- [ ] Create repo with automation templates
|
||||||
|
- [ ] Deploy
|
||||||
|
|
||||||
|
Data docs
|
||||||
|
- [ ] Start working on internal docs for ways of working around DWH, VPN, dbt, etc.
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
- Data Catalogue
|
||||||
|
- [ ] Add CosmosDB to data catalogue
|
||||||
|
|
||||||
|
- #13121
|
||||||
|
- [ ] Get access from Ray
|
||||||
|
- [ ] Set query parameters in the athena thingy
|
||||||
|
- [ ] Add the new tab with totals
|
||||||
|
|
||||||
|
|
||||||
|
- [ ] Ask IT for <data-team@superhog.com> mail
|
||||||
|
- [ ] Place this in Backlog: <https://superhogteam.slack.com/archives/C06A52YUDM1/p1702557374922989>
|
||||||
|
- [ ] Story about checking for accounts with no deal id
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
- Data literacy roadmap
|
- Data literacy roadmap
|
||||||
- Data consumer
|
- Data consumer
|
||||||
- Anyone who requests it
|
- Anyone who requests it
|
||||||
|
|
|
||||||
Loading…
Add table
Add a link
Reference in a new issue