stuff
This commit is contained in:
Pablo Martin
parent
09c7d86a91
commit
2e396f4407
1 changed files with 140 additions and 46 deletions
186
log.md
186
log.md
|
|
@ -1916,44 +1916,36 @@ Stuff I want to read:;
|
|||
# 2024-02-01
|
||||
|
||||
- [X] Invoicing
|
||||
- [ ] Ryan
|
||||
- [ ] Leo
|
||||
- [ ] Invoicing backlog
|
||||
- [X] Ryan
|
||||
- [X] Leo
|
||||
- [X] Invoicing headsup to Stakeholders
|
||||
- [X] Invoicing backlog
|
||||
- [ ] Wireguard
|
||||
- [ ] Create interface for all traffic
|
||||
- [ ] Document
|
||||
- [ ]
|
||||
|
||||
|
||||
|
||||
# Now
|
||||
# 2024-02-02
|
||||
|
||||
- [X] Document SH invoicing as Data Product
|
||||
- [X] Document Stripe reports as Data Product
|
||||
- [X] DOcument Stripe as a data source
|
||||
- [X] Try out Stripe integration with Airbyte
|
||||
- [ ] Set up dbt runner machine in dev
|
||||
- [ ] Document infra
|
||||
|
||||
|
||||
Wireguard VPN
|
||||
- Delete everything
|
||||
- Set up firewall listening on 52420
|
||||
- Create point to point
|
||||
- Set it up
|
||||
- Hit dummy HTTP from Windows
|
||||
- Hit dummy HTTP from Linux
|
||||
- Hit coredns from Windows
|
||||
- Hit coredns from Linux
|
||||
- Unsuccessfully try to jump anywhere else
|
||||
- Extend to point to site
|
||||
- Change server config
|
||||
- Try to reach airbyte web from windows
|
||||
- Try to reach airbyte web from linux
|
||||
- Try to reach airbyte ssh from windows
|
||||
- Try to reach airbyte ssh from linux
|
||||
- Try to reach DWH from windows
|
||||
- Try to reach DWH from linux
|
||||
- Try to reach Core from windows
|
||||
- Try to reach Core from linux
|
||||
|
||||
# 2024-02-05
|
||||
|
||||
Architecture
|
||||
- [ ] Document dev ramblings
|
||||
- Data Catalogue
|
||||
- [X] Make big announcement + Quick Presentation Video
|
||||
- [X] Add Stripe reporting as data product in data catalogue
|
||||
- [X] Add invoicing tool as Data Product
|
||||
|
||||
Invoicing Reformation
|
||||
|
||||
- [ ] Finish stage descriptions
|
||||
- [X] Finish stage descriptions
|
||||
- [X] Drop existing knowledge on inputs
|
||||
- [X] Drop details on Stripe exports
|
||||
- [X] Get access to Stripe from Louise
|
||||
|
|
@ -1967,7 +1959,7 @@ Invoicing Reformation
|
|||
- [X] Backlog discussion
|
||||
- [X] Quick coffee with Amanda
|
||||
- [X] Phase 1 summary in Notion + Email
|
||||
- [ ] January processing
|
||||
- [X] January processing
|
||||
- New improvements
|
||||
- Automated Stripe Transaction Exports
|
||||
- Exchange Rate Formalization
|
||||
|
|
@ -1985,27 +1977,129 @@ Invoicing Reformation
|
|||
- Pablo runs the exports
|
||||
- Everything gets shared with Finance team
|
||||
|
||||
- Data Catalogue
|
||||
- [X] Make big announcement + Quick Presentation Video
|
||||
- [ ] Add CosmosDB to data catalogue
|
||||
- [ ] Add Stripe reporting as data product in data catalogue
|
||||
- [ ] Add invoicing tool as Data Product
|
||||
|
||||
|
||||
|
||||
- [ ] Ask IT for <data-team@superhog.com> mail
|
||||
- [ ] Place this in Backlog: <https://superhogteam.slack.com/archives/C06A52YUDM1/p1702557374922989>
|
||||
- [ ] Add story to backlog about rough DealId compliance monitoring through quick and dirty weekly extract
|
||||
- [ ] Story about checking for accounts with no deal id
|
||||
- [ ] Story about report to spot non-matching pricing details across Dashboard Superhog.
|
||||
- [ ] Story about having a currency exchange rate database in DWH
|
||||
|
||||
- [ ] Equipment
|
||||
- [X] Equipment
|
||||
- Bookcase for private office <https://www.ikea.com/es/es/p/billy-libreria-blanco-00263850/> x 1
|
||||
- Organizers for private office <https://www.ikea.com/es/es/p/tjena-organizador-escritorio-blanco-60395452/> x 4
|
||||
- Organizers for private office <https://www.ikea.com/es/es/p/vattenkar-estante-escritorio-blanco-20541568/> x 4
|
||||
- Post-its, pens, pencils, a couple boxes of white A4, scissors, tape, etc
|
||||
|
||||
|
||||
- [X] Bitcoin competencies test
|
||||
- [X] Organize week
|
||||
- [X] Marcel afternoon
|
||||
- [X] Schedule with candidate
|
||||
- [X] sh-invoicing issue
|
||||
- [X] dbt runner
|
||||
|
||||
## Permissions for dbt user in DWH
|
||||
|
||||
- It needs
|
||||
- To be able to create schemas and operate on them
|
||||
- To be able to ONLY READ from the sync schema
|
||||
- Nothing else
|
||||
|
||||
|
||||
Strategy:
|
||||
- Create a `modeler` role.
|
||||
- The modeler can:
|
||||
- Read anything in the database
|
||||
- Do anything in the `staging`, `intermediate` and `reporting` schemas.
|
||||
|
||||
|
||||
```sql
|
||||
|
||||
CREATE ROLE modeler WITH NOLOGIN;
|
||||
ALTER SCHEMA staging OWNER TO modeler;
|
||||
ALTER SCHEMA intermediate OWNER TO modeler;
|
||||
ALTER SCHEMA reporting OWNER TO modeler;
|
||||
|
||||
CREATE ROLE dbt_user WITH PASSWORD <password-here>;
|
||||
GRANT ROLE modeler to dbt_user;
|
||||
|
||||
|
||||
```
|
||||
|
||||
|
||||
|
||||
Testing
|
||||
```sql
|
||||
-- Reading from sync schema should work
|
||||
-- Create on sync schema should not work
|
||||
-- Inserting on sync schema should not work
|
||||
|
||||
-- Creating schema should work
|
||||
-- Creating new table in new schema should work
|
||||
-- Insert data in new table in news chema should work
|
||||
-- Reading from that new table should work
|
||||
-- Creating a view in new schema should work
|
||||
|
||||
-- Creating new table in schema `staging` should work
|
||||
-- Inserting data in new table in newschema should work
|
||||
-- Reading from the new table in `staging` should work
|
||||
-- Creating a view in `staging` should work
|
||||
|
||||
-- Doing anything on `public schema should not work`
|
||||
|
||||
```
|
||||
|
||||
## Product weekly
|
||||
|
||||
- Finance
|
||||
- They are using things smoothly, success
|
||||
- Also collecting small details
|
||||
- Any updates from other teams on this?
|
||||
- Infra
|
||||
- Working on it
|
||||
- I want to go to production this week
|
||||
|
||||
|
||||
- Louise
|
||||
- Stripe US account almost ready
|
||||
- Is that going to take waiver payments? Are we synced with finance? What's the rollout plan?
|
||||
- Yes, all payments in US dollar will go through there from Monday onwards.
|
||||
- I will receive access to the Stripe US account.
|
||||
- Who's doing this?
|
||||
- Louise, but she will hand things over to Amanda.
|
||||
|
||||
## 2024-02-06
|
||||
|
||||
- [X] Add silly shell script to git repo + instructions on how to deploy
|
||||
|
||||
# Now
|
||||
|
||||
- [ ] document infra
|
||||
- Continue
|
||||
|
||||
- [ ] Plan holidays
|
||||
|
||||
|
||||
Architecture
|
||||
- [ ] Set up dbt runner properly
|
||||
- [ ] Document deployment plan
|
||||
- [ ] Learn how the hell does the Azure cli work with the templates
|
||||
- [ ] Create repo with automation templates
|
||||
- [ ] Deploy
|
||||
|
||||
Data docs
|
||||
- [ ] Start working on internal docs for ways of working around DWH, VPN, dbt, etc.
|
||||
|
||||
|
||||
|
||||
- Data Catalogue
|
||||
- [ ] Add CosmosDB to data catalogue
|
||||
|
||||
- #13121
|
||||
- [ ] Get access from Ray
|
||||
- [ ] Set query parameters in the athena thingy
|
||||
- [ ] Add the new tab with totals
|
||||
|
||||
|
||||
- [ ] Ask IT for <data-team@superhog.com> mail
|
||||
- [ ] Place this in Backlog: <https://superhogteam.slack.com/archives/C06A52YUDM1/p1702557374922989>
|
||||
- [ ] Story about checking for accounts with no deal id
|
||||
|
||||
|
||||
|
||||
- Data literacy roadmap
|
||||
- Data consumer
|
||||
- Anyone who requests it
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue