diff --git a/log.md b/log.md index 63ffbc6..33e39da 100644 --- a/log.md +++ b/log.md @@ -1745,8 +1745,6 @@ Direct debits, perhaps finance could use this with lower fees? - Alex's role once we have a DWH - He can work great - - # 2023-01-24 ## Product catchup @@ -1762,8 +1760,6 @@ Direct debits, perhaps finance could use this with lower fees? - We have a contractor coming along for infrastructure for the next 3 months - Interview with Jeremy - - ## Meeting with Leo - Middleware PBI @@ -1772,29 +1768,24 @@ Direct debits, perhaps finance could use this with lower fees? ## Product KPIs - - - - # 2024-01-26 +## Interview with Jeremy + ETL Role - Informatica Then Support on ETL -Also doing reporting SAS, SAP BO, - +Also doing reporting SAS, SAP BO, Systems Analyst SQL with MSSQL and Oracle SSRS Tableau - - - git - SSRS - SQL queries - Issues with merges - - Doesn't + - Doesn't - postgres - No - dbt @@ -1804,7 +1795,6 @@ Tableau - He likes quite a bit the DE side - He prefers the Data Analysis - - Data contracts - Organizing data consumption @@ -1830,13 +1820,12 @@ Tableau # 2023-01-29 Today's priorities: + - [ ] Invoicing tool - [X] Data News - [X] Invoicing Documentation - [ ] Wireguard - - # 2023-01-30 - [X] Invoicing tool @@ -1853,10 +1842,9 @@ Today's priorities: - Alex will chase people when they forget stuff - Deal Id consistency - It's mandatory - - + - - Offering and pricing - ## Call with Leo - KYG stuff: please align with Ben C. @@ -1875,9 +1863,10 @@ Today's priorities: Meeting with Suzanne and Finnian Suzanne - - Account Manager + +- Account Manager Finnian - - Customer Engineer +- Customer Engineer - Usual infrastructure pattern - Integrations with AD for permission handling @@ -1895,7 +1884,7 @@ Finnian - Can scheduled sends be version controled or programmatically defined? - Can it be integrated with Azure Devops? -- Automatic +- Automatic - Three layers - Reporting @@ -1905,13 +1894,13 @@ Finnian - Scheduled email/slack alerts with conditions on visuals - Scheduled reports into slack? - We are eligible for startup credits. 100K awarded for 1 year team Stuff I want to read:; -- https://honeydew.ai/uncategorized/dbt-semantic-layer-vs-looker-lookml/ -- https://www.rittmananalytics.com/blog/2022/2/1/lightdash-looker-and-dbt-as-the-bi-tool-metrics-layer/ -- https://www.getdbt.com/coalesce-2020/perfect-complements-using-dbt-with-looker-for-effective-data-governance + +- +- +- # 2024-02-01 @@ -1923,9 +1912,7 @@ Stuff I want to read:; - [ ] Wireguard - [ ] Create interface for all traffic - [ ] Document - - [ ] - - + - [ ] # 2024-02-02 @@ -1936,7 +1923,6 @@ Stuff I want to read:; - [ ] Set up dbt runner machine in dev - [ ] Document infra - # 2024-02-05 - Data Catalogue @@ -1945,6 +1931,7 @@ Stuff I want to read:; - [X] Add invoicing tool as Data Product Invoicing Reformation + - [X] Finish stage descriptions - [X] Drop existing knowledge on inputs - [X] Drop details on Stripe exports @@ -1983,9 +1970,8 @@ Invoicing Reformation - Organizers for private office x 4 - Post-its, pens, pencils, a couple boxes of white A4, scissors, tape, etc - - [X] Bitcoin competencies test -- [X] Organize week +- [X] Organize week - [X] Marcel afternoon - [X] Schedule with candidate - [X] sh-invoicing issue @@ -1998,14 +1984,13 @@ Invoicing Reformation - To be able to ONLY READ from the sync schema - Nothing else - Strategy: + - Create a `modeler` role. - The modeler can: - Read anything in the database - Do anything in the `staging`, `intermediate` and `reporting` schemas. - ```sql CREATE ROLE modeler WITH NOLOGIN; @@ -2019,9 +2004,8 @@ GRANT ROLE modeler to dbt_user; ``` - - Testing + ```sql -- Reading from sync schema should work -- Create on sync schema should not work @@ -2052,7 +2036,6 @@ Testing - Working on it - I want to go to production this week - - Louise - Stripe US account almost ready - Is that going to take waiver payments? Are we synced with finance? What's the rollout plan? @@ -2065,26 +2048,122 @@ Testing - [X] Add silly shell script to git repo + instructions on how to deploy -# Now +# 2024-02-07 -- [ ] document infra - - Continue +## Product catchup + +- Full steam ahead with infrastructure + - Planning on deploying production on friday +- Exploring cosmosdb +- Invoicing + - Moving forward fine. + - Small issues that we will improve, nothing critical. + - I will timebox 4 working days towards the end of the month for improvements and data quality topics + - Data quality and strictness + +## Cancellations intro + +- Guests can opt-in into cancellation protection +- Guest pays. If he needs to cancel, he does and gets his booking money back. +- We initially wanted to do the nice +- Plays nicely if the host has a super strict cancellation policy, like no + +- They will define more reporting needs so we can decide what we implement +- They will loop me in when the database model is being designed so I can provide advice on it + +## Invoicing issues + +- Several accounts with same hostname, the export files are being overwritten. +- We will find a way to deal with this in next cycle. + +# 2024-02-09 + +## Interview with Barnabas + +- git? + - No experience +- python? +- sql/postgres? + - SQL + - Infopool? + +- dbt +- reporting/pbi? + - Power BI + - Which tool do you prefer? PBI + - Why? + - "Chemistry" + - Modeling in DWH vs PBI + +- Alteryx! + +- His example project was a pretty standard ERP thingy... +- "I prefer SQL to Python, but I use python for ad-hoc" + +- Data contracts + +- Organizing data consumption? +- How do you handle unrealistic expectations? + +- Technical vs Business +- What are things that you don't like about the current work and that you would hope would be different? +- How do you keep business people familiar with the reports and metrics you produce? +- Any times you've seen teams failing and why do you think it happened? +- What's the favourite reporting tool? +- Have you experience with ELT? + +- 14 years of experience +- SQL +- Python +- PBI/Tableau +- Olympus Portal (show transactions live in POS) +- Specter (self-service learning platform) +- Personalized contextual messaging towards customer for Lloyds + - Tell me more about this + - What was the team like + - What was your exact role +- Migrating from DWH to GCP + +- I'll get together with the Ben and Ben and we will be back in touch. + +My impression: + +- Reading his profile from screen, that wasn't nice. +- Avoided question on technical skills a lot, I'm very unsure about his basic technical skills. +- Very confusing explanations. +- I don't think he can organize company-wide data organization. + +# 2024-02-12 + +- [ ] Last week update +- [X] Send slot to Leo +- [ ] Infra +- [ ] Get service account from Ben R. + +## Product weekly + +- Got tangled with finance +- Infra is very close. + - Today I'll finish the script + - If invoicing doesn't give me troubles, production should be deployed before thursday + +## 2024-02-13 + +- Infra + - [ ] Make E2E visually with Infratest to check everything is connected properly + - [ ] Tear down infratest resource group + - [ ] Deploy the Production environment +- [ ] Last week update + +# Now - [ ] Plan holidays -Architecture -- [ ] Set up dbt runner properly -- [ ] Document deployment plan -- [ ] Learn how the hell does the Azure cli work with the templates -- [ ] Create repo with automation templates -- [ ] Deploy - Data docs + - [ ] Start working on internal docs for ways of working around DWH, VPN, dbt, etc. - - - Data Catalogue - [ ] Add CosmosDB to data catalogue @@ -2093,13 +2172,10 @@ Data docs - [ ] Set query parameters in the athena thingy - [ ] Add the new tab with totals - - [ ] Ask IT for mail - [ ] Place this in Backlog: - [ ] Story about checking for accounts with no deal id - - - Data literacy roadmap - Data consumer - Anyone who requests it