From 09c7d86a91d52bc0f4b328ff9100500ca67fff4c Mon Sep 17 00:00:00 2001 From: Pablo Martin Date: Thu, 1 Feb 2024 10:48:09 +0100 Subject: [PATCH] thingies --- log.md | 260 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 256 insertions(+), 4 deletions(-) diff --git a/log.md b/log.md index 71b596b..1501873 100644 --- a/log.md +++ b/log.md @@ -1718,10 +1718,236 @@ Invoicing Reformation Direct debits, perhaps finance could use this with lower fees? +# 2023-01-22 + +## Product weekly + +- Hiring +- Finance + - Great success + - Elaine happy + - Amanda happy + - Jamie happy + - I'm happy + - Tomorrow we make orphan stories get adopted by someone + - I'll start working on some tools myself for January cycle + - [X] Stripe exports + - [ ] Deposit exports +- Infra/Azure + - [ ] This week I document and script everything out and I'll book some time from Ben to go to production + +# 2023-01-23 + +## Meeting with Matt + +- Data quality for Account Managers +- Hubspot reporting +- Alex's role once we have a DWH + - He can work great + + + +# 2023-01-24 + +## Product catchup + +- Finance + - I'm rushing some developments + - We very much need for it to not rot again or it will be nasty + - Lourdes, Stripe, let's get things clean + - Ben R. and I are aligned long term philosophy +- Azure + - Document things this week + - Go to production next week with Ben if possible + - We have a contractor coming along for infrastructure for the next 3 months +- Interview with Jeremy + + + +## Meeting with Leo + +- Middleware PBI + - He sends me a list of partners + - I check why are they not appearing in the report + +## Product KPIs + + + + + +# 2024-01-26 + +ETL Role - Informatica +Then Support on ETL +Also doing reporting SAS, SAP BO, + + +Systems Analyst +SQL with MSSQL and Oracle +SSRS +Tableau + + + +- git + - SSRS + - SQL queries + - Issues with merges + - Doesn't +- postgres + - No +- dbt + - No +- pbi + +- He likes quite a bit the DE side +- He prefers the Data Analysis + + +- Data contracts + +- Organizing data consumption + - he suggests kanban approach + - each department has a product owner from that department + - g + +- How do you handle unrealistic expectations? + +- What are things that you don't like about the current work and that you would hope would be different? + - Not much time to dive into analysis beyond preparing reports. Would like to have more time for free-form exploration. + - Data Quality issues, a lot of shadow-data. With people playing games with Excel sheets. + - Very far away from the business. + +- How do you keep business people familiar with the reports and metrics you produce? +- Any times you've seen teams failing and why do you think it happened +- What's the favourite reporting tool? + +- Have you experience with ELT? + +- What don't you like about your current position/role/responsibilities? + +# 2023-01-29 + +Today's priorities: +- [ ] Invoicing tool +- [X] Data News +- [X] Invoicing Documentation +- [ ] Wireguard + + + +# 2023-01-30 + +- [X] Invoicing tool +- [ ] Wireguard +- [X] Send December export +- [X] Do update query + +## Hubspot x Finance + +- DQ + - Pricing + - Payaway percentages + - Consistency across hubspot and dashboard + - Alex will chase people when they forget stuff + - Deal Id consistency + - It's mandatory + - + - Offering and pricing + + +## Call with Leo + +- KYG stuff: please align with Ben C. + +- Guesty/Athena/edeposit dashboard + - It doesn't say it's Guesty + - Be able to filter by clients (Guesty, Hopper) for when we get more stuff + - Have more totals easily visible + - Total nights + - Total bookings cost + - Total records + - Include listings data + +## Call with Looker + +Meeting with Suzanne and Finnian + +Suzanne + - Account Manager +Finnian + - Customer Engineer + +- Usual infrastructure pattern +- Integrations with AD for permission handling + - Content access on team basis + - Row level access + - Column level security +- Reading from SQL Server +- How does this play with `dbt` +- Migrating from PBI +- How do you compare against Lightdash, Preset/Superset +- Caching options? + +- Version control + - Only data models, or also visual artifacts? + - Can scheduled sends be version controled or programmatically defined? + - Can it be integrated with Azure Devops? + +- Automatic + +- Three layers + - Reporting + - Self-service + - Modeling + +- Scheduled email/slack alerts with conditions on visuals +- Scheduled reports into slack? + + +We are eligible for startup credits. 100K awarded for 1 year team + +Stuff I want to read:; +- https://honeydew.ai/uncategorized/dbt-semantic-layer-vs-looker-lookml/ +- https://www.rittmananalytics.com/blog/2022/2/1/lightdash-looker-and-dbt-as-the-bi-tool-metrics-layer/ +- https://www.getdbt.com/coalesce-2020/perfect-complements-using-dbt-with-looker-for-effective-data-governance + +# 2024-02-01 + +- [X] Invoicing +- [ ] Ryan +- [ ] Leo +- [ ] Invoicing backlog +- [ ] Wireguard + # Now + +Wireguard VPN +- Delete everything +- Set up firewall listening on 52420 +- Create point to point + - Set it up + - Hit dummy HTTP from Windows + - Hit dummy HTTP from Linux + - Hit coredns from Windows + - Hit coredns from Linux + - Unsuccessfully try to jump anywhere else +- Extend to point to site + - Change server config + - Try to reach airbyte web from windows + - Try to reach airbyte web from linux + - Try to reach airbyte ssh from windows + - Try to reach airbyte ssh from linux + - Try to reach DWH from windows + - Try to reach DWH from linux + - Try to reach Core from windows + - Try to reach Core from linux + + Architecture - [ ] Document dev ramblings @@ -1741,6 +1967,23 @@ Invoicing Reformation - [X] Backlog discussion - [X] Quick coffee with Amanda - [X] Phase 1 summary in Notion + Email +- [ ] January processing + - New improvements + - Automated Stripe Transaction Exports + - Exchange Rate Formalization + - Automated Stripe Waiver Records management + - Automated Waiver Computation + - Plan + - 29 Morning + - Pablo shares with Jaime new export + - Jamie checks a representative sample of reports and shares errors issues with Pablo + - 29/30/31 + - Pablo applies any hotfixes possible + - On the 31st, we decide which new improvements we feel confident about and include them + - 1 + - Robinson runs the classic export + - Pablo runs the exports + - Everything gets shared with Finance team - Data Catalogue - [X] Make big announcement + Quick Presentation Video @@ -1748,8 +1991,7 @@ Invoicing Reformation - [ ] Add Stripe reporting as data product in data catalogue - [ ] Add invoicing tool as Data Product - - I found this: - - Running this command fixed it for Lourdes: `/usr/sbin/softwareupdate --install-rosetta --agree-to-license`. All is fine if user sees `Install of Rosetta 2 finished successfully.` + - [ ] Ask IT for mail - [ ] Place this in Backlog: @@ -1758,10 +2000,20 @@ Invoicing Reformation - [ ] Story about report to spot non-matching pricing details across Dashboard Superhog. - [ ] Story about having a currency exchange rate database in DWH - - - [ ] Equipment - Bookcase for private office x 1 - Organizers for private office x 4 - Organizers for private office x 4 - Post-its, pens, pencils, a couple boxes of white A4, scissors, tape, etc + +- Data literacy roadmap + - Data consumer + - Anyone who requests it + - Report builder + - Who can manage PBI (Louise, Lourdes, Ben C., Alex, Dagmara, Jamie, dev team) + - SQL Writer (Sandbox, reporting layer) + - Who can write SQL (Lourdes, Ben R., dev team) + - DWH Modeler + - Me, other Data team members + - Engineer + - God mode access