My challenge in Superhog is going to come with a lot more freedom and lot more responsibility than the previous positions I've held. The crown is heavy, and so it's both my time to shine and to not fuck up.
One of the habits that I value a lot, have almost never seen successfully applied in companies, and have seen successfully applied to some degree in Mercad~ao (thanks João) is writing things down. A lot. Everywhere. All the time. Organized repositories of text all over the place. Basically, every time you are about to open your mouth to say something, you should instead or also write it down.
I'll start on the eve of my arrival by writing down some thoughts on stuff I need to do.
- Social wise
- Make contacts: I need to have a couple of coffees with a lot of people:
- Global KPIs : <https://guardhog-my.sharepoint.com/:x:/g/personal/ben_cotte_superhog_com/EU9p7oaSBGRFoYGlJkfsMqwBt6sCRfF-jd0_3zkUYJxV2Q?e=k7ad02>
- Data used in Sales and Account management : <https://guardhog.sharepoint.com/:x:/s/Guardhoggroup/EZXCzc8ftM5JrOm43UpCh5QBrB05Rrq6MDJ7-j-B8pjHJQ?e=4o4JnG>
- Data used in Finance : <https://guardhog-my.sharepoint.com/:x:/g/personal/ben_cotte_superhog_com/ERZWA02zJlxAi8akJmzVz40BQ3sjwARt1EYUnP6Y4-sbZg?e=maj0dU>
- Data used in Product : <https://guardhog-my.sharepoint.com/:x:/g/personal/ben_cotte_superhog_com/Ed1Wt4-T-9FIodoJEkmYUzMB3VzalxzPTsiYcrQOm1ik1g?e=qZhW9Z>
- Invoicing Process : <https://miro.com/app/board/uXjVMirYfDg=/>
- Make current operations work properly now that we have resources (backfill). Andrew's opinion is they were understaffed for long and that explains why things are not working out smoothly.
- Andrew is very concerned about the invoicing process and the choice of accounting tool and now I understand why. If any of those two, are not worked out properly, they will lead to a very nasty situation. I need to judge my role in those fast and get evereyone aligned.
---
Stuff we couldn't go through, let's discuss next meeting:
- Is the P&L somehow decentralized?
- How do you and Humphrey split responsibilities?
- Things I've liked so far
- Good attitudes, good spirit
- Product is clear
- Lots of room for improvement
- If the product is making money with the current simplicity, we can surely do better
- Systems are growing uncontrolled, this will bite us in the ass soon and will hurt people's work
- The huge proportion of new joiners is going to be a cultural challenge
- Data is in a bad shape, I don't have data
- What do you think about my plan?
- Couple of weeks listening in
- Ceremony to kickstart the team
- Team proposal
- Backlog
- Building out our infrastructure as well as working on hot things. Balance between immediate stuff and building for the future
- Is there something that concerns you? Anything you are scared of? Any issue that keeps you up at night?
- I'll be eager to grab a lunch or to hear more about both you and Humphrey. I care about the plans at the top, it's one of the main reasons I'm here and not at a big corp
## Meeting with Clay
Onboarding and PMSs
- Every host gets manual onboarding. This is a pain point
- Clay fully takes care of PMS integrations. The most relevant bit of this happens to be managing onboardings, since there isn't much trouble once the integration is done for each PMC.
- Mixpanel: invite sent. Note we’re working out a bug at the moment which is blocking us from judging our accurate usage and moving forward with upgrading to the paid version/using it properly!
- I'm worried about only having two cycles to play around + Elaine leaving. I dislike this being the first thing to jump to, but I think it will have to be so
- I'm afraid untangling this doesn't look like only a data issue and it's going to be complex dancing altogether
- Ben thinks there is indeed a high level of complexity, which is natural due to the way the bookings and money flows work and due to resolution process
- The issues with payment problems and manual retries are also part of the issue. The current dual payment providers situation is part of the problem.
- Ben has a very good acceptance of the fact that product changes in dashboard might be required to get this in good shape.
- I see a general pattern of shadow IT and short-comings in Dashboard being tackled with system+organization patches everywhere. This will bite us back eventually (is kind of already)
- [X] Basic set of cutlery to keep in our office <https://www.amazon.es/BEWOS-cubiertos-cuberteria-inoxidable-Lavavajillas/dp/B0BZXZJ31Y/ref=sr_1_10?__mk_es_ES=%C3%85M%C3%85%C5%BD%C3%95%C3%91&crid=13L3AVL2USBKQ&keywords=cuberteria&qid=1701420382&sprefix=cuberteria%2Caps%2C131&sr=8-10&th=1>
- [X] A few locks for lockers to make life easier for new joiners: <https://www.amazon.es/Combinacion-Diyife-Impermeable-Archivadores-Herramientas/dp/B08Y6D1Y38/ref=cs_sr_dp_n?__mk_es_ES=%EF%BF%BDM%EF%BF%BD%7D%EF%BF%BD%EF%BF%BD&crid=8ESI3Q3APIBJ&keywords=candado%2Btaquilla&qid=1701419900&sprefix=candado%2Btaquilla%2Caps%2C104&sr=8-6&th=1>
- [X] Sit down with Ana (once her scope is more clear and she's in control)
## Ben Cotte initial explorations
Ben sent me some of the materials he has been building around assessing data in superhog last week. I'm listing some of them there to make sure they don't fall through the cracks and I can come back to them when needed.
Links to the videos shared by Ben with different areas:
Ajunto igualmente el inicial draft hecho por Trust Keith, pero tendras que hablar con Louisa de legal para que te pase la ultima version (o que pida ella la ultima version)
The Data Team is going to be a slightly different squad with a few members capable of doing technical work and building stuff. As all squads, is going to have a limited capacity and an infinite demand. Hence, like any other team, we will have to build a backlog, groom it, and prioritize it.
Because of this, I strongly believe that Data requires Product Management and its mindset. I’ve seen many companies fail at this, providing PO to most squads but not to Data “because it’s different”.
Nevertheless, I think Data it’s truly different. I think Product Management looks a bit different for Data than it will for the other tech squads for the different product domains. This is because of things like:
- In my experience, building data products like dashboards or one-shot, adhoc analysis is very different than building “regular” software products. There are commonalities, but also quite a few differences.
- There is a lot of foundational work that is necessary, yet doesn’t really fit nicely into one single product (like grunt work in the guts of a Datawarehouse, which is necessary to support many different reports for many different areas).
- The cross-functional nature of the team means that it has many different clients with different needs. This is challenging from a product management POV because you are fighting a multiheaded hydra.
- Data is less free than many other tech teams because it depends a lot on existing systems and teams to build it’s own stuff. We build on top of other’s work, and so have to manage these dependency relationships.
- Should there be a dedicated PM for Data? Is this an impossible task because it’s simply too much for a single brain?
- Should instead all PMs be able to add stories to Data’s backlog? What about needs that go across domains/teams? And about areas that don’t have a PM (like Finance)?
- Perhaps Data should have multiple backlogs? Or perhaps it’s an overkill?
- Should I simply act as the Data PM for now? How sustainable will that be?
1. Well, it’s an important topic, it should be well thought and I think your take on it is vital.
2. If there are headcount implications, we need to agree on them and get things rolling. Whatever we decide, I would like to bring to the Data Kickoff meeting on the 14th.
- Data Analyst with a strong product view acting as a product owner
- Begin with 3 and add more by justifying with a roadmap
## Meeting with Andrew about headcount
My notes:
---
Hi Andrew,
On the Data kick-off meeting for the 14th, I would like to present some plans in terms of what the data team should look like, people wise. I think it would be unwise to wait until then to show my cards, because hopefully that meeting is more about aligning visions that about discussing a lot. So, I would like to share my current thoughts to see how they align with yours and with overall budget/headcount constraints and plans.
@Ben Cotte, @Ben Robinson also looping you in for transparency, feel free to chime in.
- I think there is a lot of Data work to get done (which is great news because it’s a lot of good coming for the company).
- I’m obviously not gonna get much done on my own. I see backlog for a nice 4 heads (including myself). I would look for another Data Engineer and a couple of Data Analysts.
o The additional Data Engineer would be more on the strong code, modelling, infra side of things. This is important because we have a lot to build in this area, and because solely relying on me is a terrible idea for many reasons.
o The Data Analysts would be sitting closer to Product/RevOps/Finance/etc, working on top of.
o Nevertheless, there is some degree of overlap between the skillset and capabilities of both profiles, and I would like to get people with a good multi-hat attitude for these early stages.
o I would also be in favour of going for experienced profiles at this point. I don’t think this is a good time for someone to learn on the job. More junior profiles might provide better value/cost when the team is more mature and things have settled a bit.
- BTW, I’m aware there was an open search for an experienced analyst but, from the lack of news, I’ve assumed that’s not progressing or at least not close to completion?
- Besides this, I see the need for some extra work to be done in Product (somehow closer to Ben C.) and Infrastructure (somehow closer to Ben R.) The lines here are fuzzy. This doesn’t need to necessarily be part of the Data Team strictly speaking, but the Data Team will need some of this. I’m planning on discussing with both Ben’s next week to align.
Please, feel free to provide feedback and tear it to the ground if necessary. We can sit and talk whenever if needed. I would like to have some, even if it’s rough, we have agreed on for Thursday.
---
- Thursday plan
Senior Data Analyst (the brains: sitting closer to the business)
Let's adjust the third position depending on what we find for the Senior Data Analyst
- Exporting data out of Iplicit doesn't seem very important. We are going to end up doing the billing and charging logic outside of the system and then dump it in through the API, so it will be out anyways.
- I've started working on cataloguing our data and data products
- Diagnostics
- General vibe: we are in a sweet moment. The child is turning into a man, and we are starting to see some cracks in his clothing (Hulk picture)
- Concerns
- Operations issues: non-standardized process, hard to measure, hard to improve on
- Product issues: common-sense is the only driver, no analytical approach. No mature pricing strategy.
- Sales: nose driven methodology. Lack of basic data to drive marketing and sales (CLV, segments, analytics on channels, sales agents performance metrics, etc)
- Finance: invoicing.
- Technology: lack of data infra. People are integrating data with blood, sweat and tears.
- Good stuff
- Product works
- Good tech culture, mature engineering team
- Human capital is looking great to me
- Strong bet on product management feels promising
- Sales is nose driven, but it seems to be a very good nose given the results
- Do you think I've missed any important points? Any disagreements
- Data Team vision
- Responsibilities
- Supervise data across the org: catalogue and document all relevant items
- Build and own infrastructure to service needs that don't fall nicely within anyone's domain: DWH, integration of external data sources, etc.
- Build stable data products: reports, dashboards, APIs, etc
- Provide brainpower for highly complex analysis
- Build data literacy across the company
- Not:
- Own every little reporting need and become a blocker
- Act as a release valve to product short-comings
- Overcome bad process and system designs with miracles
- What does success look like
- Data is easily accessible for those who need it
- We have capacity to build data products with speed and quality
- Production systems run with good uptime
- We have the foundations to attempt more sophisticated projects (ML/AI, advanced statistical and optimization models, etc)
- Powers
- Be involved early when needed
- Impose certain standards on data owners in terms of documentation and practices
- Enforce agreements with upstream dependency owners (Robinson, don't break my columns please)
- Other parts of my philosophy
- We have a backlog, limited capacity and thus we need to prioritize, like any other team
- Not every question deserves being answered
- I'll be annoying on my dependencies
- Tell me if my vision fits what you expect of my team
- Plans
- Short-term (weeks)
- First batch of cataloguing and documenting
- Take control of important reports
- PBI
- Hubspot
- Global KPIs
- etc.
- Assist in the invoicing issue from the data side of things
- Get into important loops to provide the data point of view
- Product design
- Standardization of operational processes
- Company software decisions: stripe vs acquired, new accounting software, etc.
- Perform a first approach at grooming a data products backlog
- Start hiring
- Mid-term (months)
- Build new data products to cover different areas
- Build data infrastructure: DWH, integration solutions, better reporting tools
- Data trainings outside the data team
- Future
- Maintain and scale
- Advanced Data Products: ML&AI
- I don't know what else, I don't like looking that forward unless strictly necessary. Life's just too messy.
- Hiring
- Another data engineer
- 2 Data Analysts
- Seniority. No place for people who don't know what they are doing at this stage. I would rather have a few experienced hands than an army of newbies.
- Some timings and next actions
- Hiring
- Timebox data cataloguing and documenting
- Controling existing reports
- Timebox
- First round of grooming with the different areas to build up backlog
> Something's wrong with one or more fields: (VerificationPayment) ListingFriendlyName, (VerificationPayment) ListingAddressLine1, (VerificationPayment) BookingCheckout, (VerificationPayment) BookingCheckin
I agree with them that I will try to pullout the data of the table manually first and send it over so they become unblocked since they need this ASAP. Afterwards, I'll fix the report.
### Exporting the data
The table is using the following columns from the following Semantic Model entities:
The relationship between the entities is as follows:
- VerificationPayment N - 1 Host (on HostUser)
- VerficiationPayment N - 1 Payee (on PayeeUserId)
There are no significant transformations in the PBI queries.
I composed the following query to replicate what the table is building:
```sql
/****** Script for SelectTopNRows command from SSMS ******/
SELECT
Host.BillingCountryName,
Host.DealId,
Host.Email,
Host.Name,
Payee.BillingCountryName,
Payee.Email,
Payee.Name,
VerPay.DepositWithheld,
VerPay.FailedAttemptCount,
VerPay.IsPayAway,
VerPay.OrderId,
VerPay.PayAwayGrossAmount,
VerPay.PayAwayMinimumCommission,
VerPay.PayAwayPercentage,
VerPay.PayeeCurrencyIso,
VerPay.PayeeGrossAmount,
VerPay.PaymentDate,
VerPay.PaymentDueDate,
VerPay.PaymentProvider,
VerPay.PaymentRef,
VerPay.RefundDate,
VerPay.RefundDueDate,
VerPay.RefundRef,
VerPay.RetryCount,
VerPay.Status,
VerPay.SuperhogGrossAmount,
VerPay.SuperhognetAmount,
VerPay.SuperhogVatAmount,
VerPay.SuperhogVatRate,
VerPay.Type
FROM [report].[VerificationPayment] VerPay
LEFT JOIN [report].[User] Host ON VerPay.HostUser = Host.Id
LEFT JOIN [report].[User] Payee ON VerPay.PayeeUserId = Payee.Id
WHERE VerPay.PaymentDate > '2023-11-25' -- Jamie requested payments after 25th November
```
And exported the data and sent it over to Jamie.
Jamie then asked for more columns (specially, the broken ones):
> Hi Pablo - sorry, is it possible to have a report pulled with the columns that PowerBI is saying 'needs fixing'. So, adding columns for ListingFriendlyName, ListingAddressLine1, BookingCheckout and BookingCheckin for each of those payment.
I extend the query and it turns into the following:
```sql
SELECT
Host.BillingCountryName,
Host.DealId,
Host.Email,
Host.Name,
Payee.BillingCountryName,
Payee.Email,
Payee.Name,
VerPay.DepositWithheld,
VerPay.FailedAttemptCount,
VerPay.IsPayAway,
VerPay.OrderId,
VerPay.PayAwayGrossAmount,
VerPay.PayAwayMinimumCommission,
VerPay.PayAwayPercentage,
VerPay.PayeeCurrencyIso,
VerPay.PayeeGrossAmount,
VerPay.PaymentDate,
VerPay.PaymentDueDate,
VerPay.PaymentProvider,
VerPay.PaymentRef,
VerPay.RefundDate,
VerPay.RefundDueDate,
VerPay.RefundRef,
VerPay.RetryCount,
VerPay.Status,
VerPay.SuperhogGrossAmount,
VerPay.SuperhognetAmount,
VerPay.SuperhogVatAmount,
VerPay.SuperhogVatRate,
VerPay.Type,
Booking.CheckIn AS BookingCheckin,
Booking.CheckOut AS BookingCheckOut,
Acc.AddressLine1 AS ListingAddressLine1,
Acc.FriendlyName AS ListingFriendlyName
FROM [report].[VerificationPayment] VerPay
LEFT JOIN [report].[User] Host ON VerPay.HostUser = Host.Id
LEFT JOIN [report].[User] Payee ON VerPay.PayeeUserId = Payee.Id
LEFT JOIN [dbo].[VerificationRequest] VerReq ON VerPay.VerificationRequestId = VerReq.Id
LEFT JOIN dbo.Booking Booking ON VerPay.VerificationRequestId = Booking.VerificationRequestId
LEFT JOIN dbo.Accommodation Acc ON Booking.AccommodationId = Acc.AccommodationId
The first thing that comes to mind is: is the report broken in staging? I'll go check that out. -> Yes it is.
So, either it has been broken since the last time someone released something, or something in the underlying data model has broken.
I'm gonna check it the PBI refreshes have worked fine lately in PRD. -> They have
Apparently the visual (a table) is complaining about the following four fields not being available in the model. The fields are:
-`Listing Friendly Name`
-`Listing Address Line 1`
-`Booking Checkin`
-`Booking Checkout`
I had this quick conversation with Ben R.:
Pablo Martin
1:26 PM
Hi Ben, I have a little fire (broken PBI report). I was hoping you might have a couple of quick answers for a couple of quick questions that might save me some time
1:26
The broken report is the Payments > Client Pay Away Information. The main table is complaining about some missing fields
1:27
these are the fields
1:27
image.png
image.png
1:28
Do they ring a bell? Could they be related to that little release we looked at last month together with Lawrence?
If we don't have more context or ideas, I will assume these are safe to remove and call it day. If someone comes in asking for them again, then we put the detective hat on and work on it.
1:28
thanks
Ben Robinson
1:32 PM
They were broken some time ago and have been needed to be fixed. Those fields were added into the view a while back as a temp but never persisted in source control so got wiped out. They no longer exist in the view. Solution one remove them from the model and re publish the report without them. Solution two add in the listing and booking views into the exisitng model and map them - this is what we tried quickly but due to the relationship it was not playing ball and we had to abandon due to time
1:32
Happy to walk you through it though.
Pablo Martin
1:38 PM
Thanks Ben. Then I think I'll go the route of simply removing them for now.
Solution two add in the listing and booking views into the existing model and map them
Do you recall how need this for what? I'm wondering on whether to proactively do this or simply let it go to sleep unless someone brings it up.
Ben Robinson
1:42 PM
Alex is aware of it and there was a need for the fields quite a important one but they are working around the issue atm. All down to reconciliation and waiver payments i believe.
Pablo Martin
2:28 PM
:+1: thanks
So I'll go ahead, remove them, and then wait for whoever needs them to bring this back to my attention.
Here’s a rough sketch of the changes lifecycle, from start to release:
- Prep
- [X] Ensure no one else is working on the reports and announce you are going to do it.
- [X] Make a copy of the existing state of the report files, and back it up with the date.
- Build and test
- [X] Make the changes you need on the reports on your laptop.
- [X] Once changes are ready, publish the report to the staging workspace.
- [X] Make all the necessary checks in staging to ensure your changes are working as intended and nothing else broke. This is specially important if you have modified a semantic model that feeds multiple reports.
- Deploy
- [X] Once you feel confident that all is good, publish the report again to production.
- Again, make all the necessary checks in production to ensure your changes are working fine.
- Clean up
- [X] Announce that you are not working on the reporting suite anymore.
- [ ] Make an announcement to the relevant audience about the changes so that people can start using them.
- I tried with my contacts but got no interest so far
## Iplicit kickoff meeting
We meet with Kieran May from their side. Project manager from their side. He was an accountant.
Solution design meeting on 17/01. Deep dive in Accounting processes. Full blown detail. Sales processes, purchase processes, financial transactions, bank accounts, etc.
PID Document starts today.
After we create sandbox <-Usedfortrainingandalsofortechnicaltesting.Willalsoneedstaticmasterdata,wewillgettemplatesforloadingthat.
Workshops. We design and parametrize, work through how we use the tool. Also chance to modify final configs.
User base training with the end users.
Then, UAT before we go live, final sign off.
Production environment gets copied over from UAT. Configs get copied over, but actual books and transactions get deleted so Production stands clean.
After go live, we have support calls still with the implementation team.
|- Superhog Limited (UK) (has VAT, quarterly, standard not cash-based, has multiple currencies (USD, GBP, EUR, AUD, USD, AED, ZAR)) (<accounts@superhog.com>) used to get all sales
- Do we have somewhere to check timeline/milestone/status?
- Are we only starting on April 1st live? Yep.
- Do we keep the sandbox env forever? - We definitely keep it for a bit. It costs money, so we should judge if it's worth keeping it forever.
- Currency conversion we need to agree on exchange rates -> Iplicit comes with currency data, we should look into how to load it in the DWH and use that for all reporting so we have a unified vision on exchange rates.
# 2024-01-09
## Call with Humphrey
- Most important stuff
- Invoicing needs fixing and finance needs to be brain-available
- Resolutions needs to stop being a dark, obscure corner
- Lots of improvement operations wise to be more efficient
- Marketing needs data to look at hosts like users (LTV, CAC, active users)
- Pricing can be a massive profit lever if we reach the maturity of insurance companies, we need to aim for it
# 2024-01-10
## Product catchup
- Azure
- WIP
- Finance
- WIP, expected a million stories coming out of this
- Hiring
- I tried with my contacts but got no interest so far
- But the history will always be there for each guest journey, because the cover settings will be in the PaymentVerification stuff, and each VerificationRequest
- I've built some stuff, you need to know how to dance with it
- Starting Point
- The Data Scene two months ago
- Different systems with their own little reporting tools attached
- Issues with it
- Mixing Data from different places
- Limited modelling
- Decentralized modelling
- Now
- DWH and ELT model
- Centralize stuff in DWH
- The logical structure of the DWH
- Syncing sources
- Problems we have solved
- Mixing data from different places
- Advanced modelling
- Centralized Modelling
- Self-contained reporting will still be a thing though
- What you should know
- As a producer
- Keep us in the loop when building new stuff
- We need your help to document your data to make it usable for others
- We are a dependency: please don't make our life hell
- As a consumer
- You should be aware of whether Data is already synced
- You will have documentation to understand the available data
- Request stuff with time, it will help us plan capacity
- Data champions
- The more knowledge you have, the less you depend on us
# 2024-02-22
- [X] For Ben C.: template for Data Product Requests
- [X] Also book time for the week after invoicing
## Cancellation Update with Matt
- We are aiming for the 3rd monday of march (18/3)
## Cancellation Data Model
- We can know if the cover was offered by looking at the VerificationSet
- We can know if the cover was taken
- We can know if it was paid
There is no info at all on whether a guest has requested the execution of the cover
There is no info at all on the lifecycle of the execution of the cover
# 2024-02-23
## Iplicit Workshop
- How do we create Sales Invoices
- I need an API Key
- Veerinder
- Who do I call if I need help?
- Veerinder
- How do we add lines?
- Should we create in draft, or should it be submitted?
- We create in Draft, finance reviews
- How do we want to bulk send? Do we want to bulk send
- Not yet
- I would like to have this running in Sandbox mid-march
- How do we get exchange rates
---
We can import documents through an Excel template
## Invoicing Planning for February processing cycle
- How did it go?
- Plan
- Feature discussion
# 2024-02-26
## Product catch-up
- Finance
- iplicit
- I'm getting API access to the Sandbox to start integrating
- invoicing
- my plans
- Data Platform
- Doing presentations, this week some of you + tech team
- Syncing stripe US data
- Next will depend on cancellation needs
- e-deposit
- PBI in place, no more needs on sight unless you let me know
- Cancellation
- In touch with eng. for data model, conversation with Lawrence around tech debt
- I'll be defining a bit more granularly the needs
- I'm concerned with the payouts side of things
- Collecting data will be hard if it's not managed within the app itself
- Short term, that will make reporting hard
- Long term, that will make pricing hard to adjust to reality
- Interviews
## Interview with Miguel Angel Avalos
- Technical bits
- SQL
- What is your favourite?
- How do you deal with monster queries?
- Tell me the story of the last time you had a slow performant query and what you did with it
- Python
- Git
- Tell me the story of the latest conflict that was a mess
- What branching strategy do you usually go for?
- Visualization tools
- What is your favourite? Tableau, but doesn't care much
- How do you decide whether to do some logic in the modeling layer or in the visual layer?
- dbt
- Data Analyst
- Imagine you start. You have an entire company waiting for you. What's your agenda? Where would you start?
- Meet everyone
- Planner, business impact/value,
- How do you make sure that we don't get lost in an ocean of reports?
- How far do you like to go into data engineering tasks?
- Started ecology, got in love with R
- Didn't really like ecology related jobs and
- Ironhack
- How was the experience?
- How was the merger with Yandex
- "I own all reporting"
- "I like refactoring SQL code" BLING BLING BLING
- "We use a custom git server in Yandex" Lol
## Interview with Alvaro Lluis
- ADE
- 14 years in Banking
- Crisis wasn't temporal
- Left the bank because there were no opportunities to work in Data there
- "I would prefer not working with SAP again"
- ETL with Python + SQL snippets
- Technical bits
- SQL
- What is your favourite?
- favourite Snowflake
- also worked MySQL / SQL Server / Postgres
- How do you deal with monster queries?
-
- Tell me the story of the last time you had a slow performant query and what you did with it
- Python
- SQL
- Git
- Tell me the story of the latest conflict that was a mess
- What branching strategy do you usually go for?
- Visualization tools
- What is your favourite?
- How do you decide whether to do some logic in the modeling layer or in the visual layer?
- Data Analyst
- Imagine you start. You have an entire company waiting for you. What's your agenda? Where would you start?
- Prioritize needs
- How do you make sure that we don't get lost in an ocean of reports?
- How far into the Data Engineer responsibilities do you feel comfortable venturing in?
# 2024-02-27
## Interview with Mario LaTorre
"I enjoyed fundamental knowledge more than engineering."
""
- Technical bits
- SQL
- What is your favourite?
- How do you deal with monster queries?
- Tell me the story of the last time you had a slow performant query and what you did with it
- Python
- Git
- Tell me the story of the latest conflict that was a mess
- What branching strategy do you usually go for?
- Visualization tools
- What is your favourite?
- How do you decide whether to do some logic in the modeling layer or in the visual layer?
- Data Analyst
- Imagine you start. You have an entire company waiting for you. What's your agenda? Where would you start?
- Assess data
- Talk with the Product Managers and assess need
- How do you make sure that we don't get lost in an ocean of reports?
-
- How far into the Data Engineer responsibilities do you feel comfortable venturing in?
- Not infra comfy
- Writing ETLs with Airflow, S3 Data storage
- What is the Yearly Return? 30%
- Very different
- Different domain
- More descriptive
- Notice
- Barcelona office?
## sh-invoicing refund issues #13692
- Refund failed
- Make sure superhog fee is positive not negative
- refund
- Make superhog fee is negative not positive
- That should also make the amount due to host be corrected (it should be absolute smaller than the amount in host currency, not bigger)
## Cancellation reports for AM: meeting with Kayla
- She was not up to date with the needs, she will discuss with Matt and come back to me sometime next week.
## Cancellation reports for Resolutions
- They need a list of guest and booking details + money facts (cover, how much they paid, what percentage)
- Kind of what we have for e-deposit
- Questions
- How will you trigger cancellations
- How will you monitor workflow
- How will you monitor payouts
All the payout process will be managed through Hubspot tickets. The tickets will be generated by guests filling in a form if they want to execute the cover. We agreed with Lou that Lou and Alex need to ensure that the tickets need to have enough data around guest/booking in order to be able to match them with the data present in Superhog in order to perform E2E reporting of the lifecycle of cancellations.
# 2024-02-28
## AD groups and power bi with Ben R
- Ben will look into it with Mike
- Ideally data would have freedom to create groups and put people in them
- Otherwise, at least have freedom to put people in groups
- We would then link people to groups
## OKR Meeting
- I like it
- I love profitability
- Ambitious, I want my analysts
- I don't like/concern
- I hope we can strike a balance in measuring and doing
- Data many times is supporting on other goals, it's a bit hard to relate
- I'm against Speedy
# 2024-03-04
- Finance
- Run is moving forward
- My only
- No advances on other things, like changing culture or changing backend
- Looking forward for interviews, but shouldn't I make more?
- Help Lou A with Resolutions data
- Cancellations: I'm moving nothing
- Data Platform
- Presentations done
- Gonna establish rules of engagement with Robinson
## 2024-03-06
- [X] Finish update
- dbt
- [X] Add dbt deps to deployment guide
- [X] Add dbt seed to run script
- [X] Send interview invites
## Schema changes in Airbyte
- Create table
- Sync
- It works
- Scenarios
- New column gets added
- Column data type gets modified
- Column gets removed
- Column gets renamed
- Table disappears
- Write conclusions and drop them in dirty confluence page
## SQL <> DWH integration
- Schema change management
- General, keep me in the loop
- PR approval in migrations, codeowners implementation?
- Subtle changes are also important (i.e. schema doesn't change but semantics of the field do)
- Job failure ownership
- Data team monitors alerts, but someone needs to be ready on the other side as well
- We might want to monitor it
- Policy: just bother from Ben
- Data modeling
- Proper historical data in source to avoid statefulness in DWH
- Created/Updated/ general strategies for allowing incrementality
- Load
- It will get heavy at some point
- Please let me know if you start seeing issues and we can work together on fixing them
- Actions
- Create some basic docs (Pablo)
- Perhaps doing a session for people who work on SQL? (Pablo)
- Achieve clarity on who is the green flag in engineering (Ben)
- Make any necessary changes regarding code owners in repos (Ben)
# 2024-03-07
## Interview with Pol Marin
- Technical bits
- SQL
- What is your favourite?
- How do you deal with monster queries?
- Tell me the story of the last time you had a slow performant query and what you did with it
- Python
- Git
- Tell me the story of the latest conflict that was a mess
- What branching strategy do you usually go for?
- Visualization tools
- What is your favourite?
- How do you decide whether to do some logic in the modeling layer or in the visual layer?
- Data Analyst
- Imagine you start. You have an entire company waiting for you. What's your agenda? Where would you start?
- Meet stakeholders
- Getting familiar with the product
- Getting familiar with the databases
- Most important and urgent tasks
- How do you make sure that we don't get lost in an ocean of reports?
- Documentation
- How?
- Structure and Organization
-
- How far into the Data Engineer responsibilities do you feel comfortable venturing in?
"I love data and creating analysis"
# 2024-03-12
- [X] Review Lawrence's thingy
- [X] Share issues with Lou
- [X] Align with Lou D
# 2024-03-13
- [X] Fill in timesheet
- [X] Royal Enfield
- [X] Pol feedback here: <https://www.notion.so/native/Pol-Marin-fbe4aa4721d14120819e766390d0dbd2?pvs=0&deepLinkOpenNewTab=true>
- [X] Drop feedback on Colin:
- [X] Document new products as data products in catalogue
- [X] And add link in the DBT exposure yaml
- [X] Build weekly update
- [ ] Think of OKRs and book some time with Ben to go over those
- [ ] I have to discuss this with Ben, because pretty much every OKR is "give role X data Y so he can chase the actual goal"
- [X] Check when should the 1:1 with Ben happen
## Interview with Colin
- Remember to drop feedback here <https://www.notion.so/knowyourguest-superhog/Colin-Ratcliffe-9d311225db5d41febd620b75e5b5dcc1?pvs=4>
- Technical bits
- SQL
- What is your favourite?
- How do you deal with monster queries?
- Tell me the story of the last time you had a slow performant query and what you did with it
- Python
- Git
- Tell me the story of the latest conflict that was a mess
- What branching strategy do you usually go for?
- Visualization tools
- What is your favourite?
- How do you decide whether to do some logic in the modeling layer or in the visual layer?
- Data Analyst
- Imagine you start. You have an entire company waiting for you. What's your agenda? Where would you start?
- Learn what exists
- Understanding data usage across diferent silos
- Put a plan in place
- How do you make sure that we don't get lost in an ocean of reports?
- How far into the Data Engineer responsibilities do you feel comfortable venturing in?
- How do you structure sophisticated ad-hoc requests?
- Data championship
- How did you do that?
- Notice
- Available
- Salary expectations
- 75K-80K
- Location
- Remote
- Next steps
# 2024-03-18
- [X] Add waivers to dashboard
## Booking fess for host
- Bookings:
- Bring over all bookings
- Bring over all price plans
- Join them to enrich each booking with the right booking fee
- PricePlanChargedByType (to know the right date to determine the chargeability of the booking)
- Verification (to know the VerificationStartDate in the cases where it's needed)
- [X] Brought over all tables to DWH
- [X] Create sources and staging models for the new tables
- [X] Wouldn't hurt to compare the fees and waivers against what appears in the old KPI report excel, would it?
- Roughly the same, not bad
- We also need to bring over
- BookingStatus
- UserVerificationStatus
## Issues with Payment model
- All payments should be there
- Hard picking up all movements
- Pending payments
- Manual Payments
# 2024-03-20
## Interview with Yana
- Technical bits
- SQL
- What is your favourite?
Postgres, pgadmin
- How do you deal with monster queries?
- Tell me the story of the last time you had a slow performant query and what you did with it
- Python
- Git
- Tell me the story of the latest conflict that was a mess
- What branching strategy do you usually go for?
- Visualization tools
- What is your favourite?
- How do you decide whether to do some logic in the modeling layer or in the visual layer?
- Quicksight
- Data Analyst
- Imagine you start. You have an entire company waiting for you. What's your agenda? Where would you start?
- How do you make sure that we don't get lost in an ocean of reports?
- How far into the Data Engineer responsibilities do you feel comfortable venturing in?
- First in New York, then to Barcelona (10 years)
- Smart PhD
-
- How was the story of switching from academia to private company
- How was the story of new york to barcelona?
- How was Codeop? Haven't heard from them
## Invoicing debrief for February cycle
- Talk with Clay and discuss bug
- Do the new accounts report together with Jamie
# 2024-03-21
Good stuff:
- Happy with candidates and pace, let's keep it up
- Happy with getting something out with the business overview dashboard
- Happy to see tackling tech debt slowly crawl up the backlogs
- Happy about Suzanna joining
Bad stuff:
- Still overwhelmed by demand and wearing all hats
- Concerns about Joan
Any more feedback?
OKRs?
## Audit Data
- Two steps
- Free trials
- Half stuff from Hubspot
- Export Deals where the Free trial ending in the previous month
- Half is from the report section in dashboard
- Use email to find account (deal id is not always reliable) and look generally
- Check Deal Id is properly informed
- Check that price plan / pay away settings are fine
- User summary for the dashboard
- Accounts created during last ~45 days
- Filter to the ones that have properties (property=Yes)
- Make the ones in a free trial go away (is in free trial = No)
- Make sure Deal ID is informed in Dashboard
- Make sure Price Plans are informed
- Make sure Waiver Payaway details are informed properly
## Interview with Cristina
- Technical bits
- SQL
- What is your favourite?
- How do you deal with monster queries?
- Tell me the story of the last time you had a slow performant query and what you did with it
- Python
- Git
- Tell me the story of the latest conflict that was a mess
- What branching strategy do you usually go for?
- Visualization tools
- What is your favourite?
- How do you decide whether to do some logic in the modeling layer or in the visual layer?
- Data Analyst
- Imagine you start. You have an entire company waiting for you. What's your agenda? Where would you start?
- How do you make sure that we don't get lost in an ocean of reports?
- How far into the Data Engineer responsibilities do you feel comfortable venturing in?
- Starting out in Audit.
- Excel skills and managing a handful of people.
- Decided to pivot towards Data.
- Did Master in Data Science
- Switched to Sanofi
- Service Manager to BI Tool, liasing between business teams and Indian contractors building the tool
- SLA and contracts for service providers
- Too easy, got tired
- Switched to Taurus
- Small company, did a lot of things
- Started out with data entry, turned a manual pipeline into an R pipeline
- Report building for managers
- Build some PBI dashboards, but company dropped it because... it was too expensive?!
- Got bored, now is doing real state valuations. Business is slow
- Can build SQL queries
- Python: she's more biased for R
- PBI, did a bootcamp
- Prioritise finance because it's almost quarters end
- Organize people on how to ask for stuff
- Lacks experience in Engineering side of things
- But likes it
## Intro with Suzannah
Open, current issues:
- Finance willingness to patch any short-coming of other teams manually is a escape valve for poor data quality in other areas
- Finance lacks character to set priorities and draw lines in the sand
- Finance processes are an afterthought in product and operational conversations
- Superhog application is not feature rich towards Finance/Invoicing profiles
Practical issues at hand:
- Invoicing process
- Accounting software choice
- Sunsetting of Acquired
- Lack of exchange rates source of truth
Strategic topics for the long-term
- Complete lack of cost allocation processes, it's impossible to reason on the cost side of the business
- Superhog keeps on piling risk through protection commitments, but no one monitors the at-hand risk
- As a consequence of the previous two, pricing is not optimized at all
- Lack of strategic treasury management and currency management
- Payment processing fees
Content for you:
- Notion pages on invoicing effort we did earlier this year: <https://www.notion.so/knowyourguest-superhog/Start-here-93981184e2154dee9a4800f51d8c6e89?pvs=4>
- Data catalogue: <https://www.notion.so/knowyourguest-superhog/Data-Catalogue-78d91434aa1442cbb6cc13b73c7fb664?pvs=4>
- Home page for Data: <https://www.notion.so/knowyourguest-superhog/Data-Homepage-0ac0a2e52a8940c7ba4f31e5ffcc33e8?pvs=4>
# 2024-03-25
## Interview with Maud Briels
- Technical bits
- SQL
- What is your favourite?
- How do you deal with monster queries?
- Tell me the story of the last time you had a slow performant query and what you did with it
- Python
- Git
- Tell me the story of the latest conflict that was a mess
- What branching strategy do you usually go for?
- Visualization tools
- What is your favourite?
- How do you decide whether to do some logic in the modeling layer or in the visual layer?
- Data Analyst
- Imagine you start. You have an entire company waiting for you. What's your agenda? Where would you start?
- How do you make sure that we don't get lost in an ocean of reports?
- How far into the Data Engineer responsibilities do you feel comfortable venturing in?
- How
- What would you improve on your current organization?
- How do you like Barcelona vs back home?
- "When we started we were a small team of 7"
- "I hate the word single-source-of-truth"
- Why the change from Fivetran to Airbyte?
- Money
- Why the change from Tableau to Superset?
- Money
- Much easier
- Ingest
- Used to do regular python
- Fivetran
- Airbyte
- dbt for modeling
- Visuals
- Tableau
- Superset
- Airflow
- Redshift
- Really dove in the internals
- I agree with foundations vs delivering something now
- stack
- How do you feel about not running a time?
- I'm all for transparency
- "You set an hour of Udemy"
- Notice period 1 month
- 72 currently + happy to stay there
# 2024-03-27
## Data Audit
- [X] Warn Alex and Matt
- [X] Export Deals with free trial ending and find related accounts
- [X] Export Deals with free trial currently active and find related accounts
- [ ] Export the accounts created during the past month (and that are not currently in a free trial)
- How the hell do I find the right account type?
- I'm going to get one of the accounts I've exported from Hubspot and check what role do they have
- Also, the way to know when the account was created is to use the `JoinDate` field
- Case solved, it's role Platform
- [ ] For all acounts, generate an export with:
- Informed Price Plan
- Informed Pay Away settings
- Prettify and share
---
Instructions
- In Hubspot:
- Export deals ending free trial this month
- Go to deals section
- Open all deals
- Add Free Trial End Date to table
- Add a filter to export only Deals with the free trial ending during this month
- Export to file
- Export deals created this month with active free trial
- Go to deals section
- Open all deals
- Add Free Trial End Date to table
- Add a filter to export only accounts created this month
- Add a filter to export only Deals with the free trial ending beyond this month
- Export to file
- In Core:
- Run the couple of queries I made to fetch details for the hubspot accounts and for new accounts made this month
- I've decided to pick Host & Stay. Exact filename is `2402 Report 11254198823_Host & StayNorth Yorkshire_547b.xlsx`.
- I've built an excel that holds all payment references: the ones that appear in the account report, and the ones that appear in the Stripe export. Then, the same excel shows a table that shows in which source does each payment reference appear in.
# 2024-04-03
## Interview with Evripidis
- Technical bits
- SQL
- What is your favourite?
- How do you deal with monster queries?
- Tell me the story of the last time you had a slow performant query and what you did with it
- Python
- Git
- Tell me the story of the latest conflict that was a mess
- What branching strategy do you usually go for?
- Visualization tools
- What is your favourite?
- How do you decide whether to do some logic in the modeling layer or in the visual layer?
- Data Analyst
- Imagine you start. You have an entire company waiting for you. What's your agenda? Where would you start?
- How do you make sure that we don't get lost in an ocean of reports?
- How far into the Data Engineer responsibilities do you feel comfortable venturing in?
- Started at Siemens
- Oracle
- What kind of Performance
- TIBCO
- Research
- Did you like it?
- Google Data Studio
-
## KYG Data Model definition
- Prices are per customer or system wide
- Can users be multicurrency? Can listings be multicurrency?
- Conflict between active offerings/services vs bookings independently getting services attached
- IsActive is reflecting too much stuff
- Is the listing bookable
- Is the listing integrated with PMS
- Is the listing susceptible of being serviced
## Account Manager needs with Kayla
- Typical host questions
- Export all of my bookings
- Catch email for users
- Beyond monthly
- Took waiver or took deposit?
- Export listings
- Exporting resolutions
# 2024-04-05
## Xero intro with Jamie
- Amanda needs to give me access she is admin
- We need to change the Damage Waver to damage waiver
- We have multiple xero instances, but the UK one is the one that runs all the invoices
- Always separate documents for invoices and creditnotes
- Upload files and attach them through UI
- Sending: after you build the invoice, click email option, pulls the template and fills it, we need to mark attach extra to send attached files (also there's another tick box that needs to be marked at the time of sending a file)
- Lifecycle is Draft -> Approved -> Sent -> Paid
- Then we create a credit note, associated to the sales invoice
- naming should be looking like `CN-<code for invoice>`
- By default, credit note includes line items of sales invoices. Typically, you delete this and make a single line item with damage waiver
- Amount due from the excel report is tax inclusive
- We have line items
- Split by fee type
- Split by currency
- Quantity is 1
- Amounts are tax exclusive. Xero will compute them, should match what appears in excel
- Contact companies have Currency settings, but not all of THEM
- Issue date should be final date of the month
- Due date should be 14 natural days after today
- Export currencies? Doesn't seem so.
# 2024-04-08
- [X] Business Overview: Booking Fees
- [X] Updates on Notion
- [X] Clean product board
- [X] Ask Amanda for access to Xero and API
- [X] XE briefing proposal
- [X] Keep Data Subscription alive
- [X] Interview with Maud
- [X] Data audit
- [X] Update Pricing Excel
- [X] Generate debugging examples for Stripe missing payments
## Check-in cover
- Recycling cancellation cover
- We pay for alternative accomodation
- For 3 days so we can save your ass
- Pricing around 1-10 bucks
- Guest pays for the emergency at first, and then we reimburse them
- Do we have SLAs on paying back?
- Hosts must pay-in
- Will it be a single payment or mixed with Waiver?
- Pricing history!!!
- Reporting of which bookings/guests have paid for it
- For Resolutions
- For business measurement
## Product weekly
- Finance
- Exploring Xero invoicing
- Going to raise the Currency rates project ASAP
- e-deposit
- need to make some tiny changes in Ana's dashboard
- Joan/Check-in cover
- I'll discuss with Joan and Lawrence on the data model and reporting products on the Check-in cover
- Business Overview
- Dropped bookings fees
- Next: Verification fees
- When do you want to open it up?
- Hiring
- Time to go for round 3 and find a second analyst
- Little things
- Stripe payments: Lou, that's still sitting with Ben
- Migration: it will impact me as well
# 2024-04-09
- [X] Solve export issue for Cali
# 2024-04-13
- [X] Get Xero API Key
- [X] Review candidates
- [X] Reject Maud and Evripidis
- [X] Ana's little changes
- [X] Ray's info
## Finance Screening API
- Volume invoicing
- Should they be in hubspot?
- Where do we store a master list of screening APIs customers, and their currencies?
- What's the flexibility on using screening vs watchlist vs bundle?
- Agree on
- Monthly invoicing cycle
- Pricing changes policy
- Per customer pricing
- Hubspot
- Line Items specific for this product
- Validation from my side with Ray
- Gocardless
# 2024-04-16
## Product Weekly
- e-deposit
- Done
- Ana
- I still owe you the accounts that are being invoiced for both Bookings and Verification
- Joan/Check-in cover
- I'll discuss with Joan and Lawrence on the data model and reporting products on the Check-in cover
- Hiring
- Interviews ongoing with more candidates
- Oriol is coming on 9th May
- Finance
- Unexpected turn of events: Xero invoices are now in the DWH
- Next will be creating invoices
- Currency API
- Business Overview
- I think it makes sense to switch to account view of bookings and listing fees
- Lou: we will use the app as the source of truth
- Does that mean there's a database that will start properly storing resolution processes?
## KYG Charging
- How to charge listings fees for KYG Users?
- How often do we review the tier of a customer?
- What is the business story of the listing fee?
-
- What is the business narrative?
- What are the effects on smaller hosts? Is this gonna churn low volume people?
"The system looks at..."
# 2024-04-16
## Interview with Timur
- Technical bits
- SQL
- What is your favourite?
- How do you deal with monster queries?
- Tell me the story of the last time you had a slow performant query and what you did with it
- Python
- Git
- Tell me the story of the latest conflict that was a mess
- What branching strategy do you usually go for?
- Visualization tools
- What is your favourite?
- How do you decide whether to do some logic in the modeling layer or in the visual layer?
- Data Analyst
- Imagine you start. You have an entire company waiting for you. What's your agenda? Where would you start?
- How do you make sure that we don't get lost in an ocean of reports?
- How far into the Data Engineer responsibilities do you feel comfortable venturing in?
- VK
- Data Analyst
- SQL
- PostgreSQL
- Clickhouse
- Visuals
- Metabase
- Marketing / Pricing definition: responsbility on
- Visa sponsorship
- More preferable in Barcelona
# 2024-04-17
## Interview with Javier
- Technical bits
- SQL
- What is your favourite?
- How do you deal with monster queries?
- Tell me the story of the last time you had a slow performant query and what you did with it
- Python
- Git
- Tell me the story of the latest conflict that was a mess
- What branching strategy do you usually go for?
- Visualization tools
- What is your favourite?
- How do you decide whether to do some logic in the modeling layer or in the visual layer?
- Data Analyst
- Imagine you start. You have an entire company waiting for you. What's your agenda? Where would you start?
- How do you make sure that we don't get lost in an ocean of reports?
- How far into the Data Engineer responsibilities do you feel comfortable venturing in?
- Which internal customers are you serving in Pronovias?
- Supply chain planning
- Marketing
- Sales
- Lacoste
- IT business analyst, more managerial
- Not happy with it. Misses the data part of it
- SQL
- Not much of a strong explanation
- Python
- No experience in Python at the work
- Visualization
- Qlik
- A lot of Excel
- PBI
- Looker
- Tableau
-
## Interview with Joaquin
- Technical bits
- SQL
- What is your favourite?
- How do you deal with monster queries?
- Tell me the story of the last time you had a slow performant query and what you did with it
- Python
- Git
- Tell me the story of the latest conflict that was a mess
- What branching strategy do you usually go for?
- Visualization tools
- What is your favourite?
- How do you decide whether to do some logic in the modeling layer or in the visual layer?
- Data Analyst
- Imagine you start. You have an entire company waiting for you. What's your agenda? Where would you start?
- How do you make sure that we don't get lost in an ocean of reports?
- How far into the Data Engineer responsibilities do you feel comfortable venturing in?
Module planner
- Sounds like a very scientifical job that can be ran in very unscientifical ways
Now
- 20-30H
July
- Fully available
# 2024-04-18
## Host currency
- One host, one currency?
- Changeable?
- Onboarding
- Compulsory
- How to fill the existing hosts
## Interview with Erminia
- Technical bits
- SQL
- What is your favourite?
- How do you deal with monster queries?
- Tell me the story of the last time you had a slow performant query and what you did with it
- Python
- Git
- Tell me the story of the latest conflict that was a mess
- What branching strategy do you usually go for?
- Visualization tools
- What is your favourite?
- How do you decide whether to do some logic in the modeling layer or in the visual layer?
- Data Analyst
- Imagine you start. You have an entire company waiting for you. What's your agenda? Where would you start?
- How do you make sure that we don't get lost in an ocean of reports?
- How far into the Data Engineer responsibilities do you feel comfortable venturing in?
- Focus on SAP BI tools
- What is the worst project and best project, why?
- Logic in PBI vs logic in the data mart
Issues with self service
# 2024-04-19
- [X] Send rejection to Javier
- [X] Schedule with Petronella
- [X] Schedule with Gaudenzio
- [X] Timesheet
- [X] Get my Whiteboard
- [X] Make export with all accounts that had both Verification and Booking fees at the same time during the past months
## Stripe debugging
I've looked at it together with Ben and his hunch is that all the payments are actually in the backend. He could show to me how several of the examples I have brought in the debugging data could actually be found just fine in the SQL Server.
We also spotted together that the current Python code is using a SQL Server view called `VerificationPaymentTest` instead of its more seriously-looking sibling `VerificationPayment`. We suspect this might be the issue. I stored the code to both views below.
We agreed on the following plan:
- I'll try to simply switch the view used in the Python code and see if that does the trick in making all records match.
- If not, I'll have to debug further.
- In any of the cases, I'll refactor the query present in the Python code to not depend on any of the views of the SQL server and for it to be as simple as possible.
# 2024-04-22
- [X] Ask Erminia for CI re-do
- [X] Make export for Jamie
- [X] Reply to Javier with feedback
- [X] Beth's request
- [X] Break by month
- [X] Lou's request
- [X] Update pricing calculator for Ben
## Product Weekly
- Ben
- Getting the pricing calculator in place
- Ana
- Sent the comfy-workers-like accounts
- Joan/Check-in cover
- I'll discuss with Joan and Lawrence on the data model and reporting products on the Check-in cover
- Hiring
- Interviews ongoing with more candidates, I have 3 left this week
- Ben you can start shooting
- Should we involve Will in any way or just push forward?
- Finance
- Working on reporting through Xero data
- Moving forward with Currencies, looking good
- Business Overview
- I think it makes sense to switch to accounting view of bookings and listing fees
- Lou: we will use the app as the source of truth
- Does that mean there's a database that will start properly storing resolution processes?
# 2024-04-24
## Interview with Petronella
- Almost done with PhD
- 20 months at Morgan Stanley
- Got bored of current responsibilties
- [X] Update Data News
- [X] OOF for 10th of May + heads-up with Ben
- [X] Update candidates pipeline
- [X] Provide weekly update
## Jamie's issue with 3 deals
Troublesome deals:
- 13150958399
- 16531969388
- 3041623202
For those, Jamie needs:
- a list of active listings
- a list of bookings where the verification started in march
I ended up using the following code:
```sql
/*
Deals
- 13150958399
- 16531969388
- 3041623202
*/
-- Helpers
SELECT TOP(1000) *
FROM live.dbo."User"
SELECT TOP(1000) *
FROM live.dbo.AccommodationToOwner ato
SELECT TOP(1000) *
FROM live.dbo.AccommodationTouser
-- List of active listings
SELECT a.*, u.*
FROM live.dbo.Accommodation a
LEFT JOIN live.dbo.AccommodationToUser atu ON a.AccommodationId = atu.AccommodationId
LEFT JOIN live.dbo."User" u ON atu.SuperhogUserId = u.Id
WHERE a.IsActive = 1
AND u.DealId IN ('13150958399', '16531969388', '3041623202')
-- List of bookings
SELECT DISTINCT
b.BookingId as BookingId,
guest.FirstName As GuestFirstName,
guest.LastName As GuestLastName,
guest.Email As GuestEmail,
CASE WHEN guest.joindate is null THEN null ELSE FORMAT (guest.joindate, 'dd-MM-yyyy') END As GuestJoinDate,
t.Number As GuestTel,
FORMAT (s.UpdatedDate , 'dd-MM-yyyy')As 'VerifiedDate',
case
when s.UserVerificationStatusId = 1 then 'Started'
when s.UserVerificationStatusId = 2 then 'Verified'
when s.UserVerificationStatusId = 3 then 'Revoked'
else 'Unverified'
end as 'VerifiedStatus',
bs.Fullname As BookingState,
a.AccommodationId,
a.friendlyName,
a.AddressLine1,
a.AddressLine2,
a.Town,
a.Postcode,
c.[Name] As Country,
FORMAT (b.CheckIn, 'dd-MM-yyyy') As CheckIn,
FORMAT (b.CheckOut, 'dd-MM-yyyy') As CheckOut,
DATEDIFF(day, b.CheckIn, b.CheckOut) AS 'Days',
host.FirstName As HostFirstName,
host.LastName As HostLastName,
host.Email As HostEmail,
FORMAT (Coalesce(b.CreatedDate, b.createddate), 'dd-MM-yyyy') As BookingCreatedDate
FROM Booking b
LEFT JOIN [User] guest On b.GuestUserId = guest.Id
LEFT JOIN SuperhogUser s On s.SuperhogUserId = guest.Id
LEFT JOIN Telephone t ON t.ApplicationUserId = guest.Id
LEFT JOIN Membership m ON m.SuperhogMembershipId = guest.Id
LEFT JOIN Accommodation a On a.AccommodationId = b.AccommodationId
LEFT JOIN Country c ON a.CountryId = c.Id
LEFT JOIN [User] Host ON host.id = b.CreatedByUserId
LEFT JOIN PricePlanToUser pu on pu.SuperhogUserId = host.id
LEFT JOIN BookingState bs ON bs.Id = b.BookingStateId
LEFT JOIN VerificationRequest vr ON vr.Id = b.VerificationRequestId
WHERE
host.DealId IN ('13150958399', '16531969388', '3041623202')
- [X] Check if something has happened to the Listings views
## Interview with Paula
- She wants to relocate to Barcelona, doesn't like Berlin no more.
## Interview with Gaudenzio
- How do you think your experience in Superhog would be different than your consulting?
- Why London?
- Best and worst projects in your carreer
- He wants to jump into managerial positions
- Data Analyst just, is that good enough
# 2024-04-29
- [X] Fix Payments report
- Payments
- Bookings
- Listings
## Product weekly
- PBI outage, working to prevent it in the future
-
- Working on Joan's
- Emergency help with pricing framework
- Will run finance exports this week
- No advances on investor's reporting
# 2024-04-30
## Meeting with Querio
- Javier
- Lou wants to survive adhocs
- What's the technical pattern?
- What's the typical deployment plan?
- What's the cost?
- Concerns
- What if it doesn't deliver?
- What if it answers stupidly?
- Nuanced, dies with complexity
- Most of the time is invested in preparing the "context layer" in any case.
- What does building the context layer feel like?
- What if this doesn't save that much time?
- How do you manage access control?
- Security
- Volume
They've never connected to CosmosDB
"Lou A. how about we put this in front of our own customers so they can ask questions on their own data" -> Careful here. This is more dangerous than a drunk monkey with an AK-47.
## Pricing Comparison Framework
- We have a new pricing structure NewP, different from our old pricing structure OldP.
- We want to understand how revenue of existing clients might change by switching from OldP to NewP.
- Approach
- We will grab data from Q1 from a selection of hosts in terms of bookings, listings, etc.
- We will simulate what we would have charged with NewP and compare it with what we charged under OldP.
- We will potentially iterate with new crazy pricing structures beyond NewP (NewP1, NewP2, ... and so on and so forth until we hit the spot or go mad along the process).
- We only take into account host revenue. Guest revenue behaviour under price changes is much more volatile.
- Within host revenue, the interesting questions to ask and bits to compare are:
- Would we make more or less money? - Change in Total revenue
- How would booking and listing fees contribute to that? - Change in Booking Fees and Listing Fees
- How does the distribution of fees across hosts change? - Change in Mean and Stdev of hosts revenue
- Are there any hosts that are dramatically advantaged/disadvantaged due to NewP? - Max and min % change in revenue across all hosts
- In more detail:
- For each host
- Fetch number of bookings and number of active listings for Q1.
- Obtain their would-be new prices under NewP.
- Fill in the template with both of the previous, and the new fees will be calculated.
- Once that's done
- Check the differences in the results area
- Sit down, reflect, and iterate
- If you think we should try a different price plan... start from scratch and repeat the whole thing, possibly recycling the host sample and the real data on what they did during Q1.
# 2024-05-02
- [X] KYG Vers for Emyr
- [X] Emergency framework for Pricing discussions
# 2024-05-03
Today I was trying to reduce the size the the Docker VM/WSL takes up in my disk cause it was growing like crazy. Windirstat showed that it was taking 260Gb.
I followed the advice of the top answer in this stackoverflow question (<https://stackoverflow.com/questions/70946140/docker-desktop-wsl-ext4-vhdx-too-large>) and it shrinked to 60gb.
Basically, use diskpart to compact the file.
# 2024-05-06
## Product Weekly
- CheckIn Hero Reporting
- Emergency help with pricing framework
- No news on this
- Will run finance exports this week
- No advances on investor's reporting
# 2024-05-07
# 2024-05-09
## Month end financial processes
Issues:
- e-deposit
- unreliable
- manual
- They plan on using the PBI
- We need to align pricing with Finance because there are gaps on the weird rules
- new pricing???
- disputes
# 2024-05-13
## Check-in hero reporting
### Build dirty queries to have the draft on how to get everything
- Stuff that I need
- Count how many verificationrequests included Checkin Hero
- Count how many payments for checkinhero have happened
- Sum the total value of checkinhero payments
- Have guest details for users that have paid the cover. Specifically:
- Payment timestamp
- Guest Id
- Guest name
- Guest email
- Currency
- Amount
- Limit
Here's my first shot at it. I'm waiting news from Ben R and Lawrence on the correctness of these:
```sql
-- Goal: get the guest journeys (VRs) that offered the check-in cover,
-- regardless of whether the user purchased it or not.
SELECT DISTINCT vr.Id
FROM live.dbo.VerificationRequest vr
LEFT JOIN live.dbo.VerificationSetToVerificationType vstvt ON vr.VerificationSetId = vstvt.VerificationSetId
WHERE vstvt.VerificationTypeId = 15 -- 15 is check-in cover
-- AND condition to check that the journey is finished
-- Goal: get the guest journeys (VRs) where guests have successfully paid for the
-- check-in cover.
SELECT DISTINCT vr.Id
FROM live.dbo.Payment p
LEFT JOIN live.dbo.VerificationToPayment vtp ON p.PaymentId = vtp.PaymentId
LEFT JOIN live.dbo.Verification v ON vtp.VerificationId = v.Id
LEFT JOIN live.dbo.VerificationRequest vr ON v.VerificationRequestId = vr.Id
WHERE vtp.VerificationPaymentTypeId = 5 -- 5 is check-in cover
AND p.PaymentStatusId IN (3,4) -- 3 is Paid, 4 is Refunded
-- Goal: get the finished guest journeys only
WITH unfinished_vrs as (
SELECT DISTINCT v.VerificationRequestId
FROM live.dbo.Verification v
WHERE v.VerificationStatusId NOT IN (1,2,3) -- Confirmed, Revoked and Failed
)
SELECT Id
FROM live.dbo.VerificationRequest vr
LEFT JOIN unfinished_vrs uvr ON vr.Id = uvr.VerificationRequestId
WHERE uvr.VerificationRequestId IS NULL
```
Lawrence thinks that the first two ones are ok, although he highlighted that the hardcoded values might be different in environments different than production.
As for the topic on how to identify finished verifications, Ben confirmed the third query won't make it.
Instead, he pointed me to the SQL Server function named `GetVerificationProgress`. The source code of the most recent version is here: <https://guardhog.visualstudio.com/Superhog/_git/superhog-mono-app?path=/Guardhog.Data/Functions/GetVerificationProgress/202305021429044_GetVerificationProgress.cs>
I'm gonna comment it all to make it understandable for me so I can try to replicate it in the DWH:
```sql
/*
Possible return values:
- Not Started: this happens if there is not guest user still or no verification set related to the booking
- Action Required: returned if any of the verifications
*/
CREATE OR ALTER FUNCTION [dbo].[GetVerificationProgress](@bookingId int, @verificationSetId int)
RETURNS VARCHAR(100)
AS
BEGIN
IF(NULLIF(@verificationSetId, 0) IS NULL)
BEGIN
RETURN 'Not Started'
END
DECLARE @superhogUserId varchar(200) = (SELECT GuestUserId FROM Booking WHERE BookingId = @bookingId)
-- PPP: This subquery makes a list of the expected verification TYPES
INSERT INTO @expectedVerifications
SELECT
[Name]
FROM
VerificationSetToVerificationType vsvt
INNER JOIN
VerificationType vt ON vsvt.VerificationTypeId = vt.Id
WHERE
VerificationSetId IN (SELECT Id FROM VerificationSet WHERE Id = @verificationSetId)
-- PPP: This subquery will get the state of the PaymentValidation and Contract verifications for the related VR and booking.
INSERT INTO @bookingOnlyVerifications
SELECT
v.Id, v.[Name], v.VerificationStatusId
FROM
Verification v
INNER JOIN
VerificationRequest vr ON v.VerificationRequestId = vr.Id
INNER JOIN
Booking b ON b.VerificationRequestId = vr.Id
WHERE
v.SuperhogUserId = @superhogUserId AND b.BookingId = @bookingId AND [Name] IN ('PaymentValidation', 'Contract')
-- PPP: This subquery pretty much just takes the one above and keeps the records if (1) the types were expected and (2) the verificataions are not in status 1-Confirmed
FROM @bookingOnlyVerifications WHERE [Name] IN (SELECT [Name] FROM @expectedVerifications) AND VerificationStatusId <> 1
-- PPP: This subquery will get the state of any verification that:
-- - are not PaymentValidation or Contract
-- - that are related to this user
-- - that are of the Types included in the VR
-- - that are not in status 1-Confirmed
-- these are the ones from this guest that are not part of this vr
INSERT INTO @notFinishedOrStartedVerifications
SELECT
Id, v.[Name], v.VerificationStatusId
FROM
Verification v
WHERE
SuperhogUserId = @superhogUserId AND [Name] NOT IN ('PaymentValidation', 'Contract')
AND
[Name] IN (SELECT [Name] FROM @expectedVerifications)
AND
VerificationStatusId <> 1
-- PPP: this subquery gets verifications that have simply not been started and don't even exist
-- items not started
INSERT INTO @notFinishedOrStartedVerifications
SELECT
NULL, [Name], NULL
FROM
@expectedVerifications
WHERE
[Name] NOT IN
(
SELECT v.[Name] FROM Verification v WHERE SuperhogUserId = @superhogUserId AND [Name] NOT IN ('PaymentValidation', 'Contract')
UNION
SELECT [Name] FROM @bookingOnlyVerifications WHERE [Name] IN ('PaymentValidation', 'Contract')
)
-- PPP: if any of the previous trouble-searching queries found any records, it means that some of the required verifications
-- are either not finished or not even started at all.
IF EXISTS(SELECT * FROM @notFinishedOrStartedVerifications)
BEGIN
RETURN 'Action Required'
END
-- PPP: if we reach this point, this means that all the required Verification Types exist and are in status 1-Confirmed.
-- Thus, the Verification Request is complete.
RETURN 'Complete'
END
GO
```
Okay, I think I got it.
I'm going to try 10 different VR ids and see if results match.
All results match. Works like a charm.
### Add required tables from Core to DWH
- This is the list of tables that I think are required
- VerificationRequest
- VerificationSetToVerificationType
- VerificationToPayment
- Verification
- Payment
- All of them are already in the DWH, so there's no need to add any more tables to the Airbyte sync
### Build dbt models
- Staging
- We are missing VerificationSetToVerificationType
- Intermediate
- I'm probably considering enriching the VerificationRequest table with a couple of flag fields
- Reporting
### Deployment
- schedule dbt models
- create workspace and
### Water-in-plumbing changes
- Adjust naming of fields in dbt models
- Visual and page level filters
- Sanity checks
### Final roll-out
- Training for product profiles
- Training for resolutions agents
# 2024-05-15
## Monthly 1:1 one to one with Ben C
Stuff I'm happy with:
- I'm happy with Oriol
- I'm excited about Joaquin
- I'm proud that we pulled out some proper vision for CheckIn Cover
- I'm happy about how the business overview is evolving
- I'm happy with the different proactively PMs validating data stuff with us
Stuff that concerns me/I'm not happy about:
- J
- Continuous attempt at monopolizing
- Emotional reactions to work situations
- Comments on other colleagues, doubting his own squad
- Key action: protect Oriol
- Tech debt
- CheckIn Cover situation
- Old bugs notified months ago are not addressed
- API squad situation
- Certain expectations like the ones we got from Matt today, general feeling about some colleagues being a bit disappointed of the data progress so far
# 2024-05-29
## APIs Revenue Reporting
## Booking and Listing fees through Xero
- Make a dashboard with how much we invoice for booking and listing fees
- Pretty much just two linecharts
- Data coming from Xero
Open questions:
- Are booking fees and verification fees separated in Xero, or everything is crammed together?
- Are there invoices with booking fees or verification fees that are associated with contacts that don't have a Dealid?
Checkpoint:
- I need to rename the report on PBI Service and drop the new dbt models in production to deliver the listing and verification fees.
- I need to clarify with Jamie why is there such a huge booking fee amount without deal id.
- I need to clarify with Jamie how we can have such a massive gap in some months across SH/Xero booking fee totals. July 2023 is the worst month.
- Why not just do it in DWH? If you want to do that effort, I could bring it up the backlog
- We share concerns.
- We decided to run some due diligence on alternatives in the market
- I did some reading on <https://roundup.getdbt.com/p/semantic-layer-as-the-data-interface>, which is the seminal article by dbt that brought this idea to live. The following competitors to querio are mentioned.
-<https://www.delphihq.com/> (now property of cube)
-<https://www.getdot.ai/pricing>
-<https://www.seek.ai/>
## KPI Reporting Needs with Suzannah
- Headcount manual from Sage
- Bookings and Listings
- Billable bookings
- Currently done with Jamie
- Can be automated by replicating the logic of sh-invoicing
- Listings (top of list priority wise)
- Live: booking created on the month
- Active: checkout in the last six months
- Total: bookings created in the last 12 month
End of next week we have revenue figures and the quick listings KPIs export.
I keep working on revenue, Uri works on the listings quick export.
- I'm not happy with pressure on invoicing data gaps when this gets solved when the issues have been sittign on that backlog
## Leo's survey
Leo send me a brief but dense survey, noting here my answers because I think it's an important reflection.
- What are the top three challenges you face in your role that you'd like the wider business to be aware of?
- Cattering to all the branches of Superhog is hard. First, because we need the Data Team needs to hold knowledge in all services and functional areas, as well as understand a lot of processes of different teams, and have deep technical knowledge on our own systems. Second, because we can't be working on all possible lines of work at the same time, so regularly taking in new demand, refining it, and prioritising it is tough. Sometimes I simply need to completely park big areas of work and ask people to just wait because if I try to listen to every requirement, I won't have time to actually get my hands dirty executing stuff. Thankfully we are improving on this a lot.
- The speed at which we plan and deliver products and business changes makes it very hard for me to stay aligned on all the ongoing streams of work. I think having everyone at capacity long-term might impact morale and overall quality of what we do. I feel many short-cuts are taken in different areas which might bite us back soon. Sometimes I wonder if just a tad less velocity could actually make us deliver faster by making gears roll smooth instead of grind.
- I think we are playing a dangerous game in not prioritising more the technical quality of Superhog technologies. I can see first hand how the implementations of certain features and products has been of very low quality due to developers being rushed. I'm afraid this is going to impact delivery speed very soon at a rate that can hurt a lot the company, even be deadly. It certainly makes my life more difficult and makes some of my deliveries order of magnitude more slow/require more effort, since I build things on top of these patchy implementations.
- What do you think are the top three priorities for the business?
- Achieve profitable growth.
- Leverage our current position in the market to experiment with new services towards hosts and guests. With the hope of finding new great sources of revenue that we can deliver while leveraging our existing infrastructure, unlocking economies of scale.
- Improve our systems and processes to make sure that we can scale our current services efficiently, being able to grow our customerbase and business volume exponentially without our cost structure chasing it. Improve quality so onboarding, sales, invoicing, customer service, etc. work like a clock.
# 2024-06-04
## Ticket 17132 - BUG: Missing deals in May sh-invoicing export
- Right now, the purchase details table is showing the following guest details:
- first name
- last name
- guest email
- guest phone number
- These fields are all coming from the verification request itself.
- Instead, we should grab the same data from the user details. The right fields from `int_core__unified_user` would be:
-`first_name`
-`last_name`
-`email`
-`phone_number` (currently not in `int_core__unified_user`, but present in `stg_core__user`. We just need to promote it.)
So, complete changes would include:
- Add field `phone_number`from `stg_core__user` in `int_core__unified_user`.
- Add `int_core__unified_user` as a joined table in `int_core__vr_check_in_cover`
- Replace the old VR guest fields for the data coming from `int_core__unified_user`.
- Changes should propagate nicely downstream into both the reporting model `core__vr_check_in_cover` and the PBI report itself without any changes required in them.
### Impact on existing reporting
- All changes made to `int_core__vr_check_in_cover` and downstream models will only affect the checkinhero reporting.
- The inclusion of the `phone_number` field in `int_core__unified_user` is not expected to have any impact on any of its dependants.
### Execution plan
- Set up local environment with all `sync_` dependencies of `int_core__unified_user`and `core__vr_check_in_cover`
- I need tables
- Currency
- User
- SuperhogUser
- VerificationRequest
- PricePlanChargedByType
- PricePlanToUser
- Booking
- BookingState
- PaymentValidationSetToCurrency
- Payment
- PaymentStatus
- VerificationPaymentType
- VerificationToPayment
- Verification
- VerificationType
- VerificationSetToVerificationType
- Create a branch to modify `int_core__unified_user` and implement
- Create a branch to modify `int_core__vr_check_in_cover` and implement
- Check on locally executed report before merging into production
# 2024-06-17
## Prep with Suzannah for Townhall
- We need to show Revenue figures
- Guest Payments and more lively information
- Add screenshots with data content
- Listings
-
# 1:1 with Uri
- A few words from my side
- 0 bullshit
- Mercenary, professional sport team
- How have you been so far
- What did you like what did you not
- My impression on you
- What are your plans, how can we fit with them
- Team organization and managing Joaquin
# 2024-06-27
- I've purchased the SaltoKS thingie to enter into Norrsken with a physical key.
- I paid it with my bankinter credit card, and the deposit was of 18.15.
- [X] Get salto key from Norrsken
- [X] Document how much I paid so I can get it back
- [X] Plan 1:1 with Ben before Holidays
- [X] Send heads-up to Finance team
- [X] Plan invoicing topics
- [X] Send export to Suzannah
# 2024-07-05
## 1:1 with Ben Cotte
Good
- Pace is picking up, infrastructure is paying off
- We have a good way ahead of us leveraging current resources before we need a lot more
- Excited to step beyond reporting
- Analysing, not reporting
- Data Products
Bad
- Quality issues all the time
- Tension between PMs and Devs
- Invoicing saga kind of never ends, risks of big bang change
Other
- DE, pregnant
- Make job description/business case for Ben Cotte
# 2024-07-29
## 1:1 with Ben Cotte
- bonus got inflated
- up to date payment of the bonus since I began
- from now on we get the quarterly version
- Humphrey decided to slash the actual bonus amounts by 50%, but Ben C. negotiated to keep the amount at 85% for me because they are very happy with the performance I've been having.
- back from holidays
- all good, guys did great, we just move forward
- hiring DE
- can i just lead it
- selection process requirements
- decision on seniority
- Ben C. will get green light from Humphrey and then we get going
- prepare notion page on CIH revenue share
- per host
- variable %
- backend settings
- finance settings
- offer Ana an alignment session
## Priorities check with Suzannah and Ben
-
- Guest Payments tax
- Finance needs to provide guidance on how do guest services taxes get calculated
- Then we decided where to visualize tax inclusive/tax exclusive
# Now
## Grand Welcome invoicing
- Franchise
- 80 small accounts in Superhog
- Billed as one Deal Idc
## Cali's needs
- How much does each Deal Id bring
-
## Now
- [X] Update news
- [ ] Joan's request on check-in cover dashboard for resolutions
- [X] Create new workspace and publish report
- [X] Requests permissions for the different users
- [X] Do documentation for dbt models + exposure
- [X] Exposures
- [X] Final Model
- [X] Final model int mirror
- [X] check in cover pricing
- [X] Update Data Catalog
- [X] Deploy dbt models to production
- [X] Monitor dbt runs
- [X] Apply post-release adjustments
- validate hardcodes on names and ids
-`Check-in Cover`
- Payment Type 15
- check that prices are properly updated
- add proper filters to dashboards
- [X] Perform sanity checks on the shown data after we get a bit of records coming in
- [X] Run trainings on the dashboards
- [ ] Deal with verification completeness trouble in DWH/Core
