diff --git a/.gitignore b/.gitignore deleted file mode 100644 index 8f24f11..0000000 --- a/.gitignore +++ /dev/null @@ -1,14 +0,0 @@ -# Deployment configuration (contains sensitive server details) -deploy.config - -# OS files -.DS_Store -Thumbs.db - -# Editor files -.vscode/ -.idea/ -*.swp -*.swo -*~ - diff --git a/README.md b/README.md index 32ee0c7..b448f02 100644 --- a/README.md +++ b/README.md @@ -8,6 +8,4 @@ The `index.html` is ready in the `public` folder. ## How to deploy -1. Copy `deploy.config.example` to `deploy.config` -2. Fill in your server details in `deploy.config` (host, user, remote path) -3. Run `./deploy.sh` to sync the `public` folder to your remote webserver +Somehow get the `public` folder behind a webserver manually and sort out DNS. diff --git a/deploy.config.example b/deploy.config.example deleted file mode 100644 index 1c84f53..0000000 --- a/deploy.config.example +++ /dev/null @@ -1,21 +0,0 @@ -# Deployment Configuration -# Copy this file to deploy.config and fill in your server details -# deploy.config is gitignored to keep your credentials safe - -# Remote server hostname or IP address -REMOTE_HOST="example.com" - -# SSH username for the remote server -REMOTE_USER="username" - -# Remote path where the website should be deployed -# This should be the directory served by your webserver (e.g., /var/www/html, /home/username/public_html) -REMOTE_PATH="/var/www/html" - -# Optional: Path to SSH private key (if not using default ~/.ssh/id_rsa) -# Leave empty to use default SSH key -SSH_KEY="" - -# Optional: SSH port (defaults to 22 if not specified) -# SSH_PORT="22" - diff --git a/deploy.sh b/deploy.sh deleted file mode 100755 index 5cf9692..0000000 --- a/deploy.sh +++ /dev/null @@ -1,34 +0,0 @@ -#!/bin/bash - -# Deployment script for pablohere website -# This script syncs the public folder to a remote webserver - -set -e # Exit on error - -# Load deployment configuration -if [ ! -f "deploy.config" ]; then - echo "Error: deploy.config file not found!" - echo "Please copy deploy.config.example to deploy.config and fill in your server details." - exit 1 -fi - -source deploy.config - -# Validate required variables -if [ -z "$REMOTE_HOST" ] || [ -z "$REMOTE_USER" ] || [ -z "$REMOTE_PATH" ]; then - echo "Error: Required variables not set in deploy.config" - echo "Please ensure REMOTE_HOST, REMOTE_USER, and REMOTE_PATH are set." - exit 1 -fi - -# Use rsync to sync files -echo "Deploying public folder to $REMOTE_USER@$REMOTE_HOST:$REMOTE_PATH" -rsync -avz --delete \ - --exclude='.git' \ - --exclude='.DS_Store' \ - $SSH_OPTS \ - public/ \ - $REMOTE_USER@$REMOTE_HOST:$REMOTE_PATH - -echo "Deployment complete!" - diff --git a/public/index.html b/public/index.html index 7f28526..3b400b9 100644 --- a/public/index.html +++ b/public/index.html @@ -1,285 +1,139 @@ - + - Pablo here - - - + Pablo here + + + -
-

Hi, Pablo here

-

- Welcome to my website. Here I discuss thoughts and ideas. This is mostly - professional. -

-
-

What you'll find here:

- -
-
-

About me

-

A few facts you might care about:

- -
-
-
-

Contact

-

You can contact me on:

- -

- If you are looking for my CV, no need to reach out, - you can fetch it yourself here. -

-

Good reasons to reach out include:

- -

Bad reasons to reach out include:

- -
-
-
-

My projects

-

Some of the projects I've shared publicly:

- -

- There are also some other projects that I generally keep private but - might disclose under the right circumstances. Some notable hints: -

- -
-
-
-

Writings

-

Sometimes I like to jot down ideas and drop them here.

- -
-
-
-

Talks

-

Some talks I've given:

- -
-
- +
+

+ Hi, Pablo here +

+

+ Welcome to my website. Here I discuss thoughts and ideas. This is mostly professional. +

+
+

What you'll find here:

+ +
+
+

About me

+

A few facts you might care about:

+ +
+
+
+

Contact

+

You can contact me on:

+ +

If you are looking for my CV, no need to reach out, you can fetch it + yourself here.

+

Good reason to reach out include:

+ +

Bad reasons to reach out include:

+ +
+
+
+

My projects

+

Some of the projects I've shared publicly:

+ +

There are also some other projects that I generally keep private but might disclose under the right + circumstances. Some notable hints:

+ +
+
+
+

Writings

+

Sometimes I like to jot down ideas and drop them here.

+ +
+
+ - + \ No newline at end of file diff --git a/public/keybase.txt b/public/keybase.txt deleted file mode 100644 index 43ffead..0000000 --- a/public/keybase.txt +++ /dev/null @@ -1,56 +0,0 @@ -================================================================== -https://keybase.io/pablomartincalvo --------------------------------------------------------------------- - -I hereby claim: - - * I am an admin of https://pablohere.contrapeso.xyz - * I am pablomartincalvo (https://keybase.io/pablomartincalvo) on keybase. - * I have a public key ASDgHxztDlU_R4hjxbkO21-rS4Iv1gABa3BPb_Aff7aNAgo - -To do so, I am signing this object: - -{ - "body": { - "key": { - "eldest_kid": "0120d9bde13d9012e681cef2edd668d70426f1f6ef69ce7dfae20b404096eca5b06f0a", - "host": "keybase.io", - "kid": "0120e01f1ced0e553f478863c5b90edb5fab4b822fd600016b704f6ff01f7fb68d020a", - "uid": "8e71277fbc0fb1fea28d60308f495d19", - "username": "pablomartincalvo" - }, - "merkle_root": { - "ctime": 1755635067, - "hash": "4f91af0b9c674e0f1d74a7cfad7abd15a7065cded92b96ac8a6abeb5c8553318599aa1bf7b065a3312e303506256b729b8b60b3a5dd06b68694423f4341a6a14", - "hash_meta": "6472dbf2ed33341fb30b6a0c5c5c7fb39c219dd0ffd03c6e08b68c788e0de60a", - "seqno": 27031070 - }, - "service": { - "entropy": "LEFJJ4FMmlJQWPPFEO4xHE5y", - "hostname": "pablohere.contrapeso.xyz", - "protocol": "https:" - }, - "type": "web_service_binding", - "version": 2 - }, - "client": { - "name": "keybase.io go client", - "version": "6.5.1" - }, - "ctime": 1755635082, - "expire_in": 504576000, - "prev": "37f12270050ab037897ccf6ef9451b1911cb505eca7c3842993b0b8925bc79b8", - "seqno": 31, - "tag": "signature" -} - -which yields the signature: - -hKRib2R5hqhkZXRhY2hlZMOpaGFzaF90eXBlCqNrZXnEIwEg4B8c7Q5VP0eIY8W5Dttfq0uCL9YAAWtwT2/wH3+2jQIKp3BheWxvYWTESpcCH8QgN/EicAUKsDeJfM9u+UUbGRHLUF7KfDhCmTsLiSW8ebjEIAnIWTmufZ017e9WLdI1LhKBPaZ3HzmTrgyASDvY3PwoAgHCo3NpZ8RA9a3xgkSTU6Ht7M7DCsy4ClMmoWFtDEqzX9/dqskeoH2DrJUZYVymBQE1nyB0p1GuXiZA1cP5WY5SDURWZ5bBC6hzaWdfdHlwZSCkaGFzaIKkdHlwZQildmFsdWXEIEJZ4g4HC5qXcqbFf6sJ8XuZyMtoppazFqr1zPu0LH5co3RhZ80CAqd2ZXJzaW9uAQ== - -And finally, I am proving ownership of this host by posting or -appending to this document. - -View my publicly-auditable identity here: https://keybase.io/pablomartincalvo - -================================================================== \ No newline at end of file diff --git a/public/my_cv.pdf b/public/my_cv.pdf index f3422a0..27038a6 100644 Binary files a/public/my_cv.pdf and b/public/my_cv.pdf differ diff --git a/public/static/achilles-meme.jpeg b/public/static/achilles-meme.jpeg deleted file mode 100644 index 05c0227..0000000 Binary files a/public/static/achilles-meme.jpeg and /dev/null differ diff --git a/public/static/bcc2026.pdf b/public/static/bcc2026.pdf deleted file mode 100644 index d700579..0000000 Binary files a/public/static/bcc2026.pdf and /dev/null differ diff --git a/public/static/chart1_accumulated_hashrate.png b/public/static/chart1_accumulated_hashrate.png deleted file mode 100644 index 06a1faa..0000000 Binary files a/public/static/chart1_accumulated_hashrate.png and /dev/null differ diff --git a/public/static/chart2_daily_hashrate.png b/public/static/chart2_daily_hashrate.png deleted file mode 100644 index ebbb900..0000000 Binary files a/public/static/chart2_daily_hashrate.png and /dev/null differ diff --git a/public/static/chart3_cumulative_spend.png b/public/static/chart3_cumulative_spend.png deleted file mode 100644 index d906a84..0000000 Binary files a/public/static/chart3_cumulative_spend.png and /dev/null differ diff --git a/public/static/chart4_price_per_ph.png b/public/static/chart4_price_per_ph.png deleted file mode 100644 index 655f641..0000000 Binary files a/public/static/chart4_price_per_ph.png and /dev/null differ diff --git a/public/static/chart5_daily_rewards.png b/public/static/chart5_daily_rewards.png deleted file mode 100644 index 3d6b878..0000000 Binary files a/public/static/chart5_daily_rewards.png and /dev/null differ diff --git a/public/static/chart6_daily_pnl.png b/public/static/chart6_daily_pnl.png deleted file mode 100644 index a0e85fe..0000000 Binary files a/public/static/chart6_daily_pnl.png and /dev/null differ diff --git a/public/static/computers.png b/public/static/computers.png deleted file mode 100644 index 955951a..0000000 Binary files a/public/static/computers.png and /dev/null differ diff --git a/public/static/data-alley-oop.jpeg b/public/static/data-alley-oop.jpeg deleted file mode 100644 index 3eb5709..0000000 Binary files a/public/static/data-alley-oop.jpeg and /dev/null differ diff --git a/public/static/dreaming-bids.jpeg b/public/static/dreaming-bids.jpeg deleted file mode 100644 index d4cc6b5..0000000 Binary files a/public/static/dreaming-bids.jpeg and /dev/null differ diff --git a/public/static/homophobic-socialist-drug-dealer.png b/public/static/homophobic-socialist-drug-dealer.png deleted file mode 100644 index f013ebc..0000000 Binary files a/public/static/homophobic-socialist-drug-dealer.png and /dev/null differ diff --git a/public/static/hospitals-inside.png b/public/static/hospitals-inside.png deleted file mode 100644 index 515541e..0000000 Binary files a/public/static/hospitals-inside.png and /dev/null differ diff --git a/public/static/hospitals-outside.png b/public/static/hospitals-outside.png deleted file mode 100644 index 33c1cd0..0000000 Binary files a/public/static/hospitals-outside.png and /dev/null differ diff --git a/public/static/many-phs.png b/public/static/many-phs.png deleted file mode 100644 index 0c86d59..0000000 Binary files a/public/static/many-phs.png and /dev/null differ diff --git a/public/static/order-book-bids-here.png b/public/static/order-book-bids-here.png deleted file mode 100644 index de5e5a0..0000000 Binary files a/public/static/order-book-bids-here.png and /dev/null differ diff --git a/public/static/padme-meme.jpg b/public/static/padme-meme.jpg deleted file mode 100644 index e56f94f..0000000 Binary files a/public/static/padme-meme.jpg and /dev/null differ diff --git a/public/static/peaceful-life-meme.jpeg b/public/static/peaceful-life-meme.jpeg deleted file mode 100644 index d01da78..0000000 Binary files a/public/static/peaceful-life-meme.jpeg and /dev/null differ diff --git a/public/static/plumbings.png b/public/static/plumbings.png deleted file mode 100644 index 68d61d0..0000000 Binary files a/public/static/plumbings.png and /dev/null differ diff --git a/public/static/recent-hashrate.png b/public/static/recent-hashrate.png deleted file mode 100644 index 9f17797..0000000 Binary files a/public/static/recent-hashrate.png and /dev/null differ diff --git a/public/static/stations.png b/public/static/stations.png deleted file mode 100644 index 54ad4ef..0000000 Binary files a/public/static/stations.png and /dev/null differ diff --git a/public/static/streetlamps.png b/public/static/streetlamps.png deleted file mode 100644 index 84a42d0..0000000 Binary files a/public/static/streetlamps.png and /dev/null differ diff --git a/public/styles.css b/public/styles.css index 0275b32..1b99426 100644 --- a/public/styles.css +++ b/public/styles.css @@ -7,8 +7,7 @@ body { h1, h2, -h3, -h4 { +h3 { text-align: center; } @@ -22,16 +21,7 @@ img { display: block; } -figcaption { +figcaption a { font-style: italic; font-size: small; - text-align: center; -} - -blockquote { - border-left: 3px solid #888; - margin-left: 0; - padding-left: 1em; - font-style: italic; - color: #555; } \ No newline at end of file diff --git a/public/writings/10-million-sats-into-mining.html b/public/writings/10-million-sats-into-mining.html deleted file mode 100644 index d568f40..0000000 --- a/public/writings/10-million-sats-into-mining.html +++ /dev/null @@ -1,295 +0,0 @@ - - - - - Pablo here - - - - - - - - -
-

- Hi, Pablo here -

-

back to home

-
-
-

10 million sats into mining

-

Published: 2026-05-25

-

In my previous article My first petahash I discussed my beginnings - with mining via rented hashrate from - Braiins' Hashpower market plus my own - DATUM gateway pointing to - OCEAN, with - hashbidder to automate my bidding in - the hashmarket. I also mentioned that my starting plan was to put a total of 10 million sats into this - experiment then checkpoint how things were going. Well, the 10 million sats have gone through Hashpower - (and partially come back to me), so I'll deliver the checkpoint as I promised myself.

-

Before going into it, I want to share my rationale for writing and sharing this. Most stuff I fiddle and - tinker with, I don't write about publicly. But this experiment is different. Although simply setting this - up and mining is plenty of fun and pretty much justifies doing it by its entertaining nature, the reality - is that my interest in controlling hashrate goes beyond sweet weekend hacking fun. Truth is that the - BIP110 softfork is on the table, and its success will depend on people - adopting it and hashrate being deployed to support it. My node has been running with the BIP110 patch for - quite a bit, but I didn't control more than 1TH/s until I started this experiment.

-

I think just learning how to set up this system and pointing a few petahashes to support the softfork is - enough of a contribution for my paygrade. It's been long since I thought I should save the world. - Nowadays I'm happy doing what a pleb must do, which is tending to his garden and doing his humble - part.

-
- It's a peaceful life meme -
-

Nevertheless: I wrote my previous article because I found there seems to be quite a bit of interest among - the plebs around renting hashrate, and so far I had not found much written about it beyond shit slinging - between knotzis and coretards. I'm writing this second part for the same reason, plus another specific - gap I've noticed. I see many plebs ask questions about how much money will you spend doing this, or what - amount of hashrate can you secure if you are willing to dedicate X sats per month to mining. And the - answers that come back are typically long rants on the different convoluted factors that influence the - answer. Not that those answers are wrong, for it is really a convoluted topic and it's hard to talk - specifics a priori. Hence why I want to write this article where I provide specific numbers on my - experience: so that it serves others as a very relatable and simple to understand report on how much - someone actually spent mining in a certain way. Hopefully it makes for a better data point to those who - find themselves on the fences of trying out.

-

The ultimate goal of it all is to spread knowledge, in the hopes people will be enlightened and motivated - by this and will take part instead of letting Foundry mine everything. My part will end here, and then - it's your turn to act.

-

I hope you find it interesting and useful. And I hope that, if you support BIP110, you'll roll your - sleeves and replicate this, adding the hashrate you can afford to the cause.

-

Now let's jump into the mud.

- -

Quick review of the system + how it performed

-

I'm running Knots with the BIP110 patch, DATUM and hashbidder in a server I physically host myself. All - three services are hosted in a single VM. The DATUM endpoint is publicly reachable via a VPS I rent - (with sats), which then gets redirected to the DATUM process running in my server.

-

The uptime of this has been perfect. Any bit of downtime has been related to blips in my residential - networking or host downtime due to maintenance that is not related to the mining bit. None of the - Bitcoin/mining services have had any operational hiccup (hashbidder did stop working here and there, but - that was simply because I was always using the bleeding edge version and some bugs made it to - production). Resource consumption for this stack is a non-issue: stable and predictable, so the VM this - has been running on has had 99.86% uptime during the last couple of months.

-

I encountered a few small issues with Hashpower, but they were all trivial and didn't affect my - experience. The service has had a few outages of the API and web, yet as far as I can tell the actual - delivery of hashrate has been working constantly. If that failed at any time, it was brief enough to get - camouflaged as your usual hash delivery spikeyness.

-

I did face some issues with Hashpower's API docs being - incorrect: some endpoints behaved differently than what they advertised in the documentation. I got in - touch with their support in mid April and they told me they would address the issue. I checked while - writing the article and the issue is still unsolved.

-

Regarding rejected shares, I think the setup has worked out great. I just checked, and the datum dashboard - indicates I'm sitting at 0.06% (0.0006). I wouldn't be able to judge the quality of this rate myself, but - others more experienced in mining have assured me it's great, so I'll parrot back that it's great.

-

When it comes to bidding, I can assure you it has been quite optimal. I've taken whatever prices existed in Hashpower and never said "no, I won't mine now because XYZ price is too high". But I did a pretty decent job at staying as close as possible to the cheapest price being served. It is common to see bids that are 10%/20%/50% above market price in Hashpower. In my case, I'm pretty confident I must have stayed constantly between 0-5% of the cheapest bid. I can't show you the data to back it up, and won't go into the details, but hashbidder just does a good job of doing that. You can try for yourself and verify.

- -

Facts, figures, performance

-

Okay, now let's show some serious data. Let's begin with how much I mined:

-
- Accumulated hashes computed -
Total accumulated hashes computed over the experiment
-
-
- Daily average hashrate against the 5 PH/s target -
Daily average hashrate against the 5 PH/s target
-
-

After some initial fiddling, I decided I would settle for a goal of mining at ~5PH/s. I estimated this - would allow me to spend my 10 million sats over 6-8 weeks. I wanted to mine at least that long to have a - decent statistical chance to getting close to non-extreme luck streaks with OCEAN (either good or - bad).

-

You can see in the chart the first days were more unstable due to this. It also took me a bit of time to - get hashbidder properly implemented and operational, hence why I didn't really start tracking the target - hashrate great until early May.

-

Is this enough to amount to anything significant? Well, if I had solo mined at this pace, there is less - than 2.5% probability that I would have found a block in less than a year. There is also around a 2.5% - probability that if I had mined with the same weight in the network all the way to 2140, I would still - have not found a block. Thank God for pooled mining and for OCEAN to make it fair, simple and - transparent.

-

How much did I pay for this? Well, the actual exact amount of money I've spent is not exactly 10,000,000 - sats, but rather 9,929,972 sats. The difference is mixed between mining fees I spent sending my money to - Braiins and some leftover sats I have still sitting on Hashpower because they're not enough to place an - order.

-
- Cumulative sats spent on rented hashrate -
Cumulative sats spent on rented hashrate
-
-

This is a chart showing how my expense accumulated slowly by the day. Unfortunately, it only shows some - days because the Braiins API doesn't give you back the full history, hence why you see that interpolated - bit at the start. Pretty boring. Stable hashrate, stable expense.

-

Although it's not a perfectly straight line! Prices do change quite a bit overtime in hashpower. The chart - of average sats per PH/s/d shows how the average price fluctuates over different days:

-
- Average price per PH/s per day -
Average price per PH/s per day
-
-

As you can see, prices change significantly measured in sats. Differences of +-20% in a few days do - happen.

-

On the income side: this chart shows the earned rewards day by day (not to be confused with payouts).

-
- Daily mining rewards earned -
Daily mining rewards earned, with the cumulative average
-
-

Extremely volatile, as you would expect due to the nature of - TIDES, the protocol that OCEAN follows. Having said that, the - volatility of this also relates to the size of OCEAN: as they grow their share of the total difficulty, - they hit blocks most frequently and wild spikes of luck become less frequent.

-

This is the chart that should make it clear that mining with OCEAN nowadays for a brief window of time is - russian roulette. There are entire days where NOTHING is paid. If you have any hopes of having a stable - experience, plan to mine for months at least. The cumulative average line in the chart shows you how, as - you stay longer and longer, you start stabilizing on the average.

-

Finally, this chart sums up the financial performance, showing expense and reward day by day along with - the cumulative net loss:

-
- Estimated daily P&L: expense vs rewards -
Estimated daily P&L: expense vs rewards, with accumulated net profit
-
-

So what does the final P&L look like then?

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Mining experiment P&L
(in sats)
Expenses:
Transaction fees817
OCEAN fees93,218
Rented hashrate9,929,972
Total expense10,024,007
Income:
Mining rewards9,321,406
Net−702,601
- -

What comes next

-

So, I poured 10 million sats into this and got roughly 9.3 million sats out, thus spending 700K sats.

-

Surely it's time to stop, right?

-
- And then you will stop, right? meme -
-

Unhosted Marcellus made - this reasoning I find great that if - your bags are in Bitcoin, it is in your best interest to keep hash raining on the blockchain tip. He - changes the way of looking at mining as a "business" for miners, and rather as a maintenance expense for - users (and fees and miners are just a way to organize so that each user doesn't need to run an S21 in - their kitchen and risk divorce).

-

I think it's a good POV. And I'm going to stick to it. I think putting 0.1% of my network, on a yearly - basis, towards adding hash is valuable for me. Now that I've run this couple months experiment, I have a - good feel for what is the "loss" factor of cycling sats through hashpower, so I can crunch the numbers - and find out what is the hashrate I should aim for to spend 0.1% of my bags on a yearly basis.

-

Some fellow Bitcoiners are aware of my plan and have pointed out to me that my actions are pointless. That - I'm too small to make a difference in the mining scene. It must suck to have such a terrible low-agency - mentality. Pleasure should be found in doing the right things, even if they amount to little.

-
- That's why no one will remember your nym meme -
-

And you? Will you mine? Why not?

-
-

back to home

-
-
- - - - diff --git a/public/writings/a-degraded-pool-with-a-healthy-disk.html b/public/writings/a-degraded-pool-with-a-healthy-disk.html deleted file mode 100644 index 094c699..0000000 --- a/public/writings/a-degraded-pool-with-a-healthy-disk.html +++ /dev/null @@ -1,134 +0,0 @@ - - - - - Pablo here - - - - - - - -
-

- Hi, Pablo here -

-

back to home

-
-

A degraded pool with a healthy disk

-

Published: 2026-01-04

-

Part 2 of 3 in my "First ZFS Degradation" series. See also Part 1: The Setup and Part 3: The Fix.

-

The "Oh Shit" Moment

-

I wasn't even looking for trouble. I was clicking around the Proxmox web UI, exploring some storage views I hadn't noticed before, when I saw it: my ZFS pool was in DEGRADED state.

-

I opened the details. One of my two mirrored drives was listed as FAULTED.

-

I was very surprised. This box and disks were brand new and didn't even have three months of running on them. I was not expecting HW issues to come at me that fast. I SSH'd into the server and ran the command that would become my best friend over the next 24 hours:

- 
zpool status -v proxmox-tank-1
-

No glitch. The pool was degraded. The drive had racked up over 100 read errors, 600+ write errors, and 129 checksum errors. ZFS had given up on it.

- 
  NAME                                 STATE     READ WRITE CKSUM
-  proxmox-tank-1                       DEGRADED     0     0     0
-    mirror-0                           DEGRADED     0     0     0
-      ata-ST4000NT001-3M2101_WX11TN0Z  FAULTED    108   639   129  too many errors
-      ata-ST4000NT001-3M2101_WX11TN2P  ONLINE       0     0     0
-

The good news: errors: No known data errors. ZFS was serving all my data from the healthy drive. Nothing was lost yet.

-

The bad news: I was running on a single point of failure. If AGAPITO2 decided to have a bad day too, I'd be in real trouble.

-

I tried the classic IT move: rebooting. The system came back up and ZFS immediately started trying to resilver (rebuild) the degraded drive. But within minutes, the errors started piling up again and the resilver stalled.

-

Time to actually figure out what was wrong.

-

The Diagnostic Toolbox

-

When a ZFS drive acts up, you have two main sources of truth: what the kernel sees happening at the hardware level, and what the drive itself reports about its health. This can be looked up with dmesg and smartctl.

-

dmesg: The Kernel's Diary

-

The Linux kernel maintains a ring buffer of messages about hardware events, driver activities, and system operations. The dmesg command lets you read it. For disk issues, you want to grep for SATA-related keywords:

- 
dmesg -T | egrep -i 'ata[0-9]|sata|reset|link|i/o error' | tail -100
-

The -T flag gives you human-readable timestamps instead of seconds-since-boot.

-

What I saw was... weird. Here's an excerpt:

- 
[Fri Jan  2 22:25:13 2026] ata4.00: exception Emask 0x50 SAct 0x70220001 SErr 0xe0802 action 0x6 frozen
-[Fri Jan  2 22:25:13 2026] ata4.00: irq_stat 0x08000000, interface fatal error
-[Fri Jan  2 22:25:13 2026] ata4.00: failed command: READ FPDMA QUEUED
-[Fri Jan  2 22:25:13 2026] ata4: hard resetting link
-[Fri Jan  2 22:25:14 2026] ata4: SATA link down (SStatus 0 SControl 300)
-

Let me translate: the kernel tried to read from the drive on ata4, got a "fatal error," and responded by doing a hard reset of the SATA link. Then the link went down entirely. The drive just... disappeared.

-

But it didn't stay gone. A few seconds later:

- 
[Fri Jan  2 22:25:24 2026] ata4: link is slow to respond, please be patient (ready=0)
-[Fri Jan  2 22:25:24 2026] ata4: SATA link up 6.0 Gbps (SStatus 133 SControl 300)
-

The drive came back! At full speed! But then...

- 
[Fri Jan  2 22:25:29 2026] ata4.00: qc timeout after 5000 msecs (cmd 0xec)
-[Fri Jan  2 22:25:29 2026] ata4.00: failed to IDENTIFY (I/O error, err_mask=0x4)
-[Fri Jan  2 22:25:29 2026] ata4: limiting SATA link speed to 3.0 Gbps
-

It failed again. The kernel, trying to be helpful, dropped the link speed from 6.0 Gbps to 3.0 Gbps. Maybe a slower speed would be more stable?

-

It wasn't. The pattern repeated: connect, fail, reset, reconnect at a slower speed. 6.0 Gbps, then 3.0 Gbps, then 1.5 Gbps. Eventually:

- 
[Fri Jan  2 22:27:06 2026] ata4.00: disable device
-

The kernel gave up entirely.

-

This wasn't what a dying drive looks like. A dying drive throws read errors on specific bad sectors. This drive was connecting and disconnecting like someone was jiggling the cable. The kernel was calling it "interface fatal error", emphasis on interface.

-

smartctl: Asking the Drive Directly

-

Every modern hard drive has S.M.A.R.T. (Self-Monitoring, Analysis, and Reporting Technology) — basically a built-in health monitor. The smartctl command lets you get info out of it.

-

First, the overall health check:

- 
smartctl -H /dev/sdb
- 
SMART overall-health self-assessment test result: PASSED
-

Okay, that looks great. But if the disk is healthy, what the hell is going on, and where are all those errors that ZFS was spotting coming from?

-

Let's dig deeper with the extended info:

- 
smartctl -x /dev/sdb
-

The key attributes I was looking for:

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
AttributeValueWhat it means
Reallocated_Sector_Ct0Bad sectors the drive has swapped out. Zero is good.
Current_Pending_Sector0Sectors waiting to be checked. Zero is good.
UDMA_CRC_Error_Count0Data corruption during transfer. Zero is good.
Number of Hardware Resets39Times the connection has been reset. Uh...
-

All the sector-level health metrics looked perfect. No bad blocks, no pending errors, no CRC errors. The drive's magnetic platters and read/write heads were fine.

-

But 39 hardware resets? That's not normal. That's the drive (or its connection) getting reset nearly 40 times.

-

I ran the short self-test to be sure:

- 
smartctl -t short /dev/sdb
-# Wait a minute...
-smartctl -l selftest /dev/sdb
- 
# 1  Short offline       Completed without error       00%
-

The drive passed its own self-test. The platters spin, the heads move, the firmware works, and it can read its own data just fine.

-

Hypothesis

-

At this point, the evidence was pointing clearly away from "the drive is dying" and toward "something is wrong with the connection."

-

What the kernel logs told me: the drive keeps connecting and disconnecting. Each time it reconnects, the kernel tries slower speeds. Eventually it gives up entirely. This is what you see with an unstable physical connection.

-

What SMART told me: the drive itself is healthy. No bad sectors, no media errors, no signs of wear. But there have been dozens of hardware resets — the connection keeps getting interrupted.

-

The suspects, in order of likelihood:

-
    -
  1. SATA data cable: the most common culprit for intermittent connection issues. Cables go bad, or weren't seated properly in the first place.
    2. -
  2. Power connection: if the drive isn't getting stable power, it might brown out intermittently.
    3. -
  3. SATA port on the motherboard: less likely, but possible.
    4. -
  4. PSU: power supply issues could affect the power rail feeding the drive. Unlikely, since both disks where feeding from the same cable tread, but still an option.
    5. -
-

Given that I had just built this server a few weeks earlier, and a good part of that happened after midnight... I was beginning to suspect that perhaps I simply might not have plugged in the disk properly.

-

The Verdict

-

I was pretty confident now: the drive was fine, but the connection was bad. Most likely the SATA data cable, and most probably simply not connected properly.

-

The fix would require shutting down the server, opening the case, and reseating (or replacing) cables. Before doing that, I wanted to take the drive offline cleanly and document everything.

-

In Part 3, I'll walk through exactly how I fixed it: the ZFS commands, the physical work, and the validation to make sure everything was actually okay afterward.

-

Continue to Part 3: The Fix →

-

back to home

-
-
- - - - - diff --git a/public/writings/a-note-for-the-future-the-tax-bleeding-in-2025.html b/public/writings/a-note-for-the-future-the-tax-bleeding-in-2025.html deleted file mode 100644 index 226d7da..0000000 --- a/public/writings/a-note-for-the-future-the-tax-bleeding-in-2025.html +++ /dev/null @@ -1,189 +0,0 @@ - - - - Pablo here - - - - - - -
-

Hi, Pablo here

-

back to home

-
-
-

A note for the future: the tax bleeding in 2025

-

Published: 2025-08-18

-

- I hate taxes deeply. I fell through the rabbit hole of libertarian and - anarcocapitalist ideas some years ago, and taxes have been repulsive - to me ever since. I go to great lengths to not pay them, and feel - deeply hurt everytime they sting my wallet against my will. -

-

- I know life goes by fast, and what today is vivid in your memory fades - away bit by bit until it's gone. I'm truly hoping that, some day in - the future, the world will have changed to the better and people won't - be paying as much tax as we're doing today in the West. Since in that - bright, utopical future I'm dreaming of I might have forgotten about - how bad things were on this matter in 2025, I've decided to make a - little entry here making an estimate on how many taxes I'm - theoretically bleeding on a yearly basis right now. So that we can - someday look back in time and wonder: "how the fuck did we tolerate - that pillaging". -

-

Inventory

-

- Before going hard into the number crunching let's list all the tax - items I'm aware of being subject to: -

- -

- There may be some other small, less frequent taxes that I'm not - considering. These are the ones that will hit most people in my - country. -

-

The numbers

-

- Okay, let's go compute the hideous bill. I'll make a hypothetical - profile that's roughly close to mine, with a few assumptions along the - way. -

- -

With those clear, let's see the actual figures:

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Tax€/year
Income Tax (IRPF)22,401 €
Social Security (worker + employer) - 25,375 € - (worker 4,445 € + employer 20,930 €) -
VAT (blended basket)5,250 €
Real Estate Tax (IBI)1,000 €
Vehicle Tax (1.5 vehicles)225 €
Wealth Transfer (10% home, spread 50y)1,000 €
Inheritance (7% of 250k, spread 50y)350 €
Total55,602 €
-

- So there you go. A peaceful existence as a tech professional living a - normal life leads to bleeding at least 55K€ per year, all while - getting an 85K€ salary. The tax rate sits at a wonderful 64%. How far - away is this from hardcore USSR-grade communism? -

-

- And this is generous, since I didn't model (1) what gets stolen - through inflation diluting savings and (2) any capital gains that this - profile might end up paying for whatever investments he is doing with - his savings. -

-

- Then you'll see mainstream media puppets discussing why young people - don't have children. As if it was some kind of mistery. They're being - robbed their children's bread left and right, while getting hypnotized - into believing that protecting themselves against this outrageous - robbery is somehow morally despicable. -

-

Motherfuckers.

-
-

back to home

-
-
- - diff --git a/public/writings/a-simple-solution-to-spam.html b/public/writings/a-simple-solution-to-spam.html index 345dc1a..4dd9ba0 100644 --- a/public/writings/a-simple-solution-to-spam.html +++ b/public/writings/a-simple-solution-to-spam.html @@ -18,7 +18,6 @@

A simple solution to spam

-

Published: 2025-01-14

Spam is a problem that has plagued the Internet for decades already. Given a negligible cost to reach out, and easy access to some contact method, many annoying people will choose to mass bother all of us in hopes of striking a profit. diff --git a/public/writings/bitcoin-mining-is-like-adding-the-final-piece-to-a-puzzle.html b/public/writings/bitcoin-mining-is-like-adding-the-final-piece-to-a-puzzle.html index c6a3965..79928a4 100644 --- a/public/writings/bitcoin-mining-is-like-adding-the-final-piece-to-a-puzzle.html +++ b/public/writings/bitcoin-mining-is-like-adding-the-final-piece-to-a-puzzle.html @@ -18,7 +18,6 @@

Bitcoin mining is like adding the final piece to a puzzle

-

Published: 2025-02-01

Bitcoin mining is one of those terribly explained topics that everyone and their mother has kind of heard about at some point, but no one really feels comfortable explaining. diff --git a/public/writings/busy-mans-guide-to-optimizing-dbt-models-performance.html b/public/writings/busy-mans-guide-to-optimizing-dbt-models-performance.html deleted file mode 100644 index 03c890e..0000000 --- a/public/writings/busy-mans-guide-to-optimizing-dbt-models-performance.html +++ /dev/null @@ -1,359 +0,0 @@ - - - - - Pablo here - - - - - - - -

-

- Hi, Pablo here -

-

back to home

-
-
-

Busy man's guide to optimizing dbt models performance

-

Published: 2026-02-10

-

The below guide is a copy-paste of an internal doc file I created while working in Superhog. My team of - analysts were very smart guys, but had little knowledge on Postgres internals (we used Postgres for our - DWH) and low-level query optimization. That was understandable: they were analysts, busy with answering - business questions. Their value was not in fixing pipeline performance.

-

Nevertheless, giving them some degree of freedom in fixing performance was both great to avoid me - becoming a bottleneck and to also expand their knowledge, which they were eager to. This guide was - targeted to them, and the goal was to give them as many tools as possible without having to go into the - rabbit hole of EXPLAIN ANALYZE.

-
-

You have a dbt model that takes ages to run in production. For some very valid reason, this - is a problem.

-

This is a small reference guide on things you can try. I suggest you try them from start to end, since - they are sorted in a descending way by value/complexity ratio.

-

Before you start working on a model, you might want to check the bonus guide at the - bottom to learn how to make sure you don't change the outputs of a model while refactoring it. -

-

If you've tried everything you could here and things still don't work, don't hesitate to call Pablo.

- -

1. Is your model really taking too long?

-
Before you optimize a model that is taking too long, make sure it actually takes too long. -
-

The very first step is to really assess if you do have a problem.

-

We run our DWH in a Postgres server, and Postgres is a complex system. Postgres is doing many things at - all times and it's very stateful, which means you will pretty much never see exactly the same - performance twice for some given query.

-

Before going crazy optimizing, I would advise running the model or the entire project a few times and - observing the behaviour. It might be that some day it took very long for some reason, but - usually, it runs just fine.

-

You also might want to do this in a moment where there's little activity in the DWH, like very early or - late in the day, so that other users' activity in the DWH don't pollute your observations.

-

If this is a model that is already being run regularly, we can also leverage the statistics - collected by the pg_stat_statements Postgres extension to check what are the min, avg, and - max run times for it. Ask Pablo to get this.

- -

2. Reducing the amount of data

-
Make your query only bring in the data it needs, and not more. Reduce the amount of data as - early as possible.
-

This option is a simple optimization trick that can be used in many areas and it's easy to pull off.

-

The two holy devils of slow queries are large amounts of data and monster lookups/sorts. Both can be - drastically reduced by simply reducing the amount of data that goes into the query, typically by applying - some smart WHERE or creative conditions on a JOIN clause. This can be either - done in your basic CTEs where you read from other models, or in the main SELECT of your - model.

-

Typically, try to make this as early as possible in the model. Early here refers to the steps - of your query. In your queries, you will typically:

-
    -
  • read a few tables,
    • -
  • do some SELECTs
    • -
  • then do more crazy logic downstream with more SELECTs
    • -
  • and the party goes on for as long and complex your case is
    • -
-

Reducing the amount of data at the end is pointless. You will still need to read a lot of stuff early and - have monster JOINs , window functions, DISTINCTs, etc. Ideally, you want to do - it when you first access an upstream table. If not there, then as early as possible within the logic. -

-

The specifics of how to apply this are absolutely query dependent, so I can't give you magic instructions - for the query you have at hand. But let me illustrate the concept with an example:

- -

Only hosts? Then only hosts

-

You have a table stg_my_table with a lot of data, let's say 100 million records, and each - record has the id of a host. In your model, you need to join these records with the host user data to - get some columns from there. So right now your query looks something like this (tables fictional, this - is not how things look in DWH):

- 
with
-stg_my_table as (select * from {{ ref("stg_my_table") }}),
-stg_users as (select * from {{ ref("stg_users")}})
-
-select
-    ...
-from stg_my_table t
-left join
-    stg_users u
-    on t.id_host_user = id_user
-

At the time I'm writing this, the real user table in our DWH has like 600,000 records. This means - that:

-
    -
  • The CTE stg_users will need to fetch 600,000 records, with all their data, and store - them.
    • -
  • Then the left join will have to join 100 million records from my_table with the 600,000 - user records.
    • -
-

Now, this is not working for you because it takes ages. We can easily improve the situation by applying - the principle of this section: reducing the amount of data.

-

Our user table in the DWH has both hosts and guests. Actually, it has a ~1,000 hosts and everything else - is just guests. This means that:

-
    -
  • We're fetching around 599,000 guest details that we don't care about at all.
    • -
  • Every time we join a record from my_table, we do so against 600,000 user records when - we only truly care about 1,000 of them.
    • -
-

Stupid, isn't it?

-

Well, imagining that our fictional stg_users tables had a field called - is_host, we can rewrite the query this way to get exactly the same result in only a - fraction of the time: -

- 
with
-stg_my_table as (select * from {{ ref("stg_my_table") }}),
-stg_users as (
-    select *
-    from {{ ref("stg_users")}}
-    where is_host = true
-    )
-
-select
-    ...
-from stg_my_table t
-left join
-    stg_users u
-    on t.id_host_user = id_user
-

It's simple to understand: the CTE will now only get the 1,000 records related to hosts, which means we - save performance in both fetching that data and having a much smaller join operation downstream against - stg_my_table. -

- -

3. Controlling CTE materialization

-
Tell Postgres when to cache intermediate results and when to optimize through them.
-

This one requires a tiny bit of understanding of what happens under the hood, but the payoff is big and - the fix is easy to apply.

- -

What Postgres does with your CTEs

-

When Postgres runs a CTE, it has two strategies:

-
    -
  • Materialized: Postgres runs the CTE query, stores the full result in a temporary - buffer, and every downstream reference reads from that buffer. Think of it as Postgres creating a - temporary, index-less table with the CTE's output.
    • -
  • Not materialized: Postgres treats the CTE as if it were a view. It doesn't store - anything — instead, it folds the CTE's logic into the rest of the query and optimizes everything - together. This means it can push filters down, use indexes from the original tables, and skip - reading rows it doesn't need.
    • -
-

By default, Postgres decides for you: if a CTE is referenced once, it inlines it. If it's referenced - more than once, it materializes it.

-

The problem is that this default isn't always ideal, especially with how we write dbt models.

- -

Why this matters for our dbt models

-

Following our conventions, we always import upstream refs as CTEs at the top of the file:

- 
with
-    stg_users as (select * from {{ ref("stg_users") }}),
-    stg_bookings as (select * from {{ ref("stg_bookings") }}),
-
-    some_intermediate_logic as (
-        select ...
-        from stg_users
-        join stg_bookings on ...
-        where ...
-    ),
-
-    some_other_logic as (
-        select ...
-        from stg_users
-        where ...
-    )
-
-select ...
-from some_intermediate_logic
-join some_other_logic on ...
-

Notice that stg_users is referenced twice — once in some_intermediate_logic - and once in some_other_logic. This means Postgres will materialize it by default. What - happens then is:

-
    -
  1. Postgres scans the entire stg_users table and copies all 600,000 rows into a temporary - buffer.
    2. -
  2. If the buffer exceeds available memory, it spills to disk.
    3. -
  3. Every downstream CTE that reads from stg_users does a sequential scan of that buffer. - Note this means indices can't be used, even if the original table had them.
    4. -
  4. Any filters that downstream CTEs apply to stg_users (like - where is_host = true) can't be pushed down to the original table scan. Postgres reads - all 600,000 rows first, stores them, and only then filters. -
    5. -
-

All of that, for a select * that does absolutely no computation worth caching.

- -

The fix

-

You can explicitly control this behaviour by adding MATERIALIZED or - NOT MATERIALIZED to any CTE: -

- 
with
-    stg_users as not materialized (select * from {{ ref("stg_users") }}),
-    stg_bookings as not materialized (select * from {{ ref("stg_bookings") }}),
-
-    some_intermediate_logic as (
-        ...
-    ),
-
-    some_other_logic as (
-        ...
-    )
-
-select ...
-

With NOT MATERIALIZED, Postgres treats those import CTEs as transparent aliases. It can see - straight through to the original table, use its indexes, and push filters down.

- -

When to use which

-

The rule of thumb is simple:

-
    -
  • Cheap CTE, referenced multiple times → NOT MATERIALIZED. This is the - typical case for our import CTEs at the top of the file. There's no computation to cache, so - materializing just wastes resources.
    • -
  • Expensive CTE, referenced multiple times → leave it alone (or explicit - MATERIALIZED). If a CTE does heavy aggregations, complex joins, or window functions, - materializing means that work happens once. Without it, Postgres would repeat the expensive query - every time the CTE is referenced. -
    • -
  • Any CTE referenced only once → doesn't matter. Postgres inlines it automatically. -
    • -
-

If you're unsure whether a CTE is "expensive enough" to warrant materialization, just try both and - measure. There's no shame in that.

- -

4. Change upstream materializations

-
Materialize upstream models as tables instead of views to reduce computation on the model at - hand.
-

Going back to basics, dbt offers multiple materializations strategies for our models. -

-

Typically, for reasons that we won't cover here, the preferred starting point is to use views. We only go - for tables or incremental materializations if there are good reasons for this.

-

If you have a model that is having terrible performance, it's possible that the fault doesn't sit at the - model itself, but rather at an upstream model. Let me make an example.

-

Imagine we have a situation with three models:

-
    -
  • stg_my_simple_model: a model with super simple logic and small data
    • -
  • stg_my_crazy_model: a model with a crazy complex query and lots of data
    • -
  • int_my_dependant_model: an int model that reads from both previous models.
    • -
  • Where the staging models are set to materialize as views and the int model is set to materialize as - a table.
    • -
-

Because the two staging models are set to materialize as views, this means that every time you run - int_my_dependant_model, you will also have to execute the queries of - stg_my_simple_model and stg_my_crazy_model. If the upstream views model are - fast, this is not an issue of any kind. But if a model is a heavy query, this could be an issue. -

-

The point is, you might notice that int_my_dependant_model takes 600 seconds to run and - think there's something wrong with it, when actually the fault sits at stg_my_crazy_model, - which perhaps is taking 590 seconds out of the 600.

-

How can materializations solve this? Well, if stg_my_crazy_model was materialized as a table - instead of as view, whenever you ran int_my_dependant_model you would simply read from a - table with pre-populated results, instead of having to run the stg_my_crazy_model query - each time. Typically, reading the results will be much faster than running the whole query. So, in - summary, by making stg_my_crazy_model materialize as a table, you can fix your performance - issue in int_my_dependant_model.

- -

5. Switch the model to materialization to incremental

-
Make the processing of the table happen in small batches instead of on all data to make it more - manageable.
-

Imagine we want to count how many bookings were created each month.

-

As time passes, more and more months and more and more bookings appear in our history, making the size of - this problem ever increasing. But then again, once a month has finished, we shouldn't need to go back - and revisit history: what's done is done, and only the ongoing month is relevant, right?

-

dbt offers a materialization strategy named incremental, - which allows you to only work on a subset of data. This means that every time you run - dbt run , your model only works on a certain part of the data, and not all of it. If the - nature of your data and your needs allows isolating each run to a small part of all upstream data, this - strategy can help wildly improve the performance. -

-

Explaining the inner details of incremental goes beyond the scope of this page. You can - check the official docs from dbt (here), ask the team for support or check some of the incremental - models that we already have in our project and use them as references.

-

Note that using incremental strategies makes life way harder than simple view - or table ones, so only pick this up if it's truly necessary. Don't make models incremental - without trying other optimizations first, or simply because you realise that you could use it - in a specific model.

-

6. End of the line: general optimization

-

The final tip is not really a tip. The above five things are the easy-peasy, low hanging fruit stuff that - you can try. This doesn't mean that there isn't more than you can do, just that I don't know of more - simple stuff that you can try without deep knowledge of how Postgres works beneath and a willingness to - get your hands real dirty.

-

If you've reached this point and your model is still performing poorly, you either need to put your Data - Engineer hat on and really deepen your knowledge… or call Pablo.

- -

Bonus: how to make sure you didn't screw up and change the output of the model

-

The topic we are discussing in this guide is making refactors purely for the sake of performance, without - changing the output of the given model. We simply want to make the model faster, not change what data it - generates.

-

That being the case, and considering the complexity of the strategies we've presented here, being afraid - that you messed up and accidentally changed the output of the model is a very reasonable fear to have. - That's a kind of mistake that we definitely want to avoid.

-

Doing this manually can be a PITA and very time consuming, which doesn't help at all.

-

To make your life easier, I'm going to show you a new little trick.

- -

Hashing tables and comparing them

-

I'll post a snippet of code here that you can run to compare if any pair of tables has exactly - the same contents. Emphasis on exactly. Changing the slightest bit of content will be detected.

- 
SELECT md5(array_agg(md5((t1.*)::varchar))::varchar)
-  FROM (
-        SELECT *
-          FROM my_first_table
-         ORDER BY <whatever field is unique>
-       ) AS t1
-
-SELECT md5(array_agg(md5((t2.*)::varchar))::varchar)
-  FROM (
-        SELECT *
-          FROM my_second_table
-         ORDER BY <whatever field is unique>
-       ) AS t2
-

How this works is: you execute the two queries, which will return a single value each. Some hexadecimal - gibberish.

-

If the output of the two queries is identical, it means their contents are identical. If they are - different, it means there's something different across both.

-

If you don't understand how this works, and you don't care, that's fine. Just use it.

-

If not knowing does bother, you should go down the rabbit holes of hash functions and deterministic - serialization.

- -

Including this in your refactoring workflow

-

Right, now you know how to make sure that two tables are identical.

-

This is dramatically useful for your optimization workflow. You can know simply:

-
    -
  • Keep the original model
    • -
  • Create a copy of it, which is the one you will be working on (the working copy)
    • -
  • Prepare the magic query to check their contents are identical
    • -
  • From this point on, you can enter in this loop for as long as you want/need: -
      -
    • Run the magic query to ensure you start from same-output-state
      • -
    • Modify the working copy model to attempt whatever optimization thingie you wanna try
      • -
    • Once you are done, run the magic query again.
      • -
    • If the output is not the same anymore, you screwed up. Start again and avoid whatever - mistake you made.
      • -
    • If the output is still the same, you didn't cause a change in the model output. Either keep - on optimizing or call it day.
      • -
    -
    • -
  • Finally, just copy over the working copy model code into the old one and remove the working copy. -
    • -
-

I hope that helps. I also recommend doing the loop as frequently as possible. The less things you change - between executions of the magic query, the easier is to realize what caused errors if they appear.

-
-

back to home

-
-
- - - - diff --git a/public/writings/credit-cards-affairs-and-chatgpt.html b/public/writings/credit-cards-affairs-and-chatgpt.html index 35ea222..2bec290 100644 --- a/public/writings/credit-cards-affairs-and-chatgpt.html +++ b/public/writings/credit-cards-affairs-and-chatgpt.html @@ -18,7 +18,6 @@

Credit cards, affairs and ChatGPT

-

Published: 2025-01-31

Many years ago, when I was doing my Bachelor's thesis, I had a fun conversation with Dolors, my tutor back then. She was a smart woman, and so the interesting chats diff --git a/public/writings/dont-hide-it-make-it-beautiful.html b/public/writings/dont-hide-it-make-it-beautiful.html deleted file mode 100644 index ab7d7a4..0000000 --- a/public/writings/dont-hide-it-make-it-beautiful.html +++ /dev/null @@ -1,126 +0,0 @@ - - - - - Pablo here - - - - - - - -

-

- Hi, Pablo here -

-

back to home

-
-
-

Don't hide it, make it beautiful

-

Published: 2025-03-17

-

I'm currently living in a flat, and my internet connection physically comes in through my living room. That's where - my home router is placed. However, my main workspace is not in my living room but in my working room, - which is a few meters away. I would love to have a wired internet connection for my laptop, but unfortunately, with - the router being so far away, setting it up would require running a lot of cable through the walls and - ceilings. I could either leave the cable visible or go through some serious construction work to poke holes through walls and fake ceilings and - tunnel the cable through there. The latter is out of the table, since I don't even know where would I start.

- -

The first option being the only available one, there is one fundamental and unavoidable reason I don't do this: aesthetics. My partner is very conscious about - keeping our home visually pleasing. I care too, though she probably values aesthetics even more than I - do. She likely doesn't find a wired internet connection to be as essential as I do. So, for now, I have to rely on - wifi to connect from my workspace to the home router.

- -

When I was on holiday in Thailand a few years ago, I noticed that Thai homes are far more practical than - European ones in such matters. In Thailand, plumbing, electrical systems, and other maintenance-requiring - installations are typically very visible, just out there on the wall. They don't hide these things behind - fake walls or ceilings. I believe they do this because they highly value the ability to access and work - on their home's systems themselves. Many Thai people build and maintain their own homes, so they leave - everything exposed for easy access.

- -

I sometimes envy this approach. Which is funny because I don't think they do it for pleasure but out of necessity. - Still, when I saw a Thai homeowner fixing their plumbing outside their house, I thought to myself: "Damn, you're so - in control of your home". If something bad happens—like a fallen tree damaging the plumbing—they can fix - it themselves. Meanwhile, if that happened to me, I wouldn't even know where to start. I don't even know - where my plumbing is because it's all hidden behind walls.

- -

That makes me wonder: Is there a way to make these essential systems both accessible and aesthetically - pleasing? Could we have the convenience of exposed infrastructure without it looking ugly? I believe we - can.

- -

I find the problem is that we have decided certain things—plumbing, electrical wiring, visible - infrastructure—are inherently ugly. But they don't have to be. Some household items, like lamps, must be - visible by their very nature. Since they can't be hidden, we put effort into making them look good. We choose stylish designs - that complement our home's aesthetics. Why can't we do the same for cables and pipes?

- -

Imagine if all the wiring in your home was encased in beautifully braided, colorful ropes, arranged in - elegant geometric patterns. The connections, junction boxes, and fittings could be crafted from - high-quality materials like metal and wood with artistic designs. Wouldn't that be nice?

- -

Now, you might think I'm crazy—that these things are just ugly by nature. But they're not. In fact, many - aspects of modern design have become uglier over time, and we've just accepted it.

- -

Consider street lamps. In most cities today, they are dull, industrial-looking poles—rusty, ugly, - and purely functional. Yet, in older parts of my city, we still have beautiful, ornate lamp posts from - over a hundred years ago. They were designed with care, meant to serve a purpose, to be visually - appealing, and to last ages. Take a look:

- -
- -
On the left, your ugly, could-be-anywhere post 1971 streetlamp. On the right, a 19th century bad body from Gaudí.
-
- -

The same goes for train stations. Modern stations are bleak, sterile spaces—metal, plastic, and harsh - lighting. They resemble hospital emergency rooms. But look at the older ones, like this one. - Those stations are masterpieces, designed like grand halls with chandeliers and intricate details.

- -
- -
On the left, Sants Station, built in 1975. On the right, France Station, built in 1848.
-
- -

And talking about hospitals, they are also a good example. Most modern hospitals have the same white, cold, spaceship-like - aesthetic. While cleanliness is important, there's no reason they have to be so uninviting. In my city, - there's a hospital built over a hundred years ago that's so beautiful people visit it as a tourist - attraction. On the other hand, the hospitals I visit personally are plain depressing, soviet style - atrocities.

- -
- -
A random modern clinic in Barcelona vs A small section of the outside of Hospital de Sant Pau. I can skip the left and right thing now, right?
-
-
- -
Some random room in that same modern clinic vs Your regular corridor in Sant Pau.
-
- - - -

I think we can bring things back, if we care enough.

- -

Look at computers. Most office desktop cases are dull, gray boxes—uninspired and purely functional. - Naturally, many of them end up buried inside desks, or if they are small enough, simply hidden behind - the screen on a VESA mount. But gamers, who deeply care about their PCs, go the extra mile to make their - setups look amazing. They invest in custom cases, LED lighting, and stylish cooling systems. They turn - their computers into art. They are testament to the fact that we can make practical things also be - beautiful if we choose to.

- -
- -
The all-present ugly office optiplex vs A beautiful case from a passionate man.
-
- -

If we put the same effort into our homes, we wouldn't need to hide cables and pipes. We could proudly - display them as part of our interior design. Infrastructure could be both functional and beautiful, - giving us accessibility without sacrificing aesthetics.

- -

I guess the point I want to make is... Don't hide it. Instead, make it beautiful.

- -
-

back to home

-
-
- - - - \ No newline at end of file diff --git a/public/writings/fixing-a-degraded-zfs-mirror.html b/public/writings/fixing-a-degraded-zfs-mirror.html deleted file mode 100644 index ef1e295..0000000 --- a/public/writings/fixing-a-degraded-zfs-mirror.html +++ /dev/null @@ -1,189 +0,0 @@ - - - - - Pablo here - - - - - - - -
-

- Hi, Pablo here -

-

back to home

-
-

Fixing a Degraded ZFS Mirror: Reseat, Resilver, and Scrub

-

Published: 2026-01-04

-

Part 3 of 3 in my "First ZFS Degradation" series. See also Part 1: The Setup and Part 2: Diagnosing the Problem.

-

The Game Plan

-

By now I was pretty confident about what was wrong: not a dying drive, but a flaky SATA connection. The fix should be straightforward. Just take the drive offline, shut down, reseat the cables, bring it back up, and let ZFS heal itself.

-

But I wanted to do this methodically. ZFS is forgiving, but I didn't want to make things worse by rushing.

-

Here was my plan:

-
    -
  1. Take the faulty drive offline in ZFS (tell ZFS "stop trying to use this drive")
    2. -
  2. Power down the server
    3. -
  3. Open the case, inspect and reseat cables
    4. -
  4. Boot up, verify the drive is detected
    5. -
  5. Bring the drive back online in ZFS
    6. -
  6. Let the resilver complete
    7. -
  7. Run a scrub to verify data integrity
    8. -
  8. Check SMART one more time
    9. -
-

Let's walk through each step.

-

Step 1: Taking the Drive Offline

-

Before touching hardware, I wanted ZFS to stop trying to use the problematic drive.

-

First, I set up some variables to avoid typos with that long disk ID:

- 
DISKID="ata-ST4000NT001-3M2101_WX11TN0Z"
-DISKPATH="/dev/disk/by-id/$DISKID"
-

Then I took it offline:

- 
zpool offline proxmox-tank-1 "$DISKID"
-

Checking the status afterward:

- 
zpool status -v proxmox-tank-1
- 
  NAME                                 STATE     READ WRITE CKSUM
-  proxmox-tank-1                       DEGRADED     0     0     0
-    mirror-0                           DEGRADED     0     0     0
-      ata-ST4000NT001-3M2101_WX11TN0Z  OFFLINE    108   639   129
-      ata-ST4000NT001-3M2101_WX11TN2P  ONLINE       0     0     0
-

The state changed from FAULTED to OFFLINE. ZFS knows I intentionally took it offline rather than it failing on its own. The error counts are still there as a historical record, but ZFS isn't actively trying to use the drive anymore.

-

Time to shut down and get my hands dirty.

-

Step 2: Opening the Case

-

I powered down the server and opened up the Fractal Node 804. This case has a lovely design with drive bays accessible from the side, which I love. No reaching out into weird corners in the case, just unscrew a couple screws, slide the drive bay out and there they are, handy and reachable.

-

I located AGAPITO1 (I had handwritten labels on the drives, lesson learned after many sessions of playing "which drive is which") and inspected the connections.

-

Here's the honest truth: everything looked fine. The SATA data cable was plugged in. The power connector was plugged in. Nothing was obviously loose or damaged. There was a bit of tension in the cable as it moved from one area of the case (where the motherboard is) to the drives area, but I really didn't think that was affecting the connection to either the drive or the motherboard itself.

-

But "looks fine" doesn't mean "is fine". So I did a full reseat:

-
    -
  • Unplugged and firmly replugged the SATA data cable at both ends (drive and motherboard).
    • -
  • Unplugged and firmly replugged the power connector.
    • -
  • While I was in there, checked the connections on the other disk of the mirror as well.
    • -
-

I made sure each connector clicked in solidly. Then I closed up the case and hit the power button.

-

Step 3: Verifying Detection

-

The server booted up. Would Linux see the drive?

- 
ls -l /dev/disk/by-id/ | grep WX11TN0Z
- 
lrwxrwxrwx 1 root root  9 Jan  2 23:15 ata-ST4000NT001-3M2101_WX11TN0Z -> ../../sdb
-

The drive was there, mapped to /dev/sdb.

-

I opened a second terminal and started watching the kernel log in real time:

- 
dmesg -Tw
-

This would show me immediately if the connection started acting flaky again. For now, it was quiet, showing just normal boot messages, the drive being detected successfully, etc. Nothing alarming.

-

Step 4: Bringing It Back Online

-

Moment of truth. I told ZFS to start using the drive again:

- 
zpool online proxmox-tank-1 "$DISKID"
-

Immediately checked the status:

- 
zpool status -v proxmox-tank-1
- 
  pool: proxmox-tank-1
- state: DEGRADED
-status: One or more devices is currently being resilvered.
-action: Wait for the resilver to complete.
-  scan: resilver in progress since Fri Jan  2 23:17:35 2026
-        0B resilvered, 0.00% done, no estimated completion time
-
-    NAME                                 STATE     READ WRITE CKSUM
-    proxmox-tank-1                       DEGRADED     0     0     0
-      mirror-0                           DEGRADED     0     0     0
-        ata-ST4000NT001-3M2101_WX11TN0Z  DEGRADED     0     0     0  too many errors
-        ata-ST4000NT001-3M2101_WX11TN2P  ONLINE       0     0     0
-

Two things to notice: the drive's error counters are now at zero (we're starting fresh), and ZFS immediately started resilvering. It shows "too many errors" as the reason for the degraded state, which is historical, it remembers why the drive was marked bad before.

-

I kept watching both the status and the kernel log. No errors, no link resets.

-

Step 5: The Resilver

-

Resilvering is ZFS's term for rebuilding redundancy. Copying data from the healthy drive to the one that fell behind. In my case, the drive had been desynchronized for who knows how long (the pool had drifted 524GB out of sync before I noticed), so there was a lot to copy.

-

I shut down my VMs to reduce I/O contention and let the resilver have the disk bandwidth. Progress:

- 
scan: resilver in progress since Fri Jan  2 23:17:35 2026
-      495G / 618G scanned, 320G / 618G issued at 100M/s
-      320G resilvered, 51.78% done, 00:50:12 to go
-

The kernel log stayed quiet the whole time. Everything was indicating the cable reseat had worked.

-

I went to bed and let it run overnight. The next morning:

- 
scan: resilvered 495G in 01:07:58 with 0 errors on Sat Jan  3 00:25:33 2026
-

495 gigabytes resilvered in about an hour, zero errors. But the pool still showed DEGRADED with a warning about "unrecoverable error." I was very confused about this, but I solved that with some research. Apparently, ZFS is cautious and wants human acknowledgement before declaring everything healthy again.

- 
zpool clear proxmox-tank-1 ata-ST4000NT001-3M2101_WX11TN0Z
-

This command clears the error flags. Immediately:

- 
  pool: proxmox-tank-1
- state: ONLINE
-  scan: resilvered 495G in 01:07:58 with 0 errors on Sat Jan  3 00:25:33 2026
-
-    NAME                                 STATE     READ WRITE CKSUM
-    proxmox-tank-1                       ONLINE       0     0     0
-      mirror-0                           ONLINE       0     0     0
-        ata-ST4000NT001-3M2101_WX11TN0Z  ONLINE       0     0     0
-        ata-ST4000NT001-3M2101_WX11TN2P  ONLINE       0     0     0
-

Damn, seeing this felt nice.

-

Step 6: The Scrub

-

A resilver copies data to bring the drives back in sync, but it doesn't verify that all the existing data is still good. For that, you run a scrub. ZFS reads every block on the pool, verifies checksums, and repairs anything that doesn't match.

- 
zpool scrub proxmox-tank-1
-

I let this run while I brought my VMs back up (scrubs can run in the background without blocking normal operations, though performance takes a hit). A few hours later:

- 
scan: scrub repaired 13.0M in 02:14:22 with 0 errors on Sat Jan  3 11:03:54 2026
-
-    NAME                                 STATE     READ WRITE CKSUM
-    proxmox-tank-1                       ONLINE       0     0     0
-      mirror-0                           ONLINE       0     0     0
-        ata-ST4000NT001-3M2101_WX11TN0Z  ONLINE       0     0   992
-        ata-ST4000NT001-3M2101_WX11TN2P  ONLINE       0     0     0
-

Interesting. The scrub repaired 13MB of data and found 992 checksum mismatches on AGAPITO1. From what I read, checksum errors are typically a sign of the disk being in terrible shape and needing a replacement ASAP. That sounds scary, but I took the risk and assumed those were blocks that had been written incorrectly (or not at all) during the period when the connection was flaky, and not an issue with the disk itself. ZFS detected the bad checksums and healed them using the good copies from AGAPITO2.

-

I cleared the errors again and the pool was clean:

- 
zpool clear proxmox-tank-1 ata-ST4000NT001-3M2101_WX11TN0Z
-

Step 7: Final Validation with SMART

-

One more check. I wanted to see if SMART had anything new to say about the drive after all that activity:

- 
smartctl -x /dev/sdb | egrep -i 'overall|Reallocated|Pending|CRC|Hardware Resets'
- 
SMART overall-health self-assessment test result: PASSED
-  5 Reallocated_Sector_Ct   PO--CK   100   100   010    -    0
-197 Current_Pending_Sector  -O--C-   100   100   000    -    0
-199 UDMA_CRC_Error_Count    -OSRCK   200   200   000    -    0
-0x06  0x008  4              41  ---  Number of Hardware Resets
-

Still passing. The hardware reset count went from 39 to 41 — just the reboots I did during this process.

-

For completeness, I ran the long self-test. The short test only takes a minute and does basic checks, the long test actually reads every sector on the disk, which for a 4TB drive takes... a while.

- 
smartctl -t long /dev/sdb
-

The estimated time was about 6 hours. In practice, it took closer to 12. Running VMs in parallel probably didn't help.

-

But eventually:

- 
SMART Self-test log structure revision number 1
-Num  Test_Description    Status                  Remaining  LifeTime(hours)  LBA_of_first_error
-# 1  Extended offline    Completed without error       00%      1563         -
-# 2  Short offline       Completed without error       00%      1551         -
-# 3  Short offline       Completed without error       00%      1462         -
-

The extended test passed. Every sector on the disk is readable. The drive is genuinely healthy — it was just the connection that was bad.

-

Lessons Learned

-
    -
  • ZFS did exactly what it's supposed to do: Despite 524+ gigabytes of desync and nearly a thousand checksum errors, I lost zero data and was back on action while keeping my VMs running. The healthy drive kept serving everything while the flaky drive was acting up, and once the connection was fixed, ZFS healed itself automatically. Also, I was operating for an unknown amount of time with only one drive. In this case it seems it was due to stupid me messing up cable management, but I'm very happy knowing if the disk had been genuinely faulty, services would have continued just fine.
    • -
  • Physical connections matter: It's easy to not pay that much attention when building a new box. Well, it bites back.
    • -
  • Monitor your pools. I only found this issue by accident, clicking around in the Proxmox UI. The pool had been degraded for who knows how long before I noticed. I'm already working in setting up a monitor to my Uptime Kuma instance so that next time the pool status stops being ONLINE I get notified immediately.
    • -
-

I'm happy I was able to test out recoverying from a faulty disk with such a tiny issue. I learned a lot fixing it, and now I'm even more happy than before having decided to go for this ZFS pool setup.

-

Quick Reference: The Commands

-

For future me (and anyone else who ends up here with a degraded pool):

- 
# Check pool status
-zpool status -v 
-
-# Watch kernel logs in real time
-dmesg -Tw
-
-# Check SMART health
-smartctl -H /dev/sdX
-smartctl -x /dev/sdX
-
-# Take a drive offline before physical work
-zpool offline  
-
-# Bring a drive back online
-zpool online  
-
-# Clear error flags after recovery
-zpool clear  
-
-# Run a scrub to verify all data
-zpool scrub 
-
-# Run SMART self-tests
-smartctl -t short /dev/sdX  # Quick test (~1 min)
-smartctl -t long /dev/sdX   # Full surface scan (hours)
-smartctl -l selftest /dev/sdX  # Check test results
-

Thanks for reading! This was Part 3: The Fix. You might also enjoy Part 1: The Setup and Part 2: Diagnosing the Problem.

-

back to home

-
-
- - - - - diff --git a/public/writings/gresham-law-has-nothing-to-do-with-bitcoin.html b/public/writings/gresham-law-has-nothing-to-do-with-bitcoin.html deleted file mode 100644 index ba5c4ee..0000000 --- a/public/writings/gresham-law-has-nothing-to-do-with-bitcoin.html +++ /dev/null @@ -1,139 +0,0 @@ - - - - - Pablo here - - - - - - -
-

Hi, Pablo here

-

back to home

-
-
-

Gresham's Law has nothing to do with Bitcoin

-

Published: 2025-05-26

-

- This is going to be a thorough explanation for a simple thing, but we - will take it slow since this topic somehow causes loads of confusion. -

-

- Okay, so there are a lot of people in Bitcoin circles who talk about - Gresham's - Law. They often say, “Gresham's Law states that bad money drives out - good money”, then relate it to Bitcoin and the USD, and finally - proceed to reason all sort of of things on top of that. But here's - some very much needed clarification: Gresham's law has nothing to do - with Bitcoin's relationship to the USD. In fact, it actually has - nothing to do Bitcoin, or with the current USD for that matter. -

- -

- Gresham's Law is relevant to a very specific type of monetary system: - when we used coins that contained precious metals (spoiler: we don't - live in that period of history anymore). The law states that bad money - drives out good money, but what a lot of Bitcoiners seem to miss is - the actual meaning of “good” and “bad” in this context. People tend to - interpret “good” and “bad” as meaning “hard” and "easy" money, so they - reason something like: “Because Bitcoin is harder than the USD, - Gresham's law applies here.” But that is not what Gresham's law is - about at all. -

- -

- In the context of Gresham's law, “good” and “bad” refer to face value - versus commodity value. That doesn't ring a bell? Let me explain: -

- -

- Imagine a magic land where there is only one type of coin. There's no - other money — just this one coin. These coin states on themselves that - they contain one gram of gold, and right now, they really do contain - one gram of gold. Everyone uses it, and everyone is happy. There's no - “bad” money, no “good” money — it's all nice and simple. -

- -

Now, let's spice it up a bit.

- -

- After some time, a cheeky bastard (typically, a king) comes along and - starts making coins that look exactly like the original coins. I'll - call these the bad coins. The original coins will be the good coins. - Both types of coins say on them “one gram of gold,” but the bad coins - only have half a gram of gold actually in them (hence why they are - bad). -

- -

- So, to recap:
- - Good coins: one gram of gold on the coin, and actually one gram of - gold inside.
- - Bad coins: one gram of gold on the coin, but only 0.5 grams of gold - inside. -

- -

This is where Gresham's Law applies.

- -

- People in this coiny fantasy land are not stupid — they know that the - gold content is what matters. At some point, someone will realize the - bad coins don't have as much gold as they claim and will develop a - preference for the good ones. So, if I'm John the Blacksmith and I - want to buy some iron, and I have a stash of coins — some good, some - bad — I would rather keep the good coins and spend the bad coins. Why? - Because I want to keep as much gold as possible, of course. -

- -

- What happens eventually is that people grow into the habit of trying to get - rid of the bad coins and hold on to the good coins. They exploit the - confusion created by the fact that all coins have the same face value - (it says “one gram” on all coins, so everyone assumes they're worth - the same), even though the actual commodity value (the gold inside) - differs.[1] -

- -

That is the quick explanation of Gresham's law.

- -

- Now, back to the original point: what are the face value and commodity - value of Bitcoin? -

- -

- That makes no sense! Bitcoin is not a physical coin with metal in - it. It has no concept of face and commodity value. And neither does the - USD nowadays. Therefore, Gresham's law has absolutely nothing to do - with Bitcoin, the USD and any preferences the world might develop - between the two. -

- -

- Hopefully, this explanation helps make things clear. From now on, if - you want to keep your public image intact, please refrain from - invoking Gresham's law when discussing Bitcoin and USD — because doing it - shows you don't know what Gresham's Law is actually about. Don't feel - too bad if it happened to you though: it can happen even to - massive - exchanges with a great reputation. -

- -

- [1] Not relevant to the point of this post, but it's worth noting - that Gresham's Law situation is not always sure to happen in the - described scenario. If the difference between the good and bad coins - is massive, and no force opposes it, the market might jump into - Thier's Law - instead. -

-
-

back to home

-
-
- - - \ No newline at end of file diff --git a/public/writings/how-i-write-some-articles-im-lazy-about.html b/public/writings/how-i-write-some-articles-im-lazy-about.html deleted file mode 100644 index a2c1af9..0000000 --- a/public/writings/how-i-write-some-articles-im-lazy-about.html +++ /dev/null @@ -1,107 +0,0 @@ - - - - - Pablo here - - - - - - - -
-

- Hi, Pablo here -

-

back to home

-
-
-

How I write some articles I have a hard time getting started with

-

Published: 2026-03-02

-

- I have a lot of shower thoughts. Way more than I can handle. Many times I feel like some of them - would make for a nice piece on my webpage, or I should make a personal note out of. But often times - they won't catch me in the right place and time to sit down, focus, and type them out. You have to - figure out the key points, lay out a plan, think about what you want to say and how, type it all - out, work on the text... It burns some calories. And sometimes it's enough friction to kill the idea - entirely. -

-

- Recently I found a set of AI tools that allow me to capture some of these ideas and lower the - friction to get them on ink. My goal is not to have the AI write things for me, but just to quickly - drop a pretty decent first draft with very little effort, on the spot. With this, I've happily - rescued many good ideas from getting by falling through the cracks before I even hit my desk (or - getting ignored because I don't have the energy to go through it all). -

-

Step one: recording myself

-

- I'm self-hosting an open source audio transcription app called - scriberr. - It's just a fancy little webpage where you can drop audio and get a transcript. The neat bit is that - it also allows you to simply record on the spot. I used to host - Whishper, - but you had to first record then upload a file. I really liked it, but sadly I found that was - enough friction when rushing on the phone to not use it. I can access scriberr from my laptop, my - desktop, my phone, from anywhere basically. -

-

- So anytime I feel like it, I can pull out my phone and start ranting about whatever topic is on my - mind. The recording gets sent to my server back home, and scriberr transcribes the whole thing into - text automatically. Transcription itself takes a bit because I'm not using a GPU, but I'm not in a - rush usually since the important thing is just to get the ideas out of my brain and into text easily. -

-

- The transcript that comes out is usually quite decent in terms of accuracy. I'd say around 95% of - what I say gets picked up perfectly. The whisper models also do pretty decent in different - languages, so I can record in whatever I feel like at each moment. -

-

- What's not great, in case you've never used transcription tools, is that the output is just a long - stream of words. The model tries its best at punctuation, but it's rather crappy. And obviously, no - paragraphs. Plus, with unprepared, spontaneous ranting, structure tends not to be top notch either. - Definitely not article-grade text out of the box. -

-

Letting an LLM clean it up

-

- But hey, those are not issues now that we have LLMs! I have a little script that fetches the - transcript results from my server to my laptop. Once I have my raw transcript locally, I just pass - it on to Claude, with a little prompt saying something like: "This is a transcript of me talking - about this and that. Process it for me, I want the output to be like XYZ." -

-

- Sometimes I really just ask to have the transcript nicely formatted into paragraphs and proper - sentences, with the actual sentences being totally respected. Other times I already ask for some - restructuring of the ideas, so it's not just cleaning up the writing but actually shaping it into - something that resembles an article. It depends on how clear I was during my rant or what I'm really - planning to get out of it. -

-

- After this bit, I'll either abandon the idea altogether because it wasn't as interesting as I felt - initially, or I'll really work properly in the text when I can and come up with something I'm happy - with. It will still take some desk time to get to a final result, but then the article is really my - writing and not a mix of my own slop with a layer of LLM slopification on top of it. I can't help - but think people who just copy paste LLM output and put their signature under it have little respect - for themselves and their reputation, and little love for the act of thinking and writing. -

-

A great starter

-

- And there you go. In just a few minutes, without having to focus deeply in front of a blank file, I - end up with a first version of my thoughts that I can already start polishing and thinking from. Not - all transcripts make it into clean texts in the end, but at least I ensure some of the ones which - would otherwise get lost do survive. The stuff that comes out of the LLM is not always great, to be - honest. Sometimes I'll change a lot of things from what I said in the audio. Sometimes I'll end up - adding a lot of stuff that I wasn't covering in my original rant. But the first draft helps me get - my thoughts out there and gives me something to begin with. -

-

- And that's often all I need. -

-
-

back to home

-
-
- - - - \ No newline at end of file diff --git a/public/writings/i-want-code-defined-dashboards-so-badly.html b/public/writings/i-want-code-defined-dashboards-so-badly.html index 3066357..e898c06 100644 --- a/public/writings/i-want-code-defined-dashboards-so-badly.html +++ b/public/writings/i-want-code-defined-dashboards-so-badly.html @@ -18,7 +18,6 @@

I want code defined dashboards so badly

-

Published: 2025-01-14

Analysts build dashboards. These are also called reports, data tools, data products, and another gazillion funny names.

diff --git a/public/writings/if-i-started-a-data-team-again.html b/public/writings/if-i-started-a-data-team-again.html deleted file mode 100644 index 2e9e479..0000000 --- a/public/writings/if-i-started-a-data-team-again.html +++ /dev/null @@ -1,151 +0,0 @@ - - - - - Pablo here - - - - - - - -
-

- Hi, Pablo here -

-

back to home

-
-
-

If I started a Data team again

-

Published: 2025-02-04

-

- In November 2023, I joined Truvi (back then called Superhog) as the first - member of its new Data team, effectively making my hiring and the team's birth the same thing. The CEO - and COO brought me in to help a young, upcoming and quite chaotic UK SaaS company make it out of the - void of Data darkness, where no one knew shit, figures came in with three months of delay and VLOOKUP - was the closest thing there was to integrating data from two systems. -

-

- The team is nowadays very much established and critical for the business. We've built our spot within - the org, and many colleagues wonder how life could ever exist before. We can tell many tales on - delivering value to Truvi, and we have ambitious plans to keep doing more and better. And I'm having a - blast leading it. -

-

- For me, it was quite a journey: it was the first time I started a Data team from a greenfield situation. - Some day I'll make a longish writing on the whole experience, but today, I wanted to focus on a few - regrets I have. I personally think I've done a good work, I'm proud of where the team is at today, and - the words (and actions) of the company board show that they are aligned with this. But still, there's - always room to fuck up, and I do have a few parts of the story where, looking back in retrospective, I - think I took the wrong turn. -

-

- The first thing I regret is not hiring faster which, taking personal responsibility, means I didn't - focus enough on it and I wasn't as effective as I should have been. When I started the team, I quickly - secured consensus from the founders to go hire two more members for it, and we all agreed it made sense - to spend those bullets to find two Data Analysts. We found Uri and Joaquín, which are still with us today doing a - terrific job. -

-

- The bad news is they joined in May 2024, which means it took a whopping ~6 months from decision to - result. It didn't feel terrible at that moment. I was very much busy with the onslaught of work that - comes with kickstarting the team (meeting everyone, aligning with other teams, designing and starting - out infra, doing basic, urgent starting deliveries, etc.). But now I realise some of those things would - have been easier to do with the guys already around. So there goes regret number one. Lesson: if you - have agreement to get more hands on the team, make it priority #1. -

-

- The second thing I'm not happy about is not completely dropping and rebuilding a few legacy bits I - inherited when joining. Truvi had an insane amount of unmet needs in data and reporting right before I - joined, so a bit of capacity from the development teams was derailed at some point to create a couple of - basic reporting and data exporting tools to support other teams. Those were small and done in a rather - crude way, as a result of (1) being done by backend guys with no previous experience in the Data domain, - and (2) in parallel to other very important product development work. I don't blame them, I assume they did the best they could given the circumstances. -

-

- My mistake was to decide early to inherit and continue those, instead of bulldozering them and building - things from scratch with better tools and practices. It would have been quite easy to do at the time - given that those data products were rather small (and coming back to my first regret, it would probably - have been even easier to bulldozer them if the new hires would have been around already). At the time, I - traded off leaving those be and have a continuist approach to building incrementally in order to get - going with some other fresh new scopes faster. I'm not fully sure that the call was wrong (perhaps in a - parallel universe, I would think we shouldn't have rebuild those from scratch because it delayed - delivering other important things). But I do feel it was wrong right now. As a consequence, today we - have several data products and architectural constraints which are a royal pain in the ass to maintain - and grow with the company. For example, we're currently stuck with a lot of reporting in Power BI, which - I hate for multiple reasons, one of them being how badly I want to have code defined dashboards. - Bulldozering and replacing now will be much more painful than it would have been back then, so the - velocity of the team is now paying for this mistake. -

-

- There goes another lesson: if you're inheriting a few, small legacy products that don't fit in your view - and you can afford to remove or re-build them to allow your plans to be pristine, do it now. Don't wait. -

-

- A third thing I would have done differently is to start working on Data literacy across the company - earlier and more intensely. In part of the work we do in the Data team, we come in with a very polished, - mature result: this is exactly what you need to know, or the exact pristine, read-only grade data you - had to check, or the informed decision you are after. But in many other cases, our delivery ends at the - start of what I like to call the analytical last-mile: we produce some curated piece of insight and/or - data that a colleague will grab and work a bit more before getting to the final business outcome. For - example, I might export a set of KPIs around certain client accounts, and an account manager will pivot - and fuck around with them to decide certain things around how he will handle some comms with his - accounts. -

-
- -
-

- This kind of two-step deliveries sometimes are insanely valuable: there are some analysis where - the final user of the data is the most capable of juicing the right way, such as when someone takes - the data to use it in real time in a negotiation meeting. -

-

- The bad news is that, if John from Marketing fucking sucks at Power BI, Excel, or whatever tooling you - rely on to interact with your colleagues, the whole plot falls down. I'm not expecting John to crank out - a monster workbook with thirty layers of business logic and seven modules of VBA, but pivoting a bit, - filtering, multiply this col by that row, etc. You get the gist. If you're thinking to yourself: "anyone - can do such basic things!", I would kindly invite you to sit down 15 min with some rando in your company - and see them use Excel live, potentially having chunked some tranquilizer down. -

-

- Training more colleagues outside the Data team to be more proficient working with data (let it be - working on Excel, writing SQL or just reasoning) is something we're actively working on now, but I think - we should have started out earlier. I think this because of two reasons: the first, the ROI, measured in - time, is tremendous. 5 hours of working with Jane from Finance to skill her up from 0 to Excel basics - are probably much more impactful than that extra dashboard you are building, which you already sense - won't be that used or valuable. The second is, this is one of the things that has a capped max pace to - it, and won't happen faster than that no matter how hard you would like it to, or how many resources and - money you want to throw at it. People learn at a certain pace, so it's better to start early and slow, - than to trick yourself into believing you'll be able to turn Jimmy into a Power BI God in 2 days if you - try hard enough when the time comes. -

-

- So, final lesson: I would start running Data literacy initiatives early. It doesn't need to take a big - chunk of your capacity: sprinkling a few open sessions here and there, and running some 1:1 or small - group ones with bright people, should be more than enough. It will compound over time, leading to the - company better leveraging your data and tools, and the central Data team having more capacity to keep - building since more end-users will be more independent. -

-

- There goes my non-exhaustive list of things I would do differently when starting a Data team from - scratch. I hope it serves you if you are in a similar spot. I'm personally delighted with the idea of - not screwing up in these ways if I ever find myself starting another team from scratch. I would very - much rather screw up in new ways. -

-
-

back to home

-
-
- - - - \ No newline at end of file diff --git a/public/writings/is-your-drug-dealer-a-homophobic-socialist.html b/public/writings/is-your-drug-dealer-a-homophobic-socialist.html deleted file mode 100644 index 5a5650a..0000000 --- a/public/writings/is-your-drug-dealer-a-homophobic-socialist.html +++ /dev/null @@ -1,95 +0,0 @@ - - - - - Pablo here - - - - - - -
-

Hi, Pablo here

-

back to home

-
-
-

Is your drug dealer a homophobic socialist?

-

Published: 2025-06-03

-

- Lately, I've noticed a branch of - cancel - culture - I've come to find quite disturbing. I think it has mainly extended in - the US, though I think it's starting to happen in Europe too. It's - this tendency for people at companies to politically and morally judge - business counterparties and come to the conclusion that business - shouldn't be done with them because of it. -

- -

- I experienced this first hand during some afterwork beers, and for - some reason the scene got burned into my retina. A colleague of mine, - beer in hand, said something like, “We're working with this customer, - and they're unbearable because they complain a lot and challenge us - all the time when we run the monthly reconciliation. Plus, they're - from Israel.” I was mindblown at how casually that was dropped, with - not even a footnote-like explanation deemed necessary. I played my 5 - year old child attitude card and asked, "What's the problem with them - being in Israel?" She said, "Well, you know, they're in Israel and the - whole thing is happening. It's terrible. We shouldn't deal with them." -

- -

- I couldn't hold it in: I asked her if her hairdresser was from Israel. - She looked at me completely puzzled: “I don't know. Why does that - matter?” I told her, “I don't know. Apparently, you're upset about - dealing people from Israel, so I'm assuming you need to check if - everyone you do business with is from there to not do it if that's the - case.” Silent stood and the air got thick. Someone jumped in with a - nervous joke to break up the tension that my child like questions had - somehow brought to the room, and the conversation moved on. -

- -

- Ever since that day, I've come across this kind of - social-justice-business-censor thinking pop up a lot. Since that fun - first encounter, whenever someone points out at how business should - not be done with <whatever ideology/country/demographic they don't - like>, I started jokingly triggering them by asking, “Actually, are - you making sure your drug dealer a homophobic socialist?” They - generally laugh, not grasping how their stances on politically - deciding to do or not do business with someone sound as ridicolous to - me. -

- - - -

- Here's what disturbs me: trade is a very civilized act. When we - trade—whether it's goods, services, or anything else—we're putting - aside our differences and doing something mutually beneficial. We both - walk away better off. We hurt no one. We make things a tiny bit better - overall. Deciding not to trade with someone because of some political - detail which is completely irrelevant to the trade itself is - backwards. Even if I didn't like communists, I wouldn't care if a - communist is selling me bananas. It just doesn't matter. -

- -

- Seeing people blow up trade over politics makes me sad. I think it's - ignorant and hateful. And I don't think they realize where that kind - of thinking can lead. -

- -

- In the end, I just hope people can leave politics out of business. - Let's do business and all be better off thanks to it. -

-
-

back to home

-
-
- - - \ No newline at end of file diff --git a/public/writings/my-first-petahash.html b/public/writings/my-first-petahash.html deleted file mode 100644 index dec24bd..0000000 --- a/public/writings/my-first-petahash.html +++ /dev/null @@ -1,128 +0,0 @@ - - - - - Pablo here - - - - - - - -
-

- Hi, Pablo here -

-

back to home

-
-
-

My first petahash

-

Published: 2026-04-16

-

I've recently started mining Bitcoin at a scale I never had before, so I thought it would be interesting - to jot down a few observations on my recent errands.

-

My friend Unhosted Marcellus has been following closely the evolution of the - OCEAN mining pool since its launch. I hadn't used it personally until - recently, so for years all the info I had on it was second hand. What he was most excited about was the - DATUM Gateway: the great innovation is that you are - building your own block templates, which is something no other pool does. By using OCEAN with DATUM, you - enjoy the benefit of more stable mining rewards as opposed to lotto mining, while still being a sovereign - miner in the sense that you rely on your own node and you do your own templating. Great news for - decentralization.

-

The reason I had not bothered with setting all of this up so far was... that I really don't mine much. I - got gifted a cute Bitaxe - Supra from the Bitronics Shop that produces some humble - 600GH/s, so it felt pointless to do all the setup for such a tiny hashrate.

-

But then, Unhosted Marcellus started to tell me about these new markets started by Braiins called - Hashpower. Other articles explain the market better, so - I'll leave it up to you to find those to learn about it. Although I must say, if learning is what you - want, nothing beats using it. The TLDR is that you can sign up, send sats, and rent hashrate that you - can point to your own DATUM gateway. And the surprise (at least for me) is how you can literally rent - petahashes for peanuts, when you account for the fact that most of the sats you put towards buying - hashrate will come back as mining rewards.

-

Unhosted finally triggered me with this tweet. Cheeky bastard.

-

I started toying around with a few PH/s, eventually trying out double digit petahashes. There is this - funny feeling to suddenly be controlling the equivalent of tens of thousands of little bitaxes.

-
- Double digit petahashes on OCEAN -
Double digit petahashes on OCEAN
-
-

The economics around it are interesting. The bidding prices in Hashpower are usually (not always!) above - hashvalue. It's common to pay a 1%-5% premium over hashvalue. So, the most probable thing is that you - end up operating at a small loss. This is not strictly guaranteed if you mine with OCEAN, since the luck - factor is important and can easily swing rewards +-10%. So unless you mine at a stable rate with a - months-long time horizon, luck is going to play a more important role than the premium on the - hashrate.

-

To optimize your outcome, it is important to constantly update your bids in Hashpower. Bids are set at a - fixed price in sats, so as the market auction moves every few seconds, you will be either overpaying or - end up unserviced.

-
- Hashpower order book -
Hashpower order book
-
-

On the first days I was using Hashpower, I would log into it multiple times a day to adjust my bids to - stay at the right height of the order book. At first it was fun, then it felt tedious, and it started to - generate this Twitter-esque addiction feeling I didn't like. I quickly concluded I wanted to automate - this out so my only task was to contemplate how pretty my OCEAN hashrate dashboard looked like, and I - could leave behind pulling levers in Hashpower's webpage like a financial monkey.

-
- Dreaming of automated bids -
Dreaming of automated bids
-
-

I solved this problem for myself with - hashbidder. It's a small CLI tool - that I run every couple minutes with cron. The TLDR is you can give it a config file that reads "I want - to mine at 5PH/s" and the tool will set your bids with two goals in mind:

-
    -
  • To bring your hashrate in line with your goal (e.g. if you want to be at 5PH/s, and you're - currently averaging 3PH/s in the last 24 hours, it will drive your bids to a total of 7PH/s. If - instead, you're scoring an average of 10PH/s, it will stop your bids completely to let your average - go lower).
    • -
  • To pay as little as possible, but guarantee you get served. The logic here is to set the price right - above the cheapest bid that is being served currently.
    • -
-

The result is quite pleasant. Delivery is choppy because, even with frequent updates, trying to be cheap - means you often get dragged into being overbid by others and you stay there for some minutes. But the - self-adjusting hashrate compensates for it: if you've been falling behind a lot recently, hashbidder - will just hash at a higher hashrate to make up for it. I'm currently targeting 5PH/s, and this is what - my OCEAN hashrate timeline looks like.

-
- OCEAN hashrate timeline -
OCEAN hashrate timeline at ~5PH/s target
-
-

There are still a few more optimizations I'll add to hashbidder to reduce cost and decrease the - volatility of delivery, but they're just marginal improvements. The gist of it is already there and it's - doing its work fine.

-

My next steps are simply to sit and watch. I've decided I will pour 10 million sats during a few months - into this setup and then stop to measure what my rewards have totalled to, so I can provide people - interested in this with a real-life report of how everything turned out.

-

Overall, I'm having lots of fun. Setting this up made me excited in a way that felt oddly similar to the - first time I was setting up lightning nodes. The night I started out my DATUM gateway and pointed some - hashrate to it felt like the night I spun up an LND and started doing some lightning triangles in - Lightning Network+.

-

Some interesting links in case you want to learn more or give it a shot at mining with rented hash - yourself:

- -
-

back to home

-
-
- - - - diff --git a/public/writings/my-fitness-journey.html b/public/writings/my-fitness-journey.html deleted file mode 100644 index cacd9a3..0000000 --- a/public/writings/my-fitness-journey.html +++ /dev/null @@ -1,203 +0,0 @@ - - - - - Pablo here - - - - - - - -
-

- Hi, Pablo here -

-

back to home

-
-
-

My fitness journey

-

Published: 2026-03-03

-

Nowadays I find myself in the best physical shape I've ever been at. I'm stronger than I've ever been. - Pretty decent cardio-wise, not at my peak but close. I'm overall very active, and I can jump into any - physical activity without having second thoughts. I sleep like a baby, perform like a machine, eat like a - pig and it's all good. I'm at a very good weight. And there are so many other things in my life that work - nicely because my body is working nicely.

-

I'm very happy about this because, for a long time, I really didn't think I would find myself in this - situation. Historically I was not an active person at all. It took many years of back and forth, making - mistakes and trying things, to get my groove in and settle for the habits I have now. And those habits - are great because it's literally costing me no effort to maintain myself in this state.

-

So I thought I could look back a little, reflect on how I got here. Perhaps you can learn from some of - the mistakes I made and some of the things that worked for me.

-

Growing up

-

When I was a kid, I was a normal kid. I liked to play and have fun, but I was pretty terrible at sports. - Not a cripple, but surely a rather clumsy kid. I wasn't really talented for most sports, usually falling - in the bottom 25% of the distribution in terms of overall skill and performance.

-

I guess part of that was just genetics and natural talent, and part of it was simply that my parents were - not the most physically active people. They never engaged in any sports, never did any physical exercise - or went to the gym. So I had no input from them in terms of physical activity or motivation around that - area. I would just play because I was a kid, not because of my parents pushing me to go for it. As I - grew up and playing slowly became less of a thing due to age, sports also slowly faded from my life (and - I was quite happy about it: it's not fun to be the loser systematically).

-

The only exception in this path is that I did do a couple of years of martial arts when I was around 11 - or 12. That was nice, it was a lot of fun and it helped me boost my confidence. And surprisingly, I got - kind of decent at it. Being used to the bottom 25%, just being average at something felt like a massive - success. But then I went into my teenage years and my head was quickly filled with other stuff. I - eventually dropped out of it.

-

The dark ages

-

During my teenage years, I didn't do shit. I played no sports nor trained my body in any way. I don't - think I ever tried to do anything until I was 18 or 19, already in university. I signed up for the gym a - couple of times, checked online for routines, downloaded them, tried to follow them. Would usually last - two or three months and then drop out. It was boring, not motivating, and I also had no clue what I was - doing. When I look back at how I was training at those times, I can see so many mistakes. I would try to - go too hard, have overly ambitious routines that were completely not sustainable. Nutrition wasn't in - place, technique wasn't in place, exercise choices were probably extremely poor.

-

I also tried jogging for a bit, but similar experience. Would do it for a while, then eventually lose - consistency. On and off for years.

-

When I was around 20 or 21, I was living in Germany and really short on money, so signing up for a gym - was not an option. But near where I was living there was a park with some bars, and some days there was - this German fella, I think his name was Leo. He was insane good: super ripped, in extremely good - condition, did all sorts of complicated exercises like it was nothing. The kind of guy you see training - and go "oh wow, that's amazing." He would give me some tips and I really enjoyed training around there - with him. Still, I wasn't extremely consistent. But it was a good intro to doing calisthenics in a bar - park, since the closest I had done was bodyweight training when I did martial arts in my younger years. -

-

And in that park I fucked up my ribcage doing dips on the parallel bars. One day I went too hard, and I - would guess the technique must have been absolutely terrible as well. That day, after finishing a set, I - just felt this little stab in my chest, which I didn't give much importance to on the spot. Then the next - morning I had super intense pain in my chest every time I tried to move or breathe. I'm pretty confident - I searched online whether I was having a heart attack. It slowly faded but not completely, and it stuck - around for at least a couple of years. That was my first big fuck-up with training and injuring myself. - After that, I learned not to exceed myself with risky exercise like dips, to respect my natural ranges of - motion, and to just take it easy for the most part.

-

That chest pain got me away from exercise for a long time because any chest exercise would hurt quite a - bit. Push-ups were a no-no. And how's a man supposed to train if he can't do push-ups, right?

-

Some progress, then rock bottom

-

Around 23 or 24, I went a bit harder with running. Started jogging more regularly, signed up for a 10k, - did a decent job, ran a couple more races, and eventually finished a half marathon in under two hours. A - year or two where I was jogging regularly. That was nice. I eventually lost my habit, but it was a good - experience and there I learned how to do resistance training decently. Also proved myself I was capable - of training for a feat like a half marathon, which was nice since I would have never identified myself as - a runner before.

-

Then, by the time Covid hit, I was not active at all. Absolutely zero gym, calisthenics, no jogging. And - then we were thrown in our cages with the curfews. Being locked down all day was absolutely terrible. I - wouldn't move around. I was working quite hard at my desk, long days of sitting. My back was hurting like - I was a grandpa. I was smoking a pack of cigarettes a day. Probably drinking more than I should. For a - few months it just spiralled down into some mornings where I would wake up and go "god, I feel like shit - every single day."

-

Actually, let me backtrace a little. A few months before Covid hit, I had a traffic accident and broke my - leg. Had to get surgery on my knee. I was bedridden for a couple of months, then had to slowly learn how - to walk again. It's hard to explain how bad it is to be bedridden for that long.

-

So Covid hit on top of that, and I just went worse and worse. Maybe one year, one year and a half into it, - I was in absolute shit shape. At that point, it started to click in my brain that this was not normal, not - good, and not something I wanted. I guess sometimes you really need to lack something to truly appreciate - it and feel the need for it. Greener grasses, yadayada.

-

The turnaround

-

First thing, I quit smoking. One morning I woke up with my usual chest congestion, you know, where you - have to go to the bathroom and spit the hell out of yourself for five or ten minutes. All morning I was - just feeling it and I was like "I'm so done with this." I quit cold turkey that day, never went back. - Took no effort. Since then, I always advise smokers who want to quit to consider as a possible strategy - to just smoke themselves out. Smoke every single minute, as much as you want. Heck, smoke even when you - don't want to. Maybe that way you'll just get sudden rejection for it like I did.

-

Then I started training a little at home with a few dumbbells. Basic stuff: rows, squats, floor presses. - And I took it really easy because I was in such terrible shape and I had learned from my leg surgery - rehab that when you're at the bottom, you really have to take it easy coming up. That was one of the most - enlightening things I figured out. When you're in a really bad shape, don't rush it. You will go up - eventually and things will work out, but don't force yourself to do stuff you're not ready to do. Do - something that feels comfy, even trivial I would say, and start doing it regularly. From that point on, - you can start pumping the numbers.

-

I was extremely busy with work, working from home, so what I would do is train during my lunch break. 15, - 20 minutes. Four exercises, a few rows, a few presses, a few squats, maybe just nine sets, done. Cook my - lunch, continue with life. Very humble training for a busy guy.

-

Precisely because what I was doing was not ambitious, it was pretty convenient. We're talking about - working out for 15 to 20 minutes in a lightweight way at home. I didn't change clothes. I didn't get in - my car and drive to a gym. I would sometimes cook as I was training, leveraging the rest time between - sets. It was extremely convenient and humble, and because of that, I was actually able to make a habit - out of it. One month, two months, three months. Eventually it just grew into a habit. I stayed with this - attitude for at least a year. That was the first time in my life I was actually consistent with strength - training.

-

The habit stuck and slowly, I increased the intensity, but with an extremely relaxed attitude. I wasn't - tracking myself heavily. I wasn't pushing hard to pump the numbers. I was just happily trying to feel - well and then every now and then I'd throw another half a kilo here, another half a kilo there. Do 4 - sets where I used to do 3. Do 8 pushups instead of 6.

-

I also started jogging casually, doing 5k's here and there. Same principle: very humble jogs with no - intense pace goals, trivial to include in daily life.

-

Calisthenics and the bars

-

Eventually I started working for a company with a nice office close to the beach, literally a few meters - from the sand. Right in front of the office there was an absolutely amazing calisthenics bar park, with - all the pull-up bars you could figure out, all the grips and heights, parallel bars, row bars, everything - you can imagine. Training in the sand was really nice.

-

I made a habit of training whenever I went to the office. Sometimes in the morning, sometimes at the end - of the day. I would train at the bars regularly and some days go for a 5k near the beach. I just kept my - sneakers at the office and grabbed them whenever I felt I had the energy.

-

At this stage I had clearly recovered all of my shitty physical condition from the curfew, smoking and - drinking. By the way, by the beach stage I had already stopped drinking as well. I've been smoke-free - and alcohol-free for a few years now.

-

Things kept improving. I was feeling so great that I could actually be a bit more ambitious with my - training. And it just worked. It didn't feel like I was pushing myself too hard and I didn't have weird - pains. Also, because I was in decent condition and not smoking or drinking, I was getting incredibly good - sleep, so recovery was much better. I also lost quite a bit of weight due to it all (though most of the - merit I think goes to quitting alcohol completely).

-

Eventually, I kind of got addicted to the bars. The habit had completely stuck with me. Since then, I - train at different bar parks depending on where I'm working or where life takes me, but I'm always - finding some bar park and going there 2-3 times a week. I even got a weighted vest, so some training - sessions I'll go quite hard with heavy pull-ups and dips.

-

And that's where I sit today. I'm stronger than I've ever been. I can crank out 10 pull-ups any day of - the week when I'm fresh, or squat cleanly with the missus on my back. And what I today call an easy day - would have been outright impossible for me to perform in 2020. That's pretty cool.

-

I'm not doing great with jogging right now because I stopped for a few months, tried to pump the numbers - too quickly when I came back (lesson learned), and hurt my Achilles tendon. Working on recovering that, - and then I'll just be very humble again with my 5k's. But right before that, I could run you a sub-hour - 10K on any normal day, even if I had not been prepping intensely for it the weeks before.

-

The only thing I might be missing is some stretch work for flexibility and mobility. I've been thinking I - should probably try some yoga classes. Not to become a master at it, but to have my joints pushed to the - extreme positions every now and then. I'll probably do that at some point.

-

Lessons

-

If I look back, here's what I would tell my past self, or anyone who's starting out or struggling to make - exercise stick.

-

The most important thing is to start ridiculously small. So little volume, so little frequency, that it - almost feels pointless. A couple of days a week, five minutes of something that doesn't even feel like a - challenge. You might think that's not getting you anywhere. You're so wrong. Listen: first, you need to - prove yourself that you're capable of doing something trivial consistently, because doing something - consistently is all that matters. Once you prove you can do that, slowly increase, but very slowly. - Don't be in a rush. Don't try to make every session a challenge. You have a lifetime of exercising ahead - of you. Get to a certain level, stay there for a few weeks until you're bored, and then bump it up a - little bit. So little you have a hard time noticing any difference. And then repeat. Same for cardio. - Never jogged? Just go out and do two kilometres. You should be able to pull that in under 15 minutes. Do - that for a month, a couple of times a week, then do 2.5 kilometres. Take years to get to something - significant. But eventually you will get there, which is the important thing. And you'll make it because - you took it easy, you didn't injure yourself, and you didn't give up because your training plan was too - hard and completely incompatible with living a normal life. It feels counterintuitive, but by being kind - of lazy and doing very little, you eventually grow into doing a lot.

-

Along the same lines, make it trivial to start exercising in your daily life. Remove every bit of - friction you can. If you can afford to have equipment at home, have it ready and accessible, not buried - in some inconvenient closet. If you just have to reach for it and start, there's no excuse. The moment - it takes a couple of minutes to prep for exercise, you can talk yourself out of it. Same for jogging: - have your sneakers and clothes ready to grab, have one route you always do, don't plan it, don't think - about it, just go. Calisthenics bar parks have worked nicely for me precisely because they need zero - prep. You show up and the bars are there, ready for you to jump to them. And don't shy away from short - workouts either. In 15 minutes at a bar park you can do a lot. You're not going to break any records, - but it's way better than doing nothing. I never do long strength training sessions. I'm pretty confident - 80-90% of my workouts sit somewhere between 15 and 30 minutes. You can get quite decent results with - that. And if someday you have the time and energy to punish yourself for an hour, be my guest. But don't - feel you have to.

-

Finally, get your weight sorted. Being at a good weight is insanely convenient for exercise. Running, - lifting, all of it. Being overweight is a fucking pain in the ass, messes with your health in many ways - and just doesn't let your body work properly. If you're clearly overweight, getting your weight down - before you start training hard will make your life a lot easier. Things like running are terribly tolling - when you're heavy, whereas at a healthy weight it feels so much more pleasant, which helps you stick - with it.

-

That's it. It took me many years to get here, but I'm rolling with it. I'm having fun with the bars, - I'll be having fun jogging with no pressure very soon, might throw the yoga in. I keep it easy. Whenever - I feel like I have some extra energy and bandwidth, I push myself a bit harder for fun. Not on a - schedule, not with any pressure, just because I feel like doing it. Whenever I seem to have no energy or - time, I just do what I can afford to. But the key is: I always do a bit, no matter what.

-
-

back to home

-
-
- - - - \ No newline at end of file diff --git a/public/writings/my-tips-and-tricks-when-using-postgres-as-a-dwh.html b/public/writings/my-tips-and-tricks-when-using-postgres-as-a-dwh.html deleted file mode 100644 index 5e335ce..0000000 --- a/public/writings/my-tips-and-tricks-when-using-postgres-as-a-dwh.html +++ /dev/null @@ -1,174 +0,0 @@ - - - - - Pablo here - - - - - - - -
-

- Hi, Pablo here -

-

back to home

-
-
-

My tips and tricks when using Postgres as a DWH

-

Published: 2025-04-24

-

In November 2023, I joined Superhog (now called Truvi) to start out the Data team. As part of that, I - also drafted and deployed the first version of its data platform. -

-

The context led me to choose Postgres for our DWH. In a time of Snowflakes, Bigqueries and Redshifts, - this might surprise some. But I can confidently say Postgres has done a great job for us, and I can even - dare to say it has provided a better experience than other, more trendy alternatives could have. I'll - jot down my rationale for picking Postgres one of these days.

-

- Back to the topic: Postgres is not intended to act as a DWH, so using it as such might feel a bit hacky - at times. There are multiple ways to make your life better with it, as well as related tools and - practices that you might enjoy, which I'll try to list here. -

-

Use unlogged tables

-

The Write Ahead Log comes active by default for the tables you create, and - for good reasons. But in the context of an ELT DWH, it is probably a good idea to deactivate it by - making your tables unlogged. Unlogged - tables will provide you with much faster writes (roughly, twice as fast) which will make data - loading and transformation jobs inside your DWH much faster. -

-

You pay a price for this with a few trade offs, the most notable being that if your Postgres server - crashes, the contents of the unlogged tables will be lost. But, - again, if you have an ELT DWH, you can survive by running a backfill. In Truvi, we made the decision to - have the landing area for our DWH be logged, and everything else unlogged. This means if we experienced - a crash (which still hasn't happened, btw), we would recover by running a full-refresh dbt run.

-

If you are using dbt, you can easily apply this by adding this bit in your dbt_project.yml - :

- 

-models:
-    +unlogged: true
-
- -

Tuning your server's parameters

-

Postgres has many parameters you can fiddle with, with plenty of - chances to either improve or destroy your server's performance.

-

Postgres ships with some default values for it, which are almost surely not the optimal ones for - your needs, specially if you are going to use it as a DWH. Simple changes like adjusting the - work_mem will do wonders to speed up some of your heavier queries. -

-

There are many parameters to get familiar with and proper adjustment must be done taking your specific - context and needs into account. If you have no clue at all, this little web app can give you some suggestions you - canstart from. -

-

Running VACUUM ANALYZE right after building your tables

-

Out of the box, Postgres will automatically run - VACUUM - and - ANALYZE - jobs automatically. The triggers that determine when each of those gets - triggered can be adjusted with a few server parameters. If you follow an ELT pattern, most surely - re-building your non-staging tables will cause Postgres to run them. -

-

But there's a detail that is easy to overlook. Postgres automatic triggers will start those quite fast, - but not right after you build each table. This poses a performance issue: if your intermediate sections - of the DWH have tables that build upon tables, rebuilding a table and then trying to rebuild a dependant - without having an ANALYZE on the first one before might hurt you.

-

Let me describe this with an example, because this one is a bit of a tongue twister: let's assume we have - tables int_orders and int_order_kpis. int_orders holds all of our - orders, and int_order_kpis derives some kpis from them. Naturally, first you will - materialize int_orders from some upstream staging tables, and once that is complete, you - will use its contents to build int_order_kpis. -

-

- Having int_orders ANALYZE-d before you start building - int_order_kpis is highly benefitial for your performance in building - int_order_kpis. Why? Because having perfectly updated statistics and metadata on - int_orders will help Postgres' query optimizer better plan the necessary query to - materialize int_order_kpis. This can improve performance by orders of magnitude in some - queries by allowing Postgres to pick the right kind of join strategy for the specific data you have, for - example. -

-

Now, will Postgres auto VACUUM ANALYZE the freshly built int_orders before you - start building int_order_kpis? Hard to tell. It depends on how you build your DWH, and how - you've tuned your server's parameters. And the most dangerous bit is you're not in full control: it can - be that sometimes it happens, and other times it doesn't. Flaky and annoying. Some day I'll - write a post on how this behaviour drove me mad for two months because it made a model sometimes built - in a few seconds, and other times in >20min. -

-

- My advice is to make sure you always VACUUM ANALYZE right after building your tables. If - you're using dbt, you can easily achieve this by adding this to your project's - dbt_project.yml: - 


-models:
-    +post-hook:
-        sql: "VACUUM ANALYZE {{ this }}"
-        transaction: false
-        # ^ This makes dbt run a VACUUM ANALYZE on the models after building each.
-        # It's pointless for views, but it doesn't matter because Postgres fails
-        # silently withour raising an unhandled exception.
-
-

-

Monitor queries with pg_stats_statements

-

pg_stats_statements is an extension that nowadays ships with Postgres - by default. If activated, it will log info on the queries executed in the server which you can check - afterward. This includes many details, with how frequently does the query get called and what's the min, - max and mean execution time being the ones you probably care about the most. Looking at those allows you - to find queries that take long each time they run, and queries that get run a lot. -

-

Another important piece of info that gets recorded is who ran the query. This is helpful - because, if you use users in a smart way, it can help you isolate expensive queries on different uses - cases or areas. For example, if you use different users to build the DWH and to give your BI tool read - access (you do that... right?), you can easily tell apart dashboard related queries from internal, DWH - transformation ones. Another example could be internal reporting vs embedded analytics in your product: - you might have stricter performance SLAs for product-embedded, customer-facing queries than for internal - dashboards. Using different users and pg_stats_statements makes it possible for you to - dissect performance issues on those separate areas independently.

-

Dalibo's wonderful execution plan visualizer

-

Sometimes you'll have some nasty query you just need to sit down with and optimize. In my experience, in - a DWH this ends up happening with queries that involve many large tables in sequential joining and - aggregation steps (as in, you join a few tables, group to some granularity, join some more, group again, - etc). -

-

You can get the query's real execution details with EXPLAIN ANALYZE, but the output's - readability is on par with morse-encoded regex patterns. I always had headaches dealing with them until - I came across Dalibo's execution plan - visualizer. You can paste the output of EXPLAIN ANALYZE there and see the query - execution presented as a diagram. No amount of words will portray accurately how awesome the UX is, so - I encourage you to try the tool with some nasty query and see for yourself.

-

Local dev env + Foreign Data Wrapper

-

One of the awesome things of using Postgres is how trivial it is to spin up an instance. This makes - goofing around much more simpler than whenever setting up a new instance means paperwork, $$$, etc.

-

Data team members at Truvi have a dockerized Postgres running in their laptops that they can use when - they are developing on our DWH dbt project. In the early days, you could grab some production dump with - some subset of tables from our staging layer and run significant chunks of our dbt DAG in your laptop if - you were patient.

-

A few hundreds of models later, this evolved to increasingly difficult and finally became impossible. -

-

Luckily, we came across Postgres' Foreign Data Wrapper. There's - quite a bit to it, but to keep it short here, just be aware that FDW allows you to make a Postgres - server give access to some table in a different Postgres server while pretending they are local. So, you - query table X in Postgres server A, even though table X is actually stored in Postgres server B. But - your query works just the same as if it was a local genuine table.

-

Setting these up is fairly trivial, and has allowed our dbt project contributors to be able to execute - hybrid dbt runs where some data and tables is local to their laptop, whereas some upstream data is being - read from production server's. The approach has been great so far, enabling them to actually test models - before commiting them to master in a convenient way.

-
-

back to home

-
-
- - - - \ No newline at end of file diff --git a/public/writings/notes-and-lessons-from-my-departure-from-superhog.html b/public/writings/notes-and-lessons-from-my-departure-from-superhog.html deleted file mode 100644 index 3b1b490..0000000 --- a/public/writings/notes-and-lessons-from-my-departure-from-superhog.html +++ /dev/null @@ -1,204 +0,0 @@ - - - - - Pablo here - - - - - - - -
-

- Hi, Pablo here -

-

back to home

-
-
-

Notes for myself during my departure from Superhog

-

Published: 2025-07-07

-

I'm writing this a few days before my last day at Superhog (now called Truvi). Having a few company - departures under my belt already, I know a bit on what will come next. I know one part of the drill is - that 99% of the details of what happened during my tenure at the company will completely disappear from - my memory for the most part, only triggered by eerily coincidental cues here and there every few years. - I will remember clearly a few crucial, exciting days and situations. I will also hold well the names and - faces of those with who I worked closely, as well as my personal impression and judgement of them. I - will remember the office, and some details of how my daily life was when I went there.

-

But most other things will be gone from my brain, surprisingly fast.

-

Knowing that experience is a great teacher, and regretting not doing this in the past, I've decided to - collect a few notes from my time at Superhog, hoping they will serve me in making the lessons I've - learnt here stick properly.

-
    -
  • Growing really fast an organization without an incredibly solid vision you're going to stick to is - terrible. Time, money and effort will be wasted left and right, and unless you have some magic tric - up your sleeve, you'll run out of money and panic. Growth is about having a great vision, going for - it and hoping for the best, not about collecting resources and hoping that they will somehow align - themselves towards making money.
    • -
  • - If you're in the leadership of a company, you make decisions, and then things go badly because of - them, people are going to think you've fucked up and won't be happy about it. Should you publicly - retrospect in an intelligent way and clearly show you've learnt the lesson, you have a chance at - some degree of redemption, and you might even make some of the employees hopeful again that there's - still a second chance to go for success. If you don't retrospect at all and pretend the mishaps have - nothing to do with your management, they won't just think you're incompentent: they simply won't - take you seriously anymore, and won't be honest to you because smart people don't invest calories in - arguing with people who they consider idiots. -
    • -
  • - If you're in a B2B business where customers will have a long-term relationship with you, and you - have sales people, giving them incentives that are all about getting people onboard, and not about - long term performance, might be an expensive mistake. I've observed sales people who only care about - scoring deals engage in undesirable behaviours such as: -
      -
    • Sell to anyone, regardless of whether there's a good fit between your offering and the - customer needs.
      • -
    • Cut corners, surely do and say things that are in moral gray areas. If you're unlucky, even - clearly cross moral red lines.
      • -
    • - Drive the people who build and deliver your offering crazy. Since they have no incentive to - care about what happens after a deal is signed, they don't care if their actions in the - sales pipeline turn into landmines during the long-term business relationship and execution - of the service. -
      • -
    - I think many of these issues get solved by structuring compensation so that sales people do well - once the leads they convert have been doing well for some time, however you want to measure that. - Not only nasty behaviour can be avoided, but even new, good and constructive actions might arise. - For example, your sales people will care more about building a great product, and so they'll - regularly feedback to engineers and operations and care deeply about collaborating in improving - things. -
    • -
  • - If you're lucky to find talented employees, go crazy about retaining them. -
    • -
  • - The unexpected death of collegues can be a great blow to the business. -
    • -
  • Non-technical founders need CTOs with strong characters nearby to protect them from themselves. - Soft-hearted CTOs with a pleasing attitude and aversion to conflict will feel sweet at first, but - their lack of fightning back on certain topics will lead to sour consequences down the line.
    • -
  • During my tenure, my team and I managed to deliver astonishing amounts of value with extremely - simple tooling that was just enough for what we needed. This had many advantages and was a silent - win. It's not sexy, but I think it should be.
    • -
  • If you are silently efficient budget wise, as in you manage to achieve something consuming way less - money than whatever is average for your context, but you don't explain it are notably noisy about - it, nobody will give a damn. Even worse, your levels of efficiency may be taken for granted and you - might encounter trouble when asking for more bucks, even if you're still way below average.
    • -
  • When there's a feeling that a ship is going down, I've observed there's a direct correlation between - how talented an employee is and the chances he departs early. The less gifted will stay until the - end.
    • -
  • - If you're a SaaS and want to scale, don't leave your Finance team orphan of IT resources. Invoicing, - gathering customer payment details, the most frequent accounting journals, etc. should be treated as - first class requirements of your architecture, not as an afterthought. Your finance team needs to - grow in engineers, not accountants. And if you have the feeling that the number of accountants is - growing linearly with the volume of the business, you are in serious trouble and need to do - something. Failing to do this will lead to some very nasty tech debt that will kill your speed and - potentially make you lose a lot of money. -
    • -
  • If you've had employees rotating through various departments in your org, doing very different jobs, - their views and opinions are worth solid gold and should be valued as such.
    • -
  • Right befores starting in this company, I had just read the book It doesn't have to be crazy at work - by Jason Fried and DHH. At that time I thought I believed by then that it's worth creating a calm - environment to think clearly, since doing the right thing is way more important than executing fast, - and fast paced environments are not great to keep your head clear. After my time here, I'm a x100 - times more of a believer.
    • -
  • Giving people in the business some basics on SQL is really useful, but that usefulness gets - multiplied by the tidyness and documentation of your DWH. If they need to call you up every time - because there's no way they can find and understand what they need in the DWH, teaching them SQL is - pointless and only leads to frustration.
    • -
  • - If you were part of the decision to hire someone, and then they decide to leave, you should talk - with them. Even if you're not working together every day anymore and the org has changed quite a - bit. You had a stake in this person's entrance, they remember it vividly, and not calling them to - grab a coffee and say bye properly will disappoint them. -
    • -
  • If a manager gets fired and you get their direct reports now reporting to you, and you know they had - strong respect for him, make sure to recognize that feeling the first thing. Saying something along - the lines of "Guys, I know you respected X and were fond of working with him, and that you might not - be happy with his departure and having to report to me instead. I understand that and think it's - natural", will go a long way in helping with the grieving and making them feel more comfortable. -
    • -
  • - Engineering leadership is quite a bit like parenting when it comes to mirroring. Regardless of what - you say should be done, people will ignore that a lot and tend to do what you do. If senior - engineers do patchy shit on the database, don't document a thing, cut corners instead of building - properly, mindlessly submit to absurd requests instead of collaborating productively with their - non-tech colleagues, etc, the rest of engineers will do it as well, regardless on how many training - sessions on best practices you run. Conversely, if you focus on quality, give time and room to do - things right, reward ingenious solutions to problems, treat incidents in professional and serious - ways, push back from stupid managerial situations to work things out in a way that is good - for everyone, document your work properly, etc. you will soon find the rest of your colleagues - (specially, the most junior ones) following your lead, often times without you even needing to - insist on good practices. -
    • -
  • People care little about having an office on the beachfront.
    • -
  • If you manage to raise a team to have team-ownership mentality (as in, they know what's their high - level goal and will always strive for it, even if you don't provide strict guidance or are not - around at all), you might get an ego hit because it suddenly feels like you are not needed. Rest - assured, even if you're not immediately needed, your team values you, and other managers and - executives will notice the work you've done with the team, even if they don't voice it. - Unfortunately, it's likely that the appreciation for your achievement only becomes visible when you - decide to leave and people panic.
    • -
  • People with cowardly and anxious characters might seem harmless. But don't undersestimate their - ability to allow terrible things to happen precisely because stepping up and acting would require - courage and they have none. Basically the way Upham gets Mellish killed in Saving Private Ryan. - The attrocities that can be tolerated or even supported by cowards in this manner can be terrible, - and even more depressing that the loud acts of bold, evil men. Plus, if there are many cowards - clustered together, group thinking will give them comfort and normalize the behaviour in the fashion - of the bystander effect. -
    • -
  • - Only badmouth a colleague in front of others if you are going to eventually raise the issue to - either him or his superiors. Don't ever pointlessly complain with third parties if you will never - act on the issue. Specially with your direct reports. It generates a nasty victim culture, and once - you get it started, it's hard to stop. -
    • -
  • - I have found that the lack of easy access to data and skills like SQL and data analysis are orders - of magnitude less of an issue to organizational data literacy compared to managers who don't expect - a data-driven approach from their reports. People start caring about data when their boss demands - they do their work using data. People continue ignoring data when their boss tolerates the lack of - data. And when observing this, it's vital to also keep in mind that reports adjust their behaviour - to what their manager actually rewards/punishes, not what their manager says he will - reward/punish. -
    • -
  • Joel Spolsky wrote this thing - you should never do. The company decided to do it anyway. We fucked up and we paid for the - consequences exactly as Spolsky lays out.
    • -
  • Employees with little retrospective capacity are dangerous. Partly related to the previous point. - This quote from Spolsky hits right on the nail: It's important to remember that when you start - from scratch there is absolutely no reason to believe that you are going to do a better job than - you did the first time. First of all, you probably don't even have the same programming team - that worked on version one, so you don't actually have “more experience”. You're just going to - make most of the old mistakes again, and introduce some new problems that weren't in the - original version. I would only nuance that if you can retrospect deeply and learn from the - mistakes you made on the first run, perhaps there's some hope you may do a better job starting from - scratch. But I guess if you manage to retrospect and learn properly, you could also do a better job - working yourself out incrementally from the fucked up situation you failed yourself into.
    • -
  • If you set up a variable/bonus scheme, refrain from changing the structure frequently, even if you - are not really making it more stingy. Too much shuffling on that area will get employees thinking - you're playing games on them, even if it's not the case.
    • -
  • When an engineer who designed, deployed, and since then operated a non-trivial production system is - about to leave, ask him to finish his handover and lock himself out days before his actual last - working day. Challenge him to a nice treat by making it clear that, once he's locked out, he doesn't - have any other duties other than help his colleagues should something not work after his departure. - This way, you provide him with an incentive to handover as perfect and fast as possible (paid - holidays), and you make sure that you get a chance to try to operate the system without him - before he leaves. Not doing this, and instead simply waiting for his last day to remove his - creds and users from the infra, is dangerous. -
    • - -
-
-

back to home

-
-
- - - - \ No newline at end of file diff --git a/public/writings/one-efective-but-risky-way-to-find-the-top-budget-for-the-vacancy.html b/public/writings/one-efective-but-risky-way-to-find-the-top-budget-for-the-vacancy.html index 2f8aaee..cbf9a9e 100644 --- a/public/writings/one-efective-but-risky-way-to-find-the-top-budget-for-the-vacancy.html +++ b/public/writings/one-efective-but-risky-way-to-find-the-top-budget-for-the-vacancy.html @@ -18,7 +18,6 @@

One efective but risky way to find the top budget for the vacancy

-

Published: 2025-02-03

I've seen a lot of people frustrated with not knowing how much a company is willing to pay for a vacancy. There are multiple ways to find out, which you probably are familiar with if you've been job diff --git a/public/writings/replacing-a-failed-disk-in-a-zfs-mirror.html b/public/writings/replacing-a-failed-disk-in-a-zfs-mirror.html deleted file mode 100644 index 0040856..0000000 --- a/public/writings/replacing-a-failed-disk-in-a-zfs-mirror.html +++ /dev/null @@ -1,243 +0,0 @@ - - - - - Pablo here - - - - - - - -

-

- Hi, Pablo here -

-

back to home

-
-

Replacing a Failed Disk in a ZFS Mirror

-

Published: 2026-02-21

-

If you've been following along, you know the story: I set up a ZFS mirror for my Proxmox VMs, then one of the drives started acting flaky, and I diagnosed and fixed what turned out to be a bad SATA connection.

-

Well, the connection wasn't the whole story. A few weeks after that fix, the same drive, AGAPITO1, started dropping off again. Same symptoms: link resets, speed downgrades, kernel giving up on the connection. I went through the cable swap dance again, tried different SATA ports on the motherboard, tried different cables. Nothing helped. The SATA PHY on the drive itself was failing.

-

I contacted PcComponentes (where I bought it), RMA'd the drive, and ran degraded on AGAPITO2 alone for about two weeks. Then the replacement arrived. This article covers the process of physically installing a new drive and getting it into the ZFS mirror, from "box on the desk" to "pool healthy, mirror whole."

- -

The starting point

-

Before doing anything, this is what the pool looked like:

- 
  pool: proxmox-tank-1
- state: DEGRADED
-status: One or more devices have been removed.
-        Sufficient replicas exist for the pool to continue functioning in a
-        degraded state.
-action: Online the device using zpool online' or replace the device with
-        'zpool replace'.
-  scan: scrub repaired 0B in 06:55:06 with 0 errors on Tue Feb 17 20:40:50 2026
-config:
-
-        NAME                                 STATE     READ WRITE CKSUM
-        proxmox-tank-1                       DEGRADED     0     0     0
-          mirror-0                           DEGRADED     0     0     0
-            ata-ST4000NT001-3M2101_WX11TN0Z  REMOVED      0     0     0
-            ata-ST4000NT001-3M2101_WX11TN2P  ONLINE       0     0     0
-
-errors: No known data errors
-

DEGRADED with one drive REMOVED. The old drive (WX11TN0Z) was physically gone, shipped back to PcComponentes. AGAPITO2 (WX11TN2P) was holding down the fort alone.

-

This is the beauty and the terror of a degraded mirror: everything works fine. Your VMs keep running, your data is intact, reads and writes happen normally. But you have zero redundancy. If that surviving drive has a bad day, you lose everything. Two weeks of running like this was two weeks of hoping AGAPITO2 stayed healthy.

- -

Before you touch hardware

-

Before doing anything physical, I wanted to capture the current state. When things go wrong during maintenance, you want to be able to compare "before" and "after."

-

Three things to record while the server is still running:

-

Pool status, the zpool status output above. You want to know exactly what ZFS thinks the world looks like right now.

-

SATA layout, which drive is on which port:

- 
dmesg -T | grep -E 'ata[0-9]+\.[0-9]+: ATA-|ata[0-9]+: SATA link up'
-

In my case, AGAPITO2 was on ata4 and ata3 was empty (the old drive's port). This matters because after you install the new drive, you want to confirm it shows up on the expected port.

-

Surviving drive health, to make sure the drive you're depending on is actually healthy before you start:

- 
smartctl -H /dev/disk/by-id/ata-ST4000NT001-3M2101_WX11TN2P
- 
SMART overall-health self-assessment test result: PASSED
-

If this says anything other than PASSED, stop and deal with that first. You don't want to discover your only remaining copy of data is on a failing drive while you're in the middle of hardware work.

-

Once you've got your reference snapshots, shut down the server gracefully:

- 
shutdown now
- -

Physical installation

-

I won't write a hardware installation tutorial, every case and drive bay is different. But a few practical tips for homelabbers doing this for the first time:

-
    -
  • Inspect your cables before connecting them. If the SATA data cable has been sitting disconnected in the case, check the connector pins. Bent pins or dust can cause exactly the kind of intermittent issues that started this whole saga.
    • -
  • Label the new drive. I labeled mine "TOMMY" with its serial number (WX120LHQ) written on a sticker. Yes, I name my drives. It makes debugging much easier than squinting at serial numbers.
    • -
  • Push connectors until they click. Both SATA data and power. Then do the wiggle test: grab the connector gently and try to move it. If it shifts at all, it's not fully seated.
    • -
-

Seat the drive, connect both cables, close the case, and power on.

- -

Boot and verify detection

-

First thing after boot: did the kernel see the new drive?

- 
dmesg -T | grep -E 'ata[0-9]+\.[0-9]+: ATA-|ata[0-9]+: SATA link up'
- 
[Fri Feb 20 22:57:06 2026] ata3: SATA link up 6.0 Gbps (SStatus 133 SControl 300)
-[Fri Feb 20 22:57:06 2026] ata3.00: ATA-11: ST4000NT001-3M2101, EN01, max UDMA/133
-[Fri Feb 20 22:57:07 2026] ata4: SATA link up 6.0 Gbps (SStatus 133 SControl 300)
-[Fri Feb 20 22:57:07 2026] ata4.00: ATA-11: ST4000NT001-3M2101, EN01, max UDMA/133
-

Both drives detected at full 6.0 Gbps: TOMMY on ata3, AGAPITO2 on ata4.

-

Next, verify it shows up with its expected serial in /dev/disk/by-id/:

- 
ls -l /dev/disk/by-id/ | grep WX120LHQ
- 
ata-ST4000NT001-3M2101_WX120LHQ -> ../../sda
-

And confirm identity with SMART:

- 
smartctl -i /dev/disk/by-id/ata-ST4000NT001-3M2101_WX120LHQ
- 
Device Model:     ST4000NT001-3M2101
-Serial Number:    WX120LHQ
-Firmware Version: EN01
-User Capacity:    4,000,787,030,016 bytes [4.00 TB]
-SATA Version is:  SATA 3.3, 6.0 Gb/s (current: 6.0 Gb/s)
-

Correct model, serial, firmware, and running at full speed.

-

One more critical check: look for SATA errors in the kernel log.

- 
dmesg -T | grep -E 'ata[0-9]' | grep -iE 'error|fatal|reset|link down|slow|limiting'
-

I saw ata1: SATA link down and ata2: SATA link down, which are just unused ports. Nothing on ata3 or ata4. If you see errors on the port your new drive is on, stop. A brand new drive throwing SATA errors on a known-good cable is likely dead on arrival.

- -

Health-check before trusting it

-

A drive can be detected and still be dead on arrival. Before resilvering 1.3 terabytes of data onto it, I wanted to know it was actually healthy.

-

SMART overall health:

- 
smartctl -H /dev/disk/by-id/ata-ST4000NT001-3M2101_WX120LHQ
- 
SMART overall-health self-assessment test result: PASSED
-

Baseline SMART attributes, the important ones to check on a new drive:

- 
smartctl -A /dev/disk/by-id/ata-ST4000NT001-3M2101_WX120LHQ | grep -E 'Reallocated|Pending|Offline_Uncorrect|CRC'
- 
  5 Reallocated_Sector_Ct   ... -       0
-197 Current_Pending_Sector  ... -       0
-198 Offline_Uncorrectable   ... -       0
-199 UDMA_CRC_Error_Count    ... -       0
-

All zeros. Reallocated sectors would mean the drive has already had to remap bad spots. Pending sectors are blocks the drive suspects are bad but hasn't confirmed yet. CRC errors indicate data corruption during transfer. On a new or refurbished drive, all of these should be zero.

-

Short self-test:

- 
smartctl -t short /dev/disk/by-id/ata-ST4000NT001-3M2101_WX120LHQ
-# Wait ~2 minutes...
-smartctl -l selftest /dev/disk/by-id/ata-ST4000NT001-3M2101_WX120LHQ
- 
# 1  Short offline       Completed without error       00%         0         -
-

Passed with 0 power-on hours, a fresh drive. If any of these checks fail, don't proceed. Contact the seller and get another replacement.

- -

The replacement: zpool replace

-

This is the moment. One command:

- 
zpool replace proxmox-tank-1 ata-ST4000NT001-3M2101_WX11TN0Z ata-ST4000NT001-3M2101_WX120LHQ
-

This tells ZFS "the drive identified as WX11TN0Z (currently REMOVED) is being replaced by WX120LHQ." ZFS starts resilvering immediately, copying all data from the surviving drive (AGAPITO2) onto the new one (TOMMY).

-

Checking status right after:

- 
  pool: proxmox-tank-1
- state: DEGRADED
-  scan: resilver in progress since Fri Feb 20 23:10:58 2026
-
-config:
-
-        NAME                                 STATE     READ WRITE CKSUM
-        proxmox-tank-1                       DEGRADED     0     0     0
-          mirror-0                           DEGRADED     0     0     0
-            replacing-0                      DEGRADED     0     0     0
-              ata-ST4000NT001-3M2101_WX11TN0Z  REMOVED      0     0     0
-              ata-ST4000NT001-3M2101_WX120LHQ  ONLINE       0     0 7.73K
-            ata-ST4000NT001-3M2101_WX11TN2P  ONLINE       0     0     0
-

Notice the replacing-0 vdev. That's a temporary structure ZFS creates during the replacement, showing both the old (REMOVED) and new (ONLINE) drive while the resilver is in progress.

-

The 7.73K cksum count on the new drive might look alarming, but it's expected during a resilver. Those are blocks that haven't been written yet. ZFS is aware of them and they'll clear up as the resilver progresses.

-

I monitored progress with:

- 
watch -n 30 "zpool status -v proxmox-tank-1"
-

I also kept dmesg -Tw running in another terminal, watching for any SATA errors. The kernel log stayed quiet the entire time.

-

In my case, the VMs had auto-started on boot, so the resilver was competing with production I/O. It completed in about 3.5 hours: 1.34 terabytes resilvered with 0 errors. Not bad for a pair of 4TB IronWolf drives running alongside active workloads.

- -

Post-resilver verification

-

The resilver finished. Time to verify everything is actually good.

-

Pool status:

- 
  pool: proxmox-tank-1
- state: ONLINE
-  scan: resilvered 1.34T in 03:32:55 with 0 errors on Sat Feb 21 02:43:53 2026
-config:
-
-        NAME                                 STATE     READ WRITE CKSUM
-        proxmox-tank-1                       ONLINE       0     0     0
-          mirror-0                           ONLINE       0     0     0
-            ata-ST4000NT001-3M2101_WX120LHQ  ONLINE       0     0 7.73K
-            ata-ST4000NT001-3M2101_WX11TN2P  ONLINE       0     0     0
-
-errors: No known data errors
-

ONLINE. The replacing-0 vdev is gone and the mirror now has the new drive in place. The 7.73K cksum on TOMMY is a residual counter from the resilver, so let's clear it:

- 
zpool clear proxmox-tank-1
-

Now for the real test. A resilver copies data to rebuild the mirror, but a scrub reads every block on the pool, verifies all checksums, and repairs any mismatches. This is the definitive integrity check:

- 
zpool scrub proxmox-tank-1
-

This ran for about 3.5 hours across 1.34T of data:

- 
  scan: scrub repaired 0B in 03:27:50 with 0 errors on Sat Feb 21 11:38:02 2026
-
-        NAME                                 STATE     READ WRITE CKSUM
-        proxmox-tank-1                       ONLINE       0     0     0
-          mirror-0                           ONLINE       0     0     0
-            ata-ST4000NT001-3M2101_WX120LHQ  ONLINE       0     0     0
-            ata-ST4000NT001-3M2101_WX11TN2P  ONLINE       0     0     0
-
-errors: No known data errors
-

Zero bytes repaired, zero errors, both drives at 0/0/0. Clean.

-

One last thing: a post-I/O SMART check on the new drive. After hours of heavy writes during the resilver and reads during the scrub, any hardware weakness should have surfaced:

- 
smartctl -x /dev/disk/by-id/ata-ST4000NT001-3M2101_WX120LHQ | grep -E 'Reallocated|Pending|Offline_Uncorrect|CRC|Hardware Resets|COMRESET|Interface'
- 
Reallocated_Sector_Ct    ... 0
-Current_Pending_Sector   ... 0
-Offline_Uncorrectable    ... 0
-UDMA_CRC_Error_Count     ... 0
-Number of Hardware Resets ... 2
-Number of Interface CRC Errors ... 0
-COMRESET ... 2
-

All clean. The 2 hardware resets and 2 COMRESETs are just from the server booting, perfectly normal.

- -

The commands, all in one place

-

For future me and anyone else replacing a disk in a ZFS mirror:

- 
# --- Before shutdown ---
-
-# Record pool status
-zpool status -v <pool>
-
-# Record SATA layout
-dmesg -T | grep -E 'ata[0-9]+\.[0-9]+: ATA-|ata[0-9]+: SATA link up'
-
-# Check surviving drive health
-smartctl -H /dev/disk/by-id/<surviving-disk-id>
-
-# Shut down
-shutdown now
-
-# --- After boot with new drive ---
-
-# Verify detection
-dmesg -T | grep -E 'ata[0-9]+\.[0-9]+: ATA-|ata[0-9]+: SATA link up'
-ls -l /dev/disk/by-id/ | grep <new-serial>
-smartctl -i /dev/disk/by-id/<new-disk-id>
-
-# Check for SATA errors
-dmesg -T | grep -E 'ata[0-9]' | grep -iE 'error|fatal|reset|link down'
-
-# Health-check the new drive
-smartctl -H /dev/disk/by-id/<new-disk-id>
-smartctl -A /dev/disk/by-id/<new-disk-id> | grep -E 'Reallocated|Pending|Offline_Uncorrect|CRC'
-smartctl -t short /dev/disk/by-id/<new-disk-id>
-smartctl -l selftest /dev/disk/by-id/<new-disk-id>
-
-# --- Replace and resilver ---
-
-# Replace old drive with new
-zpool replace <pool> <old-disk-id> <new-disk-id>
-
-# Monitor resilver progress
-watch -n 30 "zpool status -v <pool>"
-
-# Watch kernel log for SATA errors during resilver
-dmesg -Tw
-
-# --- Post-resilver verification ---
-
-# Check final status
-zpool status -v <pool>
-
-# Clear residual cksum counters
-zpool clear <pool>
-
-# Run a full scrub
-zpool scrub <pool>
-
-# Post-I/O SMART check
-smartctl -x /dev/disk/by-id/<new-disk-id> | grep -E 'Reallocated|Pending|Offline_Uncorrect|CRC'
- -

The mirror degradation that started on February 8th is resolved. Two weeks of running on a single drive, an RMA, and one evening of work later, the pool is whole again. Full redundancy restored, zero data lost throughout the entire saga. ZFS did exactly what it was designed to do.

-

This is the fourth and final article in this series. If you're just arriving, start with Part 1: Why I Put My VMs on a ZFS Mirror, then Part 2: A Degraded Pool with a Healthy Disk, and Part 3: Fixing a Degraded ZFS Mirror.

-

back to home

-
-
- - - - diff --git a/public/writings/tech-fascination-as-a-function-of-your-age.html b/public/writings/tech-fascination-as-a-function-of-your-age.html deleted file mode 100644 index 671d18b..0000000 --- a/public/writings/tech-fascination-as-a-function-of-your-age.html +++ /dev/null @@ -1,42 +0,0 @@ - - - - - Pablo here - - - - - - - -
-

- Hi, Pablo here -

-

back to home

-
-
-

Tech fascination as a function of your age

-

Published: 2026-05-11

-

A long time ago I read a quote that I find hilariously accurate. At first it stuck with me because it was - funny. Then as time passed, I observed it was actually very accurate. I mistakenly thought the quote was - commonly known in the world, and today I realised apparently it isn't, and I can't find it posted - anywhere. So I'm storing it here for the future. The quote roughly looked like this:

-
-

Any technology that existed before you turned 18 is just boring, regular old stuff that has always - been there since the world was born and doesn't matter much.

-

Any technology that appears while you are between 18 and 30 years old is a revolution like the world - has never seen before, will change the universe in ways mankind can't even fathom and deserves your - entire devotion.

-

Any technology that appears after you've hit 30 is black magic and heresy, and you will never truly - understand it.

-
-
-

back to home

-
-
- - - - diff --git a/public/writings/the-roi-of-toilets.html b/public/writings/the-roi-of-toilets.html deleted file mode 100644 index 44d2d86..0000000 --- a/public/writings/the-roi-of-toilets.html +++ /dev/null @@ -1,79 +0,0 @@ - - - - - Pablo here - - - - - - - -
-

- Hi, Pablo here -

-

back to home

-
-
-

The ROI of toilets

-

Published: 2025-02-19

-

Years ago I worked under the organizational umbrella of this COO. He was my boss' boss. Sometimes we - bumped into each other for big meetings and presentations.

-

The COO had a background in finance and audit, which gave him certain management quirks that coupled in - rather funny ways with the nature of our data and analytics departments. There was this specific one - that was always itchy to me. At the time I was still a very junior and inexperienced professional, and - my default stance on things was to humble out, shut the fuck up and listen. But I always had my opinions - locked in my brain, and in cases like this one, I couldn't hold them back.

-

- The quirk this gentleman had was to try to measure the ROI of every little thing. He would ask for the - ROI of projects, the ROI of developments, the ROI of acquiring licenses, the ROI of going out for a - smoke. It was an understandable quirk for a financier who had never actually built or serviced anything, - but rather always looked, judged and measured from the outside. He wasn't that interested in the things - themselves, but rather in measuring them in units that would fit in his Excel sheets. -

-

- I generally thought (and still think) that assessing ROI is a good thing to aim for. But intuitively, I - found his obsession with it misplaced and counterproductive. I now have much better words to critique - and argue against his stance, but at the time I lacked those and only had a gut feeling of "this is - stupid". -

-

- One day we were in one of those meetings where he would start asking about the ROI of something while I - thought to myself: "We just need this thing and it's obviously more valuable that the money it will - cost, why are we having this conversation uuuugh". As I spiritually (not physically) rolled my eyes, I - couldn't hold it in anymore and just shot: "What's the ROI of the office toilets?" -

-

- The COO and my boss suddenly stared at me, mouths open, eyebrows pressed down as they squinted their - eyes: "What?" -

-

- "We have toilets. We have to pay for them. We could use them for desk space, but instead we put toilets. - Then we have to do plumbing and stuff. We need to pay people to clean them. it's a nuisance. How can we - know that they are the best use of shareholder funds? Who has measured the ROI of those toilets?". It - all came out naturally out of the blue. I had a great relationship with these people, but I still was - clenching my but, wondering if I had gone a bit too hard. Oh how nice it is to be young. -

-

- They chuckled and got my point. The COO stopped insisting on specific figures for the cost - element we had at hand, although he didn't surrender a good old "write a business case for this so we - can refer to it later", which probably was a wise thing to do. -

-

- I've faced similar situations a few times since then, and I've found myself in many others where it was - up to me if and how precisely should the ROI of something be measured. I now have a much clear mental - model and opinion of when it should and shouldn't be done. But that's for another day. -

-

- Nobody ever told me what was the ROI of the toilets, though. Perhaps we should remove them? -

-
-

back to home

-
-
- - - - \ No newline at end of file diff --git a/public/writings/valuing-data-teams-output.html b/public/writings/valuing-data-teams-output.html new file mode 100644 index 0000000..0cdcdd9 --- /dev/null +++ b/public/writings/valuing-data-teams-output.html @@ -0,0 +1,132 @@ + + + + + Pablo here + + + + + + + +
+

+ Hi, Pablo here +

+

back to home

+
+
+

Valuating data teams output

+

+ - Freedom and fucking up + - How work looks like + - This is an economics problem + - Fairy tail organizational designs +

+

+ In 2023, I had the chance to do something not a lot of people get to do: I started a Data team in a + startup (Truvi, formerly Superhog) from scratch. +

+

+ Being in a greenfield situation, both in organization and technical terms, was equally challenging and + rewarding. It gave me the right space and craving to spend time thinking on stuff I hadn't before. This + included very foundational questions such as... what should the Data team do? The kind of stuff you + don't think about much when you land in a cruise ship that's already been rolling for a while, and you + get told your job is to pull that lever up and down when the light tells you to. Ever since, I've had + the + chance to learn and think a lot about embedding a Data team in a small SaaS company. +

+

+ One of the hard and interesting topics is how do you measure the success of the team. How do you look at + what the team has done and answer the following questions: +

+
    +
  • How valuable is this thing we delivered?
    • +
  • Was it the most valuable think we could have done?
    • +
+

+ These are not trivial questions. Because it's easy to fuck up. Being a nimble team in a small company, + the amount of flexibility you enjoy is ecstatic. You can (and usually need to) pivot a lot, very fast. + But with freedom comes responsibility, and the pleasure of having many choices comes with the pain of + wondering if you're screwing up in what you choose. +

+

+ How work looks like +

+

+ I find experience and real situations make abstract rants like this one much more interesting, so let me + explain a bit what the Data team at Truvi faces on a daily basis to give some context. +

+

+ Truvi is a SaaS company that services short-term rentals (STR) hosts and guests. Our goal is to help + both parties reduce and manage risk in their bookings. Risk here means, for the other part, the other + party doing something nasty to you (e.g. your guest burns down your BnBs kitchen, or your host let's you + know the property you booked is flooded right when you show up at the door on a Monday night at + 11:30PM). We offer multiple services, like screening and protection, to help both parties manage this, + and we charge fees for it. +

+

+ We deliver our services through a couple of in-house developed applications and some API integrations + with PMSs, OTAs and other funky acronym-named + types of companies involved in the STR industry. +

+

+ The Data team's main responsibility, as defined by me, is to ensure people in the company know what they + need to know. We deliver this in multiple ways: +

+
    +
  • We maintain a lot of reporting. Some of it might be company-wide KPIs all management looks at, + some others are more operational detail that only affect certain teams or functions.
    • +
  • + We keep ourselves available for adhoc, quick and dirty, one off requests. We rotate this through the + different members of the team since it's quite disruptive for one's agenda and focus. +
    • +
  • + We deliver adhoc, slow and steady, brainy reports whenever people not only need Data, but someone + who knows what he's doing because the analysis requires above average data literacy. +
    • +
  • + We support data heavy projects, such as A/B testing or the acquisition of external data sources. +
    • +
+

+ Even if this categorization looks neat, the reality is more of a barrage of a million different things, + coming through the door all at once without any order. +

+

+ Given our humble capacity for delivering and our colleagues heavy appetite for asking, only a sliver + of what gets requested will be done soon. One of my jobs is to decide, together with the company + leadership, what makes it in. It's a tough job at Truvi, and it's been a tough job at previous companies + I've been at. I think that is the case because of poor organizational design. And I think we have a lot + to learn from economics. +

+

Economic calculation

+

+ The situation we have in my team is an economical one. We have lots of needs and we can't satisfy all + of them. +

+

+ This is the same situation society faces at scale: there's plenty of capital and man hours we can put up + to good use, but we have infinite options. What do we do more, hospitals, more schools or more beers? +

+

+ In society, despite what statists and bureacrats would like, these decisions are not made by a bunch of + all knowing intellectuals in their parties office. They are made on the streets, by individuals that + decide how to spend their own money and time. +

+

+ People spend their own very wisely. Even if it might look like they do stupid stuff, they don't. They do + what's good for them, with their resources and preferences. Even if we don't share their choices. Even + if we think we know better than them. +

+ +
+

back to home

+
+
+ + + + \ No newline at end of file diff --git a/public/writings/when-new-is-not-better.html b/public/writings/when-new-is-not-better.html index fe34b5f..bca6698 100644 --- a/public/writings/when-new-is-not-better.html +++ b/public/writings/when-new-is-not-better.html @@ -18,7 +18,6 @@

When new is not better

-

Published: 2025-01-19

One of the hobbies that has stuck with me for decades is photography. I've shot a few hundreds of film rolls, and some non-recorded amount of digital pics. diff --git a/public/writings/why-i-put-my-vms-on-a-zfs-mirror.html b/public/writings/why-i-put-my-vms-on-a-zfs-mirror.html deleted file mode 100644 index f3c4e7c..0000000 --- a/public/writings/why-i-put-my-vms-on-a-zfs-mirror.html +++ /dev/null @@ -1,121 +0,0 @@ - - - - - Pablo here - - - - - - - -

-

- Hi, Pablo here -

-

back to home

-
-

Why I Put My VMs on a ZFS Mirror

-

Published: 2026-01-04

-

Part 1 of 3 in my "First ZFS Degradation" series. Also read Part 2: Diagnosing the Problem and Part 3: The Fix.

-

Why This Series Exists

-

A few weeks into running my new homelab server, I stumbled upon something I wasn't expecting to see that early: my ZFS pool was in "DEGRADED" state. One of my two mirrored drives had gone FAULTED.

-

This was the first machine I had set up with a ZFS mirror, precisely to be able to deal with disk issues smoothly, without losing data and having downtime. Although it felt like a pain in the ass to spot the problem, I was also happy because it gave me a chance to drill the kind of disk maintenance I was hoping to do in this new server.

-

But here's the thing: when I was in the middle of it, I couldn't find a single resource that walked through the whole experience in detail. Plenty of docs explain what ZFS is. Plenty of forum posts have people asking "help my pool is degraded." But nothing that said "here's what it actually feels like to go through this, step by step, with all the commands and logs and reasoning behind the decisions."

-

So I wrote it down. I took a lot of notes during the process and crafted a more or less organized story from them. This three-part series is for fellow amateur homelabbers who are curious about ZFS, maybe a little intimidated by it, and want to know what happens when things go sideways. I wish I had found a very detailed log like this when I was researching ZFS initially. Hope it helps you.

-

The server and disks

-

My homelab server is a modest but capable box I built in late 2025. It has decent consumer hardware, but nothing remarkable. I'll only specify that I have currently three disks on it:

-
    -
  • OS Drive: Kingston KC3000 512GB NVMe. Proxmox lives here.
    • -
  • Data Drives: Two Seagate IronWolf Pro 4TB drives (ST4000NT001). This is where my Proxmox VMs get their disks stored.
    • -
-

The two IronWolf drives are where this story takes place. I labeled them AGAPITO1 and AGAPITO2 because... well, every pair of drives deserves a silly name. I have issues remembering serial numbers.

-

The server runs Proxmox and hosts most of my self-hosted life: personal services, testing VMs, and my Bitcoin infrastructure (which I share over at bitcoininfra.contrapeso.xyz). If this pool goes down, everything goes down.

-

Why ZFS?

-

I'll be honest: I didn't overthink this decision. ZFS is the default storage recommendation for Proxmox, it has a reputation for being rock-solid, and I'd heard enough horror stories about silent data corruption to want something with checksumming built in.

-

What I was most interested in was the ability to define RAID setups in software and deal easily with disks going in and out of them. I had never gone beyond the naive "one disk for the OS, one disk for data" setup in previous servers. After having disks failing on me in previous boxes, I decided it was time to gear up and do it proper this time. My main concern initially was just saving time: it's messy when a "simple" host has disk issues, and I hoped mirroring would allow me to invest less time in cleaning up disasters.

-

Why a Mirror?

-

When I set up the pool, I had two 4TB drives. That gave me a few options:

-
    -
  1. Single disk: Maximum space (8TB usable), zero redundancy. One bad sector and you're crying.
    2. -
  2. Mirror: Half the space (4TB usable from 8TB raw), but everything is written to both drives. One drive can completely die and you lose nothing.
    3. -
  3. RAIDZ: Needs at least 3 drives, gives you parity-based redundancy. More space-efficient than mirrors at scale.
    4. -
-

I went with the mirror for a few reasons.

-

First, I only had two drives to start with, so RAIDZ wasn't even an option yet.

-

Second, mirrors are simple. Data goes to both drives. If one dies, the other has everything. No parity calculations, no write penalties, no complexity.

-

Third (and this is the one that sold me), mirrors let you expand incrementally. With ZFS, you can add more mirror pairs (called "vdevs") to your pool later. You can even mix sizes: start with two 4TB drives, add two 8TB drives later, and ZFS will use all of it. RAIDZ doesn't give you that flexibility; once you set your vdev width, you're stuck with it.

-

When Would RAIDZ Make More Sense?

-

If you're starting with 4+ drives and you want to maximize usable space, RAIDZ starts looking attractive:

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
ConfigurationDrivesUsable SpaceFault Tolerance
Mirror250%1 drive
RAIDZ13~67%1 drive
RAIDZ1475%1 drive
RAIDZ2450%2 drives
RAIDZ26~67%2 drives
-

RAIDZ2 is popular for larger arrays because it can survive two drive failures, which matters more as you add drives (more drives = higher chance of one failing during a resilver).

-

But for a two-drive homelab that might grow to four drives someday, I felt a mirror was the right call. I can always add another mirror pair later.

-

The Pool: proxmox-tank-1

-

My ZFS pool is called proxmox-tank-1. Here's what it looks like when everything is healthy:

- 
  pool: proxmox-tank-1
- state: ONLINE
-config:
-
-    NAME                                 STATE     READ WRITE CKSUM
-    proxmox-tank-1                       ONLINE       0     0     0
-      mirror-0                           ONLINE       0     0     0
-        ata-ST4000NT001-3M2101_WX11TN0Z  ONLINE       0     0     0
-        ata-ST4000NT001-3M2101_WX11TN2P  ONLINE       0     0     0
-

That's it. One pool, one mirror vdev, two drives. The drives are identified by their serial numbers (the WX11TN0Z and WX11TN2P parts), which is important — ZFS uses stable identifiers so it doesn't get confused if Linux decides to shuffle around /dev/sda and /dev/sdb.

-

All my Proxmox VMs store their virtual disks on this pool. When I create a new VM, I point its storage at proxmox-tank-1 and ZFS handles the rest.

-

What Could Possibly Go Wrong?

-

Everything was humming along nicely. VMs were running fine and I was feeling pretty good about my setup.

-

Then, a few weeks in, I was poking around the Proxmox web UI and noticed something that caught my eye.

-

The ZFS pool was DEGRADED. One of my drives — AGAPITO1, serial WX11TN0Z — was FAULTED.

-

In Part 2, I'll walk through how I diagnosed what was actually wrong. Spoiler: the drive itself was fine. The problem was much dumber than that.

-

Continue to Part 2: Diagnosing the Problem

-

back to home

-
-
- - - - - diff --git a/public/writings/your-customers-dont-care-that-your-bathroom-is-dirty.html b/public/writings/your-customers-dont-care-that-your-bathroom-is-dirty.html deleted file mode 100644 index 4bb0d3d..0000000 --- a/public/writings/your-customers-dont-care-that-your-bathroom-is-dirty.html +++ /dev/null @@ -1,94 +0,0 @@ - - - - - Pablo here - - - - - - - -
-

- Hi, Pablo here -

-

back to home

-
-
-

Your customers don't care that your bathroom is dirty

-

Published: 2025-02-09

-

The other night I went out with the missus and we went to a fancy pants restaurants, which is unusual for - us. We prefer neighbourhood, simple places.

-

- During the dinner, she went to the bathroom and came back horrified: "God, their bathroom is fucking - disgusting". "Much worse than the usual one?", I asked. And she said: "No, but I would expect an upscale - place like this to have it squeaky clean". -

-

- I then laid down my thesis on why all restaurant bathrooms, even in really posh places, are always - terrible: "They don't care because you don't really care". "I do care!", she hit me back. "No, you - don't. You think you do: you obviously don't like it, and you would love to see it clean instead of all - filthy. But the truth is, when next month you're thinking about where to go out for dinner, you'll judge - this place and remember the meal, the waiters, how you felt. But not the bathroom. What was the last - time you discarded a restaurant because the bathroom was gross". At this point she agreed, and quickly - drew her attention to the desserts menu. Sometimes I invest too much energy and talk in things people - find boring. -

-

The bathrooms of products

-

- There are a couple of things we can learn here. -

-

- Your product surely has bathrooms. Those little corners that are not the main course, and - your customers don't care about much. You need them. Not having them would be problematic. I don't fuss - over a dirty bathroom in a restaurant, but I'm pretty confident I would remember a restaurant not having - a bathroom at all if it was responsible for some desperate run-for-it trip in search of a place to drop - my bombs. -

-

- Your product's bathrooms are those secondary features your customers kind of need, but don't care much - about. It's that export to CSV button. Your customer John needs it to push the data into his accounting - books. The formats of the date columns are weird, and the columns names are confusing, and the fact that - you send a link to his email to download it instead of just triggering a download in his browser the - moment he hits the button, make it all quite cumbersome. But, all in all, it's minor pain. The moment he - uploads it into the accounting software, he forgets about it. -

-

- I think it's important to be aware of what those are in your product, so you can prioritise accordingly - and avoid some feature-prioritisation bike shedding. Theoretically, it should be obvious, because you - know what's important (right? Right?!?), and whatever is not important, is probably not important. But - then somehow I still see mistakes made around this type of feature. -

-

- I recently had a conversation with my company's CTO about a situation like this. I had some frustration - to vent. We had invested so much time and effort in improving the UI of one of our applications. And it - was so pointless. "There's a good chunk of our customer base that pretty much never go into this UI", I - told him. "They only contact us through a form when they need the service they hired. I don't think they - care about this, and I don't think the nicer UI is going to bring any value to them, nor any money to - us". -

-

- That UI has to be there. It's where they check some settings. Reset their password. The boring stuff. - But having achieved being functional, there isn't much more value to provide in improving it. -

-

- I think it's important to identify which are your bathrooms and make sure you act accordingly. I find - it's not enough to only care about making the important stuff top priority: it helps to also make it - clear what's not important, and be explicit about it being low priority. Just like when I define the - scope for something, I like to both think in terms of what are we including, and also making a explicit - list of what we are NOT including for the sake of clarity. Theoretically, just listing the positive list - should be enough. In reality, my experience tells me making the negative helps a lot. -

-

- So, what are your bathrooms? Are you cleaning them with a toothbrush? Or you have them nice and dirty? -

-
-

back to home

-
-
- - - - \ No newline at end of file