xmrblocks: the Monero explorer, built off-box because 2 GB cannot compile it

The Onion Monero Blockchain Explorer on monerod-box, published at
exmrplore.contrapeso.xyz. Ten Gatus endpoints green across the node and the
explorer.

── It needs a compiled monero source tree, not a running monerod ───────────
xmrblocks is not an RPC client. cmake/FindMonero.cmake imports 21 STATIC
libraries out of a monero build tree and includes its headers, because it opens
the LMDB directly. The release tarball we install for the daemon ships only
executables - no .a files, no headers - so there is nothing on the box to link
against, and building it means compiling monero first.

Measured on Debian 13, building only the libraries it links:

    peak RSS, one cc1plus      ~3.2 GB
    peak RSS at -j4            ~5.5 GB
    wall time at -j4           167 s
    source + build tree        195 MB
    toolchain                  1.6 GB

That rules the box out at ANY parallelism: 768 MiB and 1800 MiB were both
OOM-killed at -j1 and -j2, because a single translation unit over 3 GB is not
something lower -j works around. My first estimate of "~10 GB of disk" was
wrong by two orders of magnitude - that figure describes upstream's full `make`,
which also builds monerod, both wallets and the test suite, none of it linked
here. Disk was never the constraint.

So scripts/build-xmrblocks.sh does the build in a debian:13 container. An
artefact only one machine can produce is one nobody can audit; this way any host
with Docker and ~6 GB reproduces it. It verifies the monero source against
binaryFate's signature exactly as roles/monero does for the daemon, and the role
asserts the resulting sha256 ON THE CONTROL NODE before copying a single byte.

── Three upstream problems, none documented anywhere ───────────────────────
Found by building, not by reading:

  1. libasio-dev is absent from upstream's dependency list. That list is
     Ubuntu-based; Crow wants standalone asio.hpp, not boost/asio.hpp, and on
     Debian 13 the build dies outright.

  2. The explorer does not compile against monero v0.18.5.1. v0.18.5.0 changed
     get_pruned_transaction_hash from returning the hash to returning bool with
     an out-param, and src/page.h still calls the old form. Upstream's
     Dockerfile pins v0.18.4.0, which is why nobody has hit it. Patched from an
     exact-line anchor, so if upstream fixes it the build STOPS rather than
     quietly doing something else. We patch instead of building against 0.18.4.0
     because the signed hashes.txt only covers the current release - using the
     older source would mean dropping the provenance chain. LMDB is VERSION 5 in
     both, so either reads the other's database.

  3. FindMonero.cmake hunts for miniupnpc inside the monero build tree with
     NO_DEFAULT_PATH, but monero stopped vendoring miniupnp as a submodule.
     Fixed with a cache override rather than patching their cmake.

USE_DEVICE_TREZOR=OFF as well - an explorer has no use for hardware wallets and
it drops the generated protobuf units. Verified that xmrblocks still links.

── Two things in the unit that look wrong and are not ──────────────────────
WorkingDirectory is load-bearing: page.h hardcodes TMPL_DIR as "./templates",
resolved relative to cwd at REQUEST time. Get it wrong and the service starts,
binds 8081, and 500s every page. That is also why the health check fetches a
real page and greps it instead of probing the port.

ReadWritePaths must include the blockchain directory even though the database is
opened read-only: monero sets mdb_flags = MDB_RDONLY WITHOUT MDB_NOLOCK, so
LMDB still registers this reader in lock.mdb. With ProtectSystem=strict and
nothing writable the open fails. "It only reads, so grant it nothing" is wrong.

── Timeouts are measured, not guessed ──────────────────────────────────────
proxmox-tank-1 is a spinning-disk mirror and xmrblocks says so on startup.
First start took 98s to open the LMDB before binding (wait_for now allows 300s),
and a COLD index page took 68 SECONDS where the next two took 0.87s and 0.012s
(the check now allows 90s, its unit 180s). The first pass used 90s and 25s and
produced exactly the false failures you would expect.

A stat() bug was caught the same way: `stat` computes a sha1 by default, so
checking that data.mdb exists was hashing the entire 40 GB blockchain. --check
found it before a real run did.

── Exposure ────────────────────────────────────────────────────────────────
Caddy on vipy terminates TLS; the upstream is reached over the tailnet, which
needed a headscale ACL entry for tag:monerod-box:8081 - without it the node is
invisible, not merely unreachable, and the symptom is a 502 "no such host".

No authentication, deliberately: it is a block explorer over public chain data.
That includes --enable-pusher, so a visitor can relay a raw transaction through
our node. A choice, not an oversight; flip xmrblocks_enable_pusher to close it.

── Also ────────────────────────────────────────────────────────────────────
monerod-box raised 2048 -> 6144 MiB. Not for the explorer: monerod alone had
outgrown 2 GB past ~39% of the chain and was OOM-killed twice in a morning at
1.74 GB anon-rss. Recorded in vms.auto.tfvars because ignore_changes = all means
tofu will not reconcile it. The box now idles at ~1.2 GB of 5.9 GB with both
services up.

No backups, deliberately - binary, templates and chain are all reproducible.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

ansible/roles/xmrblocks/templates/healthcheck.service.j2

@@ -0,0 +1,20 @@
+[Unit]
+Description=Onion Monero Blockchain Explorer Health Check
+After=network.target xmrblocks.service
+
+[Service]
+Type=oneshot
+User=root
+ExecStart=/usr/local/bin/xmrblocks-healthcheck-push.sh
+# Longer than the script's 90s curl plus its push retries. A oneshot defaults to
+# TimeoutStartSec=90, which would kill the check mid-request on a cold cache and
+# report a failure that says nothing about the service.
+TimeoutStartSec=180
+Environment=HEALTHCHECK_PUSH_URL={{ healthcheck_push_url }}
+Environment=HEALTHCHECK_PUSH_TOKEN={{ healthcheck_push_token }}
+Environment=XMRBLOCKS_URL=http://127.0.0.1:{{ xmrblocks_port }}
+StandardOutput=journal
+StandardError=journal
+
+[Install]
+WantedBy=multi-user.target

ansible/roles/xmrblocks/templates/healthcheck.sh.j2

@@ -0,0 +1,59 @@
+#!/bin/bash
+# xmrblocks health check — managed by Ansible (roles/xmrblocks)
+#
+# The exit code is the answer and systemd keeps it:
+#     systemctl is-failed xmrblocks-healthcheck.service
+# Reporting anywhere else is optional and generic.
+#
+# This fetches a real page rather than just opening the port. The binary
+# resolves its HTML as "./templates" relative to the working directory, so a
+# deploy that got WorkingDirectory wrong listens happily on 8081 and fails
+# every request. A TCP check would call that healthy.
+
+URL="${XMRBLOCKS_URL:-http://127.0.0.1:8081}"
+PUSH_URL="${HEALTHCHECK_PUSH_URL:-}"
+PUSH_TOKEN="${HEALTHCHECK_PUSH_TOKEN:-}"
+
+report() {
+    local status=$1
+    local msg=$2
+
+    # No push URL is normal, not an error: the exit code below is still a
+    # complete answer for anything reading unit state.
+    [ -n "$PUSH_URL" ] || return 0
+
+    local encoded_msg="${msg// /%20}"
+    local _ok=false
+    [ "${status}" = "up" ] && _ok=true
+    curl -s --max-time 15 --retry 2 -o /dev/null -X POST \
+        -H "Authorization: Bearer ${PUSH_TOKEN}" \
+        "${PUSH_URL}?success=${_ok}&error=${encoded_msg}" || true
+}
+
+# 90s, not the obvious 10 or 25. The index page renders ~46 KB from the LMDB,
+# and this pool is a spinning-disk mirror shared with a syncing monerod: a COLD
+# request measured 68 SECONDS, while the two immediately after it took 0.87s and
+# 0.012s. A short timeout here does not detect a slow explorer, it manufactures
+# a false DOWN every time the page cache has gone cold between runs.
+body=$(curl -s --max-time 90 -w '\n%{http_code}' "${URL}/" 2>/dev/null)
+code=$(printf '%s' "$body" | tail -1)
+
+if [ -z "$code" ] || [ "$code" = "000" ]; then
+    report "down" "xmrblocks not responding"
+    exit 1
+fi
+
+if [ "$code" != "200" ]; then
+    report "down" "xmrblocks HTTP ${code}"
+    exit 1
+fi
+
+# A 200 that is not the explorer is still a failure. The index page always
+# carries the project name in its title.
+if ! printf '%s' "$body" | grep -qi "monero"; then
+    report "down" "xmrblocks served 200 but not an explorer page (templates missing?)"
+    exit 1
+fi
+
+report "up" "OK"
+exit 0

ansible/roles/xmrblocks/templates/healthcheck.timer.j2

@@ -0,0 +1,14 @@
+[Unit]
+Description=Onion Monero Blockchain Explorer Health Check Timer
+# Deliberately NO `Requires=xmrblocks.service`. Requires on a timer stops the
+# timer when the watched unit stops - "if the thing I am watching goes down,
+# stop watching it" - leaving nothing to re-arm when it returns. Fulcrum's
+# timer carried it and sat active+enabled for seven months without firing.
+
+[Timer]
+OnBootSec=2min
+OnUnitActiveSec=5min
+Persistent=true
+
+[Install]
+WantedBy=timers.target

ansible/roles/xmrblocks/templates/xmrblocks.service.j2

@@ -0,0 +1,58 @@
+[Unit]
+Description=Onion Monero Blockchain Explorer (xmrblocks)
+Documentation=https://github.com/moneroexamples/onion-monero-blockchain-explorer
+After=network-online.target monerod.service
+Wants=network-online.target
+# The explorer reads monerod's LMDB directly. If the blockchain disk is not
+# mounted there is nothing to serve, and without this the unit would start and
+# fail on every request instead of refusing to start at all.
+RequiresMountsFor={{ xmrblocks_bc_path }}
+
+[Service]
+Type=simple
+User={{ xmrblocks_user }}
+Group={{ xmrblocks_group }}
+
+# NOT decoration: the binary hardcodes TMPL_DIR as "./templates", so it resolves
+# its HTML relative to the working directory. Without this it starts happily and
+# then 500s on the first page view.
+WorkingDirectory={{ xmrblocks_dir }}
+
+ExecStart={{ xmrblocks_dir }}/xmrblocks \
+    --bc-path {{ xmrblocks_bc_path }} \
+    --daemon-url {{ xmrblocks_daemon_url }} \
+    --daemon-login {{ xmrblocks_rpc_user }}:{{ xmrblocks_rpc_password }} \
+    --bindaddr {{ xmrblocks_bind }} \
+    --port {{ xmrblocks_port }} \
+    --concurrency {{ xmrblocks_concurrency }}{% if xmrblocks_enable_json_api %} \
+    --enable-json-api{% endif %}{% if xmrblocks_enable_autorefresh %} \
+    --enable-autorefresh-option{% endif %}{% if xmrblocks_enable_pusher %} \
+    --enable-pusher{% endif %}
+
+
+Restart=always
+RestartSec=20
+TimeoutStopSec=30
+
+StandardOutput=journal
+StandardError=journal
+
+# Hardening. The explorer serves static templates and reads the chain.
+NoNewPrivileges=true
+PrivateTmp=true
+ProtectSystem=strict
+ProtectHome=read-only
+
+# The LMDB directory must be WRITABLE even though the database is opened
+# read-only. monero sets `mdb_flags = MDB_RDONLY` without MDB_NOLOCK
+# (db_lmdb.cpp), so LMDB still registers this process in the reader table
+# inside lock.mdb. With ProtectSystem=strict and no ReadWritePaths the open
+# fails outright - the obvious "it only reads, so grant nothing" is wrong here.
+# Writing to monerod's own lock table is normal and is what any second reader
+# does; data.mdb itself stays read-only to this process.
+ReadWritePaths={{ xmrblocks_bc_path }}
+
+LimitNOFILE=16384
+
+[Install]
+WantedBy=multi-user.target