A wedged explorer produced 83 Signal messages in six hours. Nothing about that
was a monitoring success: the checks were right, the cadence was wrong.
monerod-box is a Monero node in the middle of a multi-week initial sync on a
rotating disk. A 5-minute poll tells you nothing a daily one does not, and when
something breaks it turns one fault into a message every five minutes until
somebody silences the phone - which is the failure mode that makes people stop
reading alerts entirely.
So both Monero probes move to OnCalendar:
monerod *-*-* 05:00:00 RandomizedDelaySec=900
xmrblocks *-*-* 05:30:00 RandomizedDelaySec=900
systemd units (infra/401 per-host timer) *-*-* 05:15:00
30 minutes apart on purpose: both read the same LMDB, and the explorer's check
renders a full page out of it - a cold one measured 68 seconds. Running them
together would have each paying the other's cache misses.
OnCalendar rather than OnUnitActiveSec: an interval timer re-bases on every
restart, so "daily" silently becomes "a day after whenever it last came up".
── The window and the cadence are ONE decision ─────────────────────────────
The Gatus heartbeat moves with them, to 26h. Not 24h: the window must EXCEED
the push interval, or a check reporting exactly on schedule races its own
deadline and alerts on jitter alone. With a 900s randomized delay the true
worst case is ~24h15m, so 26h leaves real headroom.
The same logic drops the pulled public-site checks to a 24h interval - and
their failure-threshold from 3 to 1 with it. Three failures at 5m is 15 minutes
of tolerance; three at 24h would be THREE DAYS before anyone heard. A daily
check has no room for a blip, so it alerts on the first.
── Scoping ────────────────────────────────────────────────────────────────
infra/401's timer is ONE PER HOST and iterates every unit in
monitored_services, so slowing it slows all of them. That is safe here only
because monerod-box runs nothing but Monero units; the knobs are therefore in
its host_vars rather than the role, and both 401 and 403 now take a per-host
(resp. per-probe) override with the old 11m as the default. Every other host
is untouched.
Host liveness and disk are deliberately NOT slowed. They answer "is the box
alive", which is still wanted within minutes.
The role-level OnCalendar defaults are prefixed (monero_healthcheck_on_calendar,
xmrblocks_healthcheck_on_calendar) because the generic `healthcheck` role owns
the unprefixed name, and an unprefixed default here would shadow it for any
play that includes both.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
||
|---|---|---|
| .. | ||
| defaults | ||
| handlers | ||
| tasks | ||
| templates | ||
| README.md | ||
xmrblocks
The Onion Monero Blockchain Explorer,
on monerod-box beside the node, published at exmrplore.contrapeso.xyz.
Why this role installs a binary instead of building one
Every other compiled service in this estate builds on its own host —
bitcoin_knots and datum_gateway both do. This one cannot, and the reason is
worth recording so nobody "fixes" it later.
xmrblocks is not a client of monerod. Its cmake/FindMonero.cmake imports 21
static libraries out of a compiled monero source tree and its headers, because
it opens the LMDB directly rather than going through RPC. Building it therefore
means compiling monero first — and the release tarball we install for the daemon
contains only executables, no .a files and no headers, so there is nothing on
the box to link against.
Measured on Debian 13, building only the libraries xmrblocks needs:
peak RSS, one cc1plus |
~3.2 GB |
peak RSS, -j4 concurrent |
~5.5 GB |
wall time, -j4 |
167 s |
| build tree | 125 MB (source 70 MB) |
| toolchain | 1.6 GB |
monerod-box had 2 GB when this was measured (it has since been raised to
6 GB, for monerod's own sake — see below). It was tried at 768 MiB and 1800 MiB,
at -j1 and -j2, and every combination was OOM-killed — a single translation unit
exceeding 3 GB is not something lower parallelism can work around. Disk was
never the constraint; the often-quoted "~10 GB" for a monero build describes
upstream's full make, which also produces monerod, both wallets and the test
suite, none of which are linked here.
So the binary is built elsewhere by scripts/build-xmrblocks.sh, which runs
the whole thing in a debian:13 container. That matters: an artifact only one
machine can produce is an artifact nobody can audit. Any host with Docker and
~6 GB of RAM reproduces it, and the script verifies the monero source against
binaryFate's GPG signature exactly as roles/monero does for the daemon.
The role refuses to ship anything it has not checked: xmrblocks_sha256 is
asserted on the control node, before the copy, against the value the build
script prints.
Three upstream problems, and the fixes
None of these are documented anywhere. All three were found by building.
-
libasio-devis missing from upstream's dependency list. The Dockerfile is Ubuntu-based and installslibboost-all-dev; Crow wants standaloneasio.hpp, notboost/asio.hpp. On Debian 13 the build dies withasio.hpp: No such file or directory. -
The explorer does not compile against monero v0.18.5.1. v0.18.5.0 changed
crypto::hash get_pruned_transaction_hash(const transaction&, const crypto::hash&); bool get_pruned_transaction_hash(const transaction&, const crypto::hash&, crypto::hash& res);and
src/page.hstill calls the old form. Upstream's Dockerfile pinsMONERO_BRANCH=v0.18.4.0, which is why nobody has hit it. The build script applies a one-line patch, anchored on the exact source line so that if upstream ever fixes it the build stops loudly instead of silently doing something else.We patch rather than build against v0.18.4.0 deliberately: the current signed
hashes.txtonly covers the current release, so using the older source would mean giving up the provenance chain. The LMDB format isVERSION 5in both, so either version reads the other's database safely. -
FindMonero.cmakelooks for miniupnpc inside the monero build tree withNO_DEFAULT_PATH, but monero stopped vendoring miniupnp as a submodule. Fixed with a-D Xmr_MINIUPNPC_LIBRARY=cache override pointing at the system library, rather than patching their cmake.
USE_DEVICE_TREZOR=OFF is also passed: an explorer has no use for hardware
wallet support, and it drops the generated protobuf units, which are among the
heaviest to compile. xmrblocks was verified to link against the resulting
libraries.
Two things the unit gets right that look wrong
WorkingDirectory is load-bearing. src/page.h hardcodes
#define TMPL_DIR "./templates" — the HTML is resolved relative to the process
working directory, at request time. Get it wrong and the service starts happily,
listens on 8081, and returns 500 for every page. That is also why the health
check fetches a real page and greps it rather than just probing the port.
ReadWritePaths must include the blockchain directory, even though the
database is opened read-only. MicroCore::init sets DBF_RDONLY, and monero's
db_lmdb.cpp turns that into mdb_flags = MDB_RDONLY without MDB_NOLOCK —
so LMDB still registers this process in the reader table inside lock.mdb. With
ProtectSystem=strict and nothing writable, the open fails outright. The
intuitive "it only reads, grant it nothing" is wrong. data.mdb itself remains
read-only to the process, and writing to monerod's lock table is what any second
reader does.
Exposure
Published by caddy_site on the edge host at exmrplore.contrapeso.xyz, with
TLS terminated there and the upstream reached over the tailnet via MagicDNS.
The explorer listens on 0.0.0.0:8081. Nothing on the LAN or the internet
reaches that directly: Tailscale's ts-input chain sits ahead of UFW, so only
tailnet peers can, and the Headscale ACL grants tag:prd-vipy exactly
tag:monerod-box:8081. A node with no ACL rule is invisible rather than
merely unreachable — it never appears in tailscale status, MagicDNS does not
resolve it, and the symptom is a Caddy 502 saying "no such host".
There is no authentication, by choice. A block explorer serves public chain
data. But note what that includes: --enable-pusher is on (upstream's default),
so any visitor can submit a raw transaction and have our node relay it. That is
a deliberate decision, not an oversight — drop xmrblocks_enable_pusher to
false to close it.
--enable-json-api and --enable-autorefresh-option are likewise upstream's
defaults.
Monitoring
| Endpoint | Source | Answers |
|---|---|---|
services/monerod-box-local/xmrblocks |
infra/401 via monitored_services |
is the unit active |
probe/xmrblocks |
this role + infra/403 |
does it serve a real page |
dns/exmrplore.contrapeso.xyz, public/exmrplore |
infra/402 |
can the internet reach it |
public/exmrplore asserts 200, unlike the Monero RPC endpoint next door
which asserts 401 — this one genuinely has no auth in front of it.
The timer carries no Requires=xmrblocks.service; see the comment in
healthcheck.timer.j2 for why that would be backwards.
Backups: none, deliberately
Everything here is derived or reproducible: the binary comes from
scripts/build-xmrblocks.sh, the templates ship with it, and the data it serves
is monerod's blockchain, which is itself excluded for the same reason. There is
no state on this service worth a backup job. Recorded as a decision so it does
not read as an oversight — the same treatment phoenixd and monero get.
Upgrading
- Edit the pins at the top of
scripts/build-xmrblocks.sh(EXPLORER_COMMIT, andMONERO_VERSION+MONERO_SRC_SHA256together). - Run it. It prints the new artifact sha256.
- Put that in
xmrblocks_sha256andxmrblocks_explorer_commithere. - Converge. The role asserts the checksum before copying anything.
If the patch anchor no longer matches, the script stops — read the diff before forcing it.
Runtime cost, and the RAM history
On 2026-09-20 the VM was raised from 2 GB to 6 GB, and that was not for the explorer. monerod alone had outgrown 2 GB once the chain passed ~39%: it was OOM-killed twice in one morning at 1.74 GB anon-rss, restarting every hour. With 6 GB the box sits at ~1.2 GB used of 5.9 GB with both services running (monerod ~1.0 GB, xmrblocks ~96 MB).
The disk is the real performance limit, not memory. proxmox-tank-1 is a
spinning-disk mirror, and xmrblocks says so on startup: "The blockchain is on a
rotating drive: this will be very slow, use an SSD if possible". Measured
consequences, both of which the role accounts for:
- first start takes ~98 seconds to open the LMDB before it binds, so
wait_forallows 300s - a COLD index page took 68 seconds, while the two requests after it took 0.87s and 0.012s — so the health check allows 90s and its unit 180s. A short timeout there does not detect a slow explorer, it manufactures false DOWNs.
Footprint note: xmrblocks mmaps the LMDB, so most of its apparent size is
reclaimable page cache rather than anonymous memory, and --concurrency is held
at 2 for the same reason.