personal_infra/ansible/services/bitcoin-knots/deploy_bitcoin_knots_playbook.yml

36 lines
1.6 KiB
YAML
Raw Normal View History

bitcoin-knots: convert to a role, de-Uptime-Kuma the health check 892-line playbook becomes 40 lines plus a role with install/build/configure/ service/healthcheck phases, five templates and one handler. bitcoin_knots_vars.yml is deleted; its content is the role's defaults. Verified after a real run: bitcoind still active since 2026-08-19 (NO restart), chain at 966844 blocks / 875 GB, DATUM config intact, dbcache still 200, health check timer firing again. changed=3, all health-check. vipy changed=0. ⚠ THE BIG ONE: the playbook would have deleted the mining integration. bitcoin.conf on the node carries a section that was hand-added and was missing from the template entirely: blockmaxsize=3985000 blockmaxweight=3985000 blocknotify=killall -USR1 datum_gateway maxmempool=1000 blockreconstructionextratxn=1000000 blocknotify is how datum_gateway learns a new block landed. Running the old playbook would have stripped all of it and solo mining would have carried on against a stale template - a silent failure that costs money rather than raising an error. Also dbcache 200 -> 3528 (hand-tuned down; the calculation wants 90% of RAM) and logging moved off the file. All now reconciled, dbcache behind bitcoin_dbcache_mb_override. bitcoin-knots and datum-gateway are ONE SYSTEM. Noted in the README. AND MY OWN FIX MADE IT MORE DANGEROUS. The `Restart bitcoind` handler was guarded by uptime_kuma_enabled, so it had been inert: bitcoin.conf and the systemd unit both notify it and neither could restart anything - a config change applied to disk, reported success, and never took effect. Ungating that is right, but it converts "wrong config sitting inertly on disk" into "node restarted onto a config that breaks mining". The ungating had to land WITH the template reconciliation, not before it. It also raises the bar permanently: any residual template/live difference now restarts a Bitcoin node on every run. Four rounds of --check --diff to reach changed=0 - the DATUM section, an explanatory comment that was rendering into the deployed config (now a {# #} Jinja comment), a "# Pruning (optional)" comment the live file had, and one trailing blank line. The build path is 32 tasks all guarded by `not bitcoind_binary_exists.stat.exists`, so a converged host skips the 30-60 minute compile and both `state: absent` deletions. Those target /opt/bitcoin-knots/{source,bitcoin-<version>}; the chain is in /mnt/knots_data and is never touched. Signature-verification tasks copied verbatim. The health check timer had last fired 2026-08-09 while reporting active/enabled - same OnBootSec + OnUnitActiveSec dead chain as fulcrum. The role runs the check once after enabling to supply the reference the timer schedules from. Ownership parity checked mechanically against `git show HEAD:`, keyed by TASK NAME rather than path - keying by path gave a false positive, because bitcoin_knots_source_dir is created with ownership and later removed with state: absent, so whichever task comes last wins and that differs between one file and five. 13/13 match. bitcoin_p2p_port and the tailscale hostname moved to services_config.yml for the socket-proxy play on the edge host. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-13 18:13:40 +02:00
---
# Bitcoin Knots: full node built from source, with PGP signature and SHA256
# verification of the release tarball. The build is guarded by a binary-exists
# check, so a converged host skips the whole 30-60 minute compile.
#
# The chain lives in bitcoin_large_data_dir (/mnt/knots_data, ~875 GB). Nothing
# here touches it; the only `state: absent` tasks target the build tree under
# /opt/bitcoin-knots and run only when the binary is missing.
2025-12-08 10:34:04 +01:00
- name: Build and Deploy Bitcoin Knots from Source
hosts: bitcoin
2025-12-08 10:34:04 +01:00
become: yes
vars:
bitcoin-knots: convert to a role, de-Uptime-Kuma the health check 892-line playbook becomes 40 lines plus a role with install/build/configure/ service/healthcheck phases, five templates and one handler. bitcoin_knots_vars.yml is deleted; its content is the role's defaults. Verified after a real run: bitcoind still active since 2026-08-19 (NO restart), chain at 966844 blocks / 875 GB, DATUM config intact, dbcache still 200, health check timer firing again. changed=3, all health-check. vipy changed=0. ⚠ THE BIG ONE: the playbook would have deleted the mining integration. bitcoin.conf on the node carries a section that was hand-added and was missing from the template entirely: blockmaxsize=3985000 blockmaxweight=3985000 blocknotify=killall -USR1 datum_gateway maxmempool=1000 blockreconstructionextratxn=1000000 blocknotify is how datum_gateway learns a new block landed. Running the old playbook would have stripped all of it and solo mining would have carried on against a stale template - a silent failure that costs money rather than raising an error. Also dbcache 200 -> 3528 (hand-tuned down; the calculation wants 90% of RAM) and logging moved off the file. All now reconciled, dbcache behind bitcoin_dbcache_mb_override. bitcoin-knots and datum-gateway are ONE SYSTEM. Noted in the README. AND MY OWN FIX MADE IT MORE DANGEROUS. The `Restart bitcoind` handler was guarded by uptime_kuma_enabled, so it had been inert: bitcoin.conf and the systemd unit both notify it and neither could restart anything - a config change applied to disk, reported success, and never took effect. Ungating that is right, but it converts "wrong config sitting inertly on disk" into "node restarted onto a config that breaks mining". The ungating had to land WITH the template reconciliation, not before it. It also raises the bar permanently: any residual template/live difference now restarts a Bitcoin node on every run. Four rounds of --check --diff to reach changed=0 - the DATUM section, an explanatory comment that was rendering into the deployed config (now a {# #} Jinja comment), a "# Pruning (optional)" comment the live file had, and one trailing blank line. The build path is 32 tasks all guarded by `not bitcoind_binary_exists.stat.exists`, so a converged host skips the 30-60 minute compile and both `state: absent` deletions. Those target /opt/bitcoin-knots/{source,bitcoin-<version>}; the chain is in /mnt/knots_data and is never touched. Signature-verification tasks copied verbatim. The health check timer had last fired 2026-08-09 while reporting active/enabled - same OnBootSec + OnUnitActiveSec dead chain as fulcrum. The role runs the check once after enabling to supply the reference the timer schedules from. Ownership parity checked mechanically against `git show HEAD:`, keyed by TASK NAME rather than path - keying by path gave a false positive, because bitcoin_knots_source_dir is created with ownership and later removed with state: absent, so whichever task comes last wins and that differs between one file and five. 13/13 match. bitcoin_p2p_port and the tailscale hostname moved to services_config.yml for the socket-proxy play on the edge host. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-13 18:13:40 +02:00
# Preserves the push URL this check has been reporting to. The role knows
# nothing about Uptime Kuma — this is just "a URL that accepts a ping".
uptime kuma: remove every live reference, repoint the probes to Gatus Nothing in the repo pushes to, authenticates against, or is gated by Uptime Kuma any more. ── The sixth instance of the banner bug ──────────────────────────────────── memos had `Restart memos` guarded by `uptime_kuma_enabled`, because the deprecation banner was placed immediately above it and swept it in. It is a HANDLER, so every memos config change since 2026-09-11 applied to disk and silently never restarted the service. Ungated. That is the same failure found in forgejo-runner's self-assert, phoenixd's timer enable, mempool's three timer enables, fulcrum's restart handler and bitcoind's restart handler. Every guard was read and asked "monitoring or deployment?" before being deleted, which is the only reason this was caught. ── What was removed ──────────────────────────────────────────────────────── 30 uptime_kuma_enabled guards across 7 unconverted service playbooks, and the 29 Kuma monitor-creation tasks they gated (embedded Python that drove the Kuma API, temp credential files, cleanup) 7 dead uptime_kuma_api_url definitions 7 stale DEPRECATED banners uptime_kuma_enabled and subdomains.uptime_kuma from group_vars/all healthcheck_push_urls from the vault - 30 push tokens services/ntfy/setup_ntfy_uptime_kuma_notification.yml -> archive/ The explanatory comments in the six converted roles are KEPT on purpose. They record why a handler is ungated, and deleting the explanation invites someone to helpfully re-add the guard. ── The probes moved rather than died ─────────────────────────────────────── Eight per-service health checks were still pushing to Kuma. They are not superseded by infra/401: that answers "is the unit running", these answer "does the service actually respond" - an RPC call to bitcoind, a TCP connect to Fulcrum's Electrum port, an HTTP fetch from Mempool's backend. A process can be perfectly `active` and useless. So they were repointed, not deleted. Gatus external endpoints take a POST with a bearer token and success=true|false where Kuma took a GET with ?status=up, so report() now maps up/down to true/false internally and no call site changed. Registered by infra/403 as the `probe` group, one token per host. Two bugs fixed while in there: * forgejo-runner's check only ever reported SUCCESS - it exited before pushing when the runner was down, so a failure was invisible until the heartbeat window expired. Reporting the failure is the entire point of a check. * All six healthcheck .service units were mode 0644 and now carry a bearer token. They are 0600. Verified: 91 endpoints, 91 UP, 0 DOWN. Every probe triggered by hand and confirmed arriving. Zero Kuma URLs left in the vault, zero live references in any playbook or role. Still standing, deliberately: the Kuma container on watchtower, its Caddy vhost, and the uptime.contrapeso.xyz DNS record. Turning the service off is a separate decision from removing the code that talked to it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-14 10:30:28 +02:00
healthcheck_push_url: "https://{{ subdomains.gatus }}.{{ root_domain }}/api/v1/endpoints/probe_bitcoin-knots/external"
healthcheck_push_token: "{{ gatus_push_tokens[inventory_hostname] }}"
bitcoin-knots: convert to a role, de-Uptime-Kuma the health check 892-line playbook becomes 40 lines plus a role with install/build/configure/ service/healthcheck phases, five templates and one handler. bitcoin_knots_vars.yml is deleted; its content is the role's defaults. Verified after a real run: bitcoind still active since 2026-08-19 (NO restart), chain at 966844 blocks / 875 GB, DATUM config intact, dbcache still 200, health check timer firing again. changed=3, all health-check. vipy changed=0. ⚠ THE BIG ONE: the playbook would have deleted the mining integration. bitcoin.conf on the node carries a section that was hand-added and was missing from the template entirely: blockmaxsize=3985000 blockmaxweight=3985000 blocknotify=killall -USR1 datum_gateway maxmempool=1000 blockreconstructionextratxn=1000000 blocknotify is how datum_gateway learns a new block landed. Running the old playbook would have stripped all of it and solo mining would have carried on against a stale template - a silent failure that costs money rather than raising an error. Also dbcache 200 -> 3528 (hand-tuned down; the calculation wants 90% of RAM) and logging moved off the file. All now reconciled, dbcache behind bitcoin_dbcache_mb_override. bitcoin-knots and datum-gateway are ONE SYSTEM. Noted in the README. AND MY OWN FIX MADE IT MORE DANGEROUS. The `Restart bitcoind` handler was guarded by uptime_kuma_enabled, so it had been inert: bitcoin.conf and the systemd unit both notify it and neither could restart anything - a config change applied to disk, reported success, and never took effect. Ungating that is right, but it converts "wrong config sitting inertly on disk" into "node restarted onto a config that breaks mining". The ungating had to land WITH the template reconciliation, not before it. It also raises the bar permanently: any residual template/live difference now restarts a Bitcoin node on every run. Four rounds of --check --diff to reach changed=0 - the DATUM section, an explanatory comment that was rendering into the deployed config (now a {# #} Jinja comment), a "# Pruning (optional)" comment the live file had, and one trailing blank line. The build path is 32 tasks all guarded by `not bitcoind_binary_exists.stat.exists`, so a converged host skips the 30-60 minute compile and both `state: absent` deletions. Those target /opt/bitcoin-knots/{source,bitcoin-<version>}; the chain is in /mnt/knots_data and is never touched. Signature-verification tasks copied verbatim. The health check timer had last fired 2026-08-09 while reporting active/enabled - same OnBootSec + OnUnitActiveSec dead chain as fulcrum. The role runs the check once after enabling to supply the reference the timer schedules from. Ownership parity checked mechanically against `git show HEAD:`, keyed by TASK NAME rather than path - keying by path gave a false positive, because bitcoin_knots_source_dir is created with ownership and later removed with state: absent, so whichever task comes last wins and that differs between one file and five. 13/13 match. bitcoin_p2p_port and the tailscale hostname moved to services_config.yml for the socket-proxy play on the edge host. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-13 18:13:40 +02:00
roles:
- bitcoin_knots
2025-12-15 23:33:08 +01:00
- name: Setup public Bitcoin P2P forwarding on the edge host
hosts: edge
2025-12-15 23:33:08 +01:00
become: yes
tasks:
bitcoin-knots, fulcrum, datum-gateway: add and use the socket_proxy role Three near-identical hosts: edge plays become one role plus three short calls. 183 lines removed, 34 added, plus a 111-line role. Verified before touching any playbook: all six live units on vipy reproduced byte-identically. Then --limit edge --check per playbook - bitcoin-knots and fulcrum changed=0; datum-gateway changed=2, both attributable to the already known caddy_site comment line and the Reload caddy handler it triggers. The 6 units and 14 Caddy files on the hosts are byte-identical afterwards. PLAN_4 claimed these three plays had "no behavioural drift at all". That was wrong - it came from a diff truncated by head -60. The live bitcoin-p2p-proxy units carry four settings this playbook never wrote: .socket Documentation=, FreeBind=true .service Documentation=, TimeoutStopSec=5, StandardOutput=journal, StandardError=journal FreeBind is the one that matters: it lets the socket bind to an address that is not up yet, so without it the socket can fail to start on boot. Running the bitcoin-knots playbook would have stripped it. Same class of hazard as headscale. The role expresses all four; bitcoin-p2p is the only caller that passes any. Also: UFW treats the rule comment as part of the rule. datum-stratum's live comment is "DATUM Gateway Stratum public access" but the role's derived default produced "DATUM Stratum public access", which rewrote the rule. Caught in the dry-run; datum now passes the comment explicitly. Two deliberate differences from the original, both documented in the README: ignore_errors: yes on the upstream check became failed_when: false, and the handler restarts the .socket, which drops connections open through it - it fires only when a unit file actually changes. The inert Uptime Kuma TCP monitor blocks stay in the playbooks rather than being pulled into a new role (12/12/18 guarded tasks). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-11 23:50:15 +02:00
- name: Expose Bitcoin P2P through a socket proxy
ansible.builtin.include_role:
name: socket_proxy
vars:
socket_proxy_name: bitcoin-p2p
socket_proxy_description: "Bitcoin P2P"
ansible: move the cross-host ports to host_vars, delete services_config.yml The four ports were the only entries in services_config.yml with a real justification: each is read twice, by the role that deploys the service on its own box AND by a socket-proxy or Caddy play that runs on the EDGE host and publishes it. A role default is invisible to that second play. But the shape was wrong in two ways. The file had to be named in vars_files: by 30 plays - opt-in configuration that someone will eventually forget - and five role defaults silently interpolated service_settings.*, so bitcoin_knots, fulcrum, datum_gateway and mempool were not self-contained: using any of them without that one vars_file entry broke it. Each port now lives in host_vars/<owning box>/main.yml: host_vars/knots_box_local/main.yml bitcoin_p2p_port, datum_gateway_api_port, datum_gateway_stratum_port host_vars/fulcrum_box_local/main.yml fulcrum_ssl_port host_vars/mempool_box_local/main.yml mempool_frontend_port host_vars auto-loads and outranks role defaults, so the owning role picks the value up with no vars_files at all, and the edge play reads the same single definition as hostvars['<host>'].<name>. The role defaults keep the protocol standard (8333, 50002, ...) so each role still works standalone, with the live deployment's value in host_vars winning. Also fixed a fourth copy of an inventory identity: the mempool Caddy play had "mempool-box:{{ ... }}" hardcoded in the upstream. It now derives the host from hostvars['mempool_box_local'].ansible_host, so inventory is the only place any box's name is written down. services_config.yml is deleted, with 25 more vars_files entries across 19 playbooks. Between this and the previous commit, 87 vars_files entries are gone and every variable in the repo now comes from group_vars/all, host_vars, inventory, a role default, or that service's own *_vars.yml. Verification: an edge-host probe resolves all eight ports and hostnames to byte-identical values to the ones services_config.yml used to supply. Each owning host resolves its own port through host_vars. All 37 playbooks' --list-tasks output is unchanged. The four edge plays that consume these values all check-diff changed=0 - the socket-proxy and Caddy units on vipy are byte-identical, which is the direct proof the rewiring landed on the same values. fulcrum and datum-gateway check-diff exactly as before (ok=28/changed=1 and ok=15/changed=1, both the known timer re-arm). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-13 21:02:57 +02:00
socket_proxy_listen_port: "{{ hostvars['knots_box_local'].bitcoin_p2p_port }}"
ansible: delete the duplicated vars files, move globals to group_vars/all Three files existed only as second copies of things group_vars/all already auto-loads, and 34 playbooks named them in vars_files: - which outranks group_vars, so the copies won. The day someone edited one and not the other, those plays would silently keep the stale value. infra_vars.yml was already drifting: group_vars/all/main.yml had grown age_backup_recipient and backup_pull_public_key that it lacked. infra_vars.yml - a strict subset of group_vars/all/main.yml infra_secrets.yml - decrypts byte-identical to group_vars/all/vault.yml infra_secrets.yml.example - documented Uptime Kuma credentials as the reason the file exists, which stopped being true Deleted, along with 62 vars_files entries across 34 playbooks (12 of which named ../../group_vars/all/main.yml directly - same defect, a vars_files entry duplicating an auto-loaded file at higher precedence than the file itself). Checked before touching anything: infra_secrets.yml was listed LAST in 10 plays, after services_config.yml, so removal would flip precedence if the two shared a key. They share none, and neither does services_config.yml with group_vars/all/main.yml, so the removal is provably inert. services_config.yml was the last one standing. It held four unrelated things: caddy_sites_dir - an identical copy of roles/caddy_site/defaults/. Deleted; the role default is now the only one. *.tailscale_hostname (x3) - a THIRD copy of each box's identity, which inventory.ini already holds as ansible_host. Deleted. Edge plays now read hostvars['<host>'].ansible_host - verified an edge play resolves that with nothing loaded and the other host in no play. Three copies of one name is how bitcoin_rpc_host ended up labelled "knots_box" while pointing at fulcrum-box. subdomains, ntfy topic, - genuinely global: their readers span managed, headscale namespace monitoring, vpn_control and edge, so no single group covers them. Moved to group_vars/all/main.yml where they auto-load. The ntfy_topic and headscale_namespace indirection through service_settings collapses to the global name. the four cross-host ports - the only entries with a real justification. Left in place; they move in the next commit. Also dead, all Uptime Kuma residue or duplication: phoenixd_monitor_name, forgejo_runner healthcheck_timeout_seconds/retries, fulcrum_tailscale_hostname, and bitcoin_knots_version - the last being a v-prefixed copy of bitcoin_knots_version_short that nothing read, two hand-maintained copies of one version string. Corrected a false comment: services_config.yml claimed the uptime_kuma subdomain "no longer resolves to anything". It resolves to 164.92.239.72 and answers HTTP 302, and 11 playbooks still template it. Same wrong premise as PLAN_3. Verification: all 37 playbooks' --list-tasks output is byte-identical before and after. A probe resolving all 22 values services_config.yml used to supply returns 21 identical and one intended deletion (caddy_sites_dir, now role-only - confirmed the role still resolves it: "Ensure Caddy sites-enabled directory exists" comes back ok against the real path). memos check-diff identical before and after. Syntax passes on every playbook. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-13 20:58:46 +02:00
socket_proxy_upstream_host: "{{ hostvars['knots_box_local'].ansible_host }}"
bitcoin-knots, fulcrum, datum-gateway: add and use the socket_proxy role Three near-identical hosts: edge plays become one role plus three short calls. 183 lines removed, 34 added, plus a 111-line role. Verified before touching any playbook: all six live units on vipy reproduced byte-identically. Then --limit edge --check per playbook - bitcoin-knots and fulcrum changed=0; datum-gateway changed=2, both attributable to the already known caddy_site comment line and the Reload caddy handler it triggers. The 6 units and 14 Caddy files on the hosts are byte-identical afterwards. PLAN_4 claimed these three plays had "no behavioural drift at all". That was wrong - it came from a diff truncated by head -60. The live bitcoin-p2p-proxy units carry four settings this playbook never wrote: .socket Documentation=, FreeBind=true .service Documentation=, TimeoutStopSec=5, StandardOutput=journal, StandardError=journal FreeBind is the one that matters: it lets the socket bind to an address that is not up yet, so without it the socket can fail to start on boot. Running the bitcoin-knots playbook would have stripped it. Same class of hazard as headscale. The role expresses all four; bitcoin-p2p is the only caller that passes any. Also: UFW treats the rule comment as part of the rule. datum-stratum's live comment is "DATUM Gateway Stratum public access" but the role's derived default produced "DATUM Stratum public access", which rewrote the rule. Caught in the dry-run; datum now passes the comment explicitly. Two deliberate differences from the original, both documented in the README: ignore_errors: yes on the upstream check became failed_when: false, and the handler restarts the .socket, which drops connections open through it - it fires only when a unit file actually changes. The inert Uptime Kuma TCP monitor blocks stay in the playbooks rather than being pulled into a new role (12/12/18 guarded tasks). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-11 23:50:15 +02:00
socket_proxy_documentation: "https://github.com/bitcoin/bitcoin"
socket_proxy_free_bind: true
socket_proxy_timeout_stop_sec: 5
socket_proxy_log_to_journal: true