counterweight/personal_infra
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
personal_infra/ansible/services_config.yml

25 lines
948 B
YAML
Raw Normal View History

ansible: delete the duplicated vars files, move globals to group_vars/all Three files existed only as second copies of things group_vars/all already auto-loads, and 34 playbooks named them in vars_files: - which outranks group_vars, so the copies won. The day someone edited one and not the other, those plays would silently keep the stale value. infra_vars.yml was already drifting: group_vars/all/main.yml had grown age_backup_recipient and backup_pull_public_key that it lacked. infra_vars.yml - a strict subset of group_vars/all/main.yml infra_secrets.yml - decrypts byte-identical to group_vars/all/vault.yml infra_secrets.yml.example - documented Uptime Kuma credentials as the reason the file exists, which stopped being true Deleted, along with 62 vars_files entries across 34 playbooks (12 of which named ../../group_vars/all/main.yml directly - same defect, a vars_files entry duplicating an auto-loaded file at higher precedence than the file itself). Checked before touching anything: infra_secrets.yml was listed LAST in 10 plays, after services_config.yml, so removal would flip precedence if the two shared a key. They share none, and neither does services_config.yml with group_vars/all/main.yml, so the removal is provably inert. services_config.yml was the last one standing. It held four unrelated things: caddy_sites_dir - an identical copy of roles/caddy_site/defaults/. Deleted; the role default is now the only one. *.tailscale_hostname (x3) - a THIRD copy of each box's identity, which inventory.ini already holds as ansible_host. Deleted. Edge plays now read hostvars['<host>'].ansible_host - verified an edge play resolves that with nothing loaded and the other host in no play. Three copies of one name is how bitcoin_rpc_host ended up labelled "knots_box" while pointing at fulcrum-box. subdomains, ntfy topic, - genuinely global: their readers span managed, headscale namespace monitoring, vpn_control and edge, so no single group covers them. Moved to group_vars/all/main.yml where they auto-load. The ntfy_topic and headscale_namespace indirection through service_settings collapses to the global name. the four cross-host ports - the only entries with a real justification. Left in place; they move in the next commit. Also dead, all Uptime Kuma residue or duplication: phoenixd_monitor_name, forgejo_runner healthcheck_timeout_seconds/retries, fulcrum_tailscale_hostname, and bitcoin_knots_version - the last being a v-prefixed copy of bitcoin_knots_version_short that nothing read, two hand-maintained copies of one version string. Corrected a false comment: services_config.yml claimed the uptime_kuma subdomain "no longer resolves to anything". It resolves to 164.92.239.72 and answers HTTP 302, and 11 playbooks still template it. Same wrong premise as PLAN_3. Verification: all 37 playbooks' --list-tasks output is byte-identical before and after. A probe resolving all 22 values services_config.yml used to supply returns 21 identical and one intended deletion (caddy_sites_dir, now role-only - confirmed the role still resolves it: "Ensure Caddy sites-enabled directory exists" comes back ok against the real path). memos check-diff identical before and after. Syntax passes on every playbook. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-13 20:58:46 +02:00
# Cross-host service settings.
#
# These exist because a value is needed by the service's own role on one host
# AND by a play that runs on the edge host. A role default is invisible to that
# second play, so it cannot live in roles/<svc>/defaults/.
#
# Everything else that used to be here has moved:
# subdomains, ntfy topic, headscale namespace -> group_vars/all/main.yml
# caddy_sites_dir -> roles/caddy_site/defaults/
# *.tailscale_hostname -> deleted; inventory already
# holds each box's identity as ansible_host, and an edge play reads it with
# hostvars['<host>'].ansible_host. Three copies of one name is how
# bitcoin_rpc_host ended up labelled "knots_box" while pointing at
# fulcrum-box.
stuff
2025-11-14 23:36:00 +01:00
service_settings:
mempool: convert to a role, de-Uptime-Kuma the health checks 745-line playbook becomes 37 lines (the role, plus the Caddy play for the edge host) and a 408-line role with docker/deploy/healthcheck phases and six templates. mempool_vars.yml is deleted; its content is the role's defaults. Three health checks are kept, not collapsed: Mempool is three moving parts and knowing which one is down is the point. Each has its own script, unit, timer and push_url, driven by a mempool_healthchecks list. The Uptime Kuma specifics are gone - the embedded Python creating monitors over the API, the /tmp credentials file, the push-URL file read back and parsed, three Environment= rewrites - and the three live push URLs are preserved from the vault, so reporting is unchanged. `Enable and start health check timers` and `Display deployment status` were both guarded by uptime_kuma_enabled despite being deployment tasks. Third service in a row with that pattern: the deprecation banner was applied to contiguous blocks, so anything sitting near the push plumbing was disabled with it. Ungated. TWO OWNERSHIP PROBLEMS, different in kind: - MINE: I wrote `owner: root` on docker-compose.yml where the original says `owner: "{{ ansible_user }}"`. A straight violation of extract-mechanically- change-nothing, caught only by reading the check-mode diff line by line. Reverted to match the original. - PRE-EXISTING, and dangerous: the playbook declared `owner: "{{ ansible_user }}"` (1000) on the MariaDB data directory, which the container owns as uid 999. Confirmed against `git show HEAD:` before concluding it was not mine. It had drifted since the containers were created and went unnoticed because the playbook had not been run since. This was not academic. The first real run pulled a newer mariadb:10.11 and recreated mempool-db; with the chown still in place MariaDB would have come back to a data directory it could not write. The role now ensures the directory exists and leaves ownership to the container. Verified after the run: /opt/mempool/mysql is still 999:999 and all three containers are healthy. This is a deliberate behaviour change, not part of the extraction. It is in this commit rather than a follow-up because the faithful version was never safe to run, so there was no intermediate state worth recording as verified. mempool_frontend_port moved to services_config.yml: two hosts need it (this role deploys the frontend, the Caddy play proxies to it from the edge host) and a role default is invisible to the second play. caddy_site's parameter assert caught this loudly - "'mempool_frontend_port' is undefined" - rather than silently. Verified: check-mode diff clean apart from unavoidable check-mode artifacts; first run ok=24 changed=5, zero failures; second run changed=2 - the two bare `command:` tasks (pull, compose up) that have no changed_when and always report changed. That is the idempotent floor. All three health checks report ExecMainStatus 0 with their push URLs intact. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-12 18:49:11 +02:00
mempool:
frontend_port: 8080
bitcoin-knots: convert to a role, de-Uptime-Kuma the health check 892-line playbook becomes 40 lines plus a role with install/build/configure/ service/healthcheck phases, five templates and one handler. bitcoin_knots_vars.yml is deleted; its content is the role's defaults. Verified after a real run: bitcoind still active since 2026-08-19 (NO restart), chain at 966844 blocks / 875 GB, DATUM config intact, dbcache still 200, health check timer firing again. changed=3, all health-check. vipy changed=0. ⚠ THE BIG ONE: the playbook would have deleted the mining integration. bitcoin.conf on the node carries a section that was hand-added and was missing from the template entirely: blockmaxsize=3985000 blockmaxweight=3985000 blocknotify=killall -USR1 datum_gateway maxmempool=1000 blockreconstructionextratxn=1000000 blocknotify is how datum_gateway learns a new block landed. Running the old playbook would have stripped all of it and solo mining would have carried on against a stale template - a silent failure that costs money rather than raising an error. Also dbcache 200 -> 3528 (hand-tuned down; the calculation wants 90% of RAM) and logging moved off the file. All now reconciled, dbcache behind bitcoin_dbcache_mb_override. bitcoin-knots and datum-gateway are ONE SYSTEM. Noted in the README. AND MY OWN FIX MADE IT MORE DANGEROUS. The `Restart bitcoind` handler was guarded by uptime_kuma_enabled, so it had been inert: bitcoin.conf and the systemd unit both notify it and neither could restart anything - a config change applied to disk, reported success, and never took effect. Ungating that is right, but it converts "wrong config sitting inertly on disk" into "node restarted onto a config that breaks mining". The ungating had to land WITH the template reconciliation, not before it. It also raises the bar permanently: any residual template/live difference now restarts a Bitcoin node on every run. Four rounds of --check --diff to reach changed=0 - the DATUM section, an explanatory comment that was rendering into the deployed config (now a {# #} Jinja comment), a "# Pruning (optional)" comment the live file had, and one trailing blank line. The build path is 32 tasks all guarded by `not bitcoind_binary_exists.stat.exists`, so a converged host skips the 30-60 minute compile and both `state: absent` deletions. Those target /opt/bitcoin-knots/{source,bitcoin-<version>}; the chain is in /mnt/knots_data and is never touched. Signature-verification tasks copied verbatim. The health check timer had last fired 2026-08-09 while reporting active/enabled - same OnBootSec + OnUnitActiveSec dead chain as fulcrum. The role runs the check once after enabling to supply the reference the timer schedules from. Ownership parity checked mechanically against `git show HEAD:`, keyed by TASK NAME rather than path - keying by path gave a false positive, because bitcoin_knots_source_dir is created with ownership and later removed with state: absent, so whichever task comes last wins and that differs between one file and five. 13/13 match. bitcoin_p2p_port and the tailscale hostname moved to services_config.yml for the socket-proxy play on the edge host. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-13 18:13:40 +02:00
bitcoin:
p2p_port: 8333
datum-gateway: convert to a role, de-Uptime-Kuma the health check 802-line playbook becomes 68 lines (three plays: the role, the Caddy dashboard, the Stratum socket proxy) plus a 345-line role. datum_gateway_vars.yml is deleted; its content is the role's defaults. Verified after a real run with zero miners connected: datum-gateway restarted cleanly onto the reformatted config, deployed config.json semantically identical to what was there (pool_address bc1qvrj3g84..., pool_pass_* false, ports unchanged), health check timer firing, and the Knots side untouched - bitcoind still up since 2026-08-19 with blocknotify intact. TWO PIECES OF DRIFT WHERE THE NODE WAS RIGHT, both confirmed with the operator: - datum_mining_address: the vault held bc1qdse9dsg... while the node had been mining to bc1qvrj3g... since 2026-08-08. This is WHERE BLOCK REWARDS ARE PAID. And unlike fulcrum and bitcoin-knots, the `Restart datum-gateway` handler here was never gated, so the stale value would have applied immediately rather than sitting inert on disk. - pool_pass_workers / pool_pass_full_users: false on the node, true in the vars file. Both corrected in the vault and role defaults with notes recording why. Comparing this config needs semantics, not text: the live file is single-line JSON and the template renders pretty-printed, so a textual diff is pure noise. Rendering it and comparing parsed JSON is what surfaced both differences. config.json carries bitcoind.rpcpassword and api.admin_password, and --diff prints rendered content - so `--check --diff` put them on the terminal. The task now sets diff: false by default (-e datum_reveal_config=true to opt in). Those two should be rotated. I also mis-reported pool_pass_workers/pool_pass_full_users as exposed credentials because my masking matched "pass" in the key name. They are BOOLEANS, and mining.pool_address is a Bitcoin address, public by nature. Only the two real passwords above were exposed. `Configure cmake build` and `Compile datum_gateway` are bare command: tasks with no changed_when, so they recompile on every run. The build is reproducible - Install datum_gateway binary sees identical content and leaves the installed binary's timestamp alone - but it is wasted work each time. Documented as the idempotent floor. Ownership parity checked mechanically against `git show HEAD:` keyed by task name: 7/7 match, 9 Kuma tasks dropped. This completes Plan 6 Stage 2: all six services in the list are roles. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-13 18:25:41 +02:00
datum_gateway:
api_port: 7152
stratum_port: 23334
fulcrum: convert to a role, de-Uptime-Kuma the health check 685-line playbook becomes 33 lines plus a 426-line role (install/service/healthcheck phases, six templates, one handler). fulcrum_vars.yml is deleted; its content is the role's defaults. Verified: fulcrum untouched - active since 2026-07-29 (no restart), 192G datadir, height 966842, bitcoind and db_mem unchanged on disk. Second run changed=1 (the arming run, changed_when: false aside). vipy changed=0. THREE PRE-EXISTING LANDMINES the check-mode diff caught, any of which a faithful extraction would have detonated: - bitcoin_rpc_host was "192.168.1.140", commented "IP of knots_box_local". But .140 is fulcrum-box ITSELF; knots-box is .135. The DHCP leases had reshuffled - the fifth instance of this same disease in this estate. The live config had been hand-corrected to knots-box; running the playbook would have reverted it and pointed Fulcrum at itself. Now addressed by Tailscale name. - The `Restart fulcrum` handler was guarded by uptime_kuma_enabled, so the three tasks that notify it (SSL cert, fulcrum.conf, systemd unit) could not restart anything. A config change applied to disk, reported success, and silently never took effect. That is worse than the other banner casualties: it makes the deployment itself lie. Ungated. - db_mem was about to go 2048 -> 4448 (75% of 5931MB RAM), leaving ~1.4GB for the OS and Fulcrum's non-cache memory. The live value had been hand-tuned down. fulcrum_db_mem_mb_override pins it. Note set_fact outranks role defaults, so the calculation itself has to honour the override. MY OWN ERROR, third instance: retyping `copy:` as `template:` lost `owner:` on the banner and on fulcrum.conf. Rather than keep catching these by eye, every managed path's owner/group/mode is now compared against `git show HEAD:` mechanically - 12/12 match. The health check timer had not fired since 2026-02-17 while reporting `active` and `enabled`. It is OnBootSec + OnUnitActiveSec with no OnCalendar: OnBootSec elapses once, and OnUnitActiveSec needs the SERVICE to have run this boot to have anything to schedule from. Restarting the timer does not supply that; running the service does, so the role now runs the check once after enabling. Also dropped `Requires=fulcrum.service` from the timer - on a timer that means "stop watching when the watched thing stops". Diagnostic note: NextElapseUSecRealtime is always empty for a monotonic timer, so it reads as broken even when healthy. I misread it once and wrongly called the timer dead. Use NextElapseUSecMonotonic or systemctl list-timers. fulcrum_ssl_port and fulcrum_tailscale_hostname moved to services_config.yml - the socket-proxy play on the edge host needs them and a role default cannot reach a second play. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-13 18:02:00 +02:00
fulcrum:
ssl_port: 50002
Reference in a new issue Copy permalink