datum-gateway: convert to a role, de-Uptime-Kuma the health check

802-line playbook becomes 68 lines (three plays: the role, the Caddy dashboard,
the Stratum socket proxy) plus a 345-line role. datum_gateway_vars.yml is
deleted; its content is the role's defaults.

Verified after a real run with zero miners connected: datum-gateway restarted
cleanly onto the reformatted config, deployed config.json semantically identical
to what was there (pool_address bc1qvrj3g84..., pool_pass_* false, ports
unchanged), health check timer firing, and the Knots side untouched - bitcoind
still up since 2026-08-19 with blocknotify intact.

TWO PIECES OF DRIFT WHERE THE NODE WAS RIGHT, both confirmed with the operator:

- datum_mining_address: the vault held bc1qdse9dsg... while the node had been
  mining to bc1qvrj3g... since 2026-08-08. This is WHERE BLOCK REWARDS ARE PAID.
  And unlike fulcrum and bitcoin-knots, the `Restart datum-gateway` handler here
  was never gated, so the stale value would have applied immediately rather than
  sitting inert on disk.
- pool_pass_workers / pool_pass_full_users: false on the node, true in the vars
  file.

Both corrected in the vault and role defaults with notes recording why.

Comparing this config needs semantics, not text: the live file is single-line
JSON and the template renders pretty-printed, so a textual diff is pure noise.
Rendering it and comparing parsed JSON is what surfaced both differences.

config.json carries bitcoind.rpcpassword and api.admin_password, and --diff
prints rendered content - so `--check --diff` put them on the terminal. The task
now sets diff: false by default (-e datum_reveal_config=true to opt in). Those
two should be rotated.

I also mis-reported pool_pass_workers/pool_pass_full_users as exposed credentials
because my masking matched "pass" in the key name. They are BOOLEANS, and
mining.pool_address is a Bitcoin address, public by nature. Only the two real
passwords above were exposed.

`Configure cmake build` and `Compile datum_gateway` are bare command: tasks with
no changed_when, so they recompile on every run. The build is reproducible -
Install datum_gateway binary sees identical content and leaves the installed
binary's timestamp alone - but it is wasted work each time. Documented as the
idempotent floor.

Ownership parity checked mechanically against `git show HEAD:` keyed by task
name: 7/7 match, 9 Kuma tasks dropped.

This completes Plan 6 Stage 2: all six services in the list are roles.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
counterweight 2026-09-13 18:25:41 +02:00
parent d26dc78b3c
commit a0c23ae766
Signed by: counterweight
GPG key ID: 883EDBAA726BD96C
17 changed files with 797 additions and 1115 deletions

View file

@ -0,0 +1,73 @@
# `datum_gateway`
Builds and runs [DATUM Gateway](https://github.com/OCEAN-xyz/datum_gateway), the
solo/pooled mining gateway, on `knots-box`. The calling playbook adds two more
plays on the edge host: the dashboard via `caddy_site`, and the public Stratum
port via `socket_proxy`.
Converted from `deploy_datum_gateway_playbook.yml` (802 lines) under Plan 6. The
playbook is now 68 lines and keeps all three plays.
## ⚠ This is half of a system
The Bitcoin Knots node on the same host feeds this gateway through
`blocknotify=killall -USR1 datum_gateway` in `bitcoin.conf` — see
`roles/bitcoin_knots/README.md`, where that line was found to be missing from the
template entirely. **Changing either config means thinking about both.**
Interrupting Stratum costs mining shares. Check before any run that restarts it:
```bash
ss -tn state established '( sport = :23334 )'
```
## Two pieces of drift where the node was right
The repo and the node had diverged on values that matter, and the deployment
would have applied the repo's:
| | node (correct) | repo said |
|---|---|---|
| `datum_mining_address` | `bc1qvrj3g84…` | `bc1qdse9dsg…` |
| `pool_pass_workers` / `_full_users` | `false` | `true` |
The address is the one that would have hurt: **it is where block rewards are
paid**, and unlike fulcrum and bitcoin-knots the `Restart datum-gateway` handler
here was *never* gated, so the change would have applied immediately rather than
sitting inert. Both corrected in the vault and defaults, with notes.
Verify semantics rather than text when touching `config.json` — render it and
compare parsed JSON, because the live file is single-line and the template is
pretty-printed, so a textual diff is all noise:
```python
json.load(open('live.json')) == json.load(open('rendered.json'))
```
## `config.json` holds real secrets — diff is suppressed
The file carries `bitcoind.rpcpassword` and `api.admin_password`. `--diff`
prints rendered content, so the task sets `diff: false` by default; pass
`-e datum_reveal_config=true` to opt in.
Note `pool_pass_workers` / `pool_pass_full_users` are **booleans**, not
passwords, despite the names — they control DATUM's pool-password passthrough.
`mining.pool_address` is a Bitcoin address and public by nature.
## Expect `changed` on the compile every run
`Configure cmake build` and `Compile datum_gateway` are bare `command:` tasks
with no `changed_when`, so they always report changed and always re-run. The
build is reproducible — `Install datum_gateway binary` sees identical content and
does not replace it, so the installed binary keeps its original timestamp — but
the compile itself is wasted work on every run. That is the idempotent floor, not
drift.
## Monitoring: one variable, no product knowledge
The check tests the gateway API and records the answer in its exit code, which
systemd keeps: `systemctl is-failed datum-gateway-healthcheck.service`. Set
`healthcheck_push_url` to report anywhere accepting an HTTP ping.
Unlike the other services here, only the health-check *timer* handler was gated
by `uptime_kuma_enabled`; the main deployment restart worked throughout.

View file

@ -0,0 +1,54 @@
# DATUM Gateway Configuration Variables
# https://github.com/OCEAN-xyz/datum_gateway
# Version - pin to a specific tag
datum_gateway_version: "v0.4.1beta"
# Directories
datum_gateway_dir: /opt/datum-gateway
datum_gateway_source_dir: "{{ datum_gateway_dir }}/source"
datum_gateway_config_dir: /etc/datum-gateway
datum_gateway_log_dir: /var/log/datum-gateway
# Binary
datum_gateway_bin_path: /usr/local/bin/datum_gateway
# Ports
datum_gateway_stratum_port: "{{ service_settings.datum_gateway.stratum_port }}"
datum_gateway_api_port: "{{ service_settings.datum_gateway.api_port }}"
# Stratum settings
datum_vardiff_min: 524288 # Minimum share difficulty (must be power of 2; OCEAN floor overrides if higher)
# Service user
datum_gateway_user: datum
datum_gateway_group: datum
# Build options
datum_gateway_build_jobs: 4
# Bitcoin node connection
# The gateway runs on the same host as Bitcoin Knots so localhost RPC works.
# datum_bitcoin_rpc_url should include http:// and port.
datum_bitcoin_rpc_url: "http://127.0.0.1:8332"
# Note: bitcoin_rpc_user and bitcoin_rpc_password come from infra_secrets.yml
# Mining config
datum_coinbase_tag_primary: "DATUM"
datum_coinbase_tag_secondary: "BY ORDER OF BIP110"
# Both false on the node; the vars file said true. Corrected 2026-09-13 to
# match reality, on the same basis as datum_mining_address: the running node
# is authoritative. These control DATUM's pool-password passthrough.
datum_pool_pass_workers: false
datum_pool_pass_full_users: false
datum_pooled_mining_only: true
# --- Health check -----------------------------------------------------------
# Checks the DATUM Gateway API and records the answer in its exit code, which
# systemd keeps: `systemctl is-failed datum-gateway-healthcheck.service`.
#
# WHERE TO REPORT HEALTH — the one place to plug in monitoring. Empty means
# check, exit honestly, report nowhere.
healthcheck_push_url: ""

View file

@ -0,0 +1,15 @@
---
- name: Restart datum-gateway
systemd:
name: datum-gateway
state: restarted
daemon_reload: yes
# Ungated. This one carried `when: uptime_kuma_enabled | default(false)` while
# the main Restart datum-gateway handler above did not — so on this service the
# deployment restart worked and only the health-check timer restart was dead.
- name: Restart datum-gateway health check timer
systemd:
name: datum-gateway-healthcheck.timer
state: restarted
daemon_reload: yes

View file

@ -0,0 +1,25 @@
---
# Ownership copied verbatim from the playbook this replaces and verified
# mechanically against `git show HEAD:`.
- name: Write DATUM Gateway config.json
ansible.builtin.template:
src: config.json.j2
dest: "{{ datum_gateway_config_dir }}/config.json"
owner: "{{ datum_gateway_user }}"
group: "{{ datum_gateway_group }}"
mode: '0640'
# config.json carries the bitcoind RPC password, the API admin password and
# the pool passwords. `--diff` prints rendered content, so running with --diff
# put all of them on the terminal and into any log capturing it. Suppressed by
# default; pass -e datum_reveal_config=true when you genuinely need the diff.
diff: "{{ datum_reveal_config | default(false) | bool }}"
notify: Restart datum-gateway
- name: Create datum-gateway systemd service
ansible.builtin.template:
src: datum-gateway.service.j2
dest: /etc/systemd/system/datum-gateway.service
owner: root
group: root
mode: '0644'
notify: Restart datum-gateway

View file

@ -0,0 +1,50 @@
---
# Everything here answers "is DATUM Gateway healthy" and records the answer. The
# Uptime Kuma specifics that used to follow — an embedded Python script creating
# monitors over the API, a /tmp credentials file, a push-URL file read back and
# parsed, and a systemd Environment= rewrite — are gone. Where it reports is now
# one variable, healthcheck_push_url.
- name: Create DATUM Gateway health check script
ansible.builtin.template:
src: healthcheck.sh.j2
dest: /usr/local/bin/datum-gateway-healthcheck-push.sh
owner: root
group: root
mode: '0755'
validate: "bash -n %s"
- name: Create datum-gateway health check systemd service
ansible.builtin.template:
src: healthcheck.service.j2
dest: /etc/systemd/system/datum-gateway-healthcheck.service
owner: root
group: root
mode: '0644'
notify: Restart datum-gateway health check timer
- name: Create datum-gateway health check systemd timer
ansible.builtin.template:
src: healthcheck.timer.j2
dest: /etc/systemd/system/datum-gateway-healthcheck.timer
owner: root
group: root
mode: '0644'
notify: Restart datum-gateway health check timer
- name: Reload systemd daemon after health check units
systemd:
daemon_reload: yes
# Ungated: enabling a timer is deployment, not monitoring.
- name: Enable and restart the datum-gateway health check timer
systemd:
name: datum-gateway-healthcheck.timer
enabled: yes
state: restarted
daemon_reload: yes
# Arms the timer and smoke-tests the check. See roles/bitcoin_knots/README.md for
# why restarting the timer alone is not enough with OnBootSec + OnUnitActiveSec.
- name: Run the DATUM Gateway health check once to arm the timer
command: systemctl start datum-gateway-healthcheck.service
changed_when: false

View file

@ -0,0 +1,74 @@
---
- name: Install DATUM Gateway build dependencies
apt:
name:
- cmake
- build-essential
- git
- libjansson-dev
- libmicrohttpd-dev
- libsodium-dev
- libcurl4-openssl-dev
# Runtime-only (netcat for health check)
- netcat-openbsd
state: present
update_cache: yes
# ===========================================
# System User and Directories
# ===========================================
- name: Create datum system user
user:
name: "{{ datum_gateway_user }}"
system: yes
shell: /usr/sbin/nologin
home: "{{ datum_gateway_dir }}"
create_home: no
comment: "DATUM Gateway"
- name: Create DATUM Gateway directories
file:
path: "{{ item.path }}"
state: directory
owner: "{{ item.owner }}"
group: "{{ datum_gateway_group }}"
mode: "{{ item.mode }}"
loop:
- { path: "{{ datum_gateway_dir }}", owner: root, mode: "0755" }
- { path: "{{ datum_gateway_source_dir }}", owner: root, mode: "0755" }
- { path: "{{ datum_gateway_config_dir }}", owner: "{{ datum_gateway_user }}", mode: "0750" }
- { path: "{{ datum_gateway_log_dir }}", owner: "{{ datum_gateway_user }}", mode: "0750" }
# ===========================================
# Build from Source
# ===========================================
- name: Clone DATUM Gateway repository at {{ datum_gateway_version }}
git:
repo: https://github.com/OCEAN-xyz/datum_gateway.git
dest: "{{ datum_gateway_source_dir }}"
version: "{{ datum_gateway_version }}"
force: yes
register: git_clone
- name: Configure cmake build
command: cmake . -DCMAKE_BUILD_TYPE=Release
args:
chdir: "{{ datum_gateway_source_dir }}"
- name: Compile datum_gateway
command: make -j{{ datum_gateway_build_jobs }}
args:
chdir: "{{ datum_gateway_source_dir }}"
- name: Install datum_gateway binary
copy:
src: "{{ datum_gateway_source_dir }}/datum_gateway"
dest: "{{ datum_gateway_bin_path }}"
remote_src: yes
owner: root
group: root
mode: "0755"
notify: Restart datum-gateway
# ===========================================
# Configuration

View file

@ -0,0 +1,6 @@
---
# import_tasks, not include_tasks: static imports stay visible to --list-tasks.
- ansible.builtin.import_tasks: install.yml
- ansible.builtin.import_tasks: configure.yml
- ansible.builtin.import_tasks: service.yml
- ansible.builtin.import_tasks: healthcheck.yml

View file

@ -0,0 +1,16 @@
---
- name: Reload systemd daemon
systemd:
daemon_reload: yes
- name: Enable and start datum-gateway
systemd:
name: datum-gateway
enabled: yes
state: started
# ===========================================
# Health Check Script + Systemd Timer
# ===========================================
# ═════════════════════════════════════════════════════════════════════════
# DEPRECATED — Uptime Kuma was decommissioned on 2026-09-11.

View file

@ -0,0 +1,35 @@
{
"bitcoind": {
"rpcuser": "{{ bitcoin_rpc_user }}",
"rpcpassword": "{{ bitcoin_rpc_password }}",
"rpcurl": "{{ datum_bitcoin_rpc_url }}",
"notify_fallback": true
},
"stratum": {
"listen_port": {{ datum_gateway_stratum_port }},
"vardiff_min": {{ datum_vardiff_min }}
},
"mining": {
"pool_address": "{{ datum_mining_address }}",
"coinbase_tag_primary": "{{ datum_coinbase_tag_primary }}",
"coinbase_tag_secondary": "{{ datum_coinbase_tag_secondary }}"
},
"api": {
"admin_password": "{{ datum_gateway_admin_password }}",
"listen_port": {{ datum_gateway_api_port }},
"modify_conf": false
},
"logger": {
"log_to_console": true,
"log_to_file": true,
"log_file": "{{ datum_gateway_log_dir }}/datum_gateway.log",
"log_rotate_daily": true,
"log_level_console": 2,
"log_level_file": 1
},
"datum": {
"pool_pass_workers": {{ datum_pool_pass_workers | lower }},
"pool_pass_full_users": {{ datum_pool_pass_full_users | lower }},
"pooled_mining_only": {{ datum_pooled_mining_only | lower }}
}
}

View file

@ -0,0 +1,22 @@
[Unit]
Description=DATUM Gateway - Bitcoin Mining Gateway
Documentation=https://github.com/OCEAN-xyz/datum_gateway
After=network.target bitcoind.service
Wants=bitcoind.service
[Service]
User={{ datum_gateway_user }}
Group={{ datum_gateway_group }}
Type=simple
ExecStart={{ datum_gateway_bin_path }} --config {{ datum_gateway_config_dir }}/config.json
Restart=on-failure
RestartSec=10
StandardOutput=journal
StandardError=journal
# Prevent config from being read by other users
ReadWritePaths={{ datum_gateway_log_dir }}
ReadOnlyPaths={{ datum_gateway_config_dir }}
[Install]
WantedBy=multi-user.target

View file

@ -0,0 +1,14 @@
[Unit]
Description=DATUM Gateway Health Check
After=network.target datum-gateway.service
[Service]
Type=oneshot
User=root
ExecStart=/usr/local/bin/datum-gateway-healthcheck-push.sh
Environment=HEALTHCHECK_PUSH_URL={{ healthcheck_push_url }}
StandardOutput=journal
StandardError=journal
[Install]
WantedBy=multi-user.target

View file

@ -0,0 +1,32 @@
#!/bin/bash
# DATUM Gateway health check — managed by Ansible (roles/datum_gateway)
#
# The exit code is the answer and systemd keeps it:
# systemctl is-failed datum-gateway-healthcheck.service
# Reporting anywhere else is optional and generic.
PUSH_URL="${HEALTHCHECK_PUSH_URL:-}"
STRATUM_PORT={{ datum_gateway_stratum_port }}
check_datum() {
# Service must be active and stratum port must be listening
systemctl is-active --quiet datum-gateway && \
nc -z 127.0.0.1 "${STRATUM_PORT}"
}
report() {
local status=$1
local msg=$2
# No push URL is normal, not an error: the exit code below is still a
# complete answer for anything reading unit state.
[ -n "$PUSH_URL" ] || return 0
curl -s --max-time 10 --retry 2 -o /dev/null \
"${PUSH_URL}?status=${status}&msg=${msg// /%20}&ping=" || true
}
if check_datum; then
report "up" "OK"
exit 0
else
report "down" "DATUM Gateway not responding"
exit 1
fi

View file

@ -0,0 +1,10 @@
[Unit]
Description=DATUM Gateway Health Check Timer
[Timer]
OnBootSec=2min
OnUnitActiveSec=1min
Persistent=true
[Install]
WantedBy=timers.target