counterweight/personal_infra
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Compare commits

base: counterweight:master
Branches Tags
counterweight:master
counterweight:refactor-ansible
counterweight:zfs-monitoring
counterweight:nodito-setup
counterweight:before-refactor
..
compare: counterweight:zfs-monitoring
Branches Tags
counterweight:refactor-ansible
counterweight:master
counterweight:zfs-monitoring
counterweight:nodito-setup
counterweight:before-refactor

No commits in common. "master" and "zfs-monitoring" have entirely different histories.
master ... zfs-monito

15 changed files with 6 additions and 2212 deletions
Download patch file Download diff file Expand all files Collapse all files

52
02_vps_core_services_setup.md
View file

@ -198,58 +198,9 @@ Checklist:
- [ ] SSH cloning works after setting up your SSH pub key
## Phoenixd
phoenixd is the server version of the Phoenix wallet: a self-custodial Lightning node that leans on the ACINQ node for liquidity and channel management. It is the Lightning backend that LNBits talks to, so deploy it first.
It is not exposed to the internet. Its HTTP API is bound to `127.0.0.1` and only local consumers (LNBits, running on the same host) use it. There is no subdomain and no Caddy config for it.
### Deploy
* Review `ansible/services/phoenixd/phoenixd_vars.yml`. The things worth a look:
* `phoenixd_version`: pinned release tag.
* `phoenixd_auto_liquidity`: how much inbound liquidity phoenixd buys when it needs some (`off`, `2m`, `5m`, `10m`). This costs sats, it is not free.
* `phoenixd_http_bind_port`: the loopback API port LNBits will point at.
* No new secrets are needed: phoenixd generates its own API password on first start.
* Run the deployment playbook: `ansible-playbook -i inventory.ini services/phoenixd/deploy_phoenixd_playbook.yml`.
* The playbook will:
* Install `phoenixd` and `phoenix-cli` into `/usr/local/bin`
* Create the `phoenix` system user and a `0700` data dir at `/opt/phoenixd/.phoenix`
* Create and start the `phoenixd` systemd service
* Create a health check timer that runs `phoenix-cli getinfo` every minute
* Register a push monitor named `Phoenixd` in Uptime Kuma under the `services` group
### Back up the seed
**Do this immediately after the first deploy.** The seed is the only thing that recovers the funds.
* Run the backup playbook: `ansible-playbook -i inventory.ini services/phoenixd/setup_backup_phoenixd_to_lapy.yml`.
* This gpg encrypts `seed.dat` and `phoenix.conf` to Lapy daily and keeps 14 days. It does not stop the node and it does not back up the channel database on purpose: phoenixd keeps channel state with its peer, and restoring a stale channel db to a live node can force close channels and cost you a penalty.
* Also write the 12 words down offline, once: `sudo cat /opt/phoenixd/.phoenix/seed.dat`.
### Operate
* Status: `sudo systemctl status phoenixd`, logs: `sudo journalctl -u phoenixd -f`.
* CLI: `sudo PHOENIX_DATADIR=/opt/phoenixd/.phoenix phoenix-cli getinfo` (also `getbalance`, `listchannels`, `createinvoice`). If you changed `phoenixd_http_bind_port`, add `--http-bind-port <port>` before the subcommand.
* API password, needed to wire anything up to the node: `sudo grep '^http-password=' /opt/phoenixd/.phoenix/phoenix.conf`.
### Restoring on a fresh host
* Deploy phoenixd but stop the service before it generates a new seed, or just drop the file in before the first run:
* `sudo mkdir -p /opt/phoenixd/.phoenix`
* `echo "your twelve words here" | sudo tee /opt/phoenixd/.phoenix/seed.dat`
* `sudo chown -R phoenix:phoenix /opt/phoenixd/.phoenix && sudo chmod 600 /opt/phoenixd/.phoenix/seed.dat`
* Then run the deployment playbook. Never run two nodes on the same seed at once.
Checklist:
- [ ] `phoenix-cli getinfo` returns a node id
- [ ] The `Phoenixd` monitor in Uptime Kuma is green
- [ ] The seed is backed up to Lapy *and* written down offline
## LNBits
LNBits is a Lightning Network wallet and accounts system. It uses the phoenixd node deployed above as its Lightning backend.
LNBits is a Lightning Network wallet and accounts system.
### Deploy
@ -263,7 +214,6 @@ LNBits is a Lightning Network wallet and accounts system. It uses the phoenixd n
* From that point on, you can configure through the Web UI.
* Some advice around specifics of LNbits:
* The default setup uses a FakeWallet backend for testing. Configure a real Lightning backend as needed by modifying the `.env` file located or using the superuser UI.
* To use the phoenixd node deployed above, set the backend to `PhoenixdWallet` with endpoint `http://127.0.0.1:9740` and the API password from `/opt/phoenixd/.phoenix/phoenix.conf`. The deployment playbook does not do this for you, since flipping the wallet backend on a live instance is not something to do behind your back.
* For security, disable the new users registration.
### Set up backups to Lapy

69
03_vm_disk_enlargement.md
View file

@ -1,69 +0,0 @@
# 03 VM Disk Enlargement
How to enlarge the disk of an existing Proxmox VM that was provisioned via the tofu project.
## Overview
Since the tofu VM resource uses `ignore_changes = all` in its lifecycle block, disk resizes are performed manually through the Proxmox UI and then expanded inside the guest OS. You should still update the tofu vars to keep them in sync with reality.
## Step 1: Update disk size in tofu vars
* Edit `tofu/nodito/terraform.tfvars` and bump the `disk_size_gb` value for the VM you want to resize.
* This won't trigger any actual change (because of `ignore_changes = all`), but keeps the declared state consistent so future readers know the real disk size.
## Step 2: Resize the disk in Proxmox
* Open the Proxmox web UI and navigate to the VM.
* Go to the **Hardware** tab.
* Select the **Hard Disk** entry (scsi0).
* Click **Disk Action** > **Resize**.
* Enter the amount of additional space you want to add (this is the increment, not the total). For example, to go from 10 GB to 30 GB, enter `20`.
* Click **Resize disk**.
The VM does **not** need to be stopped for this operation. Proxmox will grow the underlying block device while the VM is running.
## Step 3: Expand the partition and filesystem inside the VM
SSH into the VM and run these steps as root (or with `sudo`).
### Check the current layout
```bash
lsblk
```
You should see that the disk (`sda`) now shows the new total size, but the root partition (`sda1`) still has the old size.
### Grow the partition
Install `cloud-guest-utils` if `growpart` is not available:
```bash
apt install -y cloud-guest-utils
```
Then grow the root partition to fill all available space:
```bash
growpart /dev/sda 1
```
> Note the space between the device and the partition number.
### Resize the filesystem
For ext4 (the default for Debian cloud images):
```bash
resize2fs /dev/sda1
```
This works online; no reboot is required.
### Verify
```bash
df -h /
```
The root filesystem should now reflect the new size.

2
ansible/infra/420_system_healthcheck.yml
View file

@ -61,7 +61,7 @@
retries = int(sys.argv[8])
ntfy_topic = sys.argv[9] if len(sys.argv) > 9 else "alerts"
api = UptimeKumaApi(api_url, timeout=120, wait_events=2.0)
api = UptimeKumaApi(api_url, timeout=60, wait_events=2.0)
api.login(username, password)
# Get all monitors

1
ansible/infra/910_docker_playbook.yml
View file

@ -25,7 +25,6 @@
name:
- ca-certificates
- curl
- gnupg
state: present
- name: Create directory for Docker GPG key

12
ansible/infra_secrets.yml.example
View file

@ -26,15 +26,3 @@ bitcoin_rpc_password: "CHANGE_ME_TO_SECURE_PASSWORD"
# Mempool MariaDB credentials
# Used by: services/mempool/deploy_mempool_playbook.yml
mariadb_mempool_password: "CHANGE_ME_TO_SECURE_PASSWORD"
# Forgejo Runner registration token
# Used by: services/forgejo-runner/deploy_forgejo_runner_playbook.yml
# See: services/forgejo-runner/SETUP.md for how to obtain this token
forgejo_runner_registration_token: "YOUR_RUNNER_TOKEN_HERE"
# DATUM Gateway secrets
# Used by: services/datum-gateway/deploy_datum_gateway_playbook.yml
datum_mining_address: "YOUR_BITCOIN_ADDRESS_FOR_BLOCK_REWARDS"
datum_gateway_admin_password: "CHANGE_ME_TO_SECURE_PASSWORD"
datum_dashboard_username: "admin"
datum_dashboard_password_hash: "$2a$14$..." # Generate with: caddy hash-password

25
ansible/services/caddy_playbook.yml
View file

@ -13,34 +13,17 @@
state: present
update_cache: yes
# Caddy's signing subkey carries an expiry date, and Cloudsmith republishes
# the key with a refreshed binding signature before it lapses. Re-fetch on
# every run: pinning the keyring with `creates:` leaves the host stuck on a
# key that eventually expires and breaks `apt update` with an sqv error.
- name: Ensure apt keyrings directory exists
ansible.builtin.file:
path: /etc/apt/keyrings
state: directory
owner: root
group: root
mode: '0755'
- name: Download Caddy GPG armored key
ansible.builtin.get_url:
url: https://dl.cloudsmith.io/public/caddy/stable/gpg.key
dest: /etc/apt/keyrings/caddy-stable-archive-keyring.asc
dest: /tmp/caddy-stable-archive-keyring.asc
mode: '0644'
register: caddy_key_download
- name: Check for existing Caddy keyring
ansible.builtin.stat:
path: /usr/share/keyrings/caddy-stable-archive-keyring.gpg
register: caddy_keyring
- name: Convert ASCII armored key to binary keyring
ansible.builtin.command:
cmd: gpg --batch --yes --dearmor -o /usr/share/keyrings/caddy-stable-archive-keyring.gpg /etc/apt/keyrings/caddy-stable-archive-keyring.asc
when: caddy_key_download.changed or not caddy_keyring.stat.exists
cmd: gpg --dearmor -o /usr/share/keyrings/caddy-stable-archive-keyring.gpg /tmp/caddy-stable-archive-keyring.asc
args:
creates: /usr/share/keyrings/caddy-stable-archive-keyring.gpg
- name: Ensure permissions on keyring file
ansible.builtin.file:

43
ansible/services/datum-gateway/datum_gateway_vars.yml
View file

@ -1,43 +0,0 @@
# DATUM Gateway Configuration Variables
# https://github.com/OCEAN-xyz/datum_gateway
# Version - pin to a specific tag
datum_gateway_version: "v0.4.1beta"
# Directories
datum_gateway_dir: /opt/datum-gateway
datum_gateway_source_dir: "{{ datum_gateway_dir }}/source"
datum_gateway_config_dir: /etc/datum-gateway
datum_gateway_log_dir: /var/log/datum-gateway
# Binary
datum_gateway_bin_path: /usr/local/bin/datum_gateway
# Ports
datum_gateway_stratum_port: 23334 # Miners connect here via Stratum v1
datum_gateway_api_port: 7152 # Web dashboard / API
# Stratum settings
datum_vardiff_min: 524288 # Minimum share difficulty (must be power of 2; OCEAN floor overrides if higher)
# Service user
datum_gateway_user: datum
datum_gateway_group: datum
# Build options
datum_gateway_build_jobs: 4
# Bitcoin node connection
# The gateway runs on the same host as Bitcoin Knots so localhost RPC works.
# datum_bitcoin_rpc_url should include http:// and port.
datum_bitcoin_rpc_url: "http://127.0.0.1:8332"
# Note: bitcoin_rpc_user and bitcoin_rpc_password come from infra_secrets.yml
# Mining config
datum_coinbase_tag_primary: "DATUM"
datum_coinbase_tag_secondary: "BY ORDER OF BIP110"
datum_pool_pass_workers: true
datum_pool_pass_full_users: true
datum_pooled_mining_only: true

859
ansible/services/datum-gateway/deploy_datum_gateway_playbook.yml
View file

@ -1,859 +0,0 @@
---
# DATUM Gateway Deployment Playbook
#
# Deploys DATUM Gateway (https://github.com/OCEAN-xyz/datum_gateway) on the
# Bitcoin Knots host so it has direct localhost RPC access to bitcoind.
#
# What this does:
# 1. Installs build deps and compiles datum_gateway from source
# 2. Creates a dedicated system user and config/log directories
# 3. Writes /etc/datum-gateway/config.json from vars/secrets
# 4. Patches bitcoin.conf with the required blockmaxsize/blocknotify lines
# 5. Creates and enables a systemd service
# 6. Creates a push-monitor health check script + systemd timer
# 7. Registers a push monitor in Uptime Kuma
#
# Separate play: adds a Caddy reverse proxy on vipy for the dashboard.
#
# Stratum port (23334) is bound on knots_box_local. Expose it to miners via
# a firewall rule, Tailscale, or a socket proxy on vipy — not handled here.
#
# Required secrets in infra_secrets.yml:
# datum_mining_address - Bitcoin address for block rewards
# datum_gateway_admin_password - Password for the /api admin endpoint
# bitcoin_rpc_user - Shared with the bitcoin-knots deployment
# bitcoin_rpc_password - Shared with the bitcoin-knots deployment
- name: Deploy DATUM Gateway on knots_box_local
hosts: knots_box_local
become: yes
vars_files:
- ../../infra_vars.yml
- ../../services_config.yml
- ../../infra_secrets.yml
- ./datum_gateway_vars.yml
vars:
datum_gateway_subdomain: "{{ subdomains.datum_gateway }}"
datum_gateway_domain: "{{ datum_gateway_subdomain }}.{{ root_domain }}"
uptime_kuma_api_url: "https://{{ subdomains.uptime_kuma }}.{{ root_domain }}"
tasks:
# ===========================================
# Build Dependencies
# ===========================================
- name: Install DATUM Gateway build dependencies
apt:
name:
- cmake
- build-essential
- git
- libjansson-dev
- libmicrohttpd-dev
- libsodium-dev
- libcurl4-openssl-dev
# Runtime-only (netcat for health check)
- netcat-openbsd
state: present
update_cache: yes
# ===========================================
# System User and Directories
# ===========================================
- name: Create datum system user
user:
name: "{{ datum_gateway_user }}"
system: yes
shell: /usr/sbin/nologin
home: "{{ datum_gateway_dir }}"
create_home: no
comment: "DATUM Gateway"
- name: Create DATUM Gateway directories
file:
path: "{{ item.path }}"
state: directory
owner: "{{ item.owner }}"
group: "{{ datum_gateway_group }}"
mode: "{{ item.mode }}"
loop:
- { path: "{{ datum_gateway_dir }}", owner: root, mode: "0755" }
- { path: "{{ datum_gateway_source_dir }}", owner: root, mode: "0755" }
- { path: "{{ datum_gateway_config_dir }}", owner: "{{ datum_gateway_user }}", mode: "0750" }
- { path: "{{ datum_gateway_log_dir }}", owner: "{{ datum_gateway_user }}", mode: "0750" }
# ===========================================
# Build from Source
# ===========================================
- name: Clone DATUM Gateway repository at {{ datum_gateway_version }}
git:
repo: https://github.com/OCEAN-xyz/datum_gateway.git
dest: "{{ datum_gateway_source_dir }}"
version: "{{ datum_gateway_version }}"
force: yes
register: git_clone
- name: Configure cmake build
command: cmake . -DCMAKE_BUILD_TYPE=Release
args:
chdir: "{{ datum_gateway_source_dir }}"
- name: Compile datum_gateway
command: make -j{{ datum_gateway_build_jobs }}
args:
chdir: "{{ datum_gateway_source_dir }}"
- name: Install datum_gateway binary
copy:
src: "{{ datum_gateway_source_dir }}/datum_gateway"
dest: "{{ datum_gateway_bin_path }}"
remote_src: yes
owner: root
group: root
mode: "0755"
notify: Restart datum-gateway
# ===========================================
# Configuration
# ===========================================
- name: Write DATUM Gateway config.json
copy:
dest: "{{ datum_gateway_config_dir }}/config.json"
content: |
{
"bitcoind": {
"rpcuser": "{{ bitcoin_rpc_user }}",
"rpcpassword": "{{ bitcoin_rpc_password }}",
"rpcurl": "{{ datum_bitcoin_rpc_url }}",
"notify_fallback": true
},
"stratum": {
"listen_port": {{ datum_gateway_stratum_port }},
"vardiff_min": {{ datum_vardiff_min }}
},
"mining": {
"pool_address": "{{ datum_mining_address }}",
"coinbase_tag_primary": "{{ datum_coinbase_tag_primary }}",
"coinbase_tag_secondary": "{{ datum_coinbase_tag_secondary }}"
},
"api": {
"admin_password": "{{ datum_gateway_admin_password }}",
"listen_port": {{ datum_gateway_api_port }},
"modify_conf": false
},
"logger": {
"log_to_console": true,
"log_to_file": true,
"log_file": "{{ datum_gateway_log_dir }}/datum_gateway.log",
"log_rotate_daily": true,
"log_level_console": 2,
"log_level_file": 1
},
"datum": {
"pool_pass_workers": {{ datum_pool_pass_workers | lower }},
"pool_pass_full_users": {{ datum_pool_pass_full_users | lower }},
"pooled_mining_only": {{ datum_pooled_mining_only | lower }}
}
}
owner: "{{ datum_gateway_user }}"
group: "{{ datum_gateway_group }}"
mode: "0640"
notify: Restart datum-gateway
# ===========================================
# Systemd Service
# ===========================================
- name: Create datum-gateway systemd service
copy:
dest: /etc/systemd/system/datum-gateway.service
content: |
[Unit]
Description=DATUM Gateway - Bitcoin Mining Gateway
Documentation=https://github.com/OCEAN-xyz/datum_gateway
After=network.target bitcoind.service
Wants=bitcoind.service
[Service]
User={{ datum_gateway_user }}
Group={{ datum_gateway_group }}
Type=simple
ExecStart={{ datum_gateway_bin_path }} --config {{ datum_gateway_config_dir }}/config.json
Restart=on-failure
RestartSec=10
StandardOutput=journal
StandardError=journal
# Prevent config from being read by other users
ReadWritePaths={{ datum_gateway_log_dir }}
ReadOnlyPaths={{ datum_gateway_config_dir }}
[Install]
WantedBy=multi-user.target
owner: root
group: root
mode: "0644"
notify: Restart datum-gateway
- name: Reload systemd daemon
systemd:
daemon_reload: yes
- name: Enable and start datum-gateway
systemd:
name: datum-gateway
enabled: yes
state: started
# ===========================================
# Health Check Script + Systemd Timer
# ===========================================
- name: Create DATUM Gateway health check script
copy:
dest: /usr/local/bin/datum-gateway-healthcheck-push.sh
content: |
#!/bin/bash
UPTIME_KUMA_PUSH_URL="${UPTIME_KUMA_PUSH_URL}"
STRATUM_PORT={{ datum_gateway_stratum_port }}
check_datum() {
# Service must be active and stratum port must be listening
systemctl is-active --quiet datum-gateway && \
nc -z 127.0.0.1 "${STRATUM_PORT}"
}
push_to_uptime_kuma() {
local status=$1
local msg=$2
if [ -z "$UPTIME_KUMA_PUSH_URL" ]; then
echo "ERROR: UPTIME_KUMA_PUSH_URL not set"
return 1
fi
curl -s --max-time 10 --retry 2 -o /dev/null \
"${UPTIME_KUMA_PUSH_URL}?status=${status}&msg=${msg// /%20}&ping=" || true
}
if check_datum; then
push_to_uptime_kuma "up" "OK"
exit 0
else
push_to_uptime_kuma "down" "DATUM Gateway not responding"
exit 1
fi
owner: root
group: root
mode: "0755"
- name: Create datum-gateway health check systemd service
copy:
dest: /etc/systemd/system/datum-gateway-healthcheck.service
content: |
[Unit]
Description=DATUM Gateway Health Check
After=network.target datum-gateway.service
[Service]
Type=oneshot
User=root
ExecStart=/usr/local/bin/datum-gateway-healthcheck-push.sh
Environment=UPTIME_KUMA_PUSH_URL=
StandardOutput=journal
StandardError=journal
[Install]
WantedBy=multi-user.target
owner: root
group: root
mode: "0644"
- name: Create datum-gateway health check systemd timer
copy:
dest: /etc/systemd/system/datum-gateway-healthcheck.timer
content: |
[Unit]
Description=DATUM Gateway Health Check Timer
[Timer]
OnBootSec=2min
OnUnitActiveSec=1min
Persistent=true
[Install]
WantedBy=timers.target
owner: root
group: root
mode: "0644"
- name: Reload systemd daemon after health check units
systemd:
daemon_reload: yes
- name: Enable and start datum-gateway health check timer
systemd:
name: datum-gateway-healthcheck.timer
enabled: yes
state: started
# ===========================================
# Uptime Kuma Push Monitor Setup
# ===========================================
- name: Create Uptime Kuma push monitor setup script for DATUM Gateway
delegate_to: localhost
become: no
copy:
dest: /tmp/setup_datum_gateway_monitor.py
content: |
#!/usr/bin/env python3
import sys
import time
import traceback
import yaml
try:
import socketio.exceptions
except ImportError:
pass
from uptime_kuma_api import UptimeKumaApi, MonitorType
try:
with open('/tmp/ansible_datum_gateway_config.yml', 'r') as f:
config = yaml.safe_load(f)
url = config['uptime_kuma_url']
username = config['username']
password = config['password']
monitor_name = config['monitor_name']
api = UptimeKumaApi(url, timeout=30)
api.login(username, password)
monitors = api.get_monitors()
# Find or create "services" group
group = next((m for m in monitors if m.get('name') == 'services' and m.get('type') == 'group'), None)
if not group:
try:
api.add_monitor(type='group', name='services')
except Exception:
time.sleep(2)
monitors = api.get_monitors()
group = next((m for m in monitors if m.get('name') == 'services' and m.get('type') == 'group'), None)
# Get ntfy notification ID
notifications = api.get_notifications()
ntfy_notification_id = None
for notif in notifications:
if notif.get('type') == 'ntfy':
ntfy_notification_id = notif.get('id')
break
# Check if monitor already exists
existing = next((m for m in monitors if m.get('name') == monitor_name), None)
push_url = None
if existing:
print(f"Monitor '{monitor_name}' already exists (ID: {existing['id']})")
push_token = existing.get('pushToken') or existing.get('push_token')
if push_token:
push_url = f"{url}/api/push/{push_token}"
else:
print(f"Creating push monitor '{monitor_name}'...")
try:
api.add_monitor(
type=MonitorType.PUSH,
name=monitor_name,
parent=group['id'],
interval=90,
maxretries=3,
retryInterval=60,
notificationIDList={ntfy_notification_id: True} if ntfy_notification_id else {}
)
except Exception as e:
# socketio timeout: add_monitor may have succeeded server-side
print(f"add_monitor raised (possibly timeout): {e}", file=sys.stderr)
time.sleep(2)
monitors = api.get_monitors()
new_monitor = next((m for m in monitors if m.get('name') == monitor_name), None)
if new_monitor:
push_token = new_monitor.get('pushToken') or new_monitor.get('push_token')
if push_token:
push_url = f"{url}/api/push/{push_token}"
api.disconnect()
if push_url:
print(f"PUSH_URL={push_url}")
with open('/tmp/datum_gateway_push_url.txt', 'w') as f:
f.write(push_url)
print("SUCCESS")
except Exception as e:
print(f"ERROR: {str(e)}", file=sys.stderr)
traceback.print_exc(file=sys.stderr)
sys.exit(1)
mode: "0755"
- name: Create temporary config for push monitor setup
delegate_to: localhost
become: no
copy:
dest: /tmp/ansible_datum_gateway_config.yml
content: |
uptime_kuma_url: "{{ uptime_kuma_api_url }}"
username: "{{ uptime_kuma_username }}"
password: "{{ uptime_kuma_password }}"
monitor_name: "DATUM Gateway"
mode: "0644"
- name: Run Uptime Kuma push monitor setup
command: python3 /tmp/setup_datum_gateway_monitor.py
delegate_to: localhost
become: no
register: monitor_setup
changed_when: "'SUCCESS' in monitor_setup.stdout"
ignore_errors: yes
- name: Display monitor setup output
debug:
msg: "{{ monitor_setup.stdout_lines }}"
when: monitor_setup.stdout is defined
- name: Read push URL from file
slurp:
src: /tmp/datum_gateway_push_url.txt
delegate_to: localhost
become: no
register: push_url_file
ignore_errors: yes
- name: Parse push URL
set_fact:
datum_push_url: "{{ push_url_file.content | b64decode | trim }}"
when: push_url_file.content is defined
- name: Update health check service with push URL
lineinfile:
path: /etc/systemd/system/datum-gateway-healthcheck.service
regexp: "^Environment=UPTIME_KUMA_PUSH_URL="
line: "Environment=UPTIME_KUMA_PUSH_URL={{ datum_push_url }}"
when: datum_push_url is defined
notify: Restart datum-gateway health check timer
- name: Clean up temporary files
delegate_to: localhost
become: no
file:
path: "{{ item }}"
state: absent
loop:
- /tmp/setup_datum_gateway_monitor.py
- /tmp/ansible_datum_gateway_config.yml
- /tmp/datum_gateway_push_url.txt
handlers:
- name: Restart datum-gateway
systemd:
name: datum-gateway
state: restarted
daemon_reload: yes
- name: Restart datum-gateway health check timer
systemd:
name: datum-gateway-healthcheck.timer
state: restarted
daemon_reload: yes
# ===========================================
# Caddy Reverse Proxy for DATUM Dashboard (on vipy)
# ===========================================
- name: Configure Caddy reverse proxy for DATUM Gateway dashboard on vipy
hosts: vipy
become: yes
vars_files:
- ../../infra_vars.yml
- ../../services_config.yml
- ../../infra_secrets.yml
- ./datum_gateway_vars.yml
vars:
datum_gateway_subdomain: "{{ subdomains.datum_gateway }}"
datum_gateway_domain: "{{ datum_gateway_subdomain }}.{{ root_domain }}"
caddy_sites_dir: "{{ caddy_sites_dir }}"
uptime_kuma_api_url: "https://{{ subdomains.uptime_kuma }}.{{ root_domain }}"
tasks:
- name: Ensure Caddy sites-enabled directory exists
file:
path: "{{ caddy_sites_dir }}"
state: directory
owner: root
group: root
mode: "0755"
- name: Ensure Caddyfile includes import directive for sites-enabled
lineinfile:
path: /etc/caddy/Caddyfile
line: "import sites-enabled/*"
insertafter: EOF
state: present
backup: yes
create: yes
mode: "0644"
- name: Create Caddy reverse proxy config for DATUM Gateway dashboard
copy:
dest: "{{ caddy_sites_dir }}/datum-gateway.conf"
content: |
{{ datum_gateway_domain }} {
basic_auth {
{{ datum_dashboard_username }} {{ datum_dashboard_password_hash }}
}
reverse_proxy knots-box:{{ datum_gateway_api_port }} {
# Resolve via Tailscale MagicDNS
transport http {
resolvers 100.100.100.100
}
}
}
owner: root
group: root
mode: "0644"
- name: Validate Caddy config
command: caddy validate --config /etc/caddy/Caddyfile --adapter caddyfile
register: caddy_validate
changed_when: false
- name: Display Caddy validation output
debug:
msg: "{{ caddy_validate.stdout_lines + caddy_validate.stderr_lines }}"
- name: Reload Caddy
command: systemctl reload caddy
register: caddy_reload
- name: Display Caddy reload output
debug:
msg: "{{ caddy_reload.stdout_lines + caddy_reload.stderr_lines }}"
- name: Display DATUM Gateway dashboard URL
debug:
msg: "DATUM Gateway dashboard: https://{{ datum_gateway_domain }}"
# ===========================================
# Uptime Kuma HTTP Monitor for Public Dashboard
# ===========================================
- name: Create Uptime Kuma HTTP monitor setup script for DATUM dashboard
delegate_to: localhost
become: no
copy:
dest: /tmp/setup_datum_http_monitor.py
content: |
#!/usr/bin/env python3
import sys
import time
import traceback
import yaml
from uptime_kuma_api import UptimeKumaApi, MonitorType
try:
with open('/tmp/ansible_datum_http_config.yml', 'r') as f:
config = yaml.safe_load(f)
url = config['uptime_kuma_url']
username = config['username']
password = config['password']
monitor_url = config['monitor_url']
monitor_name = config['monitor_name']
api = UptimeKumaApi(url, timeout=30)
api.login(username, password)
monitors = api.get_monitors()
group = next((m for m in monitors if m.get('name') == 'services' and m.get('type') == 'group'), None)
if not group:
try:
api.add_monitor(type='group', name='services')
except Exception:
time.sleep(2)
monitors = api.get_monitors()
group = next((m for m in monitors if m.get('name') == 'services' and m.get('type') == 'group'), None)
notifications = api.get_notifications()
ntfy_notification_id = None
for notif in notifications:
if notif.get('type') == 'ntfy':
ntfy_notification_id = notif.get('id')
break
existing = next((m for m in monitors if m.get('name') == monitor_name), None)
if existing:
print(f"Monitor '{monitor_name}' already exists (ID: {existing['id']})")
else:
print(f"Creating HTTP monitor '{monitor_name}'...")
try:
api.add_monitor(
type=MonitorType.HTTP,
name=monitor_name,
url=monitor_url,
parent=group['id'],
interval=60,
maxretries=3,
retryInterval=60,
notificationIDList={ntfy_notification_id: True} if ntfy_notification_id else {}
)
except Exception as e:
print(f"add_monitor raised (possibly timeout): {e}", file=sys.stderr)
time.sleep(2)
api.disconnect()
print("SUCCESS")
except Exception as e:
print(f"ERROR: {str(e)}", file=sys.stderr)
traceback.print_exc(file=sys.stderr)
sys.exit(1)
mode: "0755"
- name: Create temporary config for HTTP monitor
delegate_to: localhost
become: no
copy:
dest: /tmp/ansible_datum_http_config.yml
content: |
uptime_kuma_url: "{{ uptime_kuma_api_url }}"
username: "{{ uptime_kuma_username }}"
password: "{{ uptime_kuma_password }}"
monitor_url: "https://{{ datum_gateway_domain }}"
monitor_name: "DATUM Gateway Dashboard"
mode: "0644"
- name: Run Uptime Kuma HTTP monitor setup
command: python3 /tmp/setup_datum_http_monitor.py
delegate_to: localhost
become: no
register: http_monitor_setup
changed_when: "'SUCCESS' in http_monitor_setup.stdout"
ignore_errors: yes
- name: Display HTTP monitor setup output
debug:
msg: "{{ http_monitor_setup.stdout_lines }}"
when: http_monitor_setup.stdout is defined
- name: Clean up HTTP monitor temporary files
delegate_to: localhost
become: no
file:
path: "{{ item }}"
state: absent
loop:
- /tmp/setup_datum_http_monitor.py
- /tmp/ansible_datum_http_config.yml
# ===========================================
# Stratum Port Forwarding on vipy via systemd-socket-proxyd
# Miners connect to vipy:23334; traffic is forwarded to knots-box:23334
# over the Tailscale network, matching the Bitcoin P2P proxy pattern.
# ===========================================
- name: Setup public Stratum port forwarding on vipy via systemd-socket-proxyd
hosts: vipy
become: yes
vars_files:
- ../../infra_vars.yml
- ../../services_config.yml
- ../../infra_secrets.yml
- ./datum_gateway_vars.yml
vars:
datum_tailscale_hostname: "knots-box"
uptime_kuma_api_url: "https://{{ subdomains.uptime_kuma }}.{{ root_domain }}"
tasks:
- name: Create Stratum proxy socket unit
copy:
dest: /etc/systemd/system/datum-stratum-proxy.socket
content: |
[Unit]
Description=DATUM Stratum Proxy Socket
[Socket]
ListenStream={{ datum_gateway_stratum_port }}
[Install]
WantedBy=sockets.target
owner: root
group: root
mode: "0644"
notify: Restart datum-stratum-proxy socket
- name: Create Stratum proxy service unit
copy:
dest: /etc/systemd/system/datum-stratum-proxy.service
content: |
[Unit]
Description=DATUM Stratum Proxy to {{ datum_tailscale_hostname }}
Requires=datum-stratum-proxy.socket
After=network.target
[Service]
Type=notify
ExecStart=/lib/systemd/systemd-socket-proxyd {{ datum_tailscale_hostname }}:{{ datum_gateway_stratum_port }}
owner: root
group: root
mode: "0644"
- name: Reload systemd daemon
systemd:
daemon_reload: yes
- name: Enable and start Stratum proxy socket
systemd:
name: datum-stratum-proxy.socket
enabled: yes
state: started
- name: Allow Stratum port through UFW
ufw:
rule: allow
port: "{{ datum_gateway_stratum_port | string }}"
proto: tcp
comment: "DATUM Gateway Stratum public access"
- name: Verify connectivity to knots-box Stratum via Tailscale
wait_for:
host: "{{ datum_tailscale_hostname }}"
port: "{{ datum_gateway_stratum_port }}"
timeout: 10
ignore_errors: yes
- name: Display public Stratum endpoint
debug:
msg: "DATUM Stratum public endpoint: {{ ansible_host }}:{{ datum_gateway_stratum_port }}"
# ===========================================
# Uptime Kuma TCP Monitor for Public Stratum
# ===========================================
- name: Create Uptime Kuma TCP monitor setup script for Stratum
delegate_to: localhost
become: no
copy:
dest: /tmp/setup_datum_stratum_tcp_monitor.py
content: |
#!/usr/bin/env python3
import sys
import time
import traceback
import yaml
from uptime_kuma_api import UptimeKumaApi, MonitorType
try:
with open('/tmp/ansible_datum_stratum_config.yml', 'r') as f:
config = yaml.safe_load(f)
url = config['uptime_kuma_url']
username = config['username']
password = config['password']
monitor_host = config['monitor_host']
monitor_port = config['monitor_port']
monitor_name = config['monitor_name']
api = UptimeKumaApi(url, timeout=30)
api.login(username, password)
monitors = api.get_monitors()
group = next((m for m in monitors if m.get('name') == 'services' and m.get('type') == 'group'), None)
if not group:
try:
api.add_monitor(type='group', name='services')
except Exception:
time.sleep(2)
monitors = api.get_monitors()
group = next((m for m in monitors if m.get('name') == 'services' and m.get('type') == 'group'), None)
notifications = api.get_notifications()
ntfy_notification_id = None
for notif in notifications:
if notif.get('type') == 'ntfy':
ntfy_notification_id = notif.get('id')
break
existing = next((m for m in monitors if m.get('name') == monitor_name), None)
if existing:
print(f"Monitor '{monitor_name}' already exists (ID: {existing['id']})")
else:
print(f"Creating TCP monitor '{monitor_name}'...")
try:
api.add_monitor(
type=MonitorType.PORT,
name=monitor_name,
hostname=monitor_host,
port=monitor_port,
parent=group['id'],
interval=60,
maxretries=3,
retryInterval=60,
notificationIDList={ntfy_notification_id: True} if ntfy_notification_id else {}
)
except Exception as e:
print(f"add_monitor raised (possibly timeout): {e}", file=sys.stderr)
time.sleep(2)
api.disconnect()
print("SUCCESS")
except Exception as e:
print(f"ERROR: {str(e)}", file=sys.stderr)
traceback.print_exc(file=sys.stderr)
sys.exit(1)
mode: "0755"
- name: Create temporary config for Stratum TCP monitor
delegate_to: localhost
become: no
copy:
dest: /tmp/ansible_datum_stratum_config.yml
content: |
uptime_kuma_url: "{{ uptime_kuma_api_url }}"
username: "{{ uptime_kuma_username }}"
password: "{{ uptime_kuma_password }}"
monitor_host: "{{ ansible_host }}"
monitor_port: {{ datum_gateway_stratum_port }}
monitor_name: "DATUM Stratum (public)"
mode: "0644"
- name: Run Uptime Kuma TCP monitor setup
command: python3 /tmp/setup_datum_stratum_tcp_monitor.py
delegate_to: localhost
become: no
register: tcp_monitor_setup
changed_when: "'SUCCESS' in tcp_monitor_setup.stdout"
ignore_errors: yes
- name: Display TCP monitor setup output
debug:
msg: "{{ tcp_monitor_setup.stdout_lines }}"
when: tcp_monitor_setup.stdout is defined
- name: Clean up Stratum TCP monitor temporary files
delegate_to: localhost
become: no
file:
path: "{{ item }}"
state: absent
loop:
- /tmp/setup_datum_stratum_tcp_monitor.py
- /tmp/ansible_datum_stratum_config.yml
handlers:
- name: Restart datum-stratum-proxy socket
systemd:
name: datum-stratum-proxy.socket
state: restarted

28
ansible/services/forgejo-runner/SETUP.md
View file

@ -1,28 +0,0 @@
# Forgejo Runner Setup
## Obtaining the Registration Token
1. Log in to the Forgejo instance at `https://forgejo.contrapeso.xyz`
2. Go to **Site Administration** > **Actions** > **Runners**
3. Click **Create new runner**
4. Copy the registration token
## Configuring the Token
Paste the token into `ansible/infra_secrets.yml`:
```yaml
forgejo_runner_registration_token: "YOUR_TOKEN_HERE"
```
## Running the Playbook
```bash
ansible-playbook ansible/services/forgejo-runner/deploy_forgejo_runner_playbook.yml
```
## Verifying
1. On the VM: `systemctl status forgejo-runner` should show active
2. In Forgejo: **Site Administration** > **Actions** > **Runners** should show the runner as online
3. In Uptime Kuma: the `forgejo-runner-healthcheck` push monitor should be receiving pings

392
ansible/services/forgejo-runner/deploy_forgejo_runner_playbook.yml
View file

@ -1,392 +0,0 @@
- name: Install Forgejo Runner on Debian 13
hosts: forgejo_runner_local
become: yes
vars_files:
- ../../infra_vars.yml
- ../../services_config.yml
- ../../infra_secrets.yml
- ./forgejo_runner_vars.yml
vars:
uptime_kuma_api_url: "https://{{ subdomains.uptime_kuma }}.{{ root_domain }}"
ntfy_topic: "{{ service_settings.ntfy.topic }}"
healthcheck_interval_seconds: 60
healthcheck_timeout_seconds: 90
healthcheck_retries: 1
healthcheck_script_dir: /opt/forgejo-runner-healthcheck
healthcheck_script_path: "{{ healthcheck_script_dir }}/forgejo_runner_healthcheck.sh"
healthcheck_log_file: "{{ healthcheck_script_dir }}/forgejo_runner_healthcheck.log"
healthcheck_service_name: forgejo-runner-healthcheck
tasks:
# ── 1. Assert Docker is available ──────────────────────────────────
- name: Check if Docker is installed
command: docker --version
register: docker_check
changed_when: false
failed_when: docker_check.rc != 0
- name: Fail if Docker is not available
assert:
that:
- docker_check.rc == 0
fail_msg: >
Docker is not installed or not in PATH.
Please install Docker before running this playbook.
# ── 2. Download forgejo-runner binary ──────────────────────────────
- name: Download forgejo-runner binary
get_url:
url: "{{ forgejo_runner_url }}"
dest: "{{ forgejo_runner_bin_path }}"
mode: '0755'
# ── 3. Create runner system user ───────────────────────────────────
- name: Create runner system user
user:
name: "{{ forgejo_runner_user }}"
system: yes
shell: /usr/sbin/nologin
home: "{{ forgejo_runner_dir }}"
create_home: no
groups: docker
append: yes
comment: 'Forgejo Runner'
# ── 4. Create working directory ────────────────────────────────────
- name: Create forgejo-runner working directory
file:
path: "{{ forgejo_runner_dir }}"
state: directory
owner: "{{ forgejo_runner_user }}"
group: "{{ forgejo_runner_user }}"
mode: '0750'
# ── 5. Generate default config ─────────────────────────────────────
- name: Check if config already exists
stat:
path: "{{ forgejo_runner_config_path }}"
register: config_stat
- name: Generate default config
shell: "{{ forgejo_runner_bin_path }} generate-config > {{ forgejo_runner_config_path }}"
args:
chdir: "{{ forgejo_runner_dir }}"
when: not config_stat.stat.exists
- name: Set config file ownership
file:
path: "{{ forgejo_runner_config_path }}"
owner: "{{ forgejo_runner_user }}"
group: "{{ forgejo_runner_user }}"
when: not config_stat.stat.exists
# ── 6. Register runner ─────────────────────────────────────────────
- name: Check if runner is already registered
stat:
path: "{{ forgejo_runner_dir }}/.runner"
register: runner_stat
- name: Register runner with Forgejo instance
command: >
{{ forgejo_runner_bin_path }} register --no-interactive
--instance {{ forgejo_instance_url }}
--token {{ forgejo_runner_registration_token }}
--name forgejo-runner-box
--labels "{{ forgejo_runner_labels }}"
args:
chdir: "{{ forgejo_runner_dir }}"
when: not runner_stat.stat.exists
- name: Set runner registration file ownership
file:
path: "{{ forgejo_runner_dir }}/.runner"
owner: "{{ forgejo_runner_user }}"
group: "{{ forgejo_runner_user }}"
when: not runner_stat.stat.exists
# ── 7. Create systemd service ──────────────────────────────────────
- name: Create forgejo-runner systemd service
copy:
dest: /etc/systemd/system/forgejo-runner.service
content: |
[Unit]
Description=Forgejo Runner
Documentation=https://forgejo.org/docs/latest/admin/actions/
After=docker.service
Requires=docker.service
[Service]
Type=simple
User={{ forgejo_runner_user }}
Group={{ forgejo_runner_user }}
WorkingDirectory={{ forgejo_runner_dir }}
ExecStart={{ forgejo_runner_bin_path }} daemon --config {{ forgejo_runner_config_path }}
Restart=on-failure
RestartSec=10
[Install]
WantedBy=multi-user.target
owner: root
group: root
mode: '0644'
# ── 8. Reload systemd, enable and start ────────────────────────────
- name: Reload systemd
systemd:
daemon_reload: yes
- name: Enable and start forgejo-runner service
systemd:
name: forgejo-runner
enabled: yes
state: started
# ── 9. Verify runner is active ─────────────────────────────────────
- name: Verify forgejo-runner is active
command: systemctl is-active forgejo-runner
register: runner_active
changed_when: false
- name: Assert runner is running
assert:
that:
- runner_active.stdout == "active"
fail_msg: "forgejo-runner service is not active: {{ runner_active.stdout }}"
# ── 10. Set up Uptime Kuma push monitor ────────────────────────────
- name: Create Uptime Kuma push monitor setup script
copy:
dest: /tmp/setup_forgejo_runner_monitor.py
content: |
#!/usr/bin/env python3
import sys
import json
from uptime_kuma_api import UptimeKumaApi
def main():
api_url = sys.argv[1]
username = sys.argv[2]
password = sys.argv[3]
group_name = sys.argv[4]
monitor_name = sys.argv[5]
monitor_description = sys.argv[6]
interval = int(sys.argv[7])
retries = int(sys.argv[8])
ntfy_topic = sys.argv[9] if len(sys.argv) > 9 else "alerts"
api = UptimeKumaApi(api_url, timeout=60, wait_events=2.0)
api.login(username, password)
# Get all monitors
monitors = api.get_monitors()
# Get all notifications and find ntfy notification
notifications = api.get_notifications()
ntfy_notification = next((n for n in notifications if n.get('name') == f'ntfy ({ntfy_topic})'), None)
notification_id_list = {}
if ntfy_notification:
notification_id_list[ntfy_notification['id']] = True
# Find or create group
group = next((m for m in monitors if m.get('name') == group_name and m.get('type') == 'group'), None)
if not group:
group_result = api.add_monitor(type='group', name=group_name)
# Refresh to get the full group object with id
monitors = api.get_monitors()
group = next((m for m in monitors if m.get('name') == group_name and m.get('type') == 'group'), None)
# Find or create/update push monitor
existing_monitor = next((m for m in monitors if m.get('name') == monitor_name), None)
monitor_data = {
'type': 'push',
'name': monitor_name,
'parent': group['id'],
'interval': interval,
'upsideDown': False,
'maxretries': retries,
'description': monitor_description,
'notificationIDList': notification_id_list
}
if existing_monitor:
monitor = api.edit_monitor(existing_monitor['id'], **monitor_data)
# Refresh to get the full monitor object with pushToken
monitors = api.get_monitors()
monitor = next((m for m in monitors if m.get('name') == monitor_name), None)
else:
monitor_result = api.add_monitor(**monitor_data)
# Refresh to get the full monitor object with pushToken
monitors = api.get_monitors()
monitor = next((m for m in monitors if m.get('name') == monitor_name), None)
result = {
'monitor_id': monitor['id'],
'push_token': monitor['pushToken'],
'group_name': group_name,
'group_id': group['id'],
'monitor_name': monitor_name
}
print(json.dumps(result))
api.disconnect()
if __name__ == '__main__':
main()
mode: '0755'
delegate_to: localhost
become: no
- name: Run Uptime Kuma push monitor setup
command: >
{{ ansible_playbook_python }}
/tmp/setup_forgejo_runner_monitor.py
"{{ uptime_kuma_api_url }}"
"{{ uptime_kuma_username }}"
"{{ uptime_kuma_password }}"
"services"
"forgejo-runner-healthcheck"
"Forgejo Runner healthcheck - ping every {{ healthcheck_interval_seconds }}s"
"{{ healthcheck_timeout_seconds }}"
"{{ healthcheck_retries }}"
"{{ ntfy_topic }}"
register: monitor_setup_result
delegate_to: localhost
become: no
changed_when: false
- name: Parse monitor setup result
set_fact:
monitor_info_parsed: "{{ monitor_setup_result.stdout | from_json }}"
- name: Set push URL
set_fact:
uptime_kuma_push_url: "{{ uptime_kuma_api_url }}/api/push/{{ monitor_info_parsed.push_token }}"
- name: Create healthcheck script directory
file:
path: "{{ healthcheck_script_dir }}"
state: directory
owner: root
group: root
mode: '0755'
- name: Create forgejo-runner healthcheck script
copy:
dest: "{{ healthcheck_script_path }}"
content: |
#!/bin/bash
# Forgejo Runner Healthcheck Script
# Checks if forgejo-runner is active and pings Uptime Kuma on success
LOG_FILE="{{ healthcheck_log_file }}"
UPTIME_KUMA_URL="{{ uptime_kuma_push_url }}"
log_message() {
echo "$(date '+%Y-%m-%d %H:%M:%S') - $1" >> "$LOG_FILE"
}
main() {
if systemctl is-active --quiet forgejo-runner; then
log_message "forgejo-runner is active, sending ping"
response=$(curl -s -w "\n%{http_code}" "$UPTIME_KUMA_URL?status=up&msg=forgejo-runner%20is%20active" 2>&1)
http_code=$(echo "$response" | tail -n1)
if [ "$http_code" = "200" ] || [ "$http_code" = "201" ]; then
log_message "Ping sent successfully (HTTP $http_code)"
else
log_message "ERROR: Failed to send ping (HTTP $http_code)"
exit 1
fi
else
log_message "ERROR: forgejo-runner is not active"
exit 1
fi
}
main
owner: root
group: root
mode: '0755'
- name: Create healthcheck systemd service
copy:
dest: "/etc/systemd/system/{{ healthcheck_service_name }}.service"
content: |
[Unit]
Description=Forgejo Runner Healthcheck
After=network.target
[Service]
Type=oneshot
ExecStart={{ healthcheck_script_path }}
User=root
StandardOutput=journal
StandardError=journal
[Install]
WantedBy=multi-user.target
owner: root
group: root
mode: '0644'
- name: Create healthcheck systemd timer
copy:
dest: "/etc/systemd/system/{{ healthcheck_service_name }}.timer"
content: |
[Unit]
Description=Run Forgejo Runner Healthcheck every minute
Requires={{ healthcheck_service_name }}.service
[Timer]
OnBootSec=30sec
OnUnitActiveSec={{ healthcheck_interval_seconds }}sec
Persistent=true
[Install]
WantedBy=timers.target
owner: root
group: root
mode: '0644'
- name: Reload systemd for healthcheck units
systemd:
daemon_reload: yes
- name: Enable and start healthcheck timer
systemd:
name: "{{ healthcheck_service_name }}.timer"
enabled: yes
state: started
- name: Test healthcheck script
command: "{{ healthcheck_script_path }}"
register: healthcheck_test
changed_when: false
- name: Verify healthcheck script works
assert:
that:
- healthcheck_test.rc == 0
fail_msg: "Healthcheck script failed to execute properly"
- name: Display deployment summary
debug:
msg: |
Forgejo Runner deployed successfully!
Runner Name: forgejo-runner-box
Instance: {{ forgejo_instance_url }}
Working Directory: {{ forgejo_runner_dir }}
Service: forgejo-runner.service ({{ runner_active.stdout }})
Healthcheck Monitor: forgejo-runner-healthcheck
Healthcheck Interval: Every {{ healthcheck_interval_seconds }}s
Timeout: {{ healthcheck_timeout_seconds }}s
- name: Clean up temporary monitor setup script
file:
path: /tmp/setup_forgejo_runner_monitor.py
state: absent
delegate_to: localhost
become: no

9
ansible/services/forgejo-runner/forgejo_runner_vars.yml
View file

@ -1,9 +0,0 @@
forgejo_runner_version: "6.3.1"
forgejo_runner_arch: "linux-amd64"
forgejo_runner_url: "https://code.forgejo.org/forgejo/runner/releases/download/v{{ forgejo_runner_version }}/forgejo-runner-{{ forgejo_runner_version }}-{{ forgejo_runner_arch }}"
forgejo_runner_bin_path: "/usr/local/bin/forgejo-runner"
forgejo_runner_user: "runner"
forgejo_runner_dir: "/opt/forgejo-runner"
forgejo_runner_config_path: "{{ forgejo_runner_dir }}/config.yml"
forgejo_runner_labels: "docker:docker://node:20-bookworm,ubuntu-latest:docker://node:20-bookworm,ubuntu-22.04:docker://node:20-bookworm,ubuntu-24.04:docker://node:20-bookworm"
forgejo_instance_url: "https://forgejo.contrapeso.xyz"

535
ansible/services/phoenixd/deploy_phoenixd_playbook.yml
View file

@ -1,535 +0,0 @@
---
# phoenixd Deployment Playbook
#
# Deploys phoenixd (https://phoenix.acinq.co/server), the server version of the
# Phoenix Lightning wallet, on vipy so LNBits can use it as a wallet backend
# over loopback.
#
# What this does:
# 1. Downloads the pinned phoenixd release and installs phoenixd + phoenix-cli
# 2. Creates a dedicated system user and a 0700 data directory
# 3. Creates and enables a systemd service
# 4. Creates a push-monitor health check script + systemd timer
# 5. Registers a push monitor in Uptime Kuma
#
# The HTTP API stays bound to 127.0.0.1 and is NOT proxied by Caddy: phoenixd
# holds funds and its API is protected by a single password. Anything that needs
# it either runs on this host or reaches it over the Tailscale mesh.
#
# ⚠️ After the first run, back up {{ phoenixd_data_dir }}/seed.dat. Losing it
# means losing the funds. See setup_backup_phoenixd_to_lapy.yml.
- name: Deploy phoenixd on vipy
hosts: vipy
become: yes
vars_files:
- ../../infra_vars.yml
- ../../services_config.yml
- ../../infra_secrets.yml
- ./phoenixd_vars.yml
vars:
uptime_kuma_api_url: "https://{{ subdomains.uptime_kuma }}.{{ root_domain }}"
tasks:
# ===========================================
# Prerequisites
# ===========================================
- name: Install phoenixd runtime dependencies
apt:
name:
- unzip
- curl
state: present
update_cache: yes
# ===========================================
# System User and Directories
# ===========================================
- name: Create phoenixd system group
group:
name: "{{ phoenixd_group }}"
system: yes
- name: Create phoenixd system user
user:
name: "{{ phoenixd_user }}"
group: "{{ phoenixd_group }}"
system: yes
shell: /usr/sbin/nologin
home: "{{ phoenixd_home }}"
create_home: yes
comment: "phoenixd Lightning node"
- name: Create phoenixd home directory
file:
path: "{{ phoenixd_home }}"
state: directory
owner: "{{ phoenixd_user }}"
group: "{{ phoenixd_group }}"
mode: "0750"
- name: Create phoenixd data directory
file:
path: "{{ phoenixd_data_dir }}"
state: directory
owner: "{{ phoenixd_user }}"
group: "{{ phoenixd_group }}"
mode: "0700"
# ===========================================
# Download and Install
# ===========================================
- name: Check if phoenixd is already installed
stat:
path: "{{ phoenixd_bin_dir }}/phoenixd"
register: phoenixd_binary
- name: Check installed phoenixd version
command: "{{ phoenixd_bin_dir }}/phoenixd --version"
register: phoenixd_installed_version
changed_when: false
failed_when: false
when: phoenixd_binary.stat.exists
- name: Decide whether phoenixd needs installing
set_fact:
phoenixd_needs_install: >-
{{ not phoenixd_binary.stat.exists
or phoenixd_version not in (phoenixd_installed_version.stdout | default('')) }}
- name: Download phoenixd {{ phoenixd_version }}
get_url:
url: "{{ phoenixd_url }}"
dest: "/tmp/phoenixd-{{ phoenixd_version }}.zip"
mode: "0644"
when: phoenixd_needs_install | bool
- name: Create temporary extraction directory
file:
path: /tmp/phoenixd-extract
state: directory
mode: "0755"
when: phoenixd_needs_install | bool
- name: Extract phoenixd archive
unarchive:
src: "/tmp/phoenixd-{{ phoenixd_version }}.zip"
dest: /tmp/phoenixd-extract
remote_src: yes
when: phoenixd_needs_install | bool
- name: Locate extracted binaries
find:
paths: /tmp/phoenixd-extract
patterns: "{{ item }}"
recurse: yes
file_type: file
register: phoenixd_extracted
loop:
- phoenixd
- phoenix-cli
when: phoenixd_needs_install | bool
- name: Fail if the archive did not contain the expected binaries
assert:
that:
- item.files | length > 0
fail_msg: "Could not find '{{ item.item }}' in the phoenixd {{ phoenixd_version }} archive"
loop: "{{ phoenixd_extracted.results }}"
loop_control:
label: "{{ item.item }}"
when: phoenixd_needs_install | bool
- name: Install phoenixd and phoenix-cli binaries
copy:
src: "{{ item.files[0].path }}"
dest: "{{ phoenixd_bin_dir }}/{{ item.item }}"
remote_src: yes
owner: root
group: root
mode: "0755"
loop: "{{ phoenixd_extracted.results }}"
loop_control:
label: "{{ item.item }}"
when: phoenixd_needs_install | bool
notify: Restart phoenixd
- name: Clean up phoenixd download artifacts
file:
path: "{{ item }}"
state: absent
loop:
- "/tmp/phoenixd-{{ phoenixd_version }}.zip"
- /tmp/phoenixd-extract
# ===========================================
# Systemd Service
# ===========================================
- name: Build phoenixd command line arguments
set_fact:
phoenixd_args: >-
{{ (['--agree-to-terms-of-service'] if phoenixd_agree_tos else [])
+ ['--chain', phoenixd_chain]
+ ['--auto-liquidity', phoenixd_auto_liquidity]
+ ['--http-bind-ip', phoenixd_http_bind_ip]
+ ['--http-bind-port', phoenixd_http_bind_port | string]
+ (['--max-mining-fee', phoenixd_max_mining_fee | string] if phoenixd_max_mining_fee else [])
+ (['--webhook', phoenixd_webhook_url] if phoenixd_webhook_url else [])
+ ['--silent'] }}
- name: Create phoenixd systemd service
copy:
dest: /etc/systemd/system/phoenixd.service
content: |
[Unit]
Description=phoenixd - Lightning Network Node
Documentation=https://phoenix.acinq.co/server
After=network-online.target
Wants=network-online.target
[Service]
Type=simple
User={{ phoenixd_user }}
Group={{ phoenixd_group }}
WorkingDirectory={{ phoenixd_home }}
Environment=PHOENIX_DATADIR={{ phoenixd_data_dir }}
ExecStart={{ phoenixd_bin_dir }}/phoenixd {{ phoenixd_args | join(' ') }}
Restart=always
RestartSec=30
TimeoutStartSec=120
TimeoutStopSec=120
StandardOutput=journal
StandardError=journal
# Hardening: the node only ever writes to its own data directory
NoNewPrivileges=true
PrivateTmp=true
ProtectSystem=strict
ProtectHome=read-only
ReadWritePaths={{ phoenixd_data_dir }}
LimitNOFILE=65535
[Install]
WantedBy=multi-user.target
owner: root
group: root
mode: "0644"
notify: Restart phoenixd
- name: Reload systemd daemon
systemd:
daemon_reload: yes
- name: Enable and start phoenixd
systemd:
name: phoenixd
enabled: yes
state: started
- name: Flush handlers so phoenixd is running before we inspect its data dir
meta: flush_handlers
# ===========================================
# First Boot Checks
# ===========================================
- name: Wait for phoenixd to write its config file
wait_for:
path: "{{ phoenixd_data_dir }}/phoenix.conf"
state: present
timeout: 120
- name: Check that the seed file exists
stat:
path: "{{ phoenixd_data_dir }}/seed.dat"
register: phoenixd_seed_file
- name: Fail if phoenixd did not create a seed
assert:
that:
- phoenixd_seed_file.stat.exists
fail_msg: "phoenixd started but {{ phoenixd_data_dir }}/seed.dat is missing - check 'journalctl -u phoenixd'"
# ===========================================
# Health Check Script + Systemd Timer
# ===========================================
- name: Create phoenixd health check script
copy:
dest: "{{ phoenixd_healthcheck_script_path }}"
content: |
#!/bin/bash
# Checks phoenixd and pushes the result to Uptime Kuma.
UPTIME_KUMA_PUSH_URL="${UPTIME_KUMA_PUSH_URL}"
export PHOENIX_DATADIR="{{ phoenixd_data_dir }}"
check_phoenixd() {
# Service must be active and the node must answer getinfo.
# phoenix-cli reads the api password from $PHOENIX_DATADIR/phoenix.conf,
# but not the bind address, so pass it explicitly.
systemctl is-active --quiet phoenixd && \
{{ phoenixd_bin_dir }}/phoenix-cli \
--http-bind-ip {{ phoenixd_http_bind_ip }} \
--http-bind-port {{ phoenixd_http_bind_port }} \
getinfo 2>/dev/null | grep -q '"nodeId"'
}
push_to_uptime_kuma() {
local status=$1
local msg=$2
if [ -z "$UPTIME_KUMA_PUSH_URL" ]; then
echo "ERROR: UPTIME_KUMA_PUSH_URL not set"
return 1
fi
curl -s --max-time 10 --retry 2 -o /dev/null \
"${UPTIME_KUMA_PUSH_URL}?status=${status}&msg=${msg// /%20}&ping=" || true
}
if check_phoenixd; then
push_to_uptime_kuma "up" "OK"
exit 0
else
push_to_uptime_kuma "down" "phoenixd not responding"
exit 1
fi
owner: root
group: root
mode: "0755"
- name: Create phoenixd health check systemd service
copy:
dest: "/etc/systemd/system/{{ phoenixd_healthcheck_service_name }}.service"
content: |
[Unit]
Description=phoenixd Health Check
After=network.target phoenixd.service
[Service]
Type=oneshot
User=root
ExecStart={{ phoenixd_healthcheck_script_path }}
Environment=UPTIME_KUMA_PUSH_URL=
StandardOutput=journal
StandardError=journal
[Install]
WantedBy=multi-user.target
owner: root
group: root
mode: "0644"
- name: Create phoenixd health check systemd timer
copy:
dest: "/etc/systemd/system/{{ phoenixd_healthcheck_service_name }}.timer"
content: |
[Unit]
Description=phoenixd Health Check Timer
[Timer]
OnBootSec=2min
OnUnitActiveSec=1min
Persistent=true
[Install]
WantedBy=timers.target
owner: root
group: root
mode: "0644"
- name: Reload systemd daemon after health check units
systemd:
daemon_reload: yes
- name: Enable and start phoenixd health check timer
systemd:
name: "{{ phoenixd_healthcheck_service_name }}.timer"
enabled: yes
state: started
# ===========================================
# Uptime Kuma Push Monitor Setup
# ===========================================
- name: Create Uptime Kuma push monitor setup script for phoenixd
delegate_to: localhost
become: no
copy:
dest: /tmp/setup_phoenixd_monitor.py
content: |
#!/usr/bin/env python3
import sys
import time
import traceback
import yaml
from uptime_kuma_api import UptimeKumaApi, MonitorType
try:
with open('/tmp/ansible_phoenixd_config.yml', 'r') as f:
config = yaml.safe_load(f)
url = config['uptime_kuma_url']
username = config['username']
password = config['password']
monitor_name = config['monitor_name']
api = UptimeKumaApi(url, timeout=30)
api.login(username, password)
monitors = api.get_monitors()
# Find or create "services" group
group = next((m for m in monitors if m.get('name') == 'services' and m.get('type') == 'group'), None)
if not group:
try:
api.add_monitor(type='group', name='services')
except Exception:
time.sleep(2)
monitors = api.get_monitors()
group = next((m for m in monitors if m.get('name') == 'services' and m.get('type') == 'group'), None)
# Get ntfy notification ID
notifications = api.get_notifications()
ntfy_notification_id = None
for notif in notifications:
if notif.get('type') == 'ntfy':
ntfy_notification_id = notif.get('id')
break
existing = next((m for m in monitors if m.get('name') == monitor_name), None)
push_url = None
if existing:
print(f"Monitor '{monitor_name}' already exists (ID: {existing['id']})")
push_token = existing.get('pushToken') or existing.get('push_token')
if push_token:
push_url = f"{url}/api/push/{push_token}"
else:
print(f"Creating push monitor '{monitor_name}'...")
try:
api.add_monitor(
type=MonitorType.PUSH,
name=monitor_name,
parent=group['id'],
interval=90,
maxretries=3,
retryInterval=60,
notificationIDList={ntfy_notification_id: True} if ntfy_notification_id else {}
)
except Exception as e:
# socketio timeout: add_monitor may have succeeded server-side
print(f"add_monitor raised (possibly timeout): {e}", file=sys.stderr)
time.sleep(2)
monitors = api.get_monitors()
new_monitor = next((m for m in monitors if m.get('name') == monitor_name), None)
if new_monitor:
push_token = new_monitor.get('pushToken') or new_monitor.get('push_token')
if push_token:
push_url = f"{url}/api/push/{push_token}"
api.disconnect()
if push_url:
print(f"PUSH_URL={push_url}")
with open('/tmp/phoenixd_push_url.txt', 'w') as f:
f.write(push_url)
print("SUCCESS")
except Exception as e:
print(f"ERROR: {str(e)}", file=sys.stderr)
traceback.print_exc(file=sys.stderr)
sys.exit(1)
mode: "0755"
- name: Create temporary config for push monitor setup
delegate_to: localhost
become: no
copy:
dest: /tmp/ansible_phoenixd_config.yml
content: |
uptime_kuma_url: "{{ uptime_kuma_api_url }}"
username: "{{ uptime_kuma_username }}"
password: "{{ uptime_kuma_password }}"
monitor_name: "{{ phoenixd_monitor_name }}"
mode: "0644"
- name: Run Uptime Kuma push monitor setup
command: python3 /tmp/setup_phoenixd_monitor.py
delegate_to: localhost
become: no
register: monitor_setup
changed_when: "'SUCCESS' in monitor_setup.stdout"
ignore_errors: yes
- name: Display monitor setup output
debug:
msg: "{{ monitor_setup.stdout_lines }}"
when: monitor_setup.stdout is defined
- name: Read push URL from file
slurp:
src: /tmp/phoenixd_push_url.txt
delegate_to: localhost
become: no
register: push_url_file
ignore_errors: yes
- name: Parse push URL
set_fact:
phoenixd_push_url: "{{ push_url_file.content | b64decode | trim }}"
when: push_url_file.content is defined
- name: Update health check service with push URL
lineinfile:
path: "/etc/systemd/system/{{ phoenixd_healthcheck_service_name }}.service"
regexp: "^Environment=UPTIME_KUMA_PUSH_URL="
line: "Environment=UPTIME_KUMA_PUSH_URL={{ phoenixd_push_url }}"
when: phoenixd_push_url is defined
notify: Restart phoenixd health check timer
- name: Clean up temporary files
delegate_to: localhost
become: no
file:
path: "{{ item }}"
state: absent
loop:
- /tmp/setup_phoenixd_monitor.py
- /tmp/ansible_phoenixd_config.yml
- /tmp/phoenixd_push_url.txt
# ===========================================
# Post-install Notes
# ===========================================
- name: Display post-install information
debug:
msg: |
✓ phoenixd {{ phoenixd_version }} deployed
Status: systemctl status phoenixd
Logs: journalctl -u phoenixd -f
CLI: sudo PHOENIX_DATADIR={{ phoenixd_data_dir }} phoenix-cli --http-bind-port {{ phoenixd_http_bind_port }} getinfo
HTTP API: http://{{ phoenixd_http_bind_ip }}:{{ phoenixd_http_bind_port }} (loopback only)
Data dir: {{ phoenixd_data_dir }}
API password (needed to wire LNBits up to this node):
sudo grep '^http-password=' {{ phoenixd_data_dir }}/phoenix.conf
⚠️ BACK UP THE SEED NOW: {{ phoenixd_data_dir }}/seed.dat
Losing it means losing the funds. Run
services/phoenixd/setup_backup_phoenixd_to_lapy.yml and also keep
the 12 words somewhere offline.
handlers:
- name: Restart phoenixd
systemd:
name: phoenixd
state: restarted
daemon_reload: yes
- name: Restart phoenixd health check timer
systemd:
name: "{{ phoenixd_healthcheck_service_name }}.timer"
state: restarted
daemon_reload: yes

52
ansible/services/phoenixd/phoenixd_vars.yml
View file

@ -1,52 +0,0 @@
# phoenixd Configuration Variables
# https://phoenix.acinq.co/server
# Version - pin to a specific release tag
phoenixd_version: "0.9.0"
phoenixd_arch: "linux-x64"
phoenixd_url: "https://github.com/ACINQ/phoenixd/releases/download/v{{ phoenixd_version }}/phoenixd-{{ phoenixd_version }}-{{ phoenixd_arch }}.zip"
# Service user
phoenixd_user: phoenix
phoenixd_group: phoenix
# Directories
phoenixd_home: /opt/phoenixd
# phoenixd resolves its data dir from PHOENIX_DATADIR; seed.dat, phoenix.conf
# and the channel db all live here.
phoenixd_data_dir: "{{ phoenixd_home }}/.phoenix"
phoenixd_bin_dir: /usr/local/bin
# Lightning settings
phoenixd_chain: mainnet
# Amount automatically requested when inbound liquidity is needed: off, 2m, 5m, 10m
phoenixd_auto_liquidity: "2m"
# Max mining fee for on-chain operations, in sats. Empty means phoenixd's default.
phoenixd_max_mining_fee: ""
# Required for non-interactive deployment
phoenixd_agree_tos: true
# HTTP API
# Stays on loopback: phoenixd is never exposed through Caddy, only local
# consumers (e.g. LNBits on the same host) talk to it.
phoenixd_http_bind_ip: "127.0.0.1"
phoenixd_http_bind_port: 9740
# Optional webhook for payment events. Leave empty to disable.
phoenixd_webhook_url: ""
# Monitoring
phoenixd_healthcheck_script_path: /usr/local/bin/phoenixd-healthcheck-push.sh
phoenixd_healthcheck_service_name: phoenixd-healthcheck
phoenixd_monitor_name: "Phoenixd"
# Remote access
remote_host_name: "vipy"
remote_host: "{{ hostvars.get(remote_host_name, {}).get('ansible_host', remote_host_name) }}"
remote_user: "{{ hostvars.get(remote_host_name, {}).get('ansible_user', 'counterweight') }}"
remote_key_file: "{{ hostvars.get(remote_host_name, {}).get('ansible_ssh_private_key_file', '') }}"
remote_port: "{{ hostvars.get(remote_host_name, {}).get('ansible_port', 22) }}"
# Local backup
local_backup_dir: "{{ lookup('env', 'HOME') }}/phoenixd-backups"
backup_script_path: "{{ lookup('env', 'HOME') }}/.local/bin/phoenixd_backup.sh"

136
ansible/services/phoenixd/setup_backup_phoenixd_to_lapy.yml
View file

@ -1,136 +0,0 @@
---
# Backs up the phoenixd seed and config from vipy to Lapy, gpg encrypted.
#
# Only seed.dat and phoenix.conf are backed up, on purpose:
# - seed.dat is what actually recovers the funds. phoenixd keeps its channel
# state with the ACINQ peer, so a wallet is restored from the seed alone.
# - restoring a *stale* channel database to a running Lightning node is
# dangerous (it can trigger a force close and a penalty), so we do not keep
# copies of phoenix.db around to be tempted by.
# - phoenix.conf holds the http api password, which is what LNBits and any
# other local consumer authenticate with.
#
# Because both files are static, phoenixd does not need to be stopped.
- name: Configure local backup for phoenixd from remote
hosts: lapy
gather_facts: no
vars_files:
- ../../infra_vars.yml
- ./phoenixd_vars.yml
vars:
remote_data_path: "{{ phoenixd_data_dir }}"
gpg_recipient: "{{ hostvars['localhost']['gpg_recipient'] | default('') }}"
gpg_key_id: "{{ hostvars['localhost']['gpg_key_id'] | default('') }}"
tasks:
- name: Debug phoenixd backup vars
debug:
msg:
- "remote_host={{ remote_host }}"
- "remote_user={{ remote_user }}"
- "remote_data_path='{{ remote_data_path }}'"
- "local_backup_dir={{ local_backup_dir }}"
- "gpg_recipient={{ gpg_recipient }}"
- "gpg_key_id={{ gpg_key_id }}"
- name: Ensure local backup directory exists
ansible.builtin.file:
path: "{{ local_backup_dir }}"
state: directory
mode: "0700"
- name: Ensure ~/.local/bin exists
ansible.builtin.file:
path: "{{ lookup('env', 'HOME') }}/.local/bin"
state: directory
mode: "0755"
- name: Create phoenixd backup script
ansible.builtin.copy:
dest: "{{ backup_script_path }}"
mode: "0750"
content: |
#!/bin/bash
set -euo pipefail
if [ -z "{{ gpg_recipient }}" ]; then
echo "GPG recipient is not configured. Aborting."
exit 1
fi
TIMESTAMP=$(date +'%Y-%m-%d')
ENCRYPTED_BACKUP="{{ local_backup_dir }}/phoenixd-backup-$TIMESTAMP.tar.gz.gpg"
{% if remote_key_file %}
SSH_CMD="ssh -i {{ remote_key_file }} -p {{ remote_port }}"
{% else %}
SSH_CMD="ssh -p {{ remote_port }}"
{% endif %}
# seed.dat + phoenix.conf only, see the header of the playbook.
echo "Creating encrypted backup archive..."
$SSH_CMD {{ remote_user }}@{{ remote_host }} \
"sudo tar -czf - -C {{ remote_data_path }} seed.dat phoenix.conf" | \
gpg --batch --yes --encrypt --recipient "{{ gpg_recipient }}" --output "$ENCRYPTED_BACKUP"
chmod 600 "$ENCRYPTED_BACKUP"
# Rotate old backups (keep 14 days)
CUTOFF_DATE=$(date -d '14 days ago' +'%Y-%m-%d')
for backup_file in "{{ local_backup_dir }}"/phoenixd-backup-*.tar.gz.gpg; do
if [ -f "$backup_file" ]; then
# Extract date from filename: phoenixd-backup-YYYY-MM-DD.tar.gz.gpg
file_date=$(basename "$backup_file" | sed -n 's/phoenixd-backup-\([0-9]\{4\}-[0-9]\{2\}-[0-9]\{2\}\)\.tar\.gz\.gpg/\1/p')
if [ -n "$file_date" ] && [ "$file_date" != "$TIMESTAMP" ] && [ "$file_date" \< "$CUTOFF_DATE" ]; then
rm -f "$backup_file"
fi
fi
done
echo "Backup completed successfully"
- name: Ensure cronjob for phoenixd backup exists
ansible.builtin.cron:
name: "phoenixd backup"
user: "{{ lookup('env', 'USER') }}"
job: "{{ backup_script_path }}"
minute: 15
hour: "9"
- name: Run phoenixd backup script to create initial backup
ansible.builtin.command: "{{ backup_script_path }}"
- name: Verify backup was created
block:
- name: Get today's date
command: date +'%Y-%m-%d'
register: today_date
changed_when: false
- name: Check if backup file exists
stat:
path: "{{ local_backup_dir }}/phoenixd-backup-{{ today_date.stdout }}.tar.gz.gpg"
register: backup_file_stat
- name: Verify backup file exists
assert:
that:
- backup_file_stat.stat.exists
- backup_file_stat.stat.isreg
fail_msg: "Backup file {{ local_backup_dir }}/phoenixd-backup-{{ today_date.stdout }}.tar.gz.gpg was not created"
success_msg: "Backup file {{ local_backup_dir }}/phoenixd-backup-{{ today_date.stdout }}.tar.gz.gpg exists"
- name: Verify backup file is not empty
assert:
that:
- backup_file_stat.stat.size > 0
fail_msg: "Backup file {{ local_backup_dir }}/phoenixd-backup-{{ today_date.stdout }}.tar.gz.gpg exists but is empty"
success_msg: "Backup file size is {{ backup_file_stat.stat.size }} bytes"
- name: Remind about the offline seed copy
debug:
msg: |
These encrypted backups are only as safe as your GPG key.
Also write the 12 words down offline once:
ssh {{ remote_user }}@{{ remote_host }} "sudo cat {{ remote_data_path }}/seed.dat"

3
ansible/services_config.yml
View file

@ -25,9 +25,6 @@ subdomains:
# Mempool Block Explorer (on mempool_box, proxied via vipy)
mempool: mempool
# DATUM Gateway dashboard (on knots_box, proxied via vipy)
datum_gateway: datum
# Caddy configuration
caddy_sites_dir: /etc/caddy/sites-enabled