navidrome: a music server on its own box, with beets in front of it

Navidrome 0.64.0 on music-box (nodito VM 8200, 1 core / 1 GB / 50 GB), public
at music.contrapeso.xyz. A single Go binary with a SQLite database; the only
external dependency is ffmpeg for transcoding. Plus beets, the CLI that fixes
tags before Navidrome ever indexes them.

── Exposed without basic auth, on purpose ─────────────────────────────────
Navidrome speaks two protocols on one port: its web UI, and the Subsonic API
(/rest/*) that every mobile client actually uses. A Subsonic client
authenticates with ?u=&t=md5(password+salt)&s= in the query string and has
nowhere to carry a second credential, so Basic Auth in front of the site is
invisible to most of them. The choice is a proxy credential or a working phone.

Same call this estate already made for monerod's restricted RPC, for the same
reason: proxied bare so Feather can reach it, with the service's own auth doing
the authenticating.

The cost is stated rather than glossed: the login page and the whole /rest/*
surface are reachable by anyone who finds the hostname, and Navidrome has no
rate limiting and no 2FA.

A fail2ban jail on vipy reads a per-vhost Caddy access log and catches WEB UI
brute force, where a bad login returns 401. It very likely does NOT catch
Subsonic brute force, because that protocol answers a bad password with HTTP
200 and puts the failure in the body - which Caddy never sees. Nor can it be
fixed on the music box: the request arrives from vipy over the tailnet, so the
only address fail2ban could ban there is vipy's, which would take down every
service behind the edge. Banning has to happen at the edge; the edge cannot see
the failure. So /rest/* is protected by password strength alone, and the README
carries a one-line curl to re-check that after every upgrade.

── Passwords are stored reversibly, and cannot not be ─────────────────────
The Subsonic API requires the server to compute md5(password+salt) per request,
so Navidrome cannot hash passwords one-way. It encrypts them at rest with
PasswordEncryptionKey instead. Two real consequences, both in the README: root
on that box means every Navidrome password in plaintext, so the password must
be unique to this service; and changing the key invalidates every stored
password, so it is set before the first account and left alone.

── beets is not a service, deliberately ───────────────────────────────────
`beet import` is interactive and it MOVES AND RENAMES FILES. A timer that
reorganises a music library unattended is a way to lose one. Installed and
configured, run by hand.

/srv/music is counterweight:navidrome mode 2750 and the setgid bit is
load-bearing: without it beets' files land in the curator's own group,
Navidrome cannot read them, and the symptom is an EMPTY LIBRARY rather than a
permission error. ReplayGain and lyrics are auto: no - on one core they turn a
two-minute import into a twenty-minute one.

── Monitoring ─────────────────────────────────────────────────────────────
Host liveness and disk (400), the systemd unit (401), a probe that fetches
/app/ and greps it (403), and public reachability plus cert expiry (402).

/app/ was chosen by testing all three candidates against 0.64.0: /ping returns
a bare "." and proves only that something listens; /rest/ping.view identifies
the server but logs two warnings per call, ~576 a day; /app/ returns
<title>Navidrome</title> and logs nothing. 402 asserts 200 on /app/ and not on
"/", because "/" answers 302 - checked live rather than assumed.

── Also ───────────────────────────────────────────────────────────────────
The headscale ACL grants tag:prd-vipy exactly tag:music-box:4533. Without a
rule the node is invisible rather than unreachable, which surfaces as a Caddy
502 "no such host".

inventory lan_ip was read off nodito's ARP table for the VM's own MAC, not
guessed - and .149 had belonged to another machine, whose stale host key made
SSH refuse the connection. That is the designed behaviour, not a fault.

50 GB is the LIBRARY ceiling, not just the OS: /srv/music is on the root disk,
so roughly 300-450 albums in FLAC. 1 GB is fine for serving and tight for the
initial scan, which is the memory peak.

No backups, deliberately. The music files would swamp the age-encrypted nightly
path, and navidrome.db is a rebuildable index. What is NOT rebuildable is
playlists, play counts, ratings and accounts - there are none today, which is
what makes the trade fair, and the README names this as the decision to revisit
if that changes.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

ansible/roles/navidrome/templates/beets-config.yaml.j2

@@ -0,0 +1,81 @@
+# beets configuration — managed by Ansible (roles/navidrome)
+# https://beets.readthedocs.io/
+#
+# Run it by hand, over SSH, when material has accumulated in {{ navidrome_incoming_dir }}:
+#     beet import {{ navidrome_incoming_dir }}
+#
+# It is interactive on purpose. `beet import -q` auto-applies only strong
+# matches and leaves the rest where they are, which makes a usable review queue
+# if the batches ever get boring.
+
+directory: {{ navidrome_music_dir }}
+library: {{ navidrome_beets_dir }}/library.db
+
+# $albumartist, NOT $artist. This is the single most important line in the
+# file: organising by $artist scatters every compilation and every "feat."
+# track into its own phantom artist, which is how a library ends up with four
+# hundred one-album artists and Navidrome gets blamed for it.
+paths:
+    default: $albumartist/$album%aunique{}/$track $title
+    singleton: Non-Album/$artist/$title
+    comp: Compilations/$album%aunique{}/$track $title
+    albumtype:soundtrack: Soundtracks/$album/$track $title
+
+import:
+    # MOVES files out of {{ navidrome_incoming_dir }} into the library. Set
+    # `copy: yes` and `move: no` instead if you would rather keep originals
+    # until you trust it.
+    move: yes
+    write: yes
+    languages: en es
+    log: {{ navidrome_beets_dir }}/import.log
+
+plugins: chroma fetchart embedart scrub duplicates mbsync missing info lastgenre replaygain lyrics permissions
+
+# THE trap this library is exposed to, and the fix for it.
+#
+# beets MOVES files into place, and rename(2) preserves the original group and
+# mode - so the setgid bit on {{ navidrome_music_dir }} does NOT apply to them
+# (it only affects files CREATED there, such as the album directories beets
+# makes, which do come out group-{{ navidrome_group }}). A track that arrives
+# mode 0600 therefore lands mode 0600 owned by the curator's own group, and
+# Navidrome cannot read it. It does not error: the track is simply ABSENT from
+# the library, which is indistinguishable from never having imported it.
+#
+# Measured on this box: a 0600 file was unreadable by the navidrome user; the
+# same file at 0644 was fine. So normalise every imported file.
+# NO leading zeros. The plugin does int(str(value), 8), and YAML would parse
+# `0644` as octal -> decimal 420, which the plugin then re-reads as octal 0o420.
+# Verified on the box: `0644` silently left a 0600 file at 0600; `644` fixes it.
+permissions:
+    file: 644
+    dir: 755
+
+# Acoustic fingerprinting: identifies a file from the audio itself when the
+# tags are too mangled to match. Needs fpcalc, installed by the role.
+chroma:
+    auto: yes
+
+fetchart:
+    auto: yes
+embedart:
+    auto: yes
+# Strip junk and stale tags before writing clean ones.
+scrub:
+    auto: yes
+
+lastgenre:
+    auto: yes
+    source: album
+
+# auto: no on both of these deliberately. This box has one core; ReplayGain
+# analysis and lyrics fetching during an import turn a two-minute job into a
+# twenty-minute one. Run them separately, after the import, when nothing is
+# waiting on you:
+#     beet replaygain
+#     beet lyrics
+replaygain:
+    auto: no
+    backend: ffmpeg
+lyrics:
+    auto: no

ansible/roles/navidrome/templates/healthcheck.service.j2

@@ -0,0 +1,18 @@
+[Unit]
+Description=Navidrome Health Check
+After=network.target navidrome.service
+
+[Service]
+Type=oneshot
+User=root
+ExecStart={{ navidrome_healthcheck_script_path }}
+# Comfortably longer than the script's 30s curl plus its push retries.
+TimeoutStartSec=120
+Environment=HEALTHCHECK_PUSH_URL={{ healthcheck_push_url }}
+Environment=HEALTHCHECK_PUSH_TOKEN={{ healthcheck_push_token }}
+Environment=NAVIDROME_URL=http://127.0.0.1:{{ navidrome_port }}
+StandardOutput=journal
+StandardError=journal
+
+[Install]
+WantedBy=multi-user.target

ansible/roles/navidrome/templates/healthcheck.sh.j2

@@ -0,0 +1,65 @@
+#!/bin/bash
+# Navidrome health check — managed by Ansible (roles/navidrome)
+#
+# The exit code is the answer and systemd keeps it:
+#     systemctl is-failed {{ navidrome_healthcheck_service_name }}.service
+# Reporting anywhere else is optional and generic.
+#
+# /app/ rather than /ping, established by testing against 0.64.0 on the box:
+#
+#   /ping            200, Content-Type text/plain, body is a single "."  - it
+#                    proves something is listening and nothing more.
+#   /rest/ping.view  identifies itself ("type":"navidrome") but logs TWO
+#                    warning lines per call. At one call per 5 minutes that is
+#                    ~576 spurious warnings a day, which is how you train
+#                    yourself to ignore the warning level.
+#   /app/            200, 2.8 KB, <title>Navidrome</title>, and SILENT.
+#
+# So /app/ is the only one that proves it is really Navidrome serving its real
+# UI without polluting the journal. Same reasoning as the xmrblocks check,
+# which fetches a real page because a missing templates dir serves 200s of
+# nothing.
+
+URL="${NAVIDROME_URL:-http://127.0.0.1:{{ navidrome_port }}}"
+PUSH_URL="${HEALTHCHECK_PUSH_URL:-}"
+PUSH_TOKEN="${HEALTHCHECK_PUSH_TOKEN:-}"
+
+report() {
+    local status=$1
+    local msg=$2
+
+    # No push URL is normal, not an error: the exit code below is still a
+    # complete answer for anything reading unit state.
+    [ -n "$PUSH_URL" ] || return 0
+
+    local encoded_msg="${msg// /%20}"
+    local _ok=false
+    [ "${status}" = "up" ] && _ok=true
+    curl -s --max-time 15 --retry 2 -o /dev/null -X POST \
+        -H "Authorization: Bearer ${PUSH_TOKEN}" \
+        "${PUSH_URL}?success=${_ok}&error=${encoded_msg}" || true
+}
+
+# 30s: Navidrome answers /ping in milliseconds normally, but this box has one
+# core and a library scan will happily eat it. A tight timeout here would
+# manufacture false DOWNs every time a scan runs.
+body=$(curl -s --max-time 30 -w '\n%{http_code}' "${URL}/app/" 2>/dev/null)
+code=$(printf '%s' "$body" | tail -1)
+
+if [ -z "$code" ] || [ "$code" = "000" ]; then
+    report "down" "navidrome not responding"
+    exit 1
+fi
+
+if [ "$code" != "200" ]; then
+    report "down" "navidrome HTTP ${code}"
+    exit 1
+fi
+
+if ! printf '%s' "$body" | grep -qi "navidrome"; then
+    report "down" "navidrome served 200 but not its own UI (something else on the port?)"
+    exit 1
+fi
+
+report "up" "OK"
+exit 0

ansible/roles/navidrome/templates/healthcheck.timer.j2

@@ -0,0 +1,14 @@
+[Unit]
+Description=Navidrome Health Check Timer
+# Deliberately NO `Requires=navidrome.service`. Requires on a timer stops the
+# timer when the watched unit stops - "if the thing I am watching goes down,
+# stop watching it" - leaving nothing to re-arm when it returns. Fulcrum's
+# timer carried it and sat active+enabled for seven months without firing.
+
+[Timer]
+OnBootSec=2min
+OnUnitActiveSec=5min
+Persistent=true
+
+[Install]
+WantedBy=timers.target

ansible/roles/navidrome/templates/navidrome.service.j2

@@ -0,0 +1,33 @@
+[Unit]
+Description=Navidrome Music Server
+Documentation=https://www.navidrome.org/docs/
+After=network-online.target
+Wants=network-online.target
+
+[Service]
+Type=simple
+User={{ navidrome_user }}
+Group={{ navidrome_group }}
+WorkingDirectory={{ navidrome_data_dir }}
+ExecStart={{ navidrome_bin_path }} --configfile {{ navidrome_config_file }}
+Restart=always
+RestartSec=20
+TimeoutStopSec=60
+StandardOutput=journal
+StandardError=journal
+
+# Hardening: Navidrome writes its database and cache, and nothing else.
+NoNewPrivileges=true
+PrivateTmp=true
+ProtectSystem=strict
+ProtectHome=read-only
+# {{ navidrome_music_dir }} is deliberately absent: ProtectSystem=strict leaves
+# it read-only, which is exactly right. Navidrome indexes the library; beets
+# writes it. If Navidrome ever appears to need write access here, something is
+# wrong with the request, not with this line.
+ReadWritePaths={{ navidrome_data_dir }}
+
+LimitNOFILE=65535
+
+[Install]
+WantedBy=multi-user.target

ansible/roles/navidrome/templates/navidrome.toml.j2

@@ -0,0 +1,24 @@
+# Navidrome configuration — managed by Ansible (roles/navidrome)
+# https://www.navidrome.org/docs/usage/configuration-options/
+#
+# Mode 0640 root:{{ navidrome_group }}: PasswordEncryptionKey is below.
+
+MusicFolder = "{{ navidrome_music_dir }}"
+DataFolder  = "{{ navidrome_data_dir }}"
+
+Address = "{{ navidrome_bind_address }}"
+Port    = {{ navidrome_port }}
+
+LogLevel = "{{ navidrome_log_level }}"
+
+# See the long note in defaults/main.yml. Changing this invalidates every
+# stored password.
+PasswordEncryptionKey = "{{ navidrome_password_encryption_key }}"
+
+EnableInsightsCollector = {{ navidrome_enable_insights | bool | lower }}
+
+# Scan schedule is left at Navidrome's own default deliberately. The option has
+# moved between releases (top-level ScanSchedule vs a [Scanner] section), and
+# pinning the wrong spelling here is silently ignored rather than rejected -
+# you would get the default anyway and believe you had configured something.
+# Check `navidrome config` on the box before adding it.