navidrome: a music server on its own box, with beets in front of it

Navidrome 0.64.0 on music-box (nodito VM 8200, 1 core / 1 GB / 50 GB), public
at music.contrapeso.xyz. A single Go binary with a SQLite database; the only
external dependency is ffmpeg for transcoding. Plus beets, the CLI that fixes
tags before Navidrome ever indexes them.

── Exposed without basic auth, on purpose ─────────────────────────────────
Navidrome speaks two protocols on one port: its web UI, and the Subsonic API
(/rest/*) that every mobile client actually uses. A Subsonic client
authenticates with ?u=&t=md5(password+salt)&s= in the query string and has
nowhere to carry a second credential, so Basic Auth in front of the site is
invisible to most of them. The choice is a proxy credential or a working phone.

Same call this estate already made for monerod's restricted RPC, for the same
reason: proxied bare so Feather can reach it, with the service's own auth doing
the authenticating.

The cost is stated rather than glossed: the login page and the whole /rest/*
surface are reachable by anyone who finds the hostname, and Navidrome has no
rate limiting and no 2FA.

A fail2ban jail on vipy reads a per-vhost Caddy access log and catches WEB UI
brute force, where a bad login returns 401. It very likely does NOT catch
Subsonic brute force, because that protocol answers a bad password with HTTP
200 and puts the failure in the body - which Caddy never sees. Nor can it be
fixed on the music box: the request arrives from vipy over the tailnet, so the
only address fail2ban could ban there is vipy's, which would take down every
service behind the edge. Banning has to happen at the edge; the edge cannot see
the failure. So /rest/* is protected by password strength alone, and the README
carries a one-line curl to re-check that after every upgrade.

── Passwords are stored reversibly, and cannot not be ─────────────────────
The Subsonic API requires the server to compute md5(password+salt) per request,
so Navidrome cannot hash passwords one-way. It encrypts them at rest with
PasswordEncryptionKey instead. Two real consequences, both in the README: root
on that box means every Navidrome password in plaintext, so the password must
be unique to this service; and changing the key invalidates every stored
password, so it is set before the first account and left alone.

── beets is not a service, deliberately ───────────────────────────────────
`beet import` is interactive and it MOVES AND RENAMES FILES. A timer that
reorganises a music library unattended is a way to lose one. Installed and
configured, run by hand.

/srv/music is counterweight:navidrome mode 2750 and the setgid bit is
load-bearing: without it beets' files land in the curator's own group,
Navidrome cannot read them, and the symptom is an EMPTY LIBRARY rather than a
permission error. ReplayGain and lyrics are auto: no - on one core they turn a
two-minute import into a twenty-minute one.

── Monitoring ─────────────────────────────────────────────────────────────
Host liveness and disk (400), the systemd unit (401), a probe that fetches
/app/ and greps it (403), and public reachability plus cert expiry (402).

/app/ was chosen by testing all three candidates against 0.64.0: /ping returns
a bare "." and proves only that something listens; /rest/ping.view identifies
the server but logs two warnings per call, ~576 a day; /app/ returns
<title>Navidrome</title> and logs nothing. 402 asserts 200 on /app/ and not on
"/", because "/" answers 302 - checked live rather than assumed.

── Also ───────────────────────────────────────────────────────────────────
The headscale ACL grants tag:prd-vipy exactly tag:music-box:4533. Without a
rule the node is invisible rather than unreachable, which surfaces as a Caddy
502 "no such host".

inventory lan_ip was read off nodito's ARP table for the VM's own MAC, not
guessed - and .149 had belonged to another machine, whose stale host key made
SSH refuse the connection. That is the designed behaviour, not a fault.

50 GB is the LIBRARY ceiling, not just the OS: /srv/music is on the root disk,
so roughly 300-450 albums in FLAC. 1 GB is fine for serving and tight for the
initial scan, which is the memory peak.

No backups, deliberately. The music files would swamp the age-encrypted nightly
path, and navidrome.db is a rebuildable index. What is NOT rebuildable is
playlists, play counts, ratings and accounts - there are none today, which is
what makes the trade fair, and the README names this as the decision to revisit
if that changes.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

ansible/roles/navidrome/tasks/beets.yml

@@ -0,0 +1,70 @@
+---
+# beets: the tag fixer that runs BEFORE Navidrome sees anything.
+#
+# Navidrome has no metadata fetcher and no tag editor by design - it trusts the
+# tags on disk completely. That is the right call, but it means a library with
+# bad tags renders as a bad library and there is nothing in Navidrome to fix it
+# with. beets is the fix, and it belongs on this box because that is where the
+# files are.
+#
+# There is no unit and no timer here on purpose: `beet import` is interactive,
+# it MOVES and RENAMES files, and a cron job that reorganises a music library
+# unattended is a way to lose one.
+
+- name: Install beets and its fingerprinting dependencies
+  apt:
+    name:
+      - beets
+      # fpcalc. Without it the chroma plugin cannot acoustically identify a
+      # file whose tags are too mangled to match, which is exactly the case
+      # beets is most useful for.
+      - libchromaprint-tools
+      - python3-acoustid
+      # lastgenre imports pylast, and Debian's beets package does NOT depend on
+      # it. Without this the plugin fails to load and every single `beet`
+      # invocation prints a ModuleNotFoundError traceback while genres are
+      # silently never fetched - configured, listed in `plugins:`, and inert.
+      - python3-pylast
+    state: present
+    update_cache: true
+  when: navidrome_beets_enabled | bool
+
+- name: Create the beets state directory
+  ansible.builtin.file:
+    path: "{{ navidrome_beets_dir }}"
+    state: directory
+    owner: "{{ navidrome_curator_user }}"
+    group: "{{ navidrome_curator_user }}"
+    mode: '0750'
+  when: navidrome_beets_enabled | bool
+
+# Both levels explicitly, parent first. The file module creates missing parents
+# like `mkdir -p`, but it creates them owned by ROOT - which would leave
+# ~/.config root-owned on a fresh cloud-init box and break every other
+# user-level tool that wants to write there later.
+- name: Create the curator's XDG config directory
+  ansible.builtin.file:
+    path: "{{ navidrome_beets_config | dirname | dirname }}"
+    state: directory
+    owner: "{{ navidrome_curator_user }}"
+    group: "{{ navidrome_curator_user }}"
+    mode: '0755'
+  when: navidrome_beets_enabled | bool
+
+- name: Create the beets config directory
+  ansible.builtin.file:
+    path: "{{ navidrome_beets_config | dirname }}"
+    state: directory
+    owner: "{{ navidrome_curator_user }}"
+    group: "{{ navidrome_curator_user }}"
+    mode: '0755'
+  when: navidrome_beets_enabled | bool
+
+- name: Write the beets configuration
+  ansible.builtin.template:
+    src: beets-config.yaml.j2
+    dest: "{{ navidrome_beets_config }}"
+    owner: "{{ navidrome_curator_user }}"
+    group: "{{ navidrome_curator_user }}"
+    mode: '0644'
+  when: navidrome_beets_enabled | bool

ansible/roles/navidrome/tasks/healthcheck.yml

@@ -0,0 +1,48 @@
+---
+# "Does Navidrome actually answer", as opposed to infra/401's "is the unit
+# active". Navidrome can be perfectly `active` and serving 500s - a music
+# folder it cannot read does exactly that, and presents as an empty library.
+
+- name: Create the navidrome health check script
+  ansible.builtin.template:
+    src: healthcheck.sh.j2
+    dest: "{{ navidrome_healthcheck_script_path }}"
+    owner: root
+    group: root
+    mode: '0755'
+    validate: "bash -n %s"
+
+# 0600: carries the Gatus bearer token.
+- name: Create the navidrome health check systemd service
+  ansible.builtin.template:
+    src: healthcheck.service.j2
+    dest: "/etc/systemd/system/{{ navidrome_healthcheck_service_name }}.service"
+    owner: root
+    group: root
+    mode: '0600'
+  notify: Restart navidrome health check timer
+
+- name: Create the navidrome health check systemd timer
+  ansible.builtin.template:
+    src: healthcheck.timer.j2
+    dest: "/etc/systemd/system/{{ navidrome_healthcheck_service_name }}.timer"
+    owner: root
+    group: root
+    mode: '0644'
+  notify: Restart navidrome health check timer
+
+- name: Reload systemd daemon after health check units
+  systemd:
+    daemon_reload: yes
+
+# Ungated: enabling a timer is deployment, not monitoring.
+- name: Enable and restart the navidrome health check timer
+  systemd:
+    name: "{{ navidrome_healthcheck_service_name }}.timer"
+    enabled: yes
+    state: restarted
+    daemon_reload: yes
+
+- name: Run the navidrome health check once to arm the timer
+  ansible.builtin.command: "systemctl start {{ navidrome_healthcheck_service_name }}.service"
+  changed_when: false

ansible/roles/navidrome/tasks/install.yml

@@ -0,0 +1,134 @@
+---
+# The user, the directories, and the binary.
+
+- name: Install Navidrome's runtime dependencies
+  apt:
+    name:
+      # Navidrome shells out to ffmpeg for on-the-fly transcoding. Without it
+      # the server starts fine and every format the client cannot play natively
+      # simply fails, which looks like a client bug.
+      - ffmpeg
+      - curl
+    state: present
+    update_cache: true
+
+- name: Create the navidrome system group
+  group:
+    name: "{{ navidrome_group }}"
+    system: yes
+    state: present
+
+- name: Create the navidrome system user
+  user:
+    name: "{{ navidrome_user }}"
+    group: "{{ navidrome_group }}"
+    system: yes
+    shell: /usr/sbin/nologin
+    home: "{{ navidrome_data_dir }}"
+    create_home: no
+    comment: "Navidrome Music Server"
+    state: present
+
+- name: Create the navidrome data directory
+  ansible.builtin.file:
+    path: "{{ navidrome_data_dir }}"
+    state: directory
+    owner: "{{ navidrome_user }}"
+    group: "{{ navidrome_group }}"
+    mode: '0750'
+
+- name: Create the navidrome config directory
+  ansible.builtin.file:
+    path: "{{ navidrome_config_dir }}"
+    state: directory
+    owner: root
+    group: "{{ navidrome_group }}"
+    mode: '0750'
+
+# Two principals share this tree and neither is root: the curator writes it
+# through beets, Navidrome only reads it. setgid (the 2) is load-bearing -
+# without it, files beets creates land in the curator's own group and Navidrome
+# cannot read them, which presents as an empty library rather than an error.
+- name: Create the music library directory
+  ansible.builtin.file:
+    path: "{{ navidrome_music_dir }}"
+    state: directory
+    owner: "{{ navidrome_curator_user }}"
+    group: "{{ navidrome_group }}"
+    mode: '2750'
+
+- name: Create the incoming staging directory
+  ansible.builtin.file:
+    path: "{{ navidrome_incoming_dir }}"
+    state: directory
+    owner: "{{ navidrome_curator_user }}"
+    group: "{{ navidrome_curator_user }}"
+    mode: '0755'
+
+# check_mode: no on the next two - this repo verifies with `--check --diff`, so
+# the role has to survive a dry run. A read-only version probe and a temp
+# directory are both harmless to perform for real, and without them every task
+# below evaluates against an undefined navidrome_tmp.path and the check aborts.
+- name: Check the installed navidrome version
+  ansible.builtin.command: "{{ navidrome_bin_path }} --version"
+  register: navidrome_installed_version
+  changed_when: false
+  failed_when: false
+  check_mode: no
+
+- name: Decide whether navidrome needs installing
+  ansible.builtin.set_fact:
+    navidrome_needs_install: "{{ navidrome_version not in (navidrome_installed_version.stdout | default('')) }}"
+
+- name: Create a temporary directory for the navidrome download
+  ansible.builtin.tempfile:
+    state: directory
+    suffix: navidrome
+  register: navidrome_tmp
+  when: navidrome_needs_install | bool
+  check_mode: no
+
+# checksum: is the verification. Upstream publishes navidrome_checksums.txt per
+# release but signs nothing, so there is no signature to check the way
+# roles/monero checks binaryfate's - this pins the exact artefact and no more.
+- name: Download the navidrome release tarball
+  ansible.builtin.get_url:
+    url: "{{ navidrome_url }}"
+    dest: "{{ navidrome_tmp.path }}/navidrome.tar.gz"
+    checksum: "sha256:{{ navidrome_sha256 }}"
+    mode: '0644'
+  when: navidrome_needs_install | bool and not ansible_check_mode
+
+- name: Unpack the navidrome tarball
+  ansible.builtin.unarchive:
+    src: "{{ navidrome_tmp.path }}/navidrome.tar.gz"
+    dest: "{{ navidrome_tmp.path }}"
+    remote_src: yes
+  when: navidrome_needs_install | bool and not ansible_check_mode
+
+- name: Install the navidrome binary
+  ansible.builtin.copy:
+    src: "{{ navidrome_tmp.path }}/navidrome"
+    dest: "{{ navidrome_bin_path }}"
+    remote_src: yes
+    owner: root
+    group: root
+    mode: '0755'
+  when: navidrome_needs_install | bool and not ansible_check_mode
+  notify: Restart navidrome
+
+- name: Remove the navidrome download directory
+  ansible.builtin.file:
+    path: "{{ navidrome_tmp.path }}"
+    state: absent
+  when: navidrome_needs_install | bool
+  check_mode: no
+
+# Check mode skips the three tasks above, so without this a dry run is silent
+# about the single most important thing it would do.
+- name: Report what a real run would install
+  ansible.builtin.debug:
+    msg: >-
+      Would install navidrome {{ navidrome_version }} to {{ navidrome_bin_path }}
+      (currently: {{ navidrome_installed_version.stdout | default('not installed') | trim }})
+  when: navidrome_needs_install | bool and ansible_check_mode

ansible/roles/navidrome/tasks/main.yml

@@ -0,0 +1,7 @@
+---
+# import_tasks, not include_tasks: static imports stay visible to --list-tasks,
+# which is the main verification tool in this repo.
+- ansible.builtin.import_tasks: install.yml
+- ansible.builtin.import_tasks: beets.yml
+- ansible.builtin.import_tasks: service.yml
+- ansible.builtin.import_tasks: healthcheck.yml

ansible/roles/navidrome/tasks/service.yml

@@ -0,0 +1,67 @@
+---
+# Config, unit, start, prove it is up.
+
+# The key encrypts every stored password. Starting without it would come up
+# fine and quietly use a built-in default, so this fails the run instead.
+- name: Assert the password encryption key arrived from the vault
+  ansible.builtin.assert:
+    that:
+      - navidrome_password_encryption_key | length > 0
+    fail_msg: >-
+      navidrome_password_encryption_key is empty. It is injected by
+      services/navidrome/deploy_navidrome_playbook.yml from
+      group_vars/all/vault.yml. Navidrome stores user passwords REVERSIBLY
+      (the Subsonic API needs the plaintext to compute md5(password+salt)),
+      and this key is what encrypts them at rest. Set it before the first
+      account exists - changing it later invalidates every stored password.
+    quiet: yes
+
+- name: Write the navidrome configuration
+  ansible.builtin.template:
+    src: navidrome.toml.j2
+    dest: "{{ navidrome_config_file }}"
+    owner: root
+    group: "{{ navidrome_group }}"
+    mode: '0640'
+  diff: "{{ navidrome_reveal_config | default(false) | bool }}"
+  notify: Restart navidrome
+
+- name: Create the navidrome systemd unit
+  ansible.builtin.template:
+    src: navidrome.service.j2
+    dest: /etc/systemd/system/navidrome.service
+    owner: root
+    group: root
+    mode: '0644'
+  notify: Restart navidrome
+
+- name: Reload systemd daemon
+  systemd:
+    daemon_reload: yes
+
+- name: Enable and start navidrome
+  systemd:
+    name: navidrome
+    enabled: yes
+    state: started
+
+- name: Flush handlers so navidrome is running before we probe it
+  meta: flush_handlers
+
+- name: Wait for the navidrome HTTP port
+  ansible.builtin.wait_for:
+    port: "{{ navidrome_port }}"
+    host: 127.0.0.1
+    delay: 5
+    timeout: 300
+  ignore_errors: yes
+
+- name: Check the navidrome service status
+  systemd:
+    name: navidrome
+  register: navidrome_status
+  changed_when: false
+
+- name: Display the navidrome service status
+  ansible.builtin.debug:
+    msg: "navidrome is {{ 'running' if navidrome_status.status.ActiveState == 'active' else 'NOT running' }}"