monero: find the chain disk by its label, and a blank one by Proxmox slot
After the chain disk moved to the SSD, a reboot on 2026-10-04 swapped sda and sdb, and monero_data_device: /dev/sdb then named the ROOT disk. The root-disk guard caught it, but a routine run should never have been reading a device name at all. Every run now looks for the filesystem labelled monero-data first. If it is there, monero_data_device is never consulted and the mount goes by that filesystem's UUID. Only a blank disk is found by name, and the name is now /dev/disk/by-id/scsi-0QEMU_QEMU_HARDDISK_drive-scsi1: Proxmox passes the slot in as the serial, so it is fixed before boot and survives `qm disk move`. The by-id path is resolved with readlink before the root-disk guard compares it, since a symlink path is never a prefix of /dev/sdb1 and the guard would otherwise always pass. The UUID assert now also fails on two disks carrying the same label. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
parent
c3a7ca0ba7
commit
71dbfbe5ff
4 changed files with 116 additions and 63 deletions
|
|
@ -12,11 +12,11 @@ monero_restricted_rpc_port: 18089
|
|||
# The explorer's HTTP port, read by the Caddy play on the edge host.
|
||||
xmrblocks_port: 8081
|
||||
|
||||
# The 500 GB disk added to this VM in Proxmox as scsi1. Named here rather than
|
||||
# left to the role default so that "which disk holds the blockchain" is a
|
||||
# property of the machine, recorded next to the machine. The role only ever
|
||||
# uses it to create the filesystem; the mount itself goes by UUID.
|
||||
monero_data_device: /dev/sdb
|
||||
# The 500 GB chain disk, attached in Proxmox as scsi1. Named by that slot, which
|
||||
# is fixed when the disk is attached and survives reboots and storage moves -
|
||||
# not /dev/sdb, which it stopped being on 2026-10-04. Only read when the disk has
|
||||
# no monero-data filesystem yet; see roles/monero/defaults/main.yml.
|
||||
monero_data_device: /dev/disk/by-id/scsi-0QEMU_QEMU_HARDDISK_drive-scsi1
|
||||
|
||||
# Systemd services deployed on this host, monitored every 5 minutes.
|
||||
#
|
||||
|
|
|
|||
|
|
@ -62,10 +62,15 @@ Three things there are load-bearing:
|
|||
|
||||
- **The whole device, no partition table.** One disk, one job; a partition table
|
||||
is just another layer that can disagree with itself.
|
||||
- **Everything after mkfs goes by UUID.** `/dev/sdX` ordering is decided by the
|
||||
kernel at boot. This estate has been bitten eight times by an identifier
|
||||
written down once whose thing then moved; `monero_data_device` is used *only*
|
||||
to create the filesystem.
|
||||
- **The device name is read once, ever.** Each run first looks for a filesystem
|
||||
labelled `monero-data`; if it exists, `monero_data_device` is not consulted
|
||||
and the mount goes by that filesystem's UUID. Only a blank disk is found by
|
||||
name, and that name is the Proxmox slot as published under `/dev/disk/by-id`
|
||||
(`...drive-scsi1`), fixed before the VM boots and kept across `qm disk move`.
|
||||
Never `/dev/sdX`: the kernel assigns those in probe order, and on 2026-10-04 a
|
||||
reboot swapped `sda` and `sdb`, leaving the old `/dev/sdb` value naming the
|
||||
root disk. The root-disk guard caught it - but a routine run should never
|
||||
have been reading the name in the first place.
|
||||
- **`RequiresMountsFor=` in the unit.** `nofail` in fstab means a missing disk
|
||||
does not drop a headless VM into an emergency shell - but without
|
||||
`RequiresMountsFor` monerod would then start, find an empty directory, and
|
||||
|
|
|
|||
|
|
@ -38,16 +38,25 @@ monero_data_dir: /var/lib/monero
|
|||
# The whole device, no partition table: this disk has one job and a partition
|
||||
# table would only add a layer that can disagree with itself.
|
||||
#
|
||||
# monero_data_device is used ONLY to create the filesystem, once. Everything
|
||||
# afterwards - fstab, the mount, the systemd unit - goes by UUID, because
|
||||
# /dev/sdX ordering is decided by the kernel at boot and is not a stable name
|
||||
# for anything. This estate has been bitten eight times by an identifier that
|
||||
# was written down once and then moved.
|
||||
monero_data_device: /dev/sdb
|
||||
# The disk is known by two names, for two different moments:
|
||||
#
|
||||
# monero_data_label is its identity once it holds a filesystem. Every run looks
|
||||
# for it first, and if it is there the device name is never consulted at all -
|
||||
# a routine run cannot be broken by a device name, because it does not read one.
|
||||
#
|
||||
# monero_data_device is used ONLY on the first run, to find the blank disk and
|
||||
# create that filesystem. It must be a name fixed before the VM boots: Proxmox
|
||||
# passes the slot ("scsi1") in as the disk's serial, and udev publishes it under
|
||||
# /dev/disk/by-id. That survives reboots and `qm disk move` alike. NEVER
|
||||
# /dev/sdX: the kernel hands those out in probe order, and on 2026-10-04 a reboot
|
||||
# after moving this disk to the SSD swapped sda and sdb - the value written here
|
||||
# then named the ROOT disk.
|
||||
monero_data_label: monero-data
|
||||
monero_data_device: /dev/disk/by-id/scsi-0QEMU_QEMU_HARDDISK_drive-scsi1
|
||||
monero_data_fstype: ext4
|
||||
# -m 0: the 5% ext4 reserve exists to stop root-owned daemons deadlocking a full
|
||||
# root filesystem. On a dedicated data disk it is 25 GB thrown away.
|
||||
monero_data_mkfs_opts: "-m 0 -L monero-data"
|
||||
monero_data_mkfs_opts: "-m 0 -L {{ monero_data_label }}"
|
||||
|
||||
# --- Network ----------------------------------------------------------------
|
||||
# P2P is OUTBOUND ONLY. No inbound port is published from the edge host, so the
|
||||
|
|
|
|||
|
|
@ -5,10 +5,16 @@
|
|||
# appears nowhere in Ansible, so a rebuild of that host would come up with the
|
||||
# chain silently missing. This role does it in code instead.
|
||||
|
||||
- name: Check the Monero data device
|
||||
ansible.builtin.stat:
|
||||
path: "{{ monero_data_device }}"
|
||||
register: monero_device
|
||||
# Does the disk already carry our filesystem? Its label is the identity from
|
||||
# here on. When it is found, the device name below is never read: everything
|
||||
# that follows works from the labelled filesystem, so a stale or reordered
|
||||
# device name cannot break a routine run. blkid exits 2 for "no such label".
|
||||
- name: Look for an existing Monero filesystem by its label
|
||||
ansible.builtin.command: "blkid -L {{ monero_data_label }}"
|
||||
register: monero_labelled_fs
|
||||
changed_when: false
|
||||
failed_when: monero_labelled_fs.rc not in [0, 2]
|
||||
check_mode: false
|
||||
|
||||
- name: Find the device backing the root filesystem
|
||||
ansible.builtin.command: findmnt -no SOURCE /
|
||||
|
|
@ -16,48 +22,77 @@
|
|||
changed_when: false
|
||||
check_mode: false
|
||||
|
||||
# A guard, not ceremony. mkfs on the wrong device is unrecoverable, and
|
||||
# /dev/sdX names are assigned by the kernel at boot - the whole reason
|
||||
# everything below this point works by UUID.
|
||||
- name: Assert the data device is a real block device and is not the root disk
|
||||
ansible.builtin.assert:
|
||||
that:
|
||||
- monero_device.stat.exists
|
||||
- monero_device.stat.isblk | default(false)
|
||||
- not monero_root_source.stdout.startswith(monero_data_device)
|
||||
quiet: true
|
||||
fail_msg: >-
|
||||
{{ monero_data_device }} is not a usable data device, or it is the disk
|
||||
the root filesystem lives on ({{ monero_root_source.stdout }}).
|
||||
Refusing to touch it. Check `lsblk` on the host.
|
||||
# First run only: the disk is blank and has to be found by name.
|
||||
- name: Prepare a blank data disk
|
||||
when: monero_labelled_fs.rc != 0
|
||||
block:
|
||||
# The configured name is a /dev/disk/by-id symlink. Resolve it before
|
||||
# comparing it to anything: the root source is a kernel name like
|
||||
# /dev/sdb1, and a symlink path is never a prefix of one, so comparing the
|
||||
# unresolved name would make the root-disk guard below always pass.
|
||||
- name: Resolve the configured data device to its kernel name
|
||||
ansible.builtin.command: "readlink -e {{ monero_data_device }}"
|
||||
register: monero_device_path
|
||||
changed_when: false
|
||||
failed_when: false
|
||||
check_mode: false
|
||||
|
||||
- name: Read any existing filesystem on the data device
|
||||
ansible.builtin.command: "blkid -o value -s TYPE {{ monero_data_device }}"
|
||||
register: monero_existing_fs
|
||||
changed_when: false
|
||||
failed_when: false
|
||||
check_mode: false
|
||||
- name: Check the Monero data device
|
||||
ansible.builtin.stat:
|
||||
path: "{{ monero_device_path.stdout | default('', true) or monero_data_device }}"
|
||||
register: monero_device
|
||||
|
||||
# community.general.filesystem is idempotent and will not rebuild a filesystem
|
||||
# that is already the requested type, but it would happily create one on a disk
|
||||
# holding something it does not recognise. Say so out loud instead.
|
||||
- name: Refuse to reformat a disk that already carries a different filesystem
|
||||
ansible.builtin.assert:
|
||||
that:
|
||||
- monero_existing_fs.stdout | trim in ['', monero_data_fstype]
|
||||
quiet: true
|
||||
fail_msg: >-
|
||||
{{ monero_data_device }} already holds a '{{ monero_existing_fs.stdout | trim }}'
|
||||
filesystem, not '{{ monero_data_fstype }}'. Refusing to overwrite it.
|
||||
- name: Find the disk the root filesystem lives on
|
||||
ansible.builtin.command: "lsblk -no PKNAME {{ monero_root_source.stdout }}"
|
||||
register: monero_root_disk
|
||||
changed_when: false
|
||||
check_mode: false
|
||||
|
||||
- name: Create the filesystem on the data device
|
||||
community.general.filesystem:
|
||||
fstype: "{{ monero_data_fstype }}"
|
||||
dev: "{{ monero_data_device }}"
|
||||
opts: "{{ monero_data_mkfs_opts }}"
|
||||
# A guard, not ceremony. mkfs on the wrong device is unrecoverable.
|
||||
- name: Assert the data device is a real block device and is not the root disk
|
||||
ansible.builtin.assert:
|
||||
that:
|
||||
- monero_device_path.rc == 0
|
||||
- monero_device.stat.exists
|
||||
- monero_device.stat.isblk | default(false)
|
||||
- monero_device_path.stdout != '/dev/' ~ (monero_root_disk.stdout | trim)
|
||||
- not monero_root_source.stdout.startswith(monero_device_path.stdout)
|
||||
quiet: true
|
||||
fail_msg: >-
|
||||
{{ monero_data_device }} (-> {{ monero_device_path.stdout | default('nothing', true) }})
|
||||
is not a usable data device, or it is the disk the root filesystem
|
||||
lives on ({{ monero_root_source.stdout }}). Refusing to touch it.
|
||||
Check `ls -l /dev/disk/by-id` on the host.
|
||||
|
||||
- name: Read any existing filesystem on the data device
|
||||
ansible.builtin.command: "blkid -o value -s TYPE {{ monero_device_path.stdout }}"
|
||||
register: monero_existing_fs
|
||||
changed_when: false
|
||||
failed_when: false
|
||||
check_mode: false
|
||||
|
||||
# No monero-data label was found, so a filesystem here is something else.
|
||||
# community.general.filesystem would leave an ext4 of any label alone but
|
||||
# happily format over a disk it does not recognise. Say so out loud instead.
|
||||
- name: Refuse to reformat a disk that already carries a different filesystem
|
||||
ansible.builtin.assert:
|
||||
that:
|
||||
- monero_existing_fs.stdout | trim in ['', monero_data_fstype]
|
||||
quiet: true
|
||||
fail_msg: >-
|
||||
{{ monero_data_device }} already holds a '{{ monero_existing_fs.stdout | trim }}'
|
||||
filesystem, not '{{ monero_data_fstype }}'. Refusing to overwrite it.
|
||||
|
||||
- name: Create the filesystem on the data device
|
||||
community.general.filesystem:
|
||||
fstype: "{{ monero_data_fstype }}"
|
||||
dev: "{{ monero_device_path.stdout }}"
|
||||
opts: "{{ monero_data_mkfs_opts }}"
|
||||
|
||||
# By label: on the first run the filesystem was just created with it, on every
|
||||
# other run it was found by it.
|
||||
- name: Read the filesystem UUID
|
||||
ansible.builtin.command: "blkid -o value -s UUID {{ monero_data_device }}"
|
||||
ansible.builtin.command: "blkid -o value -s UUID -t LABEL={{ monero_data_label }}"
|
||||
register: monero_fs_uuid
|
||||
changed_when: false
|
||||
failed_when: false
|
||||
|
|
@ -67,18 +102,22 @@
|
|||
# without one. On a --check run against a disk that has no filesystem yet this
|
||||
# is expected and unavoidable: check mode did not create the filesystem, so
|
||||
# there is no UUID to read. Say that, rather than leaving a bare blkid rc=2.
|
||||
- name: Assert the UUID looks like a UUID
|
||||
# Exactly one: two disks with this label (a cloned disk, say) would make the
|
||||
# mount below ambiguous.
|
||||
- name: Assert exactly one UUID was found
|
||||
ansible.builtin.assert:
|
||||
that:
|
||||
- monero_fs_uuid.stdout | trim | length == 36
|
||||
quiet: true
|
||||
fail_msg: >-
|
||||
No filesystem UUID on {{ monero_data_device }}.
|
||||
{% if ansible_check_mode %}This is expected under --check when the disk
|
||||
Expected exactly one filesystem labelled {{ monero_data_label }}, found:
|
||||
'{{ monero_fs_uuid.stdout | trim }}'.
|
||||
{% if ansible_check_mode %}No UUID is expected under --check when the disk
|
||||
has not been formatted yet - check mode cannot create the filesystem, so
|
||||
it cannot dry-run the mount either. Run without --check once, then
|
||||
--check is meaningful from then on.{% else %}mkfs appears to have failed;
|
||||
check `lsblk` and `blkid` on the host.{% endif %}
|
||||
--check is meaningful from then on.{% else %}If none: mkfs appears to have
|
||||
failed. If several: more than one disk carries the label. Check `lsblk -f`
|
||||
on the host.{% endif %}
|
||||
|
||||
# Same owner and mode as the "Own the mounted data directory" task below, and
|
||||
# that is deliberate rather than duplication. Before the disk is mounted this
|
||||
|
|
@ -93,7 +132,7 @@
|
|||
group: "{{ monero_group }}"
|
||||
mode: '0750'
|
||||
|
||||
# By UUID, never by /dev/sdb. nofail so a missing disk does not drop a headless
|
||||
# By UUID, never by a device name. nofail so a missing disk does not drop a headless
|
||||
# VM into an emergency shell - monerod is stopped from starting by
|
||||
# RequiresMountsFor= in its unit instead, which the systemd monitor then reports.
|
||||
# noatime because LMDB touches a lot of pages and none of it wants an atime write.
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue