veilor-os/test/TESTING.md

veilor-os — Testing Procedure

This document is the canonical procedure for validating a veilor-os ISO end-to-end. Every release that gets a tag MUST have a corresponding test-run report in test/test-runs/ linked from the release notes.

If reality forces you to deviate from the steps below, do not silently patch the procedure — open a commit that updates this file and appends an entry to test/METHOD-CHANGELOG.md explaining what changed and why. The changelog is what makes the procedure auditable; the procedure itself is just the latest snapshot.

---

Two test environments

Environment	Catches	Doesn't catch
VM (QEMU + virtio-vga)	install logic, kickstart bugs, %post failures, anaconda transaction failures, GRUB write, BLS entries, package selection, network stack	KMS / fbcon issues, real-firmware Secure Boot, USB controller quirks, GPU driver compatibility, sleep/wake, battery, thermals
Real hardware (USB → spare laptop)	everything VM doesn't	install repeatability (you only have so many spare laptops)

Both are required for any tagged release. VM first (cheap iteration), real hardware second (final sign-off).

---

VM test — hybrid procedure

The VM cannot type LUKS / admin passwords through QEMU's sendkey monitor command — plymouth's IPC ignores synthesised keystrokes (we verified this across 14+ sendkey variants in earlier sessions). The hybrid procedure splits the work: Claude/automation drives every step that doesn't need a password; the human types the two passwords (LUKS

• admin) into the QEMU window directly.

Standard test passwords (lab use only — never reuse outside this repo):

Prompt	Type
LUKS passphrase	veilortest1
Admin password	veilortest1

Both passwords identical on purpose — easier to remember mid-test, both satisfy the installer's 8-char min, neither contains shell-special chars (validate_pw rejects " $ \ \ & | / \n`).

Run a VM test

cd ~/ai-lab/_github/veilor-os
# Pull the ISO you want to test (from a CI release or local build)
ls /home/admin/Downloads/veilor-os-*.iso

# Wipe stale state, launch VM with monitor sock (no auto-inject — we
# don't want sendkey noise typing into prompts)
FRESH=1 NO_INJECT=1 DISPLAY=:0 ./test/run-vm.sh \
    /home/admin/Downloads/veilor-os-43-YYYYMMDD-HHMMSS.iso

Then either (a) drive the install yourself in the QEMU window, or (b) hand the monitor sock to Claude / a script:

• Monitor sock: test/veilor-vm.monitor.sock
• Send a key: echo "sendkey ret" | socat - "UNIX-CONNECT:$SOCK"
• Screendump: echo "screendump /tmp/x.ppm" | socat - "UNIX-CONNECT:$SOCK"; magick /tmp/x.ppm /tmp/x.png

Steps to verify

The complete checklist lives in test/boot-checklist.md — that file is the granular pass/fail list. The high-level flow is:

1. Live boot. GRUB (legacy menu, no Plymouth splash) → text scroll → veilor-installer banner on tty1 within ~30s. No "fedora" branding anywhere on screen.
2. Installer menu. "Install" highlighted by default. No phantom duplicate items, no stray characters in input fields.
3. Disk picker. /dev/vda (or whatever virtio gives you) listed with size + model.
4. Passwords. LUKS + admin prompts; user types veilortest1 twice.
5. Locale. en_GB.UTF-8 picks up.
6. Confirm. Disk shown with WILL BE ERASED, locale + LUKS/admin ticks shown.
7. Anaconda. "Installing veilor-os to /dev/vda · 10–30 min · logs on tty4". Watch for Configuring man-db — if anything fails, this is historically where it dies.
8. Reboot. VM reboots; ISO must NOT boot first this time. Kill QEMU + relaunch without ISO drive (see Boot installed disk below) to skip the GRUB-from-ISO path.
9. GRUB. Single "veilor-os" entry (no rescue, no "Fedora Linux").
10. LUKS prompt. Plymouth details theme — text-mode prompt for passphrase. User types veilortest1 in the QEMU window (sendkey will not work).
11. First boot. SDDM splash → admin user pre-filled → admin types veilortest1 → password-change prompt (chage -d 0 expired the password) → user picks new password → KDE Plasma session.
12. Hardening checks per test/boot-checklist.md (SELinux enforcing, fail2ban active, USBGuard active, tuned profile, etc.).

Boot installed disk (skip ISO)

After the install reboots, QEMU's CD-first boot order will land back in the live ISO. Easiest workaround: kill QEMU and re-launch without the -drive file=...iso line. The qcow2 retains the install:

pkill -f 'qemu-system.*veilor-os'
cd ~/ai-lab/_github/veilor-os/test
DISPLAY=:0 qemu-system-x86_64 \
    -enable-kvm -cpu host -smp 4 -m 4096 \
    -machine q35,smm=on \
    -global driver=cfi.pflash01,property=secure,value=on \
    -drive if=pflash,format=raw,readonly=on,file=/usr/share/edk2/ovmf/OVMF_CODE.fd \
    -drive if=pflash,format=raw,file=$PWD/veilor-vm.nvram \
    -drive file=$PWD/veilor-vm.qcow2,if=virtio,format=qcow2 \
    -monitor unix:$PWD/veilor-vm.monitor.sock,server,nowait \
    -netdev user,id=net0,hostfwd=tcp::2222-:22 \
    -device virtio-net-pci,netdev=net0 \
    -vga virtio -display gtk,gl=on

---

Real-hardware test — USB → spare laptop

Required for every tagged release. The VM cannot reproduce KMS / fbcon / GPU-driver issues; only real silicon will.

1. Flash USB

# 8GB+ USB stick, identified by lsblk (e.g. /dev/sda — confirm vendor)
sudo umount /dev/sdX* 2>/dev/null
sudo wipefs -a /dev/sdX
sudo dd if=/path/to/veilor-os-*.iso of=/dev/sdX bs=4M status=progress conv=fsync
sync
sudo eject /dev/sdX

Etcher / GNOME Disks also fine. Verify-after-flash is built into Etcher; for dd, run cmp on the first ISO_SIZE bytes if paranoid.

2. Boot test

• Disable Secure Boot in firmware (until we MOK-enroll our shim, which is v0.5+).
• Boot from USB.
• Walk the same numbered steps as the VM section, except:
  • On "TYPE NOW: passphrase" steps, you actually have a keyboard.
  • At step 8, the laptop will eject the USB and reboot to the installed system without intervention.
  • At step 11, do NOT use veilortest1 for the post-install admin password change — pick something real if this is your daily-driver laptop, or a throwaway if it's a test machine. The kickstart's ChainOfTrust ends here; from this prompt forward you own the password.

3. Capture findings

Fill in a fresh test/test-runs/YYYY-MM-DD-vX.Y.Z.md from the template. Always capture: GRUB title, kernel cmdline (cat /proc/cmdline), lsblk -f, getenforce, systemctl is-active fail2ban usbguard tuned auditd firewalld, journalctl -b -p err --no-pager.

If anything regressed, that goes at the top of the report under Regressions, with a screenshot if possible.

---

Per-run report template

Copy test/test-runs/_TEMPLATE.md (created when the first real test-run lands) and fill in section-by-section. Keep them brief — this is meant to be a 5-minute write-up, not a thesis.

---

When to alter this procedure

If a step turns out to be wrong, redundant, or missing:

1. Edit this file.
2. Append to test/METHOD-CHANGELOG.md with: date, version it first applied to, what changed, and why (cite a specific test-run report if the change is in response to a finding).
3. Reference the changelog entry in your commit message.

The changelog is the audit trail. Don't skip it.