minecraft-server/BACKUP-HUNT-2026-05-07.md
s8n a1cc3940cf docs: 2026-05-07 incident audit + backup strategy
Player YOU500 lost full inventory to AuthLimbo void-death at 17:13:39.
Investigation revealed deployed /opt/docker/backup.sh is an 88-line stub
missing the Minecraft block; last successful world backup 2026-05-02
(already pruned). No recoverable .dat exists.

Files:
- AUDIT-2026-05-07.md — server-side findings F-01..F-06 (P0 backups,
  no-keepInventory, AuthLimbo silent failure, chunk preload race,
  Xmx > container headroom, container hardening gaps)
- BACKUP-HUNT-2026-05-07.md — exhaustive backup scan; only 6-week-old
  archive at _archive/minecraft-old-2026-04-27.tar.gz
- BACKUP-STRATEGY.md — restic-based plan; 5min/hourly/daily classes,
  off-host to onyx via Tailscale, monthly drill
- CROSS-REFERENCE-2026-05-07.md — repo+doc landing map; flags
  pre-existing infra/STATE.md backup-broken note + HA-CLUSTER restic
  draft to extend rather than duplicate
- docs/RUNBOOK-BACKUP-RESTORE.md — operator runbook for .dat restore,
  full-world restore, host-loss restore, drill log
2026-05-07 17:33:24 +01:00

9.5 KiB

YOU500 Inventory Recovery — Backup Hunt Report

Date: 2026-05-07 Player: YOU500 (UUID c7c2df8e-8783-30b5-891c-86ec9343686b) Incident: Full inventory loss at 17:13:39 BST. AuthLimbo teleportAsync returned false, player teleport into world from auth_limbo failed → YOU500 left the confines of this world (void death). Vanilla /data/world/playerdata overwritten on respawn with empty inventory; vanilla void = no drops in world. Host: nullstone (192.168.0.100), live MC data at /home/docker/minecraft/ (== /opt/docker/minecraft/, same FS, inode 18877649 confirmed). SSH user: user (no sudo). All /opt/backups/2026* dated subdirs are root-owned 0700 → unreadable. /var/lib/docker/volumes/ unreadable.


Summary

Recoverable backup exists: YES — partial. The pre-rebrand world archive /home/user/ai-lab/_archive/minecraft-old-2026-04-27.tar.gz contains YOU500's playerdata .dat from 2026-03-25 18:53 (size 9617 B vs current 9192 B — bigger = inventory likely populated). It is the only known full-inventory snapshot for this UUID anywhere on the host.

Caveat: This is a 6-week-old snapshot. Items gained between 2026-03-25 and 2026-05-07 17:13 are NOT recoverable from any file backup. CoreProtect is installed and has been logging since 2026-05-01 → use /co inventory YOU500 and /co rollback to retrieve anything stored in containers post-2026-05-01.

No scheduled world backups exist. /opt/docker/backup.sh stopped backing up the MC world after 2026-05-02 (the world-backup branch was removed when the script was last edited; only configs/Matrix/RC are now dumped). Last world tarball that landed on disk: /opt/backups/20260430_020001/minecraft-configs-20260430_020001.tar.gz (12 KB → configs only, no playerdata).


Inventory of Backup Artifacts (oldest → newest)

When Path Size Owner Contains YOU500 .dat? Notes
2026-03-25 18:53 (file mtime inside) /home/user/ai-lab/_archive/minecraft-old-2026-04-27.tar.gz ~? large user YESminecraft/world/playerdata/c7c2df8e-…dat 9617 B + .dat_old 9616 B (2026-03-25 18:49) Best candidate. 133 player .dat files, full world tree, Essentials/LitePlaytimeRewards/LandClaim DBs, advancements, stats.
2026-04-30 02:01 /opt/backups/20260430_020001/minecraft-configs-20260430_020001.tar.gz 12 KB root (UNREADABLE) NO — configs only Cannot read without sudo; size implies no world data anyway.
2026-04-30 02:01 /opt/backups/20260430_020001/configs-20260430_020001.tar.gz 2.4 KB root NO Traefik/Matrix/RC configs.
2026-04-30 19:21 /opt/backups/mc-plugins-prerebrand-2026-04-30.tar.gz 224 MB user NO playerdata .dat files. Has plugins/AuthMe/playerdata/ (empty), plugins/AuthMe.bak-20260430-144204/playerdata/ (empty), plugins/SkinsRestorer/cache/YOU500.mojangcache. Vanilla world NOT included. Plugin trees only — useful for password DB (plugins/AuthMe.bak-…/authme.db), not inventory.
2026-05-03 02:00 /opt/backups/20260503_020001/configs-20260503_020001.tar.gz 2.4 KB root NO Configs.
2026-05-04 02:00 → 2026-05-07 02:00 /opt/backups/20260504_02000120260507_020001 0700 dirs root (UNREADABLE) Inferred NO from log: backup.log shows only "configs OK" / "Matrix Postgres skipping" / "Volumes skipping" — world not touched after 2026-05-02. All four dirs report 12 KB.
2026-05-07 17:15 /home/docker/minecraft/world/playerdata/c7c2df8e-…dat_old 9181 B uid 101000 YES — but POST-DEATH (empty inventory). Identical to live state right after first respawn.
2026-05-07 17:21 /home/docker/minecraft/world/playerdata/c7c2df8e-…dat 9192 B uid 101000 YES — current live, empty inventory.
2026-05-07 17:15 /tmp/you500.dat 9181 B user YES — but byte-identical-size to .dat_old; gunzip strings show only base attribute schema (no item/Slot tags) → already empty. Someone (you) already extracted the empty post-death dat. Useless for recovery.

Misc archives checked, NOT relevant

  • /opt/source-endpoint/source.tar.gz — Misskey AGPL source dump.
  • /opt/backups/misskey/* — Misskey DB/files.
  • /home/user/ai-lab/.stversions/_projects/_minecraft/launcher/java/java21.tar~*.gz — JDK.
  • /home/user/ai-lab/_projects/_minecraft/resources/racked.ru.-.minecraft.7z — launcher resources.
  • /home/user/ai-lab/.stversions/** — Syncthing versions hold only server config files (server.properties, bukkit.yml, purpur.yml etc.) under _github/online/minecraft-server/config/. No .dat or playerdata/ anywhere in .stversions. .stignore does not list world/, but the synced repo never contained the world dir to begin with (it's _github/minecraft-server/ = configs + docker-compose only).

CoreProtect — Live Rollback Source

Path Size Born Last modified
/data/plugins/CoreProtect/database.db (in container) 1.59 GB 2026-05-01 10:11:53 2026-05-07 17:27

CoreProtect logs container interactions, item drops, deaths, inventory changes since 2026-05-01. For YOU500's items stored in chests/shulkers/ender chests within the world, an in-game rollback can recover them:

  • Inspect deaths: /co lookup user:YOU500 action:#kill time:1d
  • Inspect inventory transactions: /co inventory YOU500 (CoreProtect-CE feature)
  • Rollback drops/voids near death: /co rollback time:1h user:YOU500 radius:#global action:-drop,#kill

(Items YOU500 carried in person and lost to void at 17:13:39 are unlikely to appear in CoreProtect — vanilla void death deletes drops without a kill event in some versions; CoreProtect's #kill may or may not have logged it. Worth a /co lookup user:YOU500 time:30m to confirm.)


Best Recovery Candidate

File: /home/user/ai-lab/_archive/minecraft-old-2026-04-27.tar.gz Internal path: minecraft/world/playerdata/c7c2df8e-8783-30b5-891c-86ec9343686b.dat Snapshot date: 2026-03-25 18:53 (~6 weeks before incident).

Extraction command (DO NOT RUN — for review only)

# Extract just the YOU500 dat to a staging area, do NOT touch live data
mkdir -p /tmp/you500-recovery
tar -xzvf /home/user/ai-lab/_archive/minecraft-old-2026-04-27.tar.gz \
  -C /tmp/you500-recovery \
  minecraft/world/playerdata/c7c2df8e-8783-30b5-891c-86ec9343686b.dat \
  minecraft/world/playerdata/c7c2df8e-8783-30b5-891c-86ec9343686b.dat_old

# Confirm and inspect (NBT viewer or zcat | strings) before any restore
ls -la /tmp/you500-recovery/minecraft/world/playerdata/
zcat /tmp/you500-recovery/minecraft/world/playerdata/c7c2df8e-8783-30b5-891c-86ec9343686b.dat \
  | strings | grep -E 'Slot|count|minecraft:diamond|minecraft:netherite' | head -40

Restore plan (operator decision — NOT executed)

  1. Stop the server (or kick YOU500) so file is not held open.
  2. With sudo (uid 101000 owns the file): copy the extracted .dat over /home/docker/minecraft/world/playerdata/c7c2df8e-8783-30b5-891c-86ec9343686b.dat, preserve mode/owner.
  3. Also overwrite .dat_old.
  4. Optional: replace Essentials/userdata/c7c2df8e-…yml from same archive if the YML matters.
  5. Restart server. Player rejoins with March 25 inventory + position.

Tradeoff: YOU500 will lose all progress 2026-03-25 → 2026-05-07. Communicate before applying. Combine with CoreProtect rollback to minimise loss.


Gaps

  • No scheduled world backups since 2026-05-02. /opt/docker/backup.sh no longer dumps world/. The 2026-04-30 daily contains a 12 KB "minecraft-configs" tarball (configs, not world). Action: re-add a world tarball to the daily script.
  • No off-host backup. No restic / borg / duplicity / rsnapshot installed. No rclone. No second host pulling MC data. Syncthing does not sync the world dir.
  • No filesystem snapshots. Root is ext4 on LVM (no LVM thinpool snapshots in use), /home is ext4 (no btrfs/ZFS).
  • /var/lib/docker/volumes/ unreadable without sudo. Confirmed via docker volume ls | grep -iE mine|back|world returning empty (named volumes not used for MC — bind mount only).
  • /opt/backups/2026*_020001 subdirs unreadable (mode 0700 root). Cannot diff their contents byte-for-byte; relied on backup.log text + indirect listing. They almost certainly contain only configs (12 KB dirs, log entries match).
  • docker exec minecraft-mc env | grep -i backup returned nothing — no env-driven autosave/backup plugin enabled (e.g. itzg/mc-backup sidecar absent, no AutomatedBackup / EasyBackup jar in /data/plugins).
  • AuthMe playerdata/ dirs are empty in both live and .bak-20260430-144204 — AuthMe is configured without inventory protection (no logged-out inv snapshots).
  • No InvSee / InventoryRollback plugin. Only CoreProtect (logs, not snapshots).

Permission-Limited Reads (no sudo via SSH)

Path What we couldn't see Likely contents
/opt/backups/20260504_020001/20260507_020001/ Directory listings (0700 root) Daily configs tarballs, ~12 KB each — confirmed via du in backup.log
/opt/backups/20260430_020001/minecraft-configs-20260430_020001.tar.gz tar listing (root-owned, 0600) MC config bind-mount tarball, 12914 B
/var/lib/docker/volumes/ Directory listing Named volumes — not used by MC (bind mount only)
/var/backups/ (host) Listing Standard Debian dpkg/apt backups, not MC
/root/ Anything

Re-run with sudo if any of these need confirmation, but content is improbable to change the conclusion.