infra/forgejo/DEPLOY.md
s8n 09d80a63f6 init: nullstone deploys + runbooks + audits
Sourced from previous audits + agent-wave outputs (2026-05-05):
  AUDIT-2026-05-05.md           — 5-agent stack synthesis
  forgejo/DEPLOY.md             — git.s8n.ru deploy runbook
  forgejo/forgejo-compose.yml   — production compose
  forgejo/runner-compose.yml    — forgejo-runner
  forgejo/migration-report-...  — GH→Forgejo migration audit (6/6 green)
  runbooks/MIGRATION-...        — nullstone→cobblestone runbook
  runbooks/DE-DECISION-...      — keep-vs-strip DE on cobblestone
  repos/REPO-AUDIT-2026-05-05.md — repo trees + ownership
2026-05-06 10:02:28 +01:00

5.3 KiB

Forgejo deploy runbook — nullstone

Self-host plan: replace GH Actions free-tier (quota-bound) with Forgejo + forgejo-runner running on nullstone. Same build-iso.yml workflow, no GH dependency.

Pre-flight

  • nullstone reachable at 192.168.0.100 (LAN) and via tailscale (mesh)
  • Traefik running, proxy docker network exists
  • Gandi API token configured in traefik env (LiveDNS scope, s8n.ru only) → letsencrypt resolver works for new hostnames automatically
  • DNS for git.s8n.ru must point at nullstone's public IP (Gandi manual web UI; API can't add new records per memory reference_gandi_api.md)

Step 1 — DNS

Add A record git.s8n.ru → <nullstone public IP> via Gandi web UI. Wait ~2min for propagation. Verify:

dig +short git.s8n.ru @1.1.1.1

Step 2 — copy compose files to nullstone

scp /home/admin/ai-lab/nullstone-server/forgejo/docker-compose.yml \
    nullstone:/tmp/forgejo-compose.yml
scp /home/admin/ai-lab/nullstone-server/forgejo/runner-compose.yml \
    nullstone:/tmp/forgejo-runner-compose.yml

ssh nullstone bash <<'EOF'
sudo mkdir -p /opt/docker/forgejo/{data,config}
sudo mkdir -p /opt/docker/forgejo-runner/{data,cache}
sudo chown -R 1000:1000 /opt/docker/forgejo
sudo mv /tmp/forgejo-compose.yml /opt/docker/forgejo/docker-compose.yml
sudo mv /tmp/forgejo-runner-compose.yml /opt/docker/forgejo-runner/docker-compose.yml
EOF

Step 3 — first-start Forgejo

ssh nullstone 'cd /opt/docker/forgejo && docker compose up -d'
ssh nullstone 'docker logs -f forgejo' &  # watch first-start

When you see Listen: http://0.0.0.0:3000, Forgejo is up. Hit https://git.s8n.ru/ in your browser. Traefik gets the LE cert automatically.

Step 4 — initial admin user

The first-time wizard at /install is disabled by env (we set FORGEJO__service__DISABLE_REGISTRATION=true). Create the admin via CLI inside the container:

ssh nullstone 'docker exec -u 1000 forgejo \
    forgejo admin user create \
        --admin \
        --username admin \
        --email <your-email> \
        --random-password \
        --must-change-password=false'

The random password gets printed once — save it somewhere safe. Login at git.s8n.ru with admin + that password, change it via the web UI's user settings.

Step 5 — generate runner registration token

ssh nullstone 'docker exec -u 1000 forgejo \
    forgejo actions generate-runner-token'

Output is a single line — copy it into .env next to the runner compose:

echo "RUNNER_TOKEN=<token>" | ssh nullstone 'sudo tee /opt/docker/forgejo-runner/.env'
ssh nullstone 'sudo chmod 600 /opt/docker/forgejo-runner/.env'

Step 6 — start runner

ssh nullstone 'cd /opt/docker/forgejo-runner && docker compose up -d'
ssh nullstone 'docker logs -f forgejo-runner'

Look for Runner registered successfully. Verify in Forgejo web UI: Site Administration → Actions → Runners — should list nullstone.

Step 7 — mirror veilor-os repo

In the Forgejo web UI:

  1. Create org veilor-org (matches GH org name).
  2. Click + → Migrate Repository.
  3. Type: GitHub. URL: https://github.com/veilor-org/veilor-os.
  4. Mirror = ON. Description: "self-hosted mirror; primary dev here".
  5. Click Migrate.

Forgejo pulls the repo + all branches + tags + actions config. Once done, push from local will go to BOTH (set as second remote):

cd ~/ai-lab/_github/veilor-os
git remote add nullstone https://git.s8n.ru/veilor-org/veilor-os
git push nullstone main v0.7-bluebuild-spike

Step 8 — flip workflow to nullstone runner

Change build-iso.yml:

runs-on: ubuntu-24.04  # before
runs-on: nullstone     # after — picks up our forgejo runner

Push to nullstone remote. Watch Forgejo Actions tab. Same workflow, runs on our hardware, no GH minutes.

Step 9 — close the loop

Mirror Forgejo → GitHub for public visibility. Forgejo settings on the repo → Mirror → Push mirror → https://github.com/veilor-org/veilor-os with a GH PAT that has write access. Forgejo pushes on every commit.

End state:

  • git push origin → GH (public mirror)
  • git push nullstone → Forgejo (primary; runs CI)
  • Forgejo auto-pushes to GH for visibility
  • ISO builds run unlimited on nullstone hardware
  • 0 GH Actions minutes consumed

Disk needs

  • Forgejo data: ~1GB initial, grows ~100MB/yr per repo
  • Runner workspace: ~80GB free recommended for ISO builds (squashfs
    • downloaded RPMs + xorriso staging)
  • Runner cache: ~20GB for actions/cache-style hits across builds

Confirm with df -h / on nullstone before kickoff.

Resource cost

  • Forgejo: ~200MB RAM idle, ~500MB during build queues
  • Runner: idle 50MB, ~4GB during ISO build (depsolve + squashfs)
  • Network: ~2GB/build (Fedora package download)

Should fit alongside existing nullstone services without contention.

Rollback

If anything breaks:

ssh nullstone 'cd /opt/docker/forgejo && docker compose down'
ssh nullstone 'cd /opt/docker/forgejo-runner && docker compose down'

Local repo origin still points at GH; nothing on the dev side changes. ISO builds fall back to GH Actions until quota cycles.

See also

  • veilor-os roadmap: _github/veilor-os/docs/ROADMAP.md
  • nullstone service inventory: ~/ai-lab/SYSTEM.md
  • Existing service patterns: /opt/docker/headscale/, /opt/docker/authentik/