Skip to main content

max / alloy

Keep the qemu harness that found the encrypted install bugs The installer is a ratatui program with no headless mode, so the only way to test what an installer medium actually does is to boot one and send it keystrokes. These four scripts do that: qemu with OVMF and a software TPM, keys and screendumps over the monitor socket, a command runner against the guest's serial root shell, and an ssh helper that sizes a pty so the TUI renders. They were written in a scratch directory and copied into a handoff bundle, which is a place that gets deleted on purpose once its work has landed. Both 2026-08-09 defects needed them, neither was visible without booting the result, and the boot-test task they unblock is only automatable because `vm.py shot` makes the recovery phrase readable. That is worth versioning next to the installer rather than losing. The README carries what cost time to learn: reset the firmware variables with the disk or the next boot lands at a grub prompt, the erase confirmation takes a second return, and the activity light means two screendumps of an unchanged screen still differ. ALLOY_ISO and VM_STATE override the paths; the defaults are repo-relative and the scratch directory is gitignored, kept out of output/ because build-iso.sh clears that at the start of every build.
Author: Max Johnson <me@maxj.phd> · 2026-08-10 01:54 UTC
Signed with PGP, not checked
Commit: 76758a1e36824e8ecf15fa35907ee00c96c44c20
Parent: 54c0345
6 files changed, +366 insertions, -0 deletions
M .gitignore +3
@@ -51,3 +51,6 @@
51 51 *.p12
52 52 *.pfx
53 53 credentials.json
54 +
55 + # vmtest scratch: qcow2 targets, firmware vars, vTPM state, screendumps.
56 + /build/vmtest/state/
A build/vmtest/README.md +71
@@ -1,0 +1,71 @@
1 + # vmtest — driving the installer in qemu
2 +
3 + Four scripts that boot an Alloy ISO headless and work the wizard from outside
4 + the guest. They exist because `alloy install` is a ratatui program with no
5 + headless mode, so the only way to test the thing an installer medium actually
6 + does is to send it keystrokes and read the screen back.
7 +
8 + Everything a run writes goes to `state/`, which is gitignored. Delete it to
9 + start clean. `build/build-iso.sh` clears `output/` at the start of every build,
10 + which is why the scratch directory is here instead.
11 +
12 + ## Requirements
13 +
14 + `qemu-system-x86_64` with KVM, OVMF, and `swtpm` plus `swtpm-tools`. The
15 + software TPM is not optional for the encrypted path: `bootc install
16 + --block-setup tpm2-luks` enrols a key into one, and without a TPM it fails for
17 + a reason that has nothing to do with what is being tested.
18 +
19 + ## Use
20 +
21 + ./run-vm.sh live # boot output/install.iso (ALLOY_ISO= to override)
22 + ./run-vm.sh installed # boot the target disk, no cdrom
23 +
24 + python3 vm.py hold 22 # hold the GRUB countdown open
25 + python3 vm.py key down down ret
26 + python3 vm.py type "some text"
27 + python3 vm.py shot name # screendump to state/name.png
28 + python3 serial.py "lsblk" 8 # run a command in the guest's serial root shell
29 + python3 ssh_pty.py ~/.ssh/id_ed25519 installer 30
30 +
31 + `vm.py shot` converts qemu's PPM to PNG with nothing but zlib and struct, so
32 + the installer screen can be read directly with no image tooling installed.
33 + `serial.py` reaches a root shell only on GRUB entry 3, which is the debug
34 + entry; the default entry runs the wizard and has no shell.
35 +
36 + ## Things that cost an afternoon to learn
37 +
38 + **Reset between runs by deleting `state/target.qcow2` and `state/OVMF_VARS.fd`
39 + together.** Leaving the firmware variables behind boots the half-written disk
40 + to a `grub>` prompt. Delete `state/tpm/` as well to simulate a cleared TPM.
41 +
42 + **Catch GRUB before the countdown expires.** The menu holds for five seconds
43 + from a cold boot, which is roughly when the harness is still starting. Reset
44 + with `vm.py cmd system_reset` and run `vm.py hold` immediately; any keypress
45 + stops the countdown, and `hold` sends one every 200ms.
46 +
47 + **Screenshot before pressing return in the GRUB menu.** Missing the countdown
48 + boots the default entry, and there is then no shell to recover through.
49 +
50 + **The activity light blinks, so two screendumps of an unchanged screen differ.**
51 + Comparing frames to detect progress means ignoring the header rows.
52 +
53 + **The erase confirmation takes a second return.** The review footer reads
54 + `enter install`, but the first return raises a modal. A script that sends one
55 + keystroke and waits sits there forever looking like a slow install.
56 +
57 + **Watch the target's size, not the clock.** A real install passes a gigabyte in
58 + the first minute; a qcow2 still at its created size means nothing has started.
59 +
60 + **The recovery phrase is legible in a screendump.** This was believed to need a
61 + person, which is why the encrypted path went untested for so long. It does not.
62 +
63 + ## What it caught
64 +
65 + The two defects fixed in `crates/alloy/src/install.rs` on 2026-08-09: the
66 + unmounted boot partition that made ostree report no deployment on every
67 + encrypted install, and bootc's `headless=true` karg that left the enrolled
68 + passphrase untypeable at boot. Neither is visible without booting the result.
69 +
70 + Wiki `alloy-build-notes` carries the longer form, including how to read a
71 + failed target before the installer's recovery erases it.
A build/vmtest/run-vm.sh +55
@@ -1,0 +1,55 @@
1 + #!/usr/bin/env bash
2 + # Boot the Alloy installer ISO in qemu, headless, with ssh forwarded to :2222.
3 + # run-vm.sh live boot the ISO (cdrom first)
4 + # run-vm.sh installed boot the target disk only (no cdrom)
5 + set -euo pipefail
6 +
7 + HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
8 + REPO_ROOT="$(cd "$HERE/../.." && pwd)"
9 +
10 + # Everything the run writes lives here, gitignored, and is safe to delete
11 + # between runs. Not under output/, which build-iso.sh clears at the start of
12 + # every build and would take a half-finished install with it.
13 + SCRATCH="${VM_STATE:-$HERE/state}"; mkdir -p "$SCRATCH"
14 + ISO=${ALLOY_ISO:-$REPO_ROOT/output/install.iso}
15 + DISK="$SCRATCH/target.qcow2"
16 + VARS="$SCRATCH/OVMF_VARS.fd"
17 + MODE="${1:-live}"
18 +
19 + [ -f "$VARS" ] || cp /usr/share/OVMF/OVMF_VARS_4M.fd "$VARS"
20 +
21 + # A software TPM, because `bootc install --block-setup tpm2-luks` enrols the
22 + # key into one and there is nothing to enrol into otherwise. State persists in
23 + # tpm/ so an installed disk can unlock on the next boot the way real hardware
24 + # would; delete that directory to simulate a cleared TPM.
25 + mkdir -p "$SCRATCH/tpm"
26 + if [ ! -S "$SCRATCH/tpm/swtpm-sock" ]; then
27 + swtpm_setup --tpm2 --tpmstate "$SCRATCH/tpm" --createek --create-ek-cert \
28 + --create-platform-cert --lock-nvram --overwrite >/dev/null 2>&1 || true
29 + swtpm socket --tpm2 --tpmstate dir="$SCRATCH/tpm" \
30 + --ctrl type=unixio,path="$SCRATCH/tpm/swtpm-sock" \
31 + --flags startup-clear --daemon
32 + fi
33 + [ -f "$DISK" ] || qemu-img create -f qcow2 "$DISK" 40G >/dev/null
34 +
35 + args=(
36 + -enable-kvm -machine q35 -cpu host -m 4608 -smp 4
37 + -drive if=pflash,format=raw,unit=0,readonly=on,file=/usr/share/OVMF/OVMF_CODE_4M.fd
38 + -drive "if=pflash,format=raw,unit=1,file=$VARS"
39 + -drive "file=$DISK,if=virtio,format=qcow2"
40 + -netdev user,id=n0,hostfwd=tcp:127.0.0.1:2222-:22
41 + -device virtio-net-pci,netdev=n0
42 + -chardev "socket,id=chrtpm,path=$SCRATCH/tpm/swtpm-sock"
43 + -tpmdev emulator,id=tpm0,chardev=chrtpm
44 + -device tpm-crb,tpmdev=tpm0
45 + -display none
46 + -chardev "socket,id=ser0,path=$SCRATCH/serial.sock,server=on,wait=off,logfile=$SCRATCH/serial-$MODE.log"
47 + -serial chardev:ser0
48 + -monitor "unix:$SCRATCH/monitor.sock,server,nowait"
49 + )
50 +
51 + if [ "$MODE" = live ]; then
52 + args+=(-drive "file=$ISO,media=cdrom,readonly=on" -boot d)
53 + fi
54 +
55 + exec qemu-system-x86_64 "${args[@]}"
A build/vmtest/serial.py +44
@@ -1,0 +1,44 @@
1 + #!/usr/bin/env python3
2 + """Send a shell command to the guest's serial debug shell and read the reply.
3 +
4 + serial.py "cmd" run and read for 5s
5 + serial.py "cmd" 20 run and read for 20s
6 + serial.py --read 5 read only
7 + """
8 + import socket, sys, time, os
9 +
10 + SCRATCH = os.environ.get(
11 + "VM_STATE", os.path.join(os.path.dirname(os.path.abspath(__file__)), "state")
12 + )
13 + s = socket.socket(socket.AF_UNIX)
14 + s.connect(os.path.join(SCRATCH, "serial.sock"))
15 + s.settimeout(0.5)
16 +
17 +
18 + def drain():
19 + out = b""
20 + try:
21 + while True:
22 + c = s.recv(65536)
23 + if not c:
24 + break
25 + out += c
26 + except (socket.timeout, BlockingIOError):
27 + pass
28 + return out
29 +
30 +
31 + if sys.argv[1] == "--read":
32 + time.sleep(float(sys.argv[2]))
33 + sys.stdout.write(drain().decode(errors="replace"))
34 + sys.exit()
35 +
36 + drain()
37 + cmd = sys.argv[1]
38 + secs = float(sys.argv[2]) if len(sys.argv) > 2 else 5.0
39 + s.sendall(("\n" + cmd + "\n").encode())
40 + end = time.time() + secs
41 + out = b""
42 + while time.time() < end:
43 + out += drain()
44 + sys.stdout.write(out.decode(errors="replace"))
A build/vmtest/ssh_pty.py +49
@@ -1,0 +1,49 @@
1 + #!/usr/bin/env python3
2 + """Run ssh under a real pty of a known size and capture what it draws.
3 +
4 + ssh_pty.py <keyfile> <user> [seconds] [keys-to-send]
5 +
6 + Keys are sent after 4s, then output is captured until the timeout.
7 + """
8 + import os, pty, sys, time, select, struct, fcntl, termios, signal
9 +
10 + key, user = sys.argv[1], sys.argv[2]
11 + seconds = float(sys.argv[3]) if len(sys.argv) > 3 else 12.0
12 + to_send = sys.argv[4] if len(sys.argv) > 4 else ""
13 +
14 + pid, fd = pty.fork()
15 + if pid == 0:
16 + os.execvp("ssh", [
17 + "ssh", "-tt",
18 + "-o", "StrictHostKeyChecking=no",
19 + "-o", "UserKnownHostsFile=/dev/null",
20 + "-o", "LogLevel=ERROR",
21 + "-p", "2222", "-i", key, f"{user}@127.0.0.1",
22 + ])
23 +
24 + fcntl.ioctl(fd, termios.TIOCSWINSZ, struct.pack("HHHH", 40, 120, 0, 0))
25 +
26 + out = b""
27 + deadline = time.time() + seconds
28 + sent = False
29 + while time.time() < deadline:
30 + if not sent and time.time() > deadline - seconds + 5:
31 + if to_send:
32 + os.write(fd, to_send.encode())
33 + sent = True
34 + r, _, _ = select.select([fd], [], [], 0.5)
35 + if r:
36 + try:
37 + chunk = os.read(fd, 65536)
38 + except OSError:
39 + break
40 + if not chunk:
41 + break
42 + out += chunk
43 +
44 + try:
45 + os.kill(pid, signal.SIGKILL)
46 + except ProcessLookupError:
47 + pass
48 + os.waitpid(pid, 0)
49 + sys.stdout.buffer.write(out)
A build/vmtest/vm.py +144
@@ -1,0 +1,144 @@
1 + #!/usr/bin/env python3
2 + """Drive the qemu monitor: sendkey a string, and screendump to PNG.
3 +
4 + Usage:
5 + vm.py type "some text" type printable ASCII into the guest
6 + vm.py key ret tab spc send named keys
7 + vm.py shot name screendump to name.png
8 + """
9 + import socket, sys, time, zlib, struct, os, re
10 +
11 + SCRATCH = os.environ.get(
12 + "VM_STATE", os.path.join(os.path.dirname(os.path.abspath(__file__)), "state")
13 + )
14 + SOCK = os.path.join(SCRATCH, "monitor.sock")
15 +
16 + NAMED = {
17 + " ": "spc", "-": "minus", "=": "equal", "[": "bracket_left",
18 + "]": "bracket_right", ";": "semicolon", "'": "apostrophe",
19 + "`": "grave_accent", "\\": "backslash", ",": "comma", ".": "dot",
20 + "/": "slash",
21 + }
22 + SHIFTED = {
23 + "!": "1", "@": "2", "#": "3", "$": "4", "%": "5", "^": "6", "&": "7",
24 + "*": "8", "(": "9", ")": "0", "_": "minus", "+": "equal",
25 + "{": "bracket_left", "}": "bracket_right", ":": "semicolon",
26 + '"': "apostrophe", "~": "grave_accent", "|": "backslash", "<": "comma",
27 + ">": "dot", "?": "slash",
28 + }
29 +
30 +
31 + class Monitor:
32 + def __init__(self):
33 + self.s = socket.socket(socket.AF_UNIX)
34 + self.s.connect(SOCK)
35 + time.sleep(0.2)
36 + self.drain()
37 +
38 + def drain(self):
39 + self.s.setblocking(False)
40 + out = b""
41 + try:
42 + while True:
43 + chunk = self.s.recv(65536)
44 + if not chunk:
45 + break
46 + out += chunk
47 + except BlockingIOError:
48 + pass
49 + self.s.setblocking(True)
50 + return out.decode(errors="replace")
51 +
52 + def cmd(self, line, wait=0.05):
53 + self.s.sendall((line + "\n").encode())
54 + time.sleep(wait)
55 + return self.drain()
56 +
57 + def sendkey(self, k):
58 + self.cmd("sendkey " + k, wait=0.04)
59 +
60 + def type(self, text):
61 + for c in text:
62 + if c.isalnum():
63 + self.sendkey(c if not c.isupper() else "shift-" + c.lower())
64 + elif c in NAMED:
65 + self.sendkey(NAMED[c])
66 + elif c in SHIFTED:
67 + self.sendkey("shift-" + SHIFTED[c])
68 + elif c == "\n":
69 + self.sendkey("ret")
70 + else:
71 + raise SystemExit("no key mapping for %r" % c)
72 +
73 +
74 + def ppm_to_png(ppm_path, png_path):
75 + with open(ppm_path, "rb") as f:
76 + data = f.read()
77 + # P6 header: magic, width height, maxval, each possibly comment-separated
78 + fields, idx = [], 2
79 + while len(fields) < 3:
80 + while data[idx:idx + 1].isspace():
81 + idx += 1
82 + if data[idx:idx + 1] == b"#":
83 + while data[idx:idx + 1] != b"\n":
84 + idx += 1
85 + continue
86 + start = idx
87 + while not data[idx:idx + 1].isspace():
88 + idx += 1
89 + fields.append(int(data[start:idx]))
90 + idx += 1
91 + w, h, _maxval = fields
92 + pixels = data[idx:]
93 + raw = b"".join(b"\x00" + pixels[y * w * 3:(y + 1) * w * 3] for y in range(h))
94 +
95 + def chunk(tag, payload):
96 + return (struct.pack(">I", len(payload)) + tag + payload
97 + + struct.pack(">I", zlib.crc32(tag + payload) & 0xFFFFFFFF))
98 +
99 + png = (b"\x89PNG\r\n\x1a\n"
100 + + chunk(b"IHDR", struct.pack(">IIBBBBB", w, h, 8, 2, 0, 0, 0))
101 + + chunk(b"IDAT", zlib.compress(raw, 6))
102 + + chunk(b"IEND", b""))
103 + with open(png_path, "wb") as f:
104 + f.write(png)
105 + return w, h
106 +
107 +
108 + def main():
109 + if len(sys.argv) < 2:
110 + raise SystemExit(__doc__)
111 + verb = sys.argv[1]
112 + if verb == "shot":
113 + name = sys.argv[2]
114 + ppm = os.path.join(SCRATCH, name + ".ppm")
115 + png = os.path.join(SCRATCH, name + ".png")
116 + m = Monitor()
117 + m.cmd("screendump " + ppm, wait=1.0)
118 + for _ in range(20):
119 + if os.path.exists(ppm) and os.path.getsize(ppm) > 1000:
120 + break
121 + time.sleep(0.3)
122 + print(ppm_to_png(ppm, png), png)
123 + return
124 + m = Monitor()
125 + if verb == "type":
126 + m.type(sys.argv[2])
127 + elif verb == "key":
128 + for k in sys.argv[2:]:
129 + m.sendkey(k)
130 + elif verb == "hold":
131 + # Keep the GRUB countdown from expiring: one connection, many keys.
132 + end = time.time() + float(sys.argv[2])
133 + while time.time() < end:
134 + m.sendkey("down")
135 + m.sendkey("up")
136 + time.sleep(0.2)
137 + elif verb == "cmd":
138 + print(m.cmd(" ".join(sys.argv[2:]), wait=0.5))
139 + else:
140 + raise SystemExit(__doc__)
141 +
142 +
143 + if __name__ == "__main__":
144 + main()