Skip to main content

max / alloy

Check the installed machine's labels, and say how to repair one that is wrong bootc container lint answers for the image. Nothing answered for what the installer wrote afterwards, which is why a mislabelled /etc survived every build and surfaced only when something finally needed DynamicUser. build/check-installed.sh runs restorecon -nvR over /etc and /usr and starts a real DynamicUser unit, with a --self-test over the filter because a wrong verdict still looks like a verdict. The manual carries the one-command repair for machines installed before the fix, which is every machine that has one.
Co-Authored-By
Claude Opus 5 (1M context) <noreply@anthropic.com>
Author: Max Johnson <me@maxj.phd> · 2026-08-16 00:15 UTC
Signed with PGP, not checked
Commit: c407be91a49b8eee7a73085f39b21b899a08f6d0
Parent: f6d4b20
4 files changed, +226 insertions, -0 deletions
@@ -60,6 +60,26 @@
60 60 **The recovery phrase is legible in a screendump.** This was believed to need a
61 61 person, which is why the encrypted path went untested for so long. It does not.
62 62
63 + ## Check the result, not only the screen
64 +
65 + An install that finishes is not an install that worked. `bootc container lint`
66 + answers for the image; nothing answered for what the installer wrote afterwards
67 + until `build/check-installed.sh`, which is why a mislabelled `/etc` survived
68 + every build and was found only when something finally needed `DynamicUser`.
69 +
70 + Run it on the installed guest once it is up:
71 +
72 + ssh -p 2222 -i ~/.ssh/id_ed25519 <user>@127.0.0.1 'sudo bash -s' \
73 + < ../check-installed.sh
74 +
75 + `run-vm.sh` forwards the guest's ssh to :2222 on both `live` and `installed`,
76 + which is the same port `ssh_pty.py` uses.
77 +
78 + Exit 0 is a labelled machine whose `DynamicUser` units start, 1 is a defect
79 + with the repair printed, and 3 means the check could not run — not root, no
80 + `restorecon`, or an install from the `selinux=0` GRUB entry, which deliberately
81 + produces an unlabelled machine.
82 +
63 83 ## What it caught
64 84
65 85 The two defects fixed in `crates/alloy/src/install.rs` on 2026-08-09: the
@@ -210,6 +210,32 @@
210 210 Layered packages are the most common way to break an otherwise reliable base,
211 211 which is the argument in chapter 7 for putting things in boxes instead.
212 212
213 + ### The machine came up with no console, and nothing will layer
214 +
215 + The symptom set is distinctive: no `alloy` and no terminal after the first boot
216 + following an install, `rpm-ostree status` failing with `Could not activate
217 + remote peer`, and `systemctl --failed` naming
218 + `alloy-layer-components.service`. Nothing in the journal explains any of it.
219 +
220 + Machines installed before 2026-08-14 are the ones that have this. The installer
221 + wrote six files in `/etc` without asking the policy what they should be
222 + labelled, one of them being the lock systemd needs to run a service under a
223 + dynamic user. `rpm-ostreed` is such a service, so the machine cannot layer a
224 + package at all, and the components that make it usable are layered at first
225 + boot.
226 +
227 + One command fixes it, from a virtual console, with no reinstall:
228 +
229 + sudo restorecon -R /etc
230 + systemctl reboot
231 +
232 + Confirm it afterwards, from a checkout of the repository:
233 +
234 + sudo build/check-installed.sh
235 +
236 + An install from a current image does not need any of this. The installer
237 + relabels the target itself now, and the check above passes on a fresh machine.
238 +
213 239 ---
214 240
215 241 ## Reporting something
@@ -2364,6 +2364,13 @@
2364 2364 /// about `/etc/passwd` rather than about `/mnt/.../etc/passwd`, which matches
2365 2365 /// nothing and would relabel nothing while exiting 0.
2366 2366 ///
2367 + /// Checked from outside rather than trusted: `build/check-installed.sh` runs
2368 + /// `restorecon -nvR` over `/etc` and `/usr` on an installed machine and starts
2369 + /// a real `DynamicUser` unit. The defect above passed every build because
2370 + /// `bootc container lint` answers for the image and nothing answered for what
2371 + /// the installer wrote afterwards, so that script is the check that would have
2372 + /// caught it on the first install rather than months later.
2373 + ///
2367 2374 /// Skipped, rather than failed, when the live system has no selinuxfs. That is
2368 2375 /// the `selinux=0` GRUB entry, which exists as the escape hatch for a live
2369 2376 /// system that will not boot otherwise (build/make-iso.sh). An install from it
@@ -1,0 +1,173 @@
1 + #!/usr/bin/env bash
2 + #
3 + # check-installed.sh — assert that an *installed* machine is labelled the way
4 + # the policy says it should be, and that the consequence of getting that wrong
5 + # is actually absent.
6 + #
7 + # `bootc container lint` checks the image. On 2026-08-14 the image was fine and
8 + # the machine was not: the installer's own configure stages run through a chroot
9 + # with no `/sys` mounted, so libselinux concluded SELinux was off and
10 + # shadow-utils wrote /etc/passwd, /etc/group, /etc/.pwd.lock, /etc/shadow and
11 + # /etc/gshadow with no label, and `systemd-firstboot --root` did the same to
12 + # /etc/hostname. Six files. Every build passed, because the defect was in what
13 + # the installer wrote afterwards.
14 + #
15 + # Six mislabelled files would be a triviality if one of them were not
16 + # `/etc/.pwd.lock`. systemd cannot take that lock as `etc_t`, so every unit with
17 + # `DynamicUser=yes` fails to start, with no AVC in the journal because the
18 + # denial is dontaudited. `rpm-ostreed.service` is one of those units, so the
19 + # machine cannot layer a package at all: `alloy-layer-components.service` fails
20 + # on the first boot after an install and the machine comes up with no console
21 + # and no terminal, `rpm-ostree status` fails with `Could not activate remote
22 + # peer`, and the hotfix channel is inert. The fix is `relabel_stages` in
23 + # `crates/alloy/src/install.rs`; this script is the check that would have caught
24 + # it the first time anyone installed, rather than on the day something finally
25 + # needed DynamicUser.
26 + #
27 + # So it checks both ends, and both are cheap:
28 + #
29 + # labels `restorecon -nvR` over /etc and /usr, which reports what it would
30 + # change and changes nothing.
31 + # dynuser a real `systemd-run -p DynamicUser=yes`, because that is the
32 + # property anyone cares about and no amount of label-reading proves
33 + # it as directly as running one.
34 + #
35 + # Usage:
36 + # sudo build/check-installed.sh # check this machine
37 + # build/check-installed.sh --self-test # check the filter; touches nothing
38 + #
39 + # Or against a VM from the harness in build/vmtest:
40 + # ssh alloy-vm 'sudo bash -s' < build/check-installed.sh
41 + #
42 + # The self-test exists for the same reason check-rust-stage.sh has one: the
43 + # filter is the part that fails invisibly, since a wrong verdict still looks
44 + # like a verdict. Run it after touching IGNORE.
45 + #
46 + # Exit codes follow check-rust-stage.sh:
47 + #
48 + # 0 the machine is labelled correctly and DynamicUser works.
49 + # 1 it is not. Every offending line is printed, with the repair.
50 + # 3 something about the run, not about the machine: not root, no
51 + # restorecon, or SELinux is not enforcing here. Prints `error:`.
52 + #
53 + # Why SELinux-off is 3 and not 0. Installing from the `selinux=0` GRUB entry
54 + # (build/make-iso.sh) deliberately produces an unlabelled machine, and the
55 + # installer skips relabelling there rather than failing, so that the escape
56 + # hatch stays an escape hatch. A machine in that state is not passing this
57 + # check; it is outside what the check can answer, and saying so is the honest
58 + # result.
59 +
60 + set -Eeuo pipefail
61 +
62 + # Lines restorecon reports that are not a defect.
63 + #
64 + # A semanage transaction leaves empty lock files under the policy store,
65 + # labelled by the process that made them rather than by the policy. They appear
66 + # on any machine that has ever layered a package — which, on Alloy, is every
67 + # machine after its first boot — and they are not related to anything here.
68 + #
69 + # Anchored on the policy store path rather than on the basename: a file called
70 + # semanage.LOCK somewhere else in /etc would be a real finding, and this filter
71 + # is the one place a real finding can be lost.
72 + IGNORE='/etc/selinux/[^/ ]+/semanage\.[^/ ]*LOCK'
73 +
74 + REPAIR='sudo restorecon -R /etc'
75 +
76 + die() { printf 'error: %s\n' "$*" >&2; exit 3; }
77 +
78 + trap 'rc=$?; [ "$rc" -eq 1 ] && exit 1; [ "$rc" -eq 3 ] && exit 3; die "unexpected failure at line ${LINENO}"' ERR
79 +
80 + # Everything restorecon would change, minus the lines above. Reads stdin so the
81 + # self-test can feed it a fixture instead of a machine.
82 + filter_findings() { grep -Ev "$IGNORE" || true; }
83 +
84 + # Fixtures are real: the six lines are the defect as it was found on 2026-08-14,
85 + # and the two locks are what a correct machine reports after it has layered its
86 + # components.
87 + self_test() {
88 + local fails=0 out
89 +
90 + out="$(printf '%s\n' \
91 + 'Would relabel /etc/passwd from unconfined_u:object_r:etc_t:s0 to system_u:object_r:passwd_file_t:s0' \
92 + 'Would relabel /etc/group from unconfined_u:object_r:etc_t:s0 to system_u:object_r:passwd_file_t:s0' \
93 + 'Would relabel /etc/.pwd.lock from unconfined_u:object_r:etc_t:s0 to system_u:object_r:passwd_file_t:s0' \
94 + 'Would relabel /etc/shadow from unconfined_u:object_r:etc_t:s0 to system_u:object_r:shadow_t:s0' \
95 + 'Would relabel /etc/gshadow from unconfined_u:object_r:etc_t:s0 to system_u:object_r:shadow_t:s0' \
96 + 'Would relabel /etc/hostname from unconfined_u:object_r:var_run_t:s0 to system_u:object_r:hostname_etc_t:s0' \
97 + | filter_findings | wc -l)"
98 + if [ "$out" -ne 6 ]; then
99 + printf 'self-test: the six known mislabels should all survive the filter, %s did\n' "$out" >&2
100 + fails=$((fails + 1))
101 + fi
102 +
103 + out="$(printf '%s\n' \
104 + 'Would relabel /etc/selinux/targeted/semanage.read.LOCK from system_u:object_r:semanage_store_t:s0 to system_u:object_r:selinux_config_t:s0' \
105 + 'Would relabel /etc/selinux/targeted/semanage.trans.LOCK from system_u:object_r:semanage_store_t:s0 to system_u:object_r:selinux_config_t:s0' \
106 + | filter_findings | wc -l)"
107 + if [ "$out" -ne 0 ]; then
108 + printf 'self-test: the semanage locks should be filtered, %s survived\n' "$out" >&2
109 + fails=$((fails + 1))
110 + fi
111 +
112 + # The filter is anchored on the policy store, so a lock-shaped name elsewhere
113 + # is a finding. This is the assertion that keeps IGNORE from being widened
114 + # into something that hides one.
115 + out="$(printf '%s\n' \
116 + 'Would relabel /etc/semanage.trans.LOCK from system_u:object_r:etc_t:s0 to system_u:object_r:selinux_config_t:s0' \
117 + | filter_findings | wc -l)"
118 + if [ "$out" -ne 1 ]; then
119 + printf 'self-test: a lock outside the policy store should survive the filter\n' >&2
120 + fails=$((fails + 1))
121 + fi
122 +
123 + if [ "$fails" -ne 0 ]; then
124 + printf 'self-test: %s failed\n' "$fails" >&2
125 + exit 1
126 + fi
127 + printf 'self-test: the filter is right about all three shapes\n'
128 + }
129 +
130 + if [ "${1:-}" = "--self-test" ]; then
131 + self_test
132 + exit 0
133 + fi
134 + [ $# -eq 0 ] || die "unknown argument: $1"
135 +
136 + [ "$(id -u)" -eq 0 ] || die "run as root: restorecon cannot stat everything under /etc otherwise, and a partial read would pass"
137 + command -v restorecon >/dev/null 2>&1 || die "no restorecon: install policycoreutils"
138 + [ -d /sys/fs/selinux ] || die "SELinux is not enforcing on this machine, so there is nothing to compare against"
139 +
140 + status=0
141 +
142 + # -n changes nothing, -v prints what it would have changed. /usr as well as
143 + # /etc: the 2026-08-14 defect was confined to /etc, and the reason anyone knew
144 + # that was a clean /usr, so checking only the half that broke would leave the
145 + # next one unmeasured.
146 + findings="$(restorecon -nvR /etc /usr 2>/dev/null | filter_findings)"
147 + if [ -n "$findings" ]; then
148 + printf 'mislabelled, %s file(s):\n' "$(printf '%s\n' "$findings" | wc -l)"
149 + printf '%s\n' "$findings"
150 + printf '\nrepair: %s\n' "$REPAIR"
151 + status=1
152 + else
153 + printf 'labels: /etc and /usr match the policy\n'
154 + fi
155 +
156 + # The consequence, measured rather than inferred. `systemd-run --wait` returns
157 + # the unit's own exit status, so a failure to *start* it is what is being
158 + # caught here and /bin/true is only there to be something to start.
159 + if command -v systemd-run >/dev/null 2>&1; then
160 + if err="$(systemd-run -q --wait --property=DynamicUser=yes /bin/true 2>&1)"; then
161 + printf 'dynuser: a DynamicUser unit starts\n'
162 + else
163 + printf 'dynuser: a DynamicUser unit does NOT start, so rpm-ostreed cannot run and nothing can be layered\n'
164 + printf '%s\n' "$err"
165 + printf '\nrepair: %s\n' "$REPAIR"
166 + status=1
167 + fi
168 + else
169 + # Not fatal: the label half is the check, and this half is its confirmation.
170 + printf 'dynuser: skipped, no systemd-run\n'
171 + fi
172 +
173 + exit "$status"