| 1 |
# 14. Troubleshooting |
| 2 |
|
| 3 |
## The general moves |
| 4 |
|
| 5 |
**Get a console that is not sway.** `Ctrl+Alt+F2` through `F6` are virtual |
| 6 |
terminals. They are themed the same way and they work when the session does |
| 7 |
not. `Ctrl+Alt+F1` goes back to the greeter. |
| 8 |
|
| 9 |
**Read the logs.** |
| 10 |
|
| 11 |
journalctl -b -p err # this boot, errors and worse |
| 12 |
journalctl --user -b # your session's services |
| 13 |
journalctl -b -u greetd # the login screen |
| 14 |
|
| 15 |
**Reload sway** after editing its config: `Mod+Shift+C`. |
| 16 |
|
| 17 |
**Boot the previous image.** Hold `Esc` or `Shift` at boot for the GRUB menu |
| 18 |
and pick the older deployment. Nothing you do to the running system can take |
| 19 |
that away, which is the point of an atomic base. |
| 20 |
|
| 21 |
--- |
| 22 |
|
| 23 |
## The session |
| 24 |
|
| 25 |
### Nothing opens when I press Mod+Return |
| 26 |
|
| 27 |
Check the config's terminal name. `~/.config/sway/config` sets `$term shop`, |
| 28 |
and if that line has been edited to something the image does not carry, the |
| 29 |
binding runs nothing and says nothing. |
| 30 |
|
| 31 |
`shop` is built into the image rather than installed from a repo, so the |
| 32 |
package manager cannot tell you whether it is there. `command -v shop` from a |
| 33 |
virtual console can. |
| 34 |
|
| 35 |
Two ways out that do not need a terminal: `Mod+D` opens the launcher, and |
| 36 |
`Ctrl+Alt+F2` gets you a virtual console where you can edit |
| 37 |
`~/.config/sway/config` and log back in. |
| 38 |
|
| 39 |
### An app exits saying it cannot open a display |
| 40 |
|
| 41 |
It is an X11 application, and there is no X server. See chapter 7: Alloy ships |
| 42 |
`xwayland disable`, and the way back is `xwayland enable` in |
| 43 |
`~/.config/sway/config.d/`. |
| 44 |
|
| 45 |
Try the Wayland flag first if it is an Electron app: |
| 46 |
|
| 47 |
org.example.SomeApp --ozone-platform=wayland |
| 48 |
|
| 49 |
### I mistyped my password too many times and now it will not take the right one |
| 50 |
|
| 51 |
Fifty consecutive failures locks the account for 24 hours. From another session, |
| 52 |
or a virtual console logged in as another user: |
| 53 |
|
| 54 |
run0 faillock --user <name> --reset |
| 55 |
|
| 56 |
`faillock --user <name>` on its own shows the count without clearing it. |
| 57 |
|
| 58 |
### I edited the sway config and the session will not start |
| 59 |
|
| 60 |
Log in on a virtual console and check it: |
| 61 |
|
| 62 |
sway --validate |
| 63 |
|
| 64 |
Then fix the file, or move it aside to fall back to the shipped one: |
| 65 |
|
| 66 |
mv ~/.config/sway/config ~/.config/sway/config.broken |
| 67 |
alloy theme apply |
| 68 |
|
| 69 |
### The screen locked and I cannot unlock it |
| 70 |
|
| 71 |
Type your password even if nothing is drawn; swaylock accepts input before it |
| 72 |
has anything to show. If the lock has genuinely crashed, switch to a virtual |
| 73 |
console and kill it: |
| 74 |
|
| 75 |
pkill swaylock |
| 76 |
|
| 77 |
That leaves an unlocked session, so do it only at a machine you trust. |
| 78 |
|
| 79 |
### My session locks while I am watching a video |
| 80 |
|
| 81 |
Only a handful of applications hold the lock off, and only when fullscreen: |
| 82 |
mpv, imv, and the browser. Add yours in `~/.config/sway/config` next to the |
| 83 |
existing `inhibit_idle fullscreen` lines. A blanket rule is deliberately not |
| 84 |
shipped, because it would make walking away from a fullscreen terminal leave |
| 85 |
the machine unlocked. |
| 86 |
|
| 87 |
--- |
| 88 |
|
| 89 |
## Hardware |
| 90 |
|
| 91 |
### The brightness keys do nothing |
| 92 |
|
| 93 |
This one is silent by design of the underlying pieces, so check both halves: |
| 94 |
|
| 95 |
groups | grep video # your account is in the video group |
| 96 |
ls -l /sys/class/backlight/*/brightness # group-writable, group video |
| 97 |
|
| 98 |
Brightness is a direct write to a sysfs file. It takes a udev rule to make that |
| 99 |
file writable and your account being in the `video` group for the rule to grant |
| 100 |
anything. Either one alone leaves the keys dead with no error anywhere: the |
| 101 |
process that would have complained has nowhere to log. |
| 102 |
|
| 103 |
Volume keys use a different path entirely (PipeWire, no privilege), which is |
| 104 |
why one can work while the other does not. |
| 105 |
|
| 106 |
### The volume overlay never appears |
| 107 |
|
| 108 |
`swayosd-server` is started by the sway config, not by a systemd unit. If it |
| 109 |
died, restart it: |
| 110 |
|
| 111 |
swayosd-server & |
| 112 |
|
| 113 |
### Print Screen seems to do nothing |
| 114 |
|
| 115 |
It probably worked. Check `~/Pictures/Screenshots`. Every capture also raises a |
| 116 |
notification, so if you get neither the file nor the notification, run the |
| 117 |
script by hand to see the error: |
| 118 |
|
| 119 |
alloy-shot output |
| 120 |
|
| 121 |
### A second monitor is not behaving |
| 122 |
|
| 123 |
This is the least tested part of the system. `swaymsg -t get_outputs` shows |
| 124 |
what sway sees, and `~/.config/sway/config.d/50-display.conf` is where to write |
| 125 |
`output` lines by hand. `alloy display` will not pick modes yet. |
| 126 |
|
| 127 |
--- |
| 128 |
|
| 129 |
## The console |
| 130 |
|
| 131 |
### alloy net cannot join a new network |
| 132 |
|
| 133 |
Connecting to a saved network needs no privilege; creating one does, and the |
| 134 |
console does not escalate yet. Use: |
| 135 |
|
| 136 |
run0 nmcli device wifi connect "Network Name" --ask |
| 137 |
|
| 138 |
### alloy pkg or alloy update says it cannot read the system |
| 139 |
|
| 140 |
Those two tabs front `rpm-ostree`, which exists only on an ostree system. If |
| 141 |
you are running the console somewhere else, that message is correct rather than |
| 142 |
broken. |
| 143 |
|
| 144 |
### The console will not start at all |
| 145 |
|
| 146 |
It needs a theme on its search path and fails loudly if there is none. Two |
| 147 |
verbs deliberately survive that, because they are on the login path: |
| 148 |
`alloy theme apply` and `alloy status --bar`. If the console is broken but the |
| 149 |
bar still draws, the theme directory is what to look at |
| 150 |
(`/usr/share/alloy/themes`). |
| 151 |
|
| 152 |
--- |
| 153 |
|
| 154 |
## Appearance |
| 155 |
|
| 156 |
### I switched to night and nothing changed |
| 157 |
|
| 158 |
The switch applies at the next login. Log out and back in, or run |
| 159 |
`alloy theme apply` and restart the affected programs. sway itself needs the |
| 160 |
re-login regardless, because it reads its config exactly once. |
| 161 |
|
| 162 |
### alloy theme apply says it skipped my files |
| 163 |
|
| 164 |
It found a config that matches neither the day nor the night render, decided it |
| 165 |
was yours, and left it alone. That is the intended behaviour. If you want the |
| 166 |
shipped render back: |
| 167 |
|
| 168 |
alloy theme apply --force |
| 169 |
|
| 170 |
which keeps your version as a `.alloy-bak` beside it. |
| 171 |
|
| 172 |
### Some characters render as boxes |
| 173 |
|
| 174 |
If they are emoji, that is expected: no emoji font is installed. Install any |
| 175 |
one and fontconfig picks it up with no further configuration. |
| 176 |
|
| 177 |
If they are not emoji, `fc-list :lang=<code>` will say whether anything in the |
| 178 |
image covers that script. |
| 179 |
|
| 180 |
### The browser is light when my desktop is dark |
| 181 |
|
| 182 |
System appearance reaches the browser through the desktop portal rather than |
| 183 |
through any file Alloy ships, and that path has not been verified on real |
| 184 |
hardware yet. If it does not work, set the theme in the browser's own settings; |
| 185 |
Alloy does not ship browser configuration to fix it with. |
| 186 |
|
| 187 |
--- |
| 188 |
|
| 189 |
## Network access |
| 190 |
|
| 191 |
### A service I started is unreachable from another machine |
| 192 |
|
| 193 |
Alloy refuses inbound connections it was not told to allow, so a server you |
| 194 |
started on a port is reachable from the machine itself and from nowhere else. |
| 195 |
See what is open: |
| 196 |
|
| 197 |
firewall-cmd --list-all |
| 198 |
|
| 199 |
Then allow yours, permanently: |
| 200 |
|
| 201 |
run0 firewall-cmd --permanent --add-port=8080/tcp |
| 202 |
run0 firewall-cmd --reload |
| 203 |
|
| 204 |
Two things that are not the firewall, checked before reaching for it. If the |
| 205 |
other machine is on your tailnet, the tailnet is exempt already, so a failure |
| 206 |
there is the service binding to localhost rather than to every interface. And a |
| 207 |
server profile does not answer mDNS, so `<name>.local` failing while the IP |
| 208 |
address works is chapter 12 rather than a firewall rule. |
| 209 |
|
| 210 |
### SSH says permission denied, and I know the password is right |
| 211 |
|
| 212 |
The password is not the problem. Alloy accepts a key and nothing else: |
| 213 |
`PasswordAuthentication` is off, so is the keyboard-interactive path behind it, |
| 214 |
and root is key-only. A correct password gets the same refusal a wrong one |
| 215 |
does. |
| 216 |
|
| 217 |
If you can reach a console on the machine, add your key there. Chapter 3, |
| 218 |
"Adding a key after the fact", is the four lines. |
| 219 |
|
| 220 |
If you cannot reach a console, this is the case with no remote fix. A headless |
| 221 |
machine installed with no key has no way in, and the way back is a screen, a |
| 222 |
keyboard, or a reinstall from a medium with a key baked into it. |
| 223 |
|
| 224 |
### I added a key and it still refuses |
| 225 |
|
| 226 |
Check the modes. sshd will not read an `authorized_keys` that anyone but its |
| 227 |
owner can write, and it declines without telling the client why: |
| 228 |
|
| 229 |
chmod 700 ~/.ssh |
| 230 |
chmod 600 ~/.ssh/authorized_keys |
| 231 |
|
| 232 |
Then check the file holds the **public** half, one key per line, starting with |
| 233 |
something like `ssh-ed25519`. The reason for a refusal is always in the |
| 234 |
server's journal even when the client is told nothing: |
| 235 |
|
| 236 |
journalctl -u sshd -n 50 |
| 237 |
|
| 238 |
## Disks and boot |
| 239 |
|
| 240 |
### The machine asks for a passphrase it never asked for before |
| 241 |
|
| 242 |
The TPM stopped answering: it was cleared, the board changed, or the disk moved |
| 243 |
to another machine. Type the passphrase you set at install. If you do not have |
| 244 |
it, this is what the eight-word recovery phrase from the installer is for. |
| 245 |
|
| 246 |
Without either one, the data is gone. That is what encryption means. |
| 247 |
|
| 248 |
### I have my password store but nothing will decrypt |
| 249 |
|
| 250 |
Two separate secrets get needed at the same moment, when a machine is lost, and |
| 251 |
they are easy to conflate. The eight-word recovery phrase above unlocks the |
| 252 |
**disk**. The age identity under `~/.config/gopass/age/` decrypts the **password |
| 253 |
store**. Neither one substitutes for the other, and neither is in the git |
| 254 |
repository holding the store, by design. |
| 255 |
|
| 256 |
So a clone of the store on a new machine is ciphertext until you put the |
| 257 |
identity back. If you still have the old disk, copy it off. If you kept a copy |
| 258 |
somewhere off the machine, restore that. If neither, the store is gone the same |
| 259 |
way an encrypted disk with no phrase is gone. |
| 260 |
|
| 261 |
Worth doing before you need it: keep the identity somewhere that is not the |
| 262 |
machine and not the store's remote. Chapter 10 covers where it lives and why it |
| 263 |
never travels with the ciphertext. |
| 264 |
|
| 265 |
### The new image is worse than the old one |
| 266 |
|
| 267 |
rpm-ostree rollback |
| 268 |
systemctl reboot |
| 269 |
|
| 270 |
Or pick the previous entry in the GRUB menu at boot. See chapter 8. |
| 271 |
|
| 272 |
### I layered a package and now the system will not boot |
| 273 |
|
| 274 |
Boot the previous deployment from the GRUB menu, then remove it: |
| 275 |
|
| 276 |
rpm-ostree uninstall <package> |
| 277 |
|
| 278 |
Layered packages are the most common way to break an otherwise reliable base, |
| 279 |
which is the argument in chapter 7 for putting things in boxes instead. |
| 280 |
|
| 281 |
### The machine came up with no console, and nothing will layer |
| 282 |
|
| 283 |
The symptom set is distinctive: no `alloy` and no terminal after the first boot |
| 284 |
following an install, `rpm-ostree status` failing with `Could not activate |
| 285 |
remote peer`, and `systemctl --failed` naming |
| 286 |
`alloy-layer-components.service`. Nothing in the journal explains any of it. |
| 287 |
|
| 288 |
Only machines installed from an older image have this. Their installer wrote |
| 289 |
six files in `/etc` without asking the policy what they should be labelled, one |
| 290 |
of them being the lock systemd needs to run a service under a dynamic user. `rpm-ostreed` is such a service, so the machine cannot layer a |
| 291 |
package at all, and the components that make it usable are layered at first |
| 292 |
boot. |
| 293 |
|
| 294 |
One command fixes it, from a virtual console, with no reinstall: |
| 295 |
|
| 296 |
sudo restorecon -R /etc |
| 297 |
systemctl reboot |
| 298 |
|
| 299 |
Confirm it afterwards, from a checkout of the repository: |
| 300 |
|
| 301 |
sudo build/check-installed.sh |
| 302 |
|
| 303 |
An install from a current image does not need any of this. The installer |
| 304 |
relabels the target itself now, and the check above passes on a fresh machine. |
| 305 |
|
| 306 |
--- |
| 307 |
|
| 308 |
## Diagnostics that are off by default |
| 309 |
|
| 310 |
Alloy's hardening posture turns three things off that a person debugging |
| 311 |
software expects to be on. Each has a way back; two of them are a command, and |
| 312 |
the third is a line in your image. |
| 313 |
|
| 314 |
### dmesg says operation not permitted |
| 315 |
|
| 316 |
The kernel ring buffer takes a capability to read |
| 317 |
(`kernel.dmesg_restrict=1`). Two ways to the same text: |
| 318 |
|
| 319 |
run0 dmesg |
| 320 |
journalctl -k |
| 321 |
|
| 322 |
`journalctl -k` is the better habit: it is the same messages, it needs no |
| 323 |
privilege, and it survives a reboot. |
| 324 |
|
| 325 |
### A program crashed and left no core file |
| 326 |
|
| 327 |
Alloy writes no cores. `kernel.core_pattern` is pointed at `/bin/false` and the |
| 328 |
hard limit is zero. A core is the process's memory written to disk, including |
| 329 |
whatever it had decrypted, which on this machine is keys and tokens. |
| 330 |
|
| 331 |
Getting one back takes root twice over, which is the intended friction: the |
| 332 |
pattern is a system setting, and the per-process hard limit is zero, so raising |
| 333 |
it needs `CAP_SYS_RESOURCE`. |
| 334 |
|
| 335 |
run0 sysctl -w kernel.core_pattern=core |
| 336 |
run0 bash -c 'ulimit -c unlimited; ./the-program-that-crashes' |
| 337 |
|
| 338 |
The core lands in the working directory, owned by root, and the program ran as |
| 339 |
root to produce it. Reboot when you are done, or set the pattern back by hand. |
| 340 |
`coredumpctl` is not the path here; `systemd-coredump` is not installed. |
| 341 |
|
| 342 |
### modprobe refuses a module that exists |
| 343 |
|
| 344 |
A short list of filesystem drivers, legacy network protocols and the FireWire |
| 345 |
stack cannot be loaded at all, rather than merely being blocked from |
| 346 |
autoloading. The list and the reason for each entry are in |
| 347 |
`/usr/lib/modprobe.d/50-alloy-blacklist.conf`. |
| 348 |
|
| 349 |
If you need one back, delete its line from that file in your image checkout and |
| 350 |
rebuild. Editing it on the running machine does not last: `/usr/lib` is replaced |
| 351 |
on every update. |
| 352 |
|
| 353 |
Attaching a debugger is unaffected. `gdb -p` and `perf record -p` against your |
| 354 |
own processes still work; Alloy deliberately did not take the setting that |
| 355 |
would have stopped them. |
| 356 |
|
| 357 |
--- |
| 358 |
|
| 359 |
## Reporting something |
| 360 |
|
| 361 |
Alloy is pre-v0 and this manual describes a moving system. If something here is |
| 362 |
wrong, the repository is at `makenot.work/git/max/alloy` and the design |
| 363 |
documents in `docs/` say what each piece is supposed to do, which is usually |
| 364 |
enough to tell a bug from a decision. |
| 365 |
|