Skip to main content

max / makenotwork

16.9 KB · 438 lines History Blame Raw
1 //! What a machine IS, declared in the topology rather than discovered.
2 //!
3 //! Bento and Sando know a host's name, ssh target, architecture and
4 //! capabilities. Neither knew what the box was, so everything downstream of
5 //! that got rediscovered by probing or by a build failing: an `ldd` guard on
6 //! the node before the symlink swap, a `glibc_check` in a recipe, a rebuilt
7 //! astra regressing silently because its toolchain lived in a `.bashrc` line.
8 //! Each of those asks "what is this machine" at runtime, on the far side of the
9 //! build.
10 //!
11 //! This is the declared half. A host or node states its base image and its libc
12 //! in the topology, and the declaration is checked against the machine before
13 //! anything is built or shipped.
14 //!
15 //! ## Declared, and verified against reality. The two are not in tension
16 //!
17 //! The rule the topology follows is DECLARED, NOT SNIFFED: the value the
18 //! pipelines reason about is the one written in config, because deriving it
19 //! from the host trades drift you can see for drift you assume away.
20 //!
21 //! Verifying that declaration against the machine is the opposite of deriving
22 //! it. Nothing here reads a host to decide what it is; it reads a host to decide
23 //! whether the config is still telling the truth. A declaration nobody checks
24 //! rots into a comment, and the failure it was written to prevent (a rebuilt box
25 //! quietly becoming something else) is exactly the one it would then miss.
26 //!
27 //! ## Silence is a skip here, not a refusal
28 //!
29 //! [`crate::base_image`] deliberately does NOT copy `Placement::check`'s
30 //! four-case rule, where one side stating and the other silent is a refusal.
31 //! That rule is right for a platform, where `linux/aarch64` and `linux/x86_64`
32 //! are a hard incompatibility and the pairing is a choice the system makes. It
33 //! is wrong here for two reasons:
34 //!
35 //! - A host that declares nothing is the fleet as it stands. macOS and Windows
36 //! build hosts have no `/etc/os-release`, and refusing them would take the
37 //! Apple and Windows pipelines down to buy nothing.
38 //! - Base-image equality is not the compatibility requirement. fw13 is
39 //! `pop/24.04` and production is `ubuntu/24.04`, two different bases that
40 //! share glibc 2.39, and binaries built on one run on the other today. A
41 //! check that demanded equal bases would refuse every MNW deploy while
42 //! describing a problem that does not exist.
43 //!
44 //! So the contract is narrow and true: a host that declares a base image must
45 //! match it. A host that declares nothing is not checked, and says so.
46 //!
47 //! ## What this does not do
48 //!
49 //! It does not compare a build host's libc against a deploy node's. That is the
50 //! real compatibility question and it is still answered at runtime, by Sando's
51 //! `ldd` guard before the symlink swap. Hoisting it to preflight is the natural
52 //! next step and needs the declarations this module adds, which is why it is
53 //! filed separately rather than smuggled in here.
54 //!
55 //! <!-- wiki: host-base-images -->
56
57 use std::fmt;
58 use std::str::FromStr;
59
60 use serde::{Deserialize, Serialize};
61
62 /// A machine's base image, as `id/version`: `alloy/0.1`, `ubuntu/24.04`,
63 /// `pop/24.04`.
64 ///
65 /// The two halves are `ID` and `VERSION_ID` from `/etc/os-release`, which every
66 /// mainstream Linux sets and which Alloy sets deliberately (`ID=alloy`,
67 /// `VERSION_ID` moving by hand on a release). Parsed rather than stringly so a
68 /// comparison is of two values and not of two spellings, on the same reasoning
69 /// as `Platform`.
70 ///
71 /// Deliberately NOT the whole of Alloy's identity. Alloy also stamps
72 /// `IMAGE_VERSION` (which build) and `ALLOY_BASE` (which Fedora), on three
73 /// separate clocks. `IMAGE_VERSION` is too fine to compare: it moves every
74 /// build, so requiring it to match would fail a deploy because the image was
75 /// rebuilt, which is not a fact about compatibility. `ID` and `VERSION_ID` are
76 /// the pair that decides what a binary can link against.
77 #[derive(Debug, Clone, PartialEq, Eq, Hash, Serialize, Deserialize)]
78 #[serde(try_from = "String", into = "String")]
79 pub struct BaseImage {
80 id: String,
81 version: String,
82 }
83
84 #[derive(Debug, thiserror::Error, PartialEq, Eq)]
85 pub enum BaseImageParseError {
86 #[error("base image `{0}` is not `id/version` (e.g. `ubuntu/24.04`, `alloy/0.1`)")]
87 BadShape(String),
88 }
89
90 impl BaseImage {
91 pub fn parse(s: &str) -> Result<Self, BaseImageParseError> {
92 let bad = || BaseImageParseError::BadShape(s.to_owned());
93 let (id, version) = s.split_once('/').ok_or_else(bad)?;
94 // Same character class `Platform` accepts. `VERSION_ID` is routinely
95 // dotted (`24.04`), and Alloy's is `0.1`.
96 let part_ok = |p: &str| {
97 !p.is_empty()
98 && p.bytes()
99 .all(|b| b.is_ascii_alphanumeric() || b == b'_' || b == b'-' || b == b'.')
100 };
101 if !part_ok(id) || !part_ok(version) {
102 return Err(bad());
103 }
104 Ok(Self {
105 id: id.to_ascii_lowercase(),
106 version: version.to_ascii_lowercase(),
107 })
108 }
109
110 pub fn id(&self) -> &str {
111 &self.id
112 }
113
114 pub fn version(&self) -> &str {
115 &self.version
116 }
117 }
118
119 impl fmt::Display for BaseImage {
120 fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
121 write!(f, "{}/{}", self.id, self.version)
122 }
123 }
124
125 impl FromStr for BaseImage {
126 type Err = BaseImageParseError;
127 fn from_str(s: &str) -> Result<Self, Self::Err> {
128 Self::parse(s)
129 }
130 }
131
132 impl TryFrom<String> for BaseImage {
133 type Error = BaseImageParseError;
134 fn try_from(s: String) -> Result<Self, Self::Error> {
135 Self::parse(&s)
136 }
137 }
138
139 impl From<BaseImage> for String {
140 fn from(b: BaseImage) -> Self {
141 b.to_string()
142 }
143 }
144
145 /// The shell one-liner a host runs to report what it is.
146 ///
147 /// Sources `/etc/os-release` rather than parsing it here, because the file is
148 /// defined as shell-sourceable and quoting varies (`VERSION_ID="24.04"` on
149 /// Ubuntu, unquoted elsewhere). Sourcing hands the quoting to the shell that
150 /// owns the format.
151 ///
152 /// Failure is reported in the output rather than in the exit status: every
153 /// branch prints its key with an empty value and the command exits 0, so a
154 /// missing file or a missing `ldd` comes back as an unreadable identity that
155 /// the caller can describe, not as an opaque non-zero from a step that also
156 /// runs other things.
157 pub fn probe_cmd() -> String {
158 // `ldd --version` writes the version line to stdout on glibc and is absent
159 // entirely on musl images; `2>/dev/null` plus the empty default covers both.
160 "if [ -r /etc/os-release ]; then . /etc/os-release; fi; \
161 printf 'id=%s\\n' \"${ID:-}\"; \
162 printf 'version_id=%s\\n' \"${VERSION_ID:-}\"; \
163 printf 'libc=%s\\n' \"$(ldd --version 2>/dev/null | head -1 | awk '{print $NF}')\""
164 .to_string()
165 }
166
167 /// What a host said about itself in answer to [`probe_cmd`].
168 ///
169 /// Every field is optional because every field can legitimately be absent: a
170 /// machine with no `/etc/os-release`, or one with no `ldd`.
171 #[derive(Debug, Clone, Default, PartialEq, Eq)]
172 pub struct ReportedIdentity {
173 pub base: Option<BaseImage>,
174 /// The glibc version as `ldd --version` states it, e.g. `2.39`. The trailing
175 /// field of that line, which is the upstream version on both spellings seen
176 /// in the fleet (`ldd (Ubuntu GLIBC 2.39-0ubuntu8.7) 2.39` and
177 /// `ldd (GNU libc) 2.42`).
178 pub libc: Option<String>,
179 }
180
181 /// Read [`probe_cmd`]'s output.
182 ///
183 /// Tolerant by construction: unknown keys are ignored and a malformed base
184 /// image reads as absent rather than as an error, because this parses a remote
185 /// machine's answer and the useful failure is "that host could not tell me what
186 /// it is", raised by [`check`] against what was declared.
187 pub fn parse_probe(stdout: &str) -> ReportedIdentity {
188 let mut id = String::new();
189 let mut version = String::new();
190 let mut libc = None;
191 for line in stdout.lines() {
192 let Some((k, v)) = line.split_once('=') else {
193 continue;
194 };
195 let v = v.trim();
196 match k.trim() {
197 "id" => id = v.to_string(),
198 "version_id" => version = v.to_string(),
199 "libc" if !v.is_empty() => libc = Some(v.to_string()),
200 _ => {}
201 }
202 }
203 let base = if id.is_empty() || version.is_empty() {
204 None
205 } else {
206 BaseImage::parse(&format!("{id}/{version}")).ok()
207 };
208 ReportedIdentity { base, libc }
209 }
210
211 /// Why a host's declaration and the host itself disagree.
212 #[derive(Debug, Clone, PartialEq, Eq, thiserror::Error)]
213 pub enum IdentityDrift {
214 #[error(
215 "`{host}` is declared as {declared} and reports {reported}. \
216 Either the machine was rebuilt or the topology is stale; fix whichever \
217 is wrong before building here"
218 )]
219 BaseMismatch {
220 host: String,
221 declared: BaseImage,
222 reported: BaseImage,
223 },
224 #[error(
225 "`{host}` is declared as {declared} but could not say what it is \
226 (no readable /etc/os-release). A declaration that cannot be checked is \
227 the drift this exists to catch; remove the declaration or fix the host"
228 )]
229 BaseUnreadable { host: String, declared: BaseImage },
230 #[error(
231 "`{host}` is declared to have glibc {declared} and reports {reported}. \
232 The libc floor a binary built here carries would be wrong"
233 )]
234 LibcMismatch {
235 host: String,
236 declared: String,
237 reported: String,
238 },
239 #[error(
240 "`{host}` is declared to have glibc {declared} and reports none \
241 (no ldd). Remove the declaration or fix the host"
242 )]
243 LibcUnreadable { host: String, declared: String },
244 }
245
246 /// Compare what a host was declared to be against what it says it is.
247 ///
248 /// `Ok(None)` means nothing was declared and nothing was checked; the caller
249 /// reports that rather than passing silently, so an unchecked host is visible
250 /// in a build log instead of looking like a checked one.
251 ///
252 /// `libc` is checked only when declared, independently of the base image. The
253 /// two are separate facts: a base image can be pinned while the point release
254 /// under it moves, and it is the libc number that decides whether a binary
255 /// loads.
256 pub fn check(
257 host: &str,
258 declared: Option<&BaseImage>,
259 declared_libc: Option<&str>,
260 reported: &ReportedIdentity,
261 ) -> Result<Option<String>, IdentityDrift> {
262 if let Some(declared) = declared {
263 match &reported.base {
264 None => {
265 return Err(IdentityDrift::BaseUnreadable {
266 host: host.to_string(),
267 declared: declared.clone(),
268 });
269 }
270 Some(reported) if reported != declared => {
271 return Err(IdentityDrift::BaseMismatch {
272 host: host.to_string(),
273 declared: declared.clone(),
274 reported: reported.clone(),
275 });
276 }
277 Some(_) => {}
278 }
279 }
280
281 if let Some(want) = declared_libc {
282 match reported.libc.as_deref() {
283 None => {
284 return Err(IdentityDrift::LibcUnreadable {
285 host: host.to_string(),
286 declared: want.to_string(),
287 });
288 }
289 Some(got) if got != want => {
290 return Err(IdentityDrift::LibcMismatch {
291 host: host.to_string(),
292 declared: want.to_string(),
293 reported: got.to_string(),
294 });
295 }
296 Some(_) => {}
297 }
298 }
299
300 Ok(match (declared, declared_libc) {
301 (None, None) => None,
302 (Some(b), None) => Some(format!("{host} is {b}")),
303 (Some(b), Some(l)) => Some(format!("{host} is {b}, glibc {l}")),
304 (None, Some(l)) => Some(format!("{host} has glibc {l}")),
305 })
306 }
307
308 #[cfg(test)]
309 mod tests {
310 use super::*;
311
312 fn img(s: &str) -> BaseImage {
313 BaseImage::parse(s).expect("test base image must parse")
314 }
315
316 #[test]
317 fn a_base_image_is_a_shape_not_a_spelling() {
318 assert_eq!(img("Ubuntu/24.04").to_string(), "ubuntu/24.04");
319 assert_eq!(img("alloy/0.1").id(), "alloy");
320 assert_eq!(img("alloy/0.1").version(), "0.1");
321 for bad in ["ubuntu", "ubuntu/", "/24.04", "ubuntu/24 04", "a/b/c"] {
322 assert!(
323 BaseImage::parse(bad).is_err(),
324 "`{bad}` must not parse as a base image"
325 );
326 }
327 }
328
329 /// The two `ldd --version` spellings actually present in the fleet. Ubuntu
330 /// and Pop!_OS put the distro and the package release in the parenthesis;
331 /// Fedora (so Alloy) does not. The trailing field is the upstream version
332 /// on both, which is why it is the field taken.
333 #[test]
334 fn libc_is_read_from_both_ldd_spellings() {
335 let ubuntu = parse_probe("id=ubuntu\nversion_id=24.04\nlibc=2.39\n");
336 assert_eq!(ubuntu.libc.as_deref(), Some("2.39"));
337 assert_eq!(ubuntu.base, Some(img("ubuntu/24.04")));
338
339 let alloy = parse_probe("id=alloy\nversion_id=0.1\nlibc=2.42\n");
340 assert_eq!(alloy.libc.as_deref(), Some("2.42"));
341 assert_eq!(alloy.base, Some(img("alloy/0.1")));
342 }
343
344 #[test]
345 fn an_unreadable_host_reports_nothing_rather_than_a_half_answer() {
346 let none = parse_probe("id=\nversion_id=\nlibc=\n");
347 assert_eq!(none, ReportedIdentity::default());
348 // A half-answer is not a base image: `ID` with no `VERSION_ID` cannot be
349 // compared against a declaration that carries both.
350 let half = parse_probe("id=ubuntu\nversion_id=\nlibc=2.39\n");
351 assert_eq!(half.base, None);
352 assert_eq!(half.libc.as_deref(), Some("2.39"));
353 }
354
355 #[test]
356 fn a_host_that_declares_nothing_is_not_checked_and_says_so() {
357 let reported = parse_probe("id=ubuntu\nversion_id=24.04\nlibc=2.39\n");
358 assert_eq!(check("windows-x86", None, None, &reported), Ok(None));
359 }
360
361 #[test]
362 fn a_declared_host_must_match() {
363 let reported = parse_probe("id=ubuntu\nversion_id=26.04\nlibc=2.43\n");
364 assert_eq!(
365 check("testnot", Some(&img("ubuntu/24.04")), None, &reported),
366 Err(IdentityDrift::BaseMismatch {
367 host: "testnot".into(),
368 declared: img("ubuntu/24.04"),
369 reported: img("ubuntu/26.04"),
370 })
371 );
372 assert!(
373 check("testnot", Some(&img("ubuntu/26.04")), None, &reported)
374 .expect("a matching declaration must pass")
375 .is_some(),
376 "a checked host must report what it was checked against"
377 );
378 }
379
380 /// The case this whole module exists for: a box rebuilt into something else
381 /// while the topology still describes the old one. Before the declaration
382 /// this surfaced as a build failing with a resolver error naming the wrong
383 /// cause.
384 #[test]
385 fn a_rebuilt_host_is_caught_rather_than_discovered_by_a_build_failing() {
386 let rebuilt = parse_probe("id=fedora\nversion_id=43\nlibc=2.42\n");
387 let err = check("astra", Some(&img("pop/24.04")), None, &rebuilt)
388 .expect_err("a rebuilt host must not pass its old declaration");
389 assert!(
390 err.to_string().contains("pop/24.04") && err.to_string().contains("fedora/43"),
391 "the refusal must name both what was declared and what is there: {err}"
392 );
393 }
394
395 /// A declaration that cannot be verified is refused rather than waved
396 /// through. Waving it through is how a declaration decays into a comment.
397 #[test]
398 fn a_declaration_that_cannot_be_checked_is_a_refusal() {
399 let silent = ReportedIdentity::default();
400 assert!(matches!(
401 check("prod", Some(&img("ubuntu/24.04")), None, &silent),
402 Err(IdentityDrift::BaseUnreadable { .. })
403 ));
404 assert!(matches!(
405 check("prod", None, Some("2.39"), &silent),
406 Err(IdentityDrift::LibcUnreadable { .. })
407 ));
408 }
409
410 /// libc is checked independently of the base image, because a point release
411 /// can move under a pinned base and it is the libc number that decides
412 /// whether a binary loads.
413 #[test]
414 fn libc_drifts_independently_of_the_base_image() {
415 let drifted = parse_probe("id=ubuntu\nversion_id=24.04\nlibc=2.41\n");
416 assert_eq!(
417 check("prod", Some(&img("ubuntu/24.04")), Some("2.39"), &drifted),
418 Err(IdentityDrift::LibcMismatch {
419 host: "prod".into(),
420 declared: "2.39".into(),
421 reported: "2.41".into(),
422 })
423 );
424 }
425
426 /// The probe must survive a host with no `/etc/os-release` without failing
427 /// the step it rides in, so the shape of the command matters: it prints all
428 /// three keys unconditionally.
429 #[test]
430 fn the_probe_prints_every_key_and_sources_rather_than_parses() {
431 let cmd = probe_cmd();
432 assert!(cmd.contains(". /etc/os-release"), "{cmd}");
433 for key in ["id=%s", "version_id=%s", "libc=%s"] {
434 assert!(cmd.contains(key), "probe must print {key}: {cmd}");
435 }
436 }
437 }
438