Skip to main content

max / makenotwork

sando: compile the frontends in the code_smoke gate Both MNW build scripts report a tsc error as a cargo::warning and let the Rust build succeed against whatever static/dist/ already holds, so a type error in a chat widget cannot stop the forum from compiling. Nothing downstream noticed either, and a deploy would rsync the previous build's bundle. The code_smoke gate now runs npm run build for each configured frontend_build before it touches a database, where that failure is fatal. A project absent from the worktree is skipped so older shas still rebuild; a missing npm is not, because a host that cannot produce the bundle cannot produce the release.
Co-Authored-By
Claude Opus 5 (1M context) <noreply@anthropic.com>
Author: Max Johnson <me@maxj.phd> · 2026-07-26 14:59 UTC
Signed with PGP, not checked
Commit: 60f3a6cde0113ee50de11db7a18222de8fc67b12
Parent: 860f43e
8 files changed, +355 insertions, -1 deletion
M sando/README.md +29 -1
@@ -105,7 +105,7 @@
105 105
106 106 | Kind | When | What it proves |
107 107 |------|------|----------------|
108 - | `code_smoke` | build | Boots the fresh binary on a throwaway DB it migrates from scratch and seeds, then probes `/health`. Runs first: green here isolates a later red as an environment problem, not a code one. |
108 + | `code_smoke` | build | Compiles every `[[frontend_build]]` (see below), then boots the fresh binary on a throwaway DB it migrates from scratch and seeds, then probes `/health`. Runs first: green here isolates a later red as an environment problem, not a code one. |
109 109 | `fmt` | build | `cargo fmt --check` over every `[[test_target]]`. No compilation, so it fails fast. |
110 110 | `cargo_test` | build | Every configured `[[test_target]]` crate's suite, in order (see below). |
111 111 | `hardening_test` | build | What `cargo_test` structurally cannot reach (see below). |
@@ -163,6 +163,34 @@
163 163 tip. If *no* target exists, the gate fails rather than reporting a pass over
164 164 zero suites.
165 165
166 + ### What `code_smoke` builds first
167 +
168 + Before it creates a database or boots anything, `code_smoke` compiles every
169 + configured frontend:
170 +
171 + ```toml
172 + [[frontend_build]]
173 + dir = "server/frontend"
174 +
175 + [[frontend_build]]
176 + dir = "multithreaded/frontend" # script = "build" by default
177 + ```
178 +
179 + These are npm projects whose compiled output the binary serves but whose failure
180 + `cargo build` will not report. Both MNW crates compile TypeScript from a build
181 + script that downgrades a `tsc` error to a `cargo::warning` and lets the Rust
182 + build succeed against whatever `static/dist/` already holds, on purpose, so a
183 + type error in a chat widget cannot stop the forum from compiling. The cost is
184 + that nothing downstream noticed either, and the deploy rsynced the previous
185 + build's bundle. This is the one place that failure is fatal.
186 +
187 + Semantics match `cargo_test`: `npm ci` first if `node_modules` is absent (usually
188 + it is not, because the build script that produced the artifact already installed
189 + it), stop at the first red project with the directory named, one deadline across
190 + the gate, and a project absent from the worktree is skipped with a log line so
191 + older shas still rebuild. Omitting the key entirely gates nothing, which is the
192 + right default for a project with no frontend.
193 +
166 194 ### The lint and supply-chain gates
167 195
168 196 `clippy` and `fmt` run over the same `[[test_target]]` list as `cargo_test`,
M sando/daemon/sando-daemon.toml +10
@@ -25,6 +25,16 @@
25 25 # manual SQL is needed on a fresh box. Must match the prod DB owner.
26 26 scratch_owner_role = "makenotwork"
27 27
28 + # TypeScript frontends the code_smoke gate compiles. Both build scripts downgrade
29 + # a tsc error to a cargo::warning so the Rust build still succeeds against a
30 + # stale static/dist/; this is where that failure is fatal instead. See the
31 + # frontend_build docs in daemon/src/config.rs.
32 + [[frontend_build]]
33 + dir = "server/frontend"
34 +
35 + [[frontend_build]]
36 + dir = "multithreaded/frontend"
37 +
28 38 # Companion crates built from the same worktree/sha as the server and staged
29 39 # into the release bundle, so a contract-coupled service can't drift out of
30 40 # lockstep. mnw-cli is the public git-SSH server that proxies to /api/internal/*
M sando/deploy/sando-daemon.toml.example +18
@@ -33,10 +33,28 @@
33 33 # sando user must be able to write it; cargo creates it if absent.
34 34 cargo_target_dir = "/srv/sando/cargo-target"
35 35
36 + # TypeScript frontends the code_smoke gate compiles (npm run build), before it
37 + # creates a database or boots anything. Both crates compile these from a build
38 + # script that reports a tsc error as a cargo::warning and lets the Rust build
39 + # succeed against whatever static/dist/ already holds — deliberately, so a type
40 + # error in a chat widget cannot stop the forum from compiling. The consequence is
41 + # that nothing downstream noticed either: the deploy rsynced the previous build's
42 + # bundle. This is the one place that failure is fatal.
43 + [[frontend_build]]
44 + dir = "server/frontend"
45 +
46 + [[frontend_build]]
47 + dir = "multithreaded/frontend"
48 +
36 49 # Non-binary content shipped as part of each release. Multiple entries can
37 50 # target the same `dst` (additive merge — used to build `docs/` from three
38 51 # worktree sources). Sources are relative to the worktree root; dsts are
39 52 # relative to <release_root>/releases/<v>/.
53 + #
54 + # error-pages is now only load-bearing for 502.html: the 404 and 500 pages are
55 + # embedded in the server binary and served at /__errors/, which Caddy proxies to
56 + # (server/deploy/Caddyfile). 502 is the app-is-down page, so it cannot come from
57 + # the app, and Caddy keeps reading it off disk.
40 58 [[release_contents]]
41 59 src = "server/deploy/error-pages"
42 60 dst = "error-pages"
M sando/daemon/src/build.rs +2
@@ -695,6 +695,7 @@
695 695 all_features: false,
696 696 scratch_db: true,
697 697 }],
698 + frontend_builds: vec![],
698 699 };
699 700
700 701 let topo = Topology {
@@ -773,6 +774,7 @@
773 774 gate_timeout_secs: 2400,
774 775 companions: Vec::new(),
775 776 test_targets: vec![],
777 + frontend_builds: vec![],
776 778 }
777 779 }
778 780
M sando/daemon/src/config.rs +72
@@ -101,6 +101,35 @@
101 101 /// that configures nothing keeps today's behavior.
102 102 #[serde(default = "default_test_targets", rename = "test_target")]
103 103 pub test_targets: Vec<TestTarget>,
104 + /// Frontend builds the `code_smoke` gate runs, in order, before it touches a
105 + /// database. Each is an npm project whose compiled output is served by the
106 + /// binary but is not produced by `cargo build` in any way cargo can fail on:
107 + /// both MNW frontends compile from a build script that reports a `tsc` error
108 + /// as a `cargo::warning` and lets the Rust build succeed against whatever
109 + /// `static/dist/` already holds, deliberately, so a type error in a chat
110 + /// widget cannot stop the forum from compiling. The cost of that choice is
111 + /// that nothing downstream noticed either, and the deploy shipped a stale
112 + /// bundle. This is where the same failure is fatal. Default empty, so a
113 + /// project that configures nothing keeps today's behavior.
114 + #[serde(default, rename = "frontend_build")]
115 + pub frontend_builds: Vec<FrontendBuild>,
116 + }
117 +
118 + /// One npm project the `code_smoke` gate compiles.
119 + #[derive(Debug, Clone, Deserialize)]
120 + pub struct FrontendBuild {
121 + /// Directory under the worktree holding `package.json`
122 + /// (e.g. `server/frontend`).
123 + pub dir: PathBuf,
124 + /// npm script to run. Defaults to `build`, which is what emits the bundle
125 + /// the release actually serves; `typecheck` would prove less (it never
126 + /// writes `static/dist/`, so it cannot catch an emit failure).
127 + #[serde(default = "default_frontend_script")]
128 + pub script: String,
129 + }
130 +
131 + fn default_frontend_script() -> String {
132 + "build".into()
104 133 }
105 134
106 135 /// One crate's test suite, as run by the `cargo_test` gate.
@@ -293,6 +322,13 @@
293 322 t.dir.display(),
294 323 );
295 324 }
325 + for f in &self.frontend_builds {
326 + anyhow::ensure!(
327 + !f.script.is_empty(),
328 + "frontend_build {} has an empty script; omit the key for the default `build`",
329 + f.dir.display(),
330 + );
331 + }
296 332 if let Some(url) = self.scratch_db_url.as_deref() {
297 333 assert_scratch_db_loopback(url)
298 334 .context("scratch_db_url must address a loopback (on-box) database")?;
@@ -320,6 +356,7 @@
320 356 gate_timeout_secs: default_gate_timeout_secs(),
321 357 companions: Vec::new(),
322 358 test_targets: default_test_targets(),
359 + frontend_builds: Vec::new(),
323 360 }
324 361 }
325 362 }
@@ -420,6 +457,41 @@
420 457 .any(|t| t.dir == std::path::Path::new("mnw-cli")),
421 458 "the companion that installs onto prod-1 must be gated",
422 459 );
460 + // Both crates serve JS their build scripts compile best-effort, so an
461 + // unlisted frontend is an ungated bundle.
462 + for dir in ["server/frontend", "multithreaded/frontend"] {
463 + assert!(
464 + cfg.frontend_builds
465 + .iter()
466 + .any(|f| f.dir == std::path::Path::new(dir)),
467 + "{dir} must be gated: its build script swallows a tsc error",
468 + );
469 + }
470 + }
471 +
472 + #[test]
473 + fn frontend_builds_default_to_empty_and_to_the_build_script() {
474 + let cfg: Config = toml::from_str(MINIMAL).unwrap();
475 + assert!(
476 + cfg.frontend_builds.is_empty(),
477 + "a project with no frontend must configure nothing"
478 + );
479 +
480 + let raw = format!("{MINIMAL}\n[[frontend_build]]\ndir = \"server/frontend\"\n");
481 + let cfg: Config = toml::from_str(&raw).unwrap();
482 + cfg.validate().unwrap();
483 + assert_eq!(cfg.frontend_builds[0].script, "build");
484 + }
485 +
486 + #[test]
487 + fn validate_rejects_an_empty_frontend_script() {
488 + let raw = format!("{MINIMAL}\n[[frontend_build]]\ndir = \"x\"\nscript = \"\"\n");
489 + let err = toml::from_str::<Config>(&raw)
490 + .unwrap()
491 + .validate()
492 + .unwrap_err()
493 + .to_string();
494 + assert!(err.contains("empty script"), "got: {err}");
423 495 }
424 496
425 497 #[test]
M sando/daemon/src/gates.rs +211
@@ -1267,6 +1267,15 @@
1267 1267 .await);
1268 1268 };
1269 1269
1270 + // Frontend builds, before anything else: they need no DB and no staged
1271 + // binary, and a `tsc` error is the one failure the Rust build deliberately
1272 + // swallows (both MNW build scripts emit `cargo::warning` and succeed against
1273 + // a stale `static/dist/`). Failing here is what stops the deploy rsyncing
1274 + // the previous build's bundle.
1275 + if let Some(outcome) = code_smoke_frontends(ctx, &mut log_buf).await {
1276 + return Ok(finish(outcome.with_log_ref(log_ref), log_buf).await);
1277 + }
1278 +
1270 1279 // Docs integrity, first and cheapest: run the staged binary's DB-free
1271 1280 // `MNW_CHECK_DOCS` mode before creating the throwaway DB. A broken internal
1272 1281 // docs link (a `[..](x.md)` resolving to a slug no page serves) fails here
@@ -1307,6 +1316,111 @@
1307 1316 Ok(finish(outcome.with_log_ref(log_ref), log_buf).await)
1308 1317 }
1309 1318
1319 + /// Compile every configured `frontend_build` in the worktree.
1320 + ///
1321 + /// Returns `Some(failed)` on the first project that does not build; `None` when
1322 + /// all of them do (or none are configured). Output is appended to `log_buf`
1323 + /// either way.
1324 + ///
1325 + /// `npm ci` runs only when `node_modules` is absent. Usually it is not: the app
1326 + /// build script installed it during the `cargo build` that produced the artifact
1327 + /// this gate is about to smoke, so the common path here is just `npm run build`
1328 + /// against a warm install — seconds. The install branch covers the gate running
1329 + /// against a worktree whose build script was skipped or failed at `npm ci`, and
1330 + /// it is as fatal as a compile failure, because the alternative is compiling
1331 + /// against whatever some earlier sha installed.
1332 + ///
1333 + /// Unlike the app build scripts, nothing here is best-effort. That asymmetry is
1334 + /// the point of the gate.
1335 + async fn code_smoke_frontends(ctx: &GateCtx, log_buf: &mut Vec<u8>) -> Option<GateOutcome> {
1336 + for fe in &ctx.cfg.frontend_builds {
1337 + let dir = ctx.worktree.join(&fe.dir);
1338 + let label = fe.dir.display().to_string();
1339 + log_buf.extend_from_slice(format!("---- frontend build ({label}) ----\n").as_bytes());
1340 +
1341 + if !dir.is_dir() {
1342 + // An older sha predating the frontend, mid-bisect. Skipping keeps
1343 + // sando able to rebuild history; the log says so out loud.
1344 + log_buf.extend_from_slice(
1345 + format!("{label} absent from this worktree; skipping\n").as_bytes(),
1346 + );
1347 + continue;
1348 + }
1349 +
1350 + if !dir.join("node_modules").is_dir()
1351 + && let Some(outcome) = run_npm(&dir, &label, &["ci"], "npm ci", ctx, log_buf).await
1352 + {
1353 + return Some(outcome);
1354 + }
1355 +
1356 + if let Some(outcome) = run_npm(
1357 + &dir,
1358 + &label,
1359 + &["run", &fe.script],
1360 + &format!("npm run {}", fe.script),
1361 + ctx,
1362 + log_buf,
1363 + )
1364 + .await
1365 + {
1366 + return Some(outcome);
1367 + }
1368 + }
1369 + None
1370 + }
1371 +
1372 + /// One `npm` invocation for [`code_smoke_frontends`], bounded by the gate
1373 + /// timeout so a wedged install cannot hold the whole pipeline (the enclosing
1374 + /// `code_smoke` ceiling would catch it eventually, but this attributes the
1375 + /// failure to the project that hung).
1376 + async fn run_npm(
1377 + dir: &std::path::Path,
1378 + label: &str,
1379 + args: &[&str],
1380 + what: &str,
1381 + ctx: &GateCtx,
1382 + log_buf: &mut Vec<u8>,
1383 + ) -> Option<GateOutcome> {
1384 + log_buf.extend_from_slice(format!("$ {what}\n").as_bytes());
1385 + let mut cmd = tokio::process::Command::new("npm");
1386 + cmd.args(args)
1387 + .current_dir(dir)
1388 + .stdout(std::process::Stdio::piped())
1389 + .stderr(std::process::Stdio::piped())
1390 + .kill_on_drop(true);
1391 + let ceiling = std::time::Duration::from_secs(ctx.cfg.gate_timeout_secs);
1392 + let out = match tokio::time::timeout(ceiling, cmd.output()).await {
1393 + Ok(Ok(out)) => out,
1394 + Ok(Err(e)) => {
1395 + // A missing `npm` lands here. Fatal, not skipped: a build host
1396 + // without Node cannot produce the bundle the release serves, and
1397 + // silently passing is how the stale bundle shipped in the first place.
1398 + log_buf.extend_from_slice(format!("{what} could not be spawned: {e}\n").as_bytes());
1399 + return Some(GateOutcome::failed(GateFailure::SpawnFailed {
1400 + message: format!("{what} in {label}: {e}"),
1401 + }));
1402 + }
1403 + Err(_elapsed) => {
1404 + log_buf.extend_from_slice(
1405 + format!("{what} timed out after {}s\n", ctx.cfg.gate_timeout_secs).as_bytes(),
1406 + );
1407 + return Some(GateOutcome::failed(GateFailure::CodeSmokeFrontend {
1408 + dir: label.to_string(),
1409 + exit_code: None,
1410 + }));
1411 + }
1412 + };
1413 + log_buf.extend_from_slice(&out.stdout);
1414 + log_buf.extend_from_slice(&out.stderr);
1415 + if out.status.success() {
1416 + return None;
1417 + }
1418 + Some(GateOutcome::failed(GateFailure::CodeSmokeFrontend {
1419 + dir: label.to_string(),
1420 + exit_code: out.status.code(),
1421 + }))
1422 + }
1423 +
1310 1424 /// Run the staged binary's DB-free docs integrity check (`MNW_CHECK_DOCS=1`).
1311 1425 ///
1312 1426 /// Returns `Some(failed)` if the check reports broken links, cannot be spawned,
@@ -2115,6 +2229,103 @@
2115 2229 assert!(matches!(f, GateFailure::SpawnFailed { .. }));
2116 2230 }
2117 2231
2232 + /// A `GateCtx` over `worktree` with the given frontend projects configured.
2233 + /// No DB, no artifact — `code_smoke_frontends` touches neither.
2234 + async fn frontend_ctx(worktree: &std::path::Path, dirs: &[&str]) -> GateCtx {
2235 + let mut cfg = crate::config::Config::for_tests();
2236 + cfg.frontend_builds = dirs
2237 + .iter()
2238 + .map(|d| crate::config::FrontendBuild {
2239 + dir: PathBuf::from(d),
2240 + script: "build".into(),
2241 + })
2242 + .collect();
2243 + cfg.logs_root = worktree.join("logs");
2244 + GateCtx {
2245 + pool: SqlitePoolOptions::new()
2246 + .max_connections(1)
2247 + .connect("sqlite::memory:")
2248 + .await
2249 + .unwrap(),
2250 + cfg: std::sync::Arc::new(cfg),
2251 + tier: TierId::new("host"),
2252 + version: "0.1.0".parse().unwrap(),
2253 + worktree: worktree.to_path_buf(),
2254 + events: events::channel(),
2255 + nodes: Vec::new(),
2256 + build_id: None,
2257 + }
2258 + }
2259 +
2260 + /// Write a minimal npm project at `worktree/<dir>` whose `build` script
2261 + /// exits with `exit_code`. Pre-creates `node_modules` so the gate skips
2262 + /// `npm ci` — these tests are about the build step, not the network.
2263 + fn fake_npm_project(worktree: &std::path::Path, dir: &str, exit_code: u8) {
2264 + let root = worktree.join(dir);
2265 + std::fs::create_dir_all(root.join("node_modules")).unwrap();
2266 + std::fs::write(
2267 + root.join("package.json"),
2268 + format!(
2269 + r#"{{"name":"fake","version":"0.0.0","private":true,
2270 + "scripts":{{"build":"exit {exit_code}"}}}}"#
2271 + ),
2272 + )
2273 + .unwrap();
2274 + }
2275 +
2276 + #[tokio::test]
2277 + async fn frontend_gate_fails_on_a_build_error_and_names_the_project() {
2278 + // The whole point of the gate: the app build scripts downgrade this to a
2279 + // cargo::warning, so if it passes here nothing stops a stale bundle.
2280 + let tmp = tempfile::tempdir().unwrap();
2281 + fake_npm_project(tmp.path(), "server/frontend", 0);
2282 + fake_npm_project(tmp.path(), "multithreaded/frontend", 2);
2283 + let ctx = frontend_ctx(tmp.path(), &["server/frontend", "multithreaded/frontend"]).await;
2284 +
2285 + let mut log = Vec::new();
2286 + let outcome = code_smoke_frontends(&ctx, &mut log)
2287 + .await
2288 + .expect("a failing tsc must fail the gate");
2289 + let crate::outcome::GateStatus::Failed { failure } = &outcome.status else {
2290 + panic!("expected a failure, got {:?}", outcome.status)
2291 + };
2292 + assert!(
2293 + matches!(
2294 + failure,
2295 + GateFailure::CodeSmokeFrontend { dir, exit_code: Some(2) }
2296 + if dir == "multithreaded/frontend"
2297 + ),
2298 + "got: {failure:?}"
2299 + );
2300 + // The passing project ran first; its output belongs in the log too.
2301 + let log = String::from_utf8_lossy(&log);
2302 + assert!(log.contains("server/frontend"), "log: {log}");
2303 + }
2304 +
2305 + #[tokio::test]
2306 + async fn frontend_gate_passes_when_every_project_builds() {
2307 + let tmp = tempfile::tempdir().unwrap();
2308 + fake_npm_project(tmp.path(), "server/frontend", 0);
2309 + let ctx = frontend_ctx(tmp.path(), &["server/frontend"]).await;
2310 + assert!(
2311 + code_smoke_frontends(&ctx, &mut Vec::new()).await.is_none(),
2312 + "a clean build must not fail the gate"
2313 + );
2314 + }
2315 +
2316 + #[tokio::test]
2317 + async fn frontend_gate_skips_a_project_absent_from_the_worktree() {
2318 + // Rebuilding an older sha that predates the frontend must stay possible.
2319 + let tmp = tempfile::tempdir().unwrap();
2320 + let ctx = frontend_ctx(tmp.path(), &["multithreaded/frontend"]).await;
2321 + let mut log = Vec::new();
2322 + assert!(code_smoke_frontends(&ctx, &mut log).await.is_none());
2323 + assert!(
2324 + String::from_utf8_lossy(&log).contains("skipping"),
2325 + "the skip must be visible in the log"
2326 + );
2327 + }
2328 +
2118 2329 #[tokio::test]
2119 2330 async fn cargo_test_fails_closed_when_no_target_exists_in_the_worktree() {
2120 2331 // A worktree missing every configured crate must not report "tests
M sando/daemon/src/outcome.rs +12
@@ -245,6 +245,14 @@
245 245 /// readiness log and the health probe — so a missing log line fails even
246 246 /// when the probe passes.
247 247 CodeSmokeNoListeningLog,
248 + /// `code_smoke`: a configured `frontend_build` failed to compile. `dir` is
249 + /// the npm project under the worktree; `exit_code` is npm's status. This is
250 + /// the failure both MNW build scripts downgrade to a `cargo::warning` on
251 + /// purpose (so a broken chat widget cannot stop the forum compiling) — which
252 + /// means this gate is the only place it is ever fatal, and the only thing
253 + /// standing between a `tsc` error and a deploy that rsyncs the previous
254 + /// build's `static/dist/`.
255 + CodeSmokeFrontend { dir: String, exit_code: Option<i32> },
248 256 /// `cargo_test` / `boot_smoke`: tokio could not spawn the child.
249 257 SpawnFailed { message: String },
250 258 /// Gate took longer than the configured ceiling.
@@ -320,6 +328,10 @@
320 328 GateFailure::CodeSmokeNoListeningLog => {
321 329 "served /health but never logged 'listening'".into()
322 330 }
331 + GateFailure::CodeSmokeFrontend { dir, exit_code } => match exit_code {
332 + Some(c) => format!("frontend build failed in {dir}: exit {c}"),
333 + None => format!("frontend build failed in {dir}"),
334 + },
323 335 GateFailure::SpawnFailed { message } => format!("spawn: {message}"),
324 336 GateFailure::Timeout { gate, after_s } => format!("{gate} timed out after {after_s}s"),
325 337 GateFailure::Unclassified {
M sando/daemon/src/routes/mod.rs +1
@@ -933,6 +933,7 @@
933 933 all_features: false,
934 934 scratch_db: true,
935 935 }],
936 + frontend_builds: vec![],
936 937 }
937 938 }
938 939