Skip to main content

max / makenotwork

Carry a finished artifact from Bento to Sando Sando's intake has been able to prove and publish an artifact it did not build since `8c72c5a5`, but `POST /intake` takes a bundle already sitting under its staging directory and nothing put one there. This moves the bytes. Push, not pull, and the measurement that decided it is that this was never a cross-host problem in the common case: sandod and bentod both run on fw13, and astra's aarch64 half already comes back at `collect`. Pull is push with an extra hop and a second credential — Sando has to be told a version exists either way, and that notification IS the POST, after which it would reach back into the build daemon's tree as a different user for what it was just told about. Push also needs nothing new from Sando: `/intake` already enforces that a bundle is staged under `release_root`, so the producer half satisfies the contract as written. `[handoff.<app>]` in the daemon config, per app, no fleet-wide default: handing an artifact to a deploy controller is a per-product decision. Daemon config rather than `bento.toml` because it names where sandod lives and which secret opens it, which is machine knowledge. The record does not travel with the bytes. It names the digest of the bundle and the digest covers every file in the bundle, so a record copied in among the artifacts changes the digest it names, and Sando would recompute, disagree, and refuse. It goes in the request body. `SyncOpts` gains `exclude` for it. The staging dir is mirrored with `--delete`, not merged, so it holds exactly this attempt. Without that, a retry after a partial transfer leaves the earlier attempt's file behind — and an extra file is a manifest mismatch, so the honest bundle would be refused for carrying a leftover. Corruption reported where nothing was corrupted is the worst failure available here. `host` is `sando@fw13` even on one machine. sandod runs as `sando` and bentod as a user unit, so ssh lands the bytes owned by the process that has to rename them, with no group or ACL on the staging directory. A failed handoff fails the target run, at a new `Step::Handoff`. Same argument the archive deposit makes, with more at stake. It is its own step because the recipe's `collect` really did succeed and failing there would contradict a green step row; it is the one step no recipe runs, since the daemon performs it after the recipe, which is also the earliest point a record exists. Appended to `Step::ALL` the way `Deploy` was, so no stored step string moves. `artifact_record::emit` returns the path it wrote. A configured handoff with no record is an error rather than a skip: for an app whose point is being deployed by Sando, a run that collected nothing is not a green run. Found on the way: `bootstrap-sandod-host.sh` never created `/srv/sando/staging`, so a freshly bootstrapped host would have answered every intake with an error about a missing path instead of about the artifact.
Co-Authored-By
Claude Opus 5 (1M context) <noreply@anthropic.com>
Author: Max Johnson <me@maxj.phd> · 2026-08-07 02:00 UTC
Signed with PGP, not checked
Commit: aaf6d23b92a5e51efc2d78833bf94db859755c81
Parent: 0d3a8c4
13 files changed, +919 insertions, -17 deletions
M bento/deploy/README.md +43
@@ -66,6 +66,49 @@
66 66 fails that target's `collect` step, before sign and publish. Leave the `[archive]`
67 67 table out to skip archiving entirely.
68 68
69 + ## Handing an artifact to Sando
70 +
71 + Bento builds and packages; Sando decides whether a thing advances a stage. For a
72 + product Sando deploys, a `[handoff.<app>]` table names where its bytes go after
73 + the recipe finishes:
74 +
75 + ```toml
76 + [handoff.pom]
77 + host = "sando@fw13"
78 + staging_root = "/srv/sando/staging"
79 + url = "http://100.103.89.95:7766"
80 + sando_app = "pom"
81 + token_file = "sando/api-token"
82 + ```
83 +
84 + Per target, the collect directory is rsynced to
85 + `<staging_root>/<app>-<version>-<target>/` and sandod is asked to take it in.
86 + Sando proves the bundle against its record before anything else happens, so a
87 + transfer that dropped or corrupted a file is refused with the file named.
88 +
89 + Two things about the transfer are load-bearing. The artifact record does **not**
90 + travel with the bytes: it names the digest of the bundle, the digest covers every
91 + file in the bundle, so a record copied in among the artifacts would change the
92 + digest it names. It goes in the request body. And the staging directory is
93 + mirrored with `--delete` rather than merged, so a retry after a partial transfer
94 + holds exactly this attempt — a leftover file is an extra file, and an extra file
95 + is a manifest mismatch that would get an honest bundle refused.
96 +
97 + `host` is usually the same machine bentod runs on, and still goes through ssh:
98 + sandod runs as `sando`, bentod as a user unit, and `sando@fw13` lands the bytes
99 + owned by the process that has to rename them without a group or an ACL on the
100 + staging directory. That means bentod's user needs an SSH key in `sando`'s
101 + `authorized_keys`, and the staging directory has to exist:
102 +
103 + ```sh
104 + ssh sando@fw13 'install -d -m 0755 /srv/sando/staging'
105 + ```
106 +
107 + A failed handoff fails that target run at a `handoff` step, which is a step no
108 + recipe runs — the daemon performs it after the recipe, and the failure needs a
109 + column of its own rather than contradicting a green `collect`. Leave the table
110 + out for every app Sando does not deploy, which is most of them.
111 +
69 112 ## Auth (CF2)
70 113
71 114 On a loopback bind, build triggers are reachable only from the daemon's own
M bento/deploy/bento-daemon.toml.example +39
@@ -45,6 +45,45 @@
45 45 host = "astra"
46 46 root = "/var/lib/bento/artifacts"
47 47
48 + # Optional, one table per app: hand that app's finished artifacts to a Sando,
49 + # which decides whether they advance a stage. Bento builds and packages; a
50 + # product Sando deploys needs its bytes to reach Sando, and this is the half
51 + # that moves them.
52 + #
53 + # Per target, after the recipe finishes: rsync the collect directory to
54 + # <staging_root>/<app>-<version>-<target>/ on `host`, then POST that path plus
55 + # the artifact record to sandod, which verifies the bundle against the record
56 + # and publishes it content-addressed, or refuses it and names the file that
57 + # drifted.
58 + #
59 + # host ssh destination of the box sandod runs on; `local` stages on this
60 + # machine. Usually the same box bentod is on, and still worth going
61 + # through ssh: sandod runs as `sando` and bentod as a user unit, so
62 + # `sando@fw13` lands the bytes owned by the process that has to
63 + # rename them, with no group or ACL on the staging directory.
64 + # staging_root Sando's release_root + /staging, ON THAT HOST. Sando refuses a
65 + # bundle staged anywhere else — publishing is an atomic rename, and
66 + # a staging dir on another filesystem would silently become a copy.
67 + # Rsynced with --delete, so name it exactly.
68 + # url sandod's base URL. Tailnet address, never a public one.
69 + # sando_app the id SANDO knows this product by, when it differs. Omit for
70 + # sandod's default product (the unprefixed mount); set it to route
71 + # to /apps/<id>/intake.
72 + # token_file sandod's bearer token, as a path relative to secrets_root. Omit
73 + # only against a loopback sandod that configured none.
74 + #
75 + # A failed handoff fails that target run, at a `handoff` step. Same argument as
76 + # the archive deposit, with more at stake: a build whose artifact never reached
77 + # the deploy controller has not done the job, and a silent skip means the release
78 + # nobody watched is the one Sando never heard about.
79 + #
80 + # [handoff.pom]
81 + # host = "sando@fw13"
82 + # staging_root = "/srv/sando/staging"
83 + # url = "http://100.103.89.95:7766"
84 + # sando_app = "pom"
85 + # token_file = "sando/api-token"
86 +
48 87 # Optional: override the per-step wall-clock budget (seconds) for EVERY step,
49 88 # replacing the per-kind defaults (build 90m, notarize 60m, sign 15m, ...). A
50 89 # step that runs past its budget fails that step and unwinds the recipe, so a
M sando/deploy/bootstrap-sandod-host.sh +5 -1
@@ -92,7 +92,11 @@
92 92 install -d -o "$SANDO_USER" -g "$SANDO_USER" -m 0750 "$SANDO_HOME"
93 93
94 94 log "4/13 /srv/sando subdirs"
95 - for sub in state work releases logs backups; do
95 + # `staging` is where an artifact built elsewhere lands before intake proves it.
96 + # The route resolves `release_root/staging` and refuses a bundle staged anywhere
97 + # else, so a host without this directory answers every intake with an error
98 + # about a missing path rather than about the artifact.
99 + for sub in state work releases staging logs backups; do
96 100 install -d -o "$SANDO_USER" -g "$SANDO_USER" -m 0750 "$SANDO_HOME/$sub"
97 101 done
98 102
M bento/daemon/src/artifact_record.rs +21 -10
@@ -10,10 +10,13 @@
10 10 //! nowhere. Writing them as one document beside the artifacts is what makes the
11 11 //! handover to Sando possible later; today nothing reads it.
12 12 //!
13 - //! Emit-only, and deliberately non-fatal. A build that produced signed,
13 + //! Non-fatal, and no longer quite emit-only. A build that produced signed,
14 14 //! notarized bytes has succeeded whether or not its paperwork could be written,
15 - //! so every failure in here is logged and swallowed. That stops being true once
16 - //! something consumes the record.
15 + //! so every failure in here is logged and swallowed. What changed is that
16 + //! [`crate::handoff`] now reads the record: it returns the path it wrote, and a
17 + //! `None` means the target has nothing to hand to Sando. The swallowing stays
18 + //! here because the reason to write a record is broader than the handoff — the
19 + //! archive keeps one for every target, including the ones no Sando deploys.
17 20
18 21 use crate::domain::{AppId, Target, Version};
19 22 use crate::engine::RecipeCtx;
@@ -21,6 +24,7 @@
21 24 use chrono::{DateTime, Utc};
22 25 use ops_artifact::{ArtifactRecord, GateRecord, Manifest, Provenance, Scope, Verdict};
23 26 use ops_exec::{Action, LogSink, Step as OpStep};
27 + use std::path::PathBuf;
24 28
25 29 /// Where a target's record lands: beside the artifacts it describes, in that
26 30 /// target's own collect directory (`dist_root/<app>/<version>/<target>/`).
@@ -40,15 +44,20 @@
40 44 pub const RECORD_FILE: &str = "record.json";
41 45
42 46 /// Build the record for a finished target run and write it beside the
43 - /// artifacts. Never fails a build: logs and returns.
44 - pub async fn emit(state: &AppState, ctx: &RecipeCtx, pinned_sha: &str) {
47 + /// artifacts. Never fails a build: logs and returns `None`.
48 + ///
49 + /// The returned path is what the handoff sends to Sando. `None` therefore means
50 + /// two different things that want the same treatment: nothing was collected, or
51 + /// the paperwork could not be written. Either way there is no artifact this
52 + /// daemon can honestly hand over.
53 + pub async fn emit(state: &AppState, ctx: &RecipeCtx, pinned_sha: &str) -> Option<PathBuf> {
45 54 if pinned_sha.is_empty() {
46 55 // Pinning is off (`pin_release_sha = false`, which is how the tests run
47 56 // against repos that are not git checkouts). There is no commit to name,
48 57 // and a record whose provenance is blank would describe bytes without
49 58 // saying where they came from, which is the thing this exists to stop.
50 59 tracing::debug!(app = %ctx.app, target = %ctx.target, "release pinning off, no record written");
51 - return;
60 + return None;
52 61 }
53 62
54 63 let hashes = ctx.artifact_hashes();
@@ -57,14 +66,14 @@
57 66 // failed at prebuild is the ordinary case here, and inventing an empty
58 67 // manifest for it would mint an identity for no bytes.
59 68 tracing::debug!(app = %ctx.app, target = %ctx.target, "no artifacts collected, no record written");
60 - return;
69 + return None;
61 70 }
62 71
63 72 let manifest = match Manifest::new(hashes) {
64 73 Ok(m) => m,
65 74 Err(e) => {
66 75 tracing::error!(app = %ctx.app, target = %ctx.target, error = %e, "could not build artifact manifest");
67 - return;
76 + return None;
68 77 }
69 78 };
70 79
@@ -88,14 +97,14 @@
88 97 // itself refuse. Loud, because it is a bug in this file, not a
89 98 // build problem.
90 99 tracing::error!(app = %ctx.app, target = %ctx.target, error = %e, "assembled an invalid artifact record");
91 - return;
100 + return None;
92 101 }
93 102 };
94 103
95 104 let path = record_path(&state.cfg.dist_root, &ctx.app, &ctx.version, ctx.target);
96 105 if let Err(e) = tokio::fs::write(&path, record.to_json()).await {
97 106 tracing::error!(path = %path.display(), error = %e, "could not write artifact record");
98 - return;
107 + return None;
99 108 }
100 109 tracing::info!(
101 110 app = %ctx.app, target = %ctx.target, digest = %record.digest.short(),
@@ -113,6 +122,8 @@
113 122 {
114 123 tracing::error!(app = %ctx.app, target = %ctx.target, error = %e, "could not archive the artifact record");
115 124 }
125 +
126 + Some(path)
116 127 }
117 128
118 129 /// `rustc --version` on the build host.
M bento/daemon/src/config.rs +167
@@ -4,6 +4,7 @@
4 4
5 5 use anyhow::{Context, Result};
6 6 use serde::Deserialize;
7 + use std::collections::HashMap;
7 8 use std::path::PathBuf;
8 9
9 10 #[derive(Debug, Clone, Deserialize)]
@@ -23,6 +24,12 @@
23 24 /// goingson 1.4.0" had no single answer to.
24 25 #[serde(default)]
25 26 pub archive: Option<Archive>,
27 + /// Which apps hand their finished artifacts to a Sando, keyed by the app id
28 + /// this daemon knows it by (see [`Handoff`]). An app with no entry builds
29 + /// and archives exactly as before; there is no fleet-wide default, because
30 + /// handing an artifact to a deploy controller is a per-product decision.
31 + #[serde(default)]
32 + pub handoff: HashMap<String, Handoff>,
26 33 /// Root for per-step run logs
27 34 /// (`<logs_root>/<app>/<version>/<target>/<step>.<run_id>.log`).
28 35 #[serde(default = "default_logs_root")]
@@ -88,6 +95,57 @@
88 95 pub root: PathBuf,
89 96 }
90 97
98 + /// Where one app's finished artifacts are handed to Sando, and how to tell it
99 + /// they arrived.
100 + ///
101 + /// The Sando/Bento boundary is "Bento builds and packages, Sando decides whether
102 + /// a thing advances a stage", and deciding requires being handed bytes. Sando's
103 + /// `POST /intake` takes a bundle that is *already* under its staging directory —
104 + /// it owns proving the bytes, and the producer owns getting them there. This is
105 + /// the producer half.
106 + ///
107 + /// Push and not pull, for a reason worth recording: Sando would have to be told
108 + /// a version exists either way (the same POST), and it would then be reaching
109 + /// into the build daemon's tree as a different user to fetch what it was just
110 + /// told about. Pull is push with an extra hop and a second credential.
111 + ///
112 + /// The transfer excludes the artifact record. Evidence names the digest of the
113 + /// bytes it vouches for, and the digest covers every file in the bundle, so a
114 + /// record copied in among the artifacts would change the digest it names and
115 + /// Sando would refuse the bundle — correctly. The record travels in the request
116 + /// body instead.
117 + #[derive(Debug, Clone, Deserialize)]
118 + pub struct Handoff {
119 + /// SSH destination of the host sandod runs on (a tailnet alias like `fw13`,
120 + /// or `sando@fw13`); `local` stages on this daemon's own filesystem.
121 + ///
122 + /// Usually the same machine bentod is on, and still worth going through ssh:
123 + /// sandod runs as `sando` and bentod as a user unit, so `sando@fw13` lands
124 + /// the bytes owned by the process that has to rename them, without a group
125 + /// or an ACL on the staging directory.
126 + pub host: String,
127 + /// Absolute path of Sando's staging directory ON THAT HOST — its
128 + /// `release_root` plus `staging`. Sando refuses a bundle staged anywhere
129 + /// else, because publishing is an atomic rename and a staging dir on another
130 + /// filesystem would silently become a copy.
131 + pub staging_root: PathBuf,
132 + /// Base URL of the sandod that will take the intake, e.g.
133 + /// `http://100.103.89.95:7766`. Tailnet address, never a public one.
134 + pub url: String,
135 + /// The app id SANDO knows this product by, when it differs from Bento's.
136 + /// Absent means the daemon's default product, which is the unprefixed mount;
137 + /// present routes to `/apps/<id>/intake`.
138 + #[serde(default)]
139 + pub sando_app: Option<String>,
140 + /// File holding sandod's bearer token, as a relative path under
141 + /// `secrets_root` — the same private layer the `secret()` host function
142 + /// reads, so the token is not a second secrets mechanism. Absent sends no
143 + /// `Authorization` header, which only works against a loopback sandod that
144 + /// configured none.
145 + #[serde(default)]
146 + pub token_file: Option<PathBuf>,
147 + }
148 +
91 149 fn default_true() -> bool {
92 150 true
93 151 }
@@ -132,6 +190,42 @@
132 190 a.root.display()
133 191 );
134 192 }
193 + for (app, h) in &self.handoff {
194 + anyhow::ensure!(
195 + !h.host.trim().is_empty(),
196 + "[handoff.{app}] host is empty; set it to an ssh destination or `local`"
197 + );
198 + // Same rule and the same reason as the archive root, with more at
199 + // stake: this destination is rsynced with `--delete`, so a relative
200 + // path or a `..` prunes a directory other than the one the config
201 + // reads as naming.
202 + anyhow::ensure!(
203 + h.staging_root.is_absolute()
204 + && !h
205 + .staging_root
206 + .components()
207 + .any(|c| c == std::path::Component::ParentDir),
208 + "[handoff.{app}] staging_root `{}` must be an absolute path with no `..`",
209 + h.staging_root.display()
210 + );
211 + anyhow::ensure!(
212 + h.url.starts_with("http://") || h.url.starts_with("https://"),
213 + "[handoff.{app}] url `{}` must be an http(s) URL for sandod",
214 + h.url
215 + );
216 + // The token is read relative to `secrets_root`, so the same
217 + // traversal guard the `secret()` host function applies belongs here:
218 + // a config that could name `../../etc/shadow` would turn a path into
219 + // a read primitive.
220 + if let Some(t) = &h.token_file {
221 + anyhow::ensure!(
222 + t.is_relative()
223 + && !t.components().any(|c| c == std::path::Component::ParentDir),
224 + "[handoff.{app}] token_file `{}` must be a relative path under secrets_root",
225 + t.display()
226 + );
227 + }
228 + }
135 229 Ok(())
136 230 }
137 231
@@ -146,6 +240,9 @@
146 240 // Off by default in tests: the archive is a second machine, and the
147 241 // tests that exercise it point it at a local directory themselves.
148 242 archive: None,
243 + // Off for the same reason as the archive: a handoff is another
244 + // daemon, and the tests that exercise it stand one up themselves.
245 + handoff: HashMap::new(),
149 246 logs_root: root.join("logs"),
150 247 step_timeout_secs: None,
151 248 notarize_backoff_secs: None,
@@ -207,4 +304,74 @@
207 304 fn an_empty_archive_host_is_rejected() {
208 305 assert!(parse("[archive]\nhost = \"\"\nroot = \"/var/lib/bento/artifacts\"\n").is_err());
209 306 }
307 +
308 + /// No handoff table at all is the ordinary case — every app but the ones
309 + /// Sando deploys builds and archives and stops there.
310 + #[test]
311 + fn handoff_is_per_app_and_absent_by_default() {
312 + assert!(parse("").unwrap().handoff.is_empty());
313 + }
314 +
315 + #[test]
316 + fn a_handoff_parses_with_its_optional_fields_absent() {
317 + let cfg = parse(
318 + r#"
319 + [handoff.pom]
320 + host = "sando@fw13"
321 + staging_root = "/srv/sando/staging"
322 + url = "http://100.103.89.95:7766"
323 + "#,
324 + )
325 + .unwrap();
326 + let h = cfg.handoff.get("pom").expect("pom hands off");
327 + assert_eq!(h.host, "sando@fw13");
328 + assert_eq!(h.staging_root, PathBuf::from("/srv/sando/staging"));
329 + // Absent means Sando's default product and no bearer header, which is
330 + // the shape a single-product sandod on loopback wants.
331 + assert!(h.sando_app.is_none());
332 + assert!(h.token_file.is_none());
333 + }
334 +
335 + /// The destination is rsynced with `--delete`. A relative path lands in the
336 + /// ssh user's home and a `..` walks out of the declared tree, and either one
337 + /// would prune a directory the config does not appear to name.
338 + #[test]
339 + fn a_relative_or_dot_dot_staging_root_is_rejected() {
340 + let with = |root: &str| {
341 + format!(
342 + "[handoff.pom]\nhost = \"fw13\"\nstaging_root = \"{root}\"\nurl = \"http://x:1\"\n"
343 + )
344 + };
345 + assert!(parse(&with("staging")).is_err());
346 + assert!(parse(&with("/srv/../etc/sando")).is_err());
347 + assert!(parse(&with("/srv/sando/staging")).is_ok());
348 + }
349 +
350 + /// `token_file` is resolved under `secrets_root`. Left unguarded it would be
351 + /// a read primitive for any file the daemon user can open.
352 + #[test]
353 + fn a_token_file_that_escapes_secrets_root_is_rejected() {
354 + let with = |token: &str| {
355 + format!(
356 + "[handoff.pom]\nhost = \"fw13\"\nstaging_root = \"/srv/sando/staging\"\n\
357 + url = \"http://x:1\"\ntoken_file = \"{token}\"\n"
358 + )
359 + };
360 + assert!(parse(&with("../../etc/shadow")).is_err());
361 + assert!(parse(&with("/etc/shadow")).is_err());
362 + assert!(parse(&with("sando/api-token")).is_ok());
363 + }
364 +
365 + /// The url is interpolated into a request. A bare host:port would be sent
366 + /// as a relative URL and fail at the first release rather than at startup.
367 + #[test]
368 + fn a_handoff_url_must_name_a_scheme() {
369 + let with = |url: &str| {
370 + format!(
371 + "[handoff.pom]\nhost = \"fw13\"\nstaging_root = \"/srv/sando/staging\"\nurl = \"{url}\"\n"
372 + )
373 + };
374 + assert!(parse(&with("100.103.89.95:7766")).is_err());
375 + assert!(parse(&with("http://100.103.89.95:7766")).is_ok());
376 + }
210 377 }
M bento/daemon/src/domain.rs +11 -1
@@ -168,6 +168,13 @@
168 168 /// last in `ALL` because a service reaches it after every gate the other kinds
169 169 /// use, and appending rather than inserting leaves the existing column order
170 170 /// (and every stored `step_runs.step` string) untouched.
171 + ///
172 + /// `Handoff` is appended for the same reason, and is the one step no recipe
173 + /// runs: the daemon performs it after the recipe finishes, sending the artifact
174 + /// to the Sando that will decide whether it advances (see [`crate::handoff`]).
175 + /// It exists as a step so a failed handoff has an honest column to fail in —
176 + /// the recipe's `collect` really did succeed, and reporting the failure there
177 + /// would contradict a green step row.
171 178 #[derive(Debug, Clone, Copy, PartialEq, Eq, Hash, Serialize, Deserialize)]
172 179 #[serde(rename_all = "snake_case")]
173 180 pub enum Step {
@@ -182,11 +189,12 @@
182 189 Publish,
183 190 Collect,
184 191 Deploy,
192 + Handoff,
185 193 }
186 194
187 195 impl Step {
188 196 /// All steps in canonical order — the matrix column set.
189 - pub const ALL: [Step; 11] = [
197 + pub const ALL: [Step; 12] = [
190 198 Step::Checkout,
191 199 Step::Prebuild,
192 200 Step::Build,
@@ -198,6 +206,7 @@
198 206 Step::Publish,
199 207 Step::Collect,
200 208 Step::Deploy,
209 + Step::Handoff,
201 210 ];
202 211
203 212 pub fn as_str(self) -> &'static str {
@@ -213,6 +222,7 @@
213 222 Step::Publish => "publish",
214 223 Step::Collect => "collect",
215 224 Step::Deploy => "deploy",
225 + Step::Handoff => "handoff",
216 226 }
217 227 }
218 228 }
M bento/daemon/src/engine.rs +10 -3
@@ -59,9 +59,12 @@
59 59 // bundle for Gatekeeper to have an opinion about.
60 60 Kind::Library | Kind::Service => Action::Build,
61 61 },
62 - // Publish/Collect run on the daemon, not through a host executor; this
63 - // label only applies if a recipe runs a bare `sh` while one is open.
64 - Step::Publish | Step::Collect => Action::Package,
62 + // Publish/Collect/Handoff run on the daemon, not through a host
63 + // executor; this label only applies if a recipe runs a bare `sh` while
64 + // one is open. `handoff` is the daemon's own post-recipe motion and no
65 + // recipe should open it at all — naming it here costs nothing and beats
66 + // a wildcard that would silently absorb the next step somebody adds.
67 + Step::Publish | Step::Collect | Step::Handoff => Action::Package,
65 68 // The one step that dispatches to a host OUTSIDE the build topology.
66 69 // Every command a recipe runs while `deploy` is open — the install, the
67 70 // restart, the health assertion — carries this action, so it reaches the
@@ -116,6 +119,10 @@
116 119 // A binary push, an install, a unit restart, and a health poll. Minutes
117 120 // of work; the ceiling is for a wedged transport, not slow work.
118 121 Step::Deploy => 15,
122 + // Unused by any recipe — the daemon runs the handoff itself, outside a
123 + // step's clock — and matched to `collect`, since it moves the same
124 + // bytes the same way and would wedge for the same reasons.
125 + Step::Handoff => 30,
119 126 };
120 127 Duration::from_secs(mins * 60)
121 128 }
M bento/daemon/src/lib.rs +1
@@ -16,6 +16,7 @@
16 16 pub mod engine;
17 17 pub mod error;
18 18 pub mod events;
19 + pub mod handoff;
19 20 pub mod metrics;
20 21 pub mod ota;
21 22 pub mod retention;
M bento/daemon/src/runner.rs +57 -1
@@ -653,10 +653,38 @@
653 653 // build that signed and collected an artifact and then failed at `publish`
654 654 // still produced bytes somebody may want the provenance of. Emit-only and
655 655 // non-fatal; nothing reads it yet.
656 - crate::artifact_record::emit(&state, &ctx, &pinned_sha).await;
656 + let record_path = crate::artifact_record::emit(&state, &ctx, &pinned_sha).await;
657 657
658 658 match outcome {
659 659 Ok(Ok(())) => {
660 + // Hand the artifact to Sando before the run is stamped ok, so a
661 + // target reported green is one whose bytes reached the controller
662 + // that decides whether they ship. Only on success: an artifact from
663 + // a failed recipe is exactly what should not be offered for a
664 + // deploy, whatever it managed to collect on the way down.
665 + if let Err(e) =
666 + handoff_for(&state, record_path.as_deref(), &app, &version, target).await
667 + {
668 + let msg = format!("{e:#}");
669 + tracing::error!(%app, %target, error = %msg, "handing the artifact to sando failed");
670 + fail_target(
671 + &state,
672 + target_run_id,
673 + &app,
674 + &version,
675 + target,
676 + Step::Handoff,
677 + &msg,
678 + )
679 + .await;
680 + crate::metrics::target_finished(
681 + &target.to_string(),
682 + "failed",
683 + started.elapsed().as_secs_f64(),
684 + );
685 + return;
686 + }
687 +
660 688 let artifacts = collected_artifacts(&state, &app, &version, target);
661 689 if let Err(e) = sqlx::query(
662 690 "UPDATE target_runs SET status = 'ok', current_step = NULL, finished_at = ? WHERE id = ?",
@@ -896,6 +924,34 @@
896 924 std::fs::read_to_string(&path).with_context(|| format!("reading recipe {}", path.display()))
897 925 }
898 926
927 + /// Send this target's finished bundle to the Sando configured for the app, if
928 + /// one is. Nothing configured is a no-op and the ordinary case.
929 + ///
930 + /// A configured handoff with no record is an error rather than a skip. It means
931 + /// the recipe succeeded and collected nothing, or that the paperwork could not
932 + /// be written — and for an app whose whole point is being deployed by Sando, a
933 + /// build that produced nothing to hand over is not a green build. Everywhere
934 + /// else a missing record stays the non-event it was.
935 + async fn handoff_for(
936 + state: &AppState,
937 + record_path: Option<&std::path::Path>,
938 + app: &AppId,
939 + version: &Version,
940 + target: Target,
941 + ) -> anyhow::Result<()> {
942 + if !state.cfg.handoff.contains_key(app.as_str()) {
943 + return Ok(());
944 + }
945 + let record_path = record_path.ok_or_else(|| {
946 + anyhow::anyhow!(
947 + "{app} hands off to sando, but this {target} run wrote no artifact record \
948 + (nothing was collected, or the record could not be written)"
949 + )
950 + })?;
951 + let dir = crate::archive::target_dir(&state.cfg.dist_root, app, version, target);
952 + crate::handoff::send(&state.cfg, &dir, record_path, app, version, target).await
953 + }
954 +
899 955 /// What this target run left in its collect directory.
900 956 ///
901 957 /// Per target: the event reports what THIS run produced, and every target of a
M shared/ops-exec/src/executor.rs +10
M shared/ops-exec/src/transport.rs +54
M sando/daemon/src/routes/mod.rs +4 -1
A bento/daemon/src/handoff.rs +497