Skip to main content

max / makenotwork

s3-storage: soak target over multipart part geometry Row 7 of astra-soak-overview, the upload and object path. The doors were counted first, as infra 8910f917 instructs, and the count decided the shape: almost all of the crate is async I/O against a live endpoint and is not fuzzable without one, while MultipartPlan is pure arithmetic with four call sites in the server and one implementation behind them. The oracle lives in the crate, following git-command's precedent, so the libFuzzer target on nightly and tests/regressions.rs on stable assert the same things and cannot drift into checking less than each other. It asserts that the parts tile the object exactly (a gap loses bytes, an overlap duplicates them, and neither shows until playback), that S3's own limits hold, that out-of-range part numbers are refused rather than answered, and that every rejection names a condition that actually does hold, which is what protects a legitimate upload from being turned away. SyncKit's client check is a one-line div_ceil restatement rather than a second parser, so this asserts the property instead of running a differential against it. Fifteen seeds at the boundaries where part arithmetic goes wrong, plus the off-by-one either side of each; regressions/ replays as an ordinary cargo test so a crash found on astra cannot come back unnoticed here. Not yet registered in witchbroom.toml, so the soak half of 8910f917's done condition is still open, as is the mutation half (1498fe2a).
Co-Authored-By
Claude Opus 5 (1M context) <noreply@anthropic.com>
Author: Max Johnson <me@maxj.phd> · 2026-08-26 22:25 UTC
Signed with PGP, not checked
Commit: 1a837bd529e3b5c98f04262a0106b4c789ffa319
Parent: 74f29ae
24 files changed, +993 insertions, -10 deletions
@@ -2083,26 +2083,70 @@
2083 2083 "syn",
2084 2084 ]
2085 2085
2086 + [[patch.unused]]
2087 + name = "synckit-client"
2088 + version = "0.9.1"
2089 +
2090 + [[patch.unused]]
2091 + name = "synckit-config"
2092 + version = "0.2.0"
2093 +
2094 + [[patch.unused]]
2095 + name = "quasi-type"
2096 + version = "0.1.0"
2097 +
2086 2098 [[patch.unused]]
2087 2099 name = "kberg"
2088 2100 version = "0.1.0"
2089 2101
2102 + [[patch.unused]]
2103 + name = "ops-status"
2104 + version = "0.1.0"
2105 +
2090 2106 [[patch.unused]]
2091 2107 name = "painhours"
2092 2108 version = "0.1.0"
2093 2109
2094 2110 [[patch.unused]]
2095 2111 name = "tagtree"
2096 - version = "0.4.0"
2097 -
2098 - [[patch.unused]]
2099 - name = "synckit-client"
2100 - version = "0.8.0"
2101 -
2102 - [[patch.unused]]
2103 - name = "synckit-config"
2104 - version = "0.2.0"
2112 + version = "0.4.1"
2105 2113
2106 2114 [[patch.unused]]
2107 2115 name = "docengine"
2108 - version = "0.4.0"
2116 + version = "0.7.0"
2117 +
2118 + [[patch.unused]]
2119 + name = "quasi-axum"
2120 + version = "0.64.0"
2121 +
2122 + [[patch.unused]]
2123 + name = "quasi-basics"
2124 + version = "0.64.0"
2125 +
2126 + [[patch.unused]]
2127 + name = "quasi-http"
2128 + version = "0.64.0"
2129 +
2130 + [[patch.unused]]
2131 + name = "quasi-immediate"
2132 + version = "0.64.0"
2133 +
2134 + [[patch.unused]]
2135 + name = "quasi-notifs"
2136 + version = "0.64.0"
2137 +
2138 + [[patch.unused]]
2139 + name = "quasi-router"
2140 + version = "0.64.0"
2141 +
2142 + [[patch.unused]]
2143 + name = "quasi-store"
2144 + version = "0.1.0"
2145 +
2146 + [[patch.unused]]
2147 + name = "quasi-tauri"
2148 + version = "0.64.0"
2149 +
2150 + [[patch.unused]]
2151 + name = "quasi-webview"
2152 + version = "0.64.0"
@@ -193,6 +193,194 @@
193 193 }
194 194 }
195 195
196 + /// What a [`MultipartPlan`] must be true of, whatever it was asked for.
197 + ///
198 + /// **The oracle lives in the crate**, so `tests/regressions.rs` on stable and
199 + /// the soak tier's libFuzzer target on nightly assert the same things and
200 + /// cannot drift into checking less than each other. The shape is
201 + /// `MNW/shared/git-command`'s, copied deliberately.
202 + ///
203 + /// ## The doors, counted before this was written
204 + ///
205 + /// Infra `8910f917` instructs it, and the answer decided the shape. Almost all
206 + /// of this crate is async I/O against a live endpoint and is not fuzzable
207 + /// without one. What IS pure, and what creator media actually depends on, is
208 + /// the part geometry: [`MultipartPlan::auto`] has four call sites, all in MNW
209 + /// server (`routes/synckit/blobs.rs` twice, `routes/api/internal/uploads.rs`
210 + /// twice), and one implementation behind them.
211 + ///
212 + /// `MultipartPlan`'s own docs say "the client computes the same boundaries
213 + /// independently", which reads like the two-parsers-of-one-grammar shape that
214 + /// made a shared crate the right answer for `git_ssh`. It is not. SyncKit's
215 + /// client check (`synckit-client/src/client/blob.rs`) is a one-line restatement
216 + /// -- `size_bytes.div_ceil(part_size) == part_count` -- so a differential
217 + /// between the two would measure nothing that asserting the property does not.
218 + /// That restatement is property (1) below, asserted against the plan itself.
219 + ///
220 + /// ## Why the properties are the ones they are
221 + ///
222 + /// This is the path creator media travels, and it carries two distinct failure
223 + /// modes. A plan whose parts do not tile the object exactly means **corrupted
224 + /// or truncated media** -- a gap loses bytes, an overlap duplicates them, and
225 + /// neither shows up until someone plays the file back. A plan that is refused
226 + /// when it should not be means a **legitimate upload rejected**, which is why
227 + /// the error paths are asserted as tightly as the success path: every rejection
228 + /// has to name a condition that actually holds.
229 + pub mod oracle {
230 + use super::{
231 + MULTIPART_MAX_OBJECT_SIZE, MULTIPART_MAX_PART_SIZE, MULTIPART_MAX_PARTS,
232 + MULTIPART_MIN_PART_SIZE, MultipartPlan,
233 + };
234 +
235 + /// Assert everything that must hold for one `(total_size, part_size)` pair.
236 + ///
237 + /// # Panics
238 + /// By design. It is an oracle, and a panic is how it reports.
239 + pub fn check_plan(total_size: u64, part_size: usize) {
240 + let outcome = MultipartPlan::new(total_size, part_size);
241 +
242 + // Whether a plan is possible at all, stated independently of the code
243 + // under test rather than read back out of it.
244 + let possible = total_size > 0
245 + && total_size <= MULTIPART_MAX_OBJECT_SIZE
246 + && part_size >= MULTIPART_MIN_PART_SIZE
247 + && part_size as u64 <= MULTIPART_MAX_PART_SIZE
248 + && total_size.div_ceil(part_size as u64) <= MULTIPART_MAX_PARTS as u64;
249 +
250 + match outcome {
251 + Err(_) => assert!(
252 + !possible,
253 + "a legitimate plan was refused: total_size {total_size}, part_size {part_size}"
254 + ),
255 + Ok(plan) => {
256 + assert!(
257 + possible,
258 + "an impossible plan was accepted: total_size {total_size}, \
259 + part_size {part_size} -> {plan:?}"
260 + );
261 + check_geometry(&plan);
262 + }
263 + }
264 + }
265 +
266 + /// Assert what [`MultipartPlan::auto`] promises: a plan for every non-empty
267 + /// object within the ceiling, with no part size for the caller to pick.
268 + ///
269 + /// # Panics
270 + /// By design.
271 + pub fn check_auto(total_size: u64) {
272 + let outcome = MultipartPlan::auto(total_size);
273 + let possible = total_size > 0 && total_size <= MULTIPART_MAX_OBJECT_SIZE;
274 +
275 + match outcome {
276 + Err(_) => assert!(
277 + !possible,
278 + "auto refused a legitimate object of {total_size} bytes"
279 + ),
280 + Ok(plan) => {
281 + assert!(
282 + possible,
283 + "auto planned an impossible object of {total_size} bytes: {plan:?}"
284 + );
285 + check_geometry(&plan);
286 + }
287 + }
288 + }
289 +
290 + /// The geometry every accepted plan must satisfy.
291 + ///
292 + /// # Panics
293 + /// By design.
294 + pub(crate) fn check_geometry(plan: &MultipartPlan) {
295 + let MultipartPlan {
296 + total_size,
297 + part_size,
298 + part_count,
299 + } = *plan;
300 +
301 + // (1) The client's independent check, asserted here so the two cannot
302 + // disagree. SyncKit refuses a session whose part_count does not equal
303 + // this, so a plan that fails it is an upload that cannot start.
304 + assert_eq!(
305 + u64::from(part_count),
306 + total_size.div_ceil(part_size as u64),
307 + "part_count disagrees with div_ceil for {plan:?}"
308 + );
309 +
310 + // (2) S3's own limits, restated because they are what the remote will
311 + // enforce whatever we believe.
312 + assert!(part_count >= 1, "a plan with no parts: {plan:?}");
313 + assert!(
314 + part_count <= MULTIPART_MAX_PARTS,
315 + "over the part limit: {plan:?}"
316 + );
317 + assert!(
318 + part_size >= MULTIPART_MIN_PART_SIZE,
319 + "part below the S3 floor: {plan:?}"
320 + );
321 + assert!(
322 + part_size as u64 <= MULTIPART_MAX_PART_SIZE,
323 + "part above the S3 ceiling: {plan:?}"
324 + );
325 +
326 + // (3) THE PARTS TILE THE OBJECT EXACTLY. This is the corruption
327 + // property: a gap loses bytes and an overlap duplicates them, and
328 + // neither is visible until someone plays the file back.
329 + let mut covered: u64 = 0;
330 + let mut expected_start: u64 = 0;
331 + for n in 1..=part_count {
332 + let len = plan.part_len(n);
333 + assert!(len > 0, "part {n} is empty in {plan:?}");
334 + assert!(
335 + len <= part_size as u64,
336 + "part {n} is longer than the part size in {plan:?}"
337 + );
338 +
339 + let (start, end) = plan
340 + .part_range(n)
341 + .unwrap_or_else(|| panic!("part {n} has no range in {plan:?}"));
342 + assert_eq!(
343 + start,
344 + expected_start,
345 + "part {n} does not abut part {} in {plan:?}",
346 + n - 1
347 + );
348 + assert_eq!(
349 + end - start + 1,
350 + len,
351 + "part {n}'s range and length disagree in {plan:?}"
352 + );
353 +
354 + covered += len;
355 + expected_start = end + 1;
356 + }
357 + assert_eq!(
358 + covered, total_size,
359 + "the parts do not cover the object exactly: {plan:?}"
360 + );
361 + assert_eq!(
362 + expected_start, total_size,
363 + "the last part does not end at the object's end: {plan:?}"
364 + );
365 +
366 + // (4) Out-of-range part numbers are refused rather than answered. A
367 + // caller that loops off the end must get nothing, not byte zero.
368 + assert_eq!(plan.part_len(0), 0, "part 0 has a length in {plan:?}");
369 + assert_eq!(plan.part_range(0), None, "part 0 has a range in {plan:?}");
370 + let past = part_count + 1;
371 + assert_eq!(
372 + plan.part_len(past),
373 + 0,
374 + "part {past} has a length in {plan:?}"
375 + );
376 + assert_eq!(
377 + plan.part_range(past),
378 + None,
379 + "part {past} has a range in {plan:?}"
380 + );
381 + }
382 + }
383 +
196 384 impl S3Client {
197 385 /// Create a new S3 client from configuration.
198 386 // Public async constructor: kept async for API stability across callers.
@@ -1199,6 +1387,54 @@
1199 1387
1200 1388 const MIB: u64 = 1024 * 1024;
1201 1389
1390 + #[test]
1391 + fn oracle_accepts_every_plan_the_crate_makes() {
1392 + // Spot sizes across the whole legal range, including the exact
1393 + // boundaries, since those are where tiling arithmetic goes wrong.
1394 + for total in [
1395 + 1,
1396 + MULTIPART_MIN_PART_SIZE as u64 - 1,
1397 + MULTIPART_MIN_PART_SIZE as u64,
1398 + MULTIPART_MIN_PART_SIZE as u64 + 1,
1399 + 25 * MIB,
1400 + MULTIPART_DEFAULT_PART_SIZE as u64 * MULTIPART_MAX_PARTS as u64,
1401 + MULTIPART_MAX_OBJECT_SIZE - 1,
1402 + MULTIPART_MAX_OBJECT_SIZE,
1403 + ] {
1404 + oracle::check_auto(total);
1405 + oracle::check_plan(total, MULTIPART_MIN_PART_SIZE);
1406 + oracle::check_plan(total, MULTIPART_DEFAULT_PART_SIZE);
1407 + }
1408 + // And the refusals, which must be refusals for a reason that holds.
1409 + oracle::check_auto(0);
1410 + oracle::check_auto(MULTIPART_MAX_OBJECT_SIZE + 1);
1411 + oracle::check_plan(25 * MIB, MULTIPART_MIN_PART_SIZE - 1);
1412 + oracle::check_plan(25 * MIB, MULTIPART_MAX_PART_SIZE as usize + 1);
1413 + }
1414 +
1415 + #[test]
1416 + #[should_panic(expected = "disagrees with div_ceil")]
1417 + fn oracle_catches_a_part_count_the_client_would_reject() {
1418 + // Hand-built, because the crate will not produce it. SyncKit refuses a
1419 + // session whose part_count is not div_ceil (synckit-client
1420 + // `client/blob.rs`), so a plan like this is an upload that can never
1421 + // start.
1422 + //
1423 + // THE TILING PROPERTY HAS NO SUCH TEST, and deliberately so: it cannot
1424 + // be broken by hand. `part_len` and `part_range` both derive from
1425 + // `part_size` and `part_count`, so any plan that satisfies the
1426 + // div_ceil check above necessarily tiles. What establishes that the
1427 + // tiling assertions are observed rather than merely present is the
1428 + // mutation run (infra `1498fe2a`), which changes the derivation itself.
1429 + // That division is the point of running both.
1430 + let plan = MultipartPlan {
1431 + total_size: 25 * MIB,
1432 + part_size: 10 * MIB as usize,
1433 + part_count: 4,
1434 + };
1435 + oracle::check_geometry(&plan);
1436 + }
1437 +
1202 1438 #[test]
1203 1439 fn multipart_plan_divides_with_remainder() {
1204 1440 // 25 MiB in 10 MiB parts -> 10 + 10 + 5.
@@ -1,0 +1,4 @@
1 + target
2 + corpus
3 + artifacts
4 + coverage
@@ -1,0 +1,2248 @@
1 + # This file is automatically @generated by Cargo.
2 + # It is not intended for manual editing.
3 + version = 4
4 +
5 + [[package]]
6 + name = "allocator-api2"
7 + version = "0.2.21"
8 + source = "registry+https://github.com/rust-lang/crates.io-index"
9 + checksum = "683d7910e743518b0e34f1186f92494becacb047c7b6bf616c96772180fef923"
10 +
11 + [[package]]
12 + name = "arbitrary"
13 + version = "1.4.2"
14 + source = "registry+https://github.com/rust-lang/crates.io-index"
15 + checksum = "c3d036a3c4ab069c7b410a2ce876bd74808d2d0888a82667669f8e783a898bf1"
16 + dependencies = [
17 + "derive_arbitrary",
18 + ]
19 +
20 + [[package]]
21 + name = "arc-swap"
22 + version = "1.9.2"
23 + source = "registry+https://github.com/rust-lang/crates.io-index"
24 + checksum = "c049c0be4daef0b145cb3555416b3b8ef5b7888a38aea1a3a155801fe7b0810b"
25 + dependencies = [
26 + "rustversion",
27 + ]
28 +
29 + [[package]]
30 + name = "atomic-waker"
31 + version = "1.1.2"
32 + source = "registry+https://github.com/rust-lang/crates.io-index"
33 + checksum = "1505bd5d3d116872e7271a6d4e16d81d0c8570876c8de68093a09ac269d8aac0"
34 +
35 + [[package]]
36 + name = "autocfg"
37 + version = "1.5.1"
38 + source = "registry+https://github.com/rust-lang/crates.io-index"
39 + checksum = "f2032f911046de80f0a198e0901378627c33f59ea0ac00e363d481118bd70a53"
40 +
41 + [[package]]
42 + name = "aws-config"
43 + version = "1.11.0"
44 + source = "registry+https://github.com/rust-lang/crates.io-index"
45 + checksum = "a767267da9e2c2e189b2f9df8b5657e850ecf5352644734ba130d4a57095cf1b"
46 + dependencies = [
47 + "aws-credential-types",
48 + "aws-runtime",
49 + "aws-sdk-sts",
50 + "aws-smithy-async",
51 + "aws-smithy-http",
52 + "aws-smithy-json",
53 + "aws-smithy-runtime",
54 + "aws-smithy-runtime-api",
55 + "aws-smithy-schema",
56 + "aws-smithy-types",
57 + "aws-types",
58 + "bytes",
59 + "fastrand",
60 + "http 1.5.0",
61 + "time",
62 + "tokio",
63 + "tracing",
64 + "url",
65 + ]
66 +
67 + [[package]]
68 + name = "aws-credential-types"
69 + version = "1.3.0"
70 + source = "registry+https://github.com/rust-lang/crates.io-index"
71 + checksum = "e93964ffdaf57857f544be3666a5f57570bb699e934700f11b49708f61bb556e"
72 + dependencies = [
73 + "aws-smithy-async",
74 + "aws-smithy-runtime-api",
75 + "aws-smithy-types",
76 + "zeroize",
77 + ]
78 +
79 + [[package]]
80 + name = "aws-runtime"
81 + version = "1.9.1"
82 + source = "registry+https://github.com/rust-lang/crates.io-index"
83 + checksum = "c9007227e10b5fed2f3e0a2beff489211e2b5604c400b7a9d5d81ca9d64c24bb"
84 + dependencies = [
85 + "aws-credential-types",
86 + "aws-sigv4",
87 + "aws-smithy-async",
88 + "aws-smithy-eventstream",
89 + "aws-smithy-http",
90 + "aws-smithy-runtime",
91 + "aws-smithy-runtime-api",
92 + "aws-smithy-types",
93 + "aws-types",
94 + "bytes",
95 + "bytes-utils",
96 + "fastrand",
97 + "http 0.2.12",
98 + "http 1.5.0",
99 + "http-body 0.4.6",
100 + "http-body 1.1.0",
101 + "percent-encoding",
102 + "pin-project-lite",
103 + "tracing",
104 + "uuid",
105 + ]
106 +
107 + [[package]]
108 + name = "aws-sdk-s3"
109 + version = "1.144.0"
110 + source = "registry+https://github.com/rust-lang/crates.io-index"
111 + checksum = "30dc8bf6baaf7d46336a0ca2c69f223d9b90d7a801fb3e28f7ea17b00dc6b1de"
112 + dependencies = [
113 + "arc-swap",
114 + "aws-credential-types",
115 + "aws-runtime",
116 + "aws-sigv4",
117 + "aws-smithy-async",
118 + "aws-smithy-checksums",
119 + "aws-smithy-eventstream",
120 + "aws-smithy-http",
121 + "aws-smithy-json",
122 + "aws-smithy-observability",
123 + "aws-smithy-runtime",
124 + "aws-smithy-runtime-api",
125 + "aws-smithy-schema",
126 + "aws-smithy-types",
127 + "aws-smithy-xml",
128 + "aws-types",
129 + "bytes",
130 + "fastrand",
131 + "hex",
132 + "hmac 0.13.0",
133 + "http 0.2.12",
134 + "http 1.5.0",
135 + "http-body 1.1.0",
136 + "lru",
137 + "percent-encoding",
138 + "regex-lite",
139 + "sha2 0.11.0",
140 + "tracing",
141 + "url",
142 + ]
143 +
144 + [[package]]
145 + name = "aws-sdk-sts"
146 + version = "1.113.0"
147 + source = "registry+https://github.com/rust-lang/crates.io-index"
148 + checksum = "68182ecb449f7537db0f4d5d25917789cf41e32074a9fe47b6a0b847fe1d2032"
149 + dependencies = [
150 + "arc-swap",
151 + "aws-credential-types",
152 + "aws-runtime",
153 + "aws-smithy-async",
154 + "aws-smithy-http",
155 + "aws-smithy-json",
156 + "aws-smithy-observability",
157 + "aws-smithy-query",
158 + "aws-smithy-runtime",
159 + "aws-smithy-runtime-api",
160 + "aws-smithy-schema",
161 + "aws-smithy-types",
162 + "aws-smithy-xml",
163 + "aws-types",
164 + "fastrand",
165 + "http 0.2.12",
166 + "http 1.5.0",
167 + "regex-lite",
168 + "tracing",
169 + ]
170 +
171 + [[package]]
172 + name = "aws-sigv4"
173 + version = "1.5.1"
174 + source = "registry+https://github.com/rust-lang/crates.io-index"
175 + checksum = "723c2234ad7511ceef63eab016b7ba6ff7c55590fefb96fa8467af014a07309f"
176 + dependencies = [
177 + "aws-credential-types",
178 + "aws-smithy-eventstream",
179 + "aws-smithy-http",
180 + "aws-smithy-runtime-api",
181 + "aws-smithy-types",
182 + "bytes",
183 + "crypto-bigint",
184 + "form_urlencoded",
185 + "hex",
186 + "hmac 0.13.0",
187 + "http 0.2.12",
188 + "http 1.5.0",
189 + "p256",
190 + "percent-encoding",
191 + "sha2 0.11.0",
192 + "subtle",
193 + "time",
194 + "tracing",
195 + "zeroize",
196 + ]
197 +
198 + [[package]]
199 + name = "aws-smithy-async"
200 + version = "1.3.0"
201 + source = "registry+https://github.com/rust-lang/crates.io-index"
202 + checksum = "f02e407fb3b54891734224b9ffac8a71fdd35f542500fa1af95754a6b2beb316"
203 + dependencies = [
204 + "futures-util",
205 + "pin-project-lite",
206 + "tokio",
207 + ]
208 +
209 + [[package]]
210 + name = "aws-smithy-checksums"
211 + version = "0.65.0"
212 + source = "registry+https://github.com/rust-lang/crates.io-index"
213 + checksum = "b67ecd999972b58e67cab052f5129906c08c25883bd0788ceefc55ef97d61307"
214 + dependencies = [
215 + "aws-smithy-http",
216 + "aws-smithy-types",
217 + "bytes",
218 + "crc-fast",
219 + "hex",
220 + "http 1.5.0",
221 + "http-body 1.1.0",
222 + "http-body-util",
223 + "md-5",
224 + "pin-project-lite",
225 + "sha1",
226 + "sha2 0.11.0",
227 + "tracing",
228 + ]
229 +
230 + [[package]]
231 + name = "aws-smithy-eventstream"
232 + version = "0.61.2"
233 + source = "registry+https://github.com/rust-lang/crates.io-index"
234 + checksum = "6de526c7b567420a31bc283657a7921b45c4cafe0827fdf2490713dcc770c28f"
235 + dependencies = [
236 + "aws-smithy-types",
237 + "bytes",
238 + "crc32fast",
239 + ]
240 +
241 + [[package]]
242 + name = "aws-smithy-http"
243 + version = "0.64.0"
244 + source = "registry+https://github.com/rust-lang/crates.io-index"
245 + checksum = "37843d9add67c3aff5856f409c6dc315d3cdff60f9c0cb5b670dab1e9920306d"
246 + dependencies = [
247 + "aws-smithy-eventstream",
248 + "aws-smithy-runtime-api",
249 + "aws-smithy-types",
250 + "bytes",
251 + "bytes-utils",
252 + "futures-core",
253 + "futures-util",
254 + "http 1.5.0",
255 + "http-body 1.1.0",
256 + "http-body-util",
257 + "percent-encoding",
258 + "pin-project-lite",
259 + "pin-utils",
260 + "tracing",
261 + ]
262 +
263 + [[package]]
264 + name = "aws-smithy-http-client"
265 + version = "1.4.0"
266 + source = "registry+https://github.com/rust-lang/crates.io-index"
267 + checksum = "ebfd138fac0337cee7516c352757ea73b9f2266e57d0bcb5bc70e9547e45aef1"
268 + dependencies = [
269 + "aws-smithy-async",
270 + "aws-smithy-runtime-api",
271 + "aws-smithy-types",
272 + "h2",
273 + "http 1.5.0",
274 + "hyper",
275 + "hyper-rustls",
276 + "hyper-util",
277 + "pin-project-lite",
278 + "rustls",
279 + "rustls-native-certs",
280 + "rustls-pki-types",
281 + "tokio",
282 + "tokio-rustls",
283 + "tower",
284 + "tracing",
285 + ]
286 +
287 + [[package]]
288 + name = "aws-smithy-json"
289 + version = "0.63.0"
290 + source = "registry+https://github.com/rust-lang/crates.io-index"
291 + checksum = "3dc65a121adb4b33729919fcfa14fa36fb33c1555a8f06bb0e2188dbfdc1d9ef"
292 + dependencies = [
293 + "aws-smithy-runtime-api",
294 + "aws-smithy-schema",
295 + "aws-smithy-types",
296 + ]
297 +
298 + [[package]]
299 + name = "aws-smithy-observability"
300 + version = "0.3.0"
301 + source = "registry+https://github.com/rust-lang/crates.io-index"
302 + checksum = "8e86338c869539a581bf161247762a6e87f92c5c075060057b5ed6d06632ed0c"
303 + dependencies = [
304 + "aws-smithy-runtime-api",
305 + ]
306 +
307 + [[package]]
308 + name = "aws-smithy-query"
309 + version = "0.62.0"
310 + source = "registry+https://github.com/rust-lang/crates.io-index"
311 + checksum = "512346c7212ab7436df2d77a16d976a468ae44a418835511d2a69269810aaf62"
312 + dependencies = [
313 + "aws-smithy-runtime-api",
314 + "aws-smithy-schema",
315 + "aws-smithy-types",
316 + "aws-smithy-xml",
317 + "urlencoding",
318 + ]
319 +
320 + [[package]]
321 + name = "aws-smithy-runtime"
322 + version = "1.14.0"
323 + source = "registry+https://github.com/rust-lang/crates.io-index"
324 + checksum = "b82e438d30e02a825d363bd639a9efaed68a8089d86101054b0081e7e0d3e606"
325 + dependencies = [
326 + "aws-smithy-async",
327 + "aws-smithy-http",
328 + "aws-smithy-observability",
329 + "aws-smithy-runtime-api",
330 + "aws-smithy-schema",
331 + "aws-smithy-types",
332 + "bytes",
333 + "fastrand",
334 + "http 0.2.12",
335 + "http 1.5.0",
336 + "http-body 0.4.6",
337 + "http-body 1.1.0",
338 + "http-body-util",
339 + "pin-project-lite",
340 + "pin-utils",
341 + "tokio",
342 + "tracing",
343 + ]
344 +
345 + [[package]]
346 + name = "aws-smithy-runtime-api"
347 + version = "1.15.0"
348 + source = "registry+https://github.com/rust-lang/crates.io-index"
349 + checksum = "954c563ce84507722d2679f07a35d21b9c6466b3872d513020d0281fc8112ac9"
350 + dependencies = [
351 + "aws-smithy-async",
352 + "aws-smithy-runtime-api-macros",
353 + "aws-smithy-types",
354 + "bytes",
355 + "http 0.2.12",
356 + "http 1.5.0",
357 + "pin-project-lite",
358 + "tokio",
359 + "tracing",
360 + "zeroize",
361 + ]
362 +
363 + [[package]]
364 + name = "aws-smithy-runtime-api-macros"
365 + version = "1.1.0"
366 + source = "registry+https://github.com/rust-lang/crates.io-index"
367 + checksum = "221eaa237ddf1ca79b60d1372aad77e47f9c0ea5b3ce5099da8c61d027dc77b3"
368 + dependencies = [
369 + "proc-macro2",
370 + "quote",
371 + "syn 2.0.119",
372 + ]
373 +
374 + [[package]]
375 + name = "aws-smithy-schema"
376 + version = "0.2.0"
377 + source = "registry+https://github.com/rust-lang/crates.io-index"
378 + checksum = "7d56e0a4e53127a632224e43633b0fe045fa9e1e3cfc68b9830f1115e103f910"
379 + dependencies = [
380 + "aws-smithy-runtime-api",
381 + "aws-smithy-types",
382 + "http 1.5.0",
383 + ]
384 +
385 + [[package]]
386 + name = "aws-smithy-types"
387 + version = "1.6.2"
388 + source = "registry+https://github.com/rust-lang/crates.io-index"
389 + checksum = "fce83ce9abbb198d25bc7131e468d0f9fe1257125e58c39f3f9fc9f5098c9647"
390 + dependencies = [
391 + "base64-simd",
392 + "bytes",
393 + "bytes-utils",
394 + "futures-core",
395 + "http 0.2.12",
396 + "http 1.5.0",
397 + "http-body 0.4.6",
398 + "http-body 1.1.0",
399 + "http-body-util",
400 + "itoa",
401 + "num-integer",
402 + "pin-project-lite",
403 + "pin-utils",
404 + "ryu",
405 + "serde",
406 + "time",
407 + "tokio",
408 + "tokio-util",
409 + ]
410 +
411 + [[package]]
412 + name = "aws-smithy-xml"
413 + version = "0.62.0"
414 + source = "registry+https://github.com/rust-lang/crates.io-index"
415 + checksum = "ce84f71c72fee2cbbadde6e7d082f5fb466e3a84733855295fa7aafd1b31b7d8"
416 + dependencies = [
417 + "aws-smithy-runtime-api",
418 + "aws-smithy-schema",
419 + "aws-smithy-types",
420 + "xmlparser",
421 + ]
422 +
423 + [[package]]
424 + name = "aws-types"
425 + version = "1.5.0"
426 + source = "registry+https://github.com/rust-lang/crates.io-index"
427 + checksum = "eec1cd5469f328c782dc3e33d4153cf118a54e33cbb3356d60d16f89883e1f94"
428 + dependencies = [
429 + "aws-credential-types",
430 + "aws-smithy-async",
431 + "aws-smithy-runtime-api",
432 + "aws-smithy-schema",
433 + "aws-smithy-types",
434 + "rustc_version",
435 + "tracing",
436 + ]
437 +
438 + [[package]]
439 + name = "base16ct"
440 + version = "0.2.0"
441 + source = "registry+https://github.com/rust-lang/crates.io-index"
442 + checksum = "4c7f02d4ea65f2c1853089ffd8d2787bdbc63de2f0d29dedbcf8ccdfa0ccd4cf"
443 +
444 + [[package]]
445 + name = "base64"
446 + version = "0.22.1"
447 + source = "registry+https://github.com/rust-lang/crates.io-index"
448 + checksum = "72b3254f16251a8381aa12e40e3c4d2f0199f8c6508fbecb9d91f575e0fbb8c6"
449 +
450 + [[package]]
451 + name = "base64-simd"
452 + version = "0.8.0"
453 + source = "registry+https://github.com/rust-lang/crates.io-index"
454 + checksum = "339abbe78e73178762e23bea9dfd08e697eb3f3301cd4be981c0f78ba5859195"
455 + dependencies = [
456 + "outref",
457 + "vsimd",
458 + ]
459 +
460 + [[package]]
461 + name = "base64ct"
462 + version = "1.8.3"
463 + source = "registry+https://github.com/rust-lang/crates.io-index"
464 + checksum = "2af50177e190e07a26ab74f8b1efbfe2ef87da2116221318cb1c2e82baf7de06"
465 +
466 + [[package]]
467 + name = "bitflags"
468 + version = "2.13.1"
469 + source = "registry+https://github.com/rust-lang/crates.io-index"
470 + checksum = "b588b76d00fde79687d7646a9b5bdf3cc0f655e0bbd080335a95d7e96f3587da"
471 +
472 + [[package]]
473 + name = "block-buffer"
474 + version = "0.10.4"
475 + source = "registry+https://github.com/rust-lang/crates.io-index"
476 + checksum = "3078c7629b62d3f0439517fa394996acacc5cbc91c5a20d8c658e77abd503a71"
477 + dependencies = [
478 + "generic-array",
479 + ]
480 +
481 + [[package]]
482 + name = "block-buffer"
483 + version = "0.12.1"
484 + source = "registry+https://github.com/rust-lang/crates.io-index"
485 + checksum = "d2f6c7dbe95a6ed67ad9f18e57daf93a2f034c524b99fd2b76d18fdfeb6660aa"
486 + dependencies = [
487 + "hybrid-array",
488 + ]
489 +
490 + [[package]]
491 + name = "bumpalo"
492 + version = "3.20.3"
493 + source = "registry+https://github.com/rust-lang/crates.io-index"
494 + checksum = "72f5acc6cb2ba439de613abc23857ec3d78374d8ed5ac84e9d11336e87da8649"
495 +
496 + [[package]]
497 + name = "bytes"
498 + version = "1.12.1"
499 + source = "registry+https://github.com/rust-lang/crates.io-index"
500 + checksum = "fc652a48c352aef3ea3aed32080501cf3ef6ed5da78602a020c991775b0aff04"
Lines truncated
@@ -1,0 +1,21 @@
1 + [package]
2 + name = "s3-storage-fuzz"
3 + version = "0.0.0"
4 + publish = false
5 + edition = "2024"
6 +
7 + [package.metadata]
8 + cargo-fuzz = true
9 +
10 + [dependencies]
11 + libfuzzer-sys = { version = "0.4", features = ["arbitrary-derive"] }
12 +
13 + [dependencies.s3-storage]
14 + path = ".."
15 +
16 + [[bin]]
17 + name = "plan"
18 + path = "fuzz_targets/plan.rs"
19 + test = false
20 + doc = false
21 + bench = false
@@ -1,0 +1,49 @@
1 + //! Structured fuzz over multipart part geometry.
2 + //!
3 + //! Row 7 of `astra-soak-overview` (upload + object path), and the fourth
4 + //! target built there. The doors were counted before this was
5 + //! written, as infra `8910f917` instructs, and the count decided the shape:
6 + //! almost all of `s3-storage` is async I/O against a live endpoint and is not
7 + //! fuzzable without one, while `MultipartPlan` is pure arithmetic with four
8 + //! call sites in MNW server and one implementation behind them.
9 + //!
10 + //! `MultipartPlan`'s docs say the client computes the same boundaries
11 + //! independently, which reads like the two-parsers shape that made a shared
12 + //! crate the answer for `git_ssh`. It is not: SyncKit's check is a one-line
13 + //! restatement (`size_bytes.div_ceil(part_size) == part_count`), so a
14 + //! differential would measure nothing the property does not. The property is
15 + //! asserted instead, and it is the first thing `oracle::check_plan` checks.
16 + //!
17 + //! ## Why this path is worth a target
18 + //!
19 + //! It is what creator media travels through, and it fails in two directions.
20 + //! Parts that do not tile the object exactly mean corrupted or truncated media,
21 + //! invisible until playback. A plan refused when it should not be means a
22 + //! legitimate upload rejected. The oracle asserts both, which is why it checks
23 + //! the error paths as tightly as the success path.
24 +
25 + #![no_main]
26 +
27 + use libfuzzer_sys::fuzz_target;
28 +
29 + /// S3's own bounds, restated so the shaped call below always lands inside them.
30 + const MIN_PART: u64 = 5 * 1024 * 1024;
31 + const MAX_PART: u64 = 5 * 1024 * 1024 * 1024;
32 +
33 + fuzz_target!(|input: (u64, u64)| {
34 + let (total_size, raw_part) = input;
35 +
36 + // The raw pair, which is mostly refusals. Worth running: "every rejection
37 + // names a condition that actually holds" is half of what the oracle
38 + // asserts, and it is the half that protects legitimate uploads.
39 + s3_storage::oracle::check_plan(total_size, raw_part as usize);
40 +
41 + // A part size guaranteed inside S3's window, so the success path is
42 + // exercised on every input rather than only when random bytes happen to
43 + // land in a range that is a vanishing fraction of u64.
44 + let shaped = MIN_PART + (raw_part % (MAX_PART - MIN_PART + 1));
45 + s3_storage::oracle::check_plan(total_size, shaped as usize);
46 +
47 + // The entry point the four real call sites actually use.
48 + s3_storage::oracle::check_auto(total_size);
49 + });
@@ -1,0 +1,10 @@
1 + # Regressions
2 +
3 + One file per input that once found a bug. `tests/regressions.rs` replays every
4 + file here on every `cargo test`, so a fix that comes undone fails immediately
5 + and locally.
6 +
7 + Nothing here is ever deleted. Name a file for the bug it found, not for its
8 + contents.
9 +
10 + Empty is the good state.
@@ -1,0 +1,14 @@
1 + # Seeds
2 +
3 + Curated, read-only, and small: the sizes where part arithmetic goes wrong.
4 + Boundaries (the 5 MiB floor, the 5 GiB part ceiling, the 5 TiB object ceiling,
5 + exactly 10,000 parts), the off-by-ones on either side of each, and the two
6 + shapes that divide differently -- an object that divides evenly into parts and
7 + one that leaves a remainder.
8 +
9 + They are NOT the corpus. Run the fuzzer with the corpus first and these second:
10 +
11 + cargo +nightly fuzz run plan fuzz/corpus/plan fuzz/seeds/plan -- -max_total_time=1800
12 +
13 + libFuzzer writes new inputs into the FIRST directory only, so this arrangement
14 + keeps the curated set curated and lets `fuzz/corpus/` grow.
Binary file
Binary file
Binary file
Binary file
Binary file
Binary file
Binary file
Binary file
Binary file
Binary file
Binary file
Binary file
Binary file
Binary file
Binary file