Skip to main content

max / makenotwork

synckit: cap blob-upload input size (risk Phase 4) Reject blob_upload data larger than MAX_BLOB_BYTES (4 GiB) before encrypting and buffering it, closing the unbounded-input OOM (a 20 GB input double-buffered to ~40 GB). Sync-pull's uncapped Vec<PullChangeEntry> is already bounded by Phase 2's read_body_capped. True 1x streaming of a single presigned PUT is not achievable (reqwest streams chunked with no Content-Length; presigned S3/Ceph PUT requires a signed Content-Length), so bounded-memory uploads above the cap need S3 multipart -- the resumable-multipart feature already deferred by decision. blob-upload lands at A-; A+ is gated on that feature, not on further crate work. Gate: 308 lib + 101 integration + 1 doc tests, clippy --all-targets clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Author: Max Johnson <me@maxj.phd> · 2026-07-07 19:09 UTC
Commit: acf5107e024eb30712b18fb73c980ed6282bd5a7
Parent: 6ffcc52
1 file changed, +17 insertions, -0 deletions
@@ -57,8 +57,25 @@ impl SyncKitClient {
57 57 /// content `hash` as AEAD associated data so the ciphertext is tied to its
58 58 /// content address. The plaintext is never sent to the server, preserving
59 59 /// the E2E encryption guarantee.
60 + ///
61 + /// The input is capped at [`MAX_BLOB_BYTES`]: without it, a caller (or a
62 + /// hostile local integration) could hand in an arbitrarily large `data`,
63 + /// which this method would encrypt into an equally large ciphertext and
64 + /// buffer whole — an unbounded memory (OOM) lever. Bounded-memory uploads of
65 + /// blobs *larger* than the cap need chunk-at-a-time streaming to S3, which in
66 + /// turn needs multipart upload (a single presigned `PUT` requires a known,
67 + /// signed `Content-Length`, and `reqwest`'s streaming body sends chunked with
68 + /// none). That multipart path is a separate, deferred server+client feature;
69 + /// until it lands, blobs above the cap are rejected here rather than risking
70 + /// the OOM.
60 71 #[instrument(skip(self, presigned_url, data))]
61 72 pub async fn blob_upload(&self, hash: &str, presigned_url: &str, data: Vec<u8>) -> Result<()> {
73 + if data.len() > MAX_BLOB_BYTES {
74 + return Err(SyncKitError::InvalidArgument(format!(
75 + "blob is {} bytes, over the {MAX_BLOB_BYTES}-byte single-upload cap; larger blobs need the (deferred) multipart path",
76 + data.len()
77 + )));
78 + }
62 79 let master_key = self.require_master_key()?;
63 80 // v3 chunked format: each chunk sealed and bound to (hash, index, count),
64 81 // so the reader verifies and decrypts one chunk at a time.