Skip to main content

max / makenotwork

Lower the browser upload cap to 2 GiB The browser path issues one presigned PUT with no resume, so a multi-GB upload that drops at 90% starts over. S3 tolerates 5 GiB, but that is the protocol limit, not a limit anyone should have to re-send against. The CLI and desktop clients chunk and resume, so the advertised 20 GB tier still has a working path and is unaffected. Renames S3_SINGLE_PUT_MAX_BYTES to BROWSER_UPLOAD_MAX_BYTES: the number no longer describes what S3 accepts, and reusing it as one would be wrong. Adds a const assert that it stays under the single-copy ceiling, so a browser upload is always promotable by one server-side copy. Also fixes the stale SCAN_SPOOL_MAX_BYTES comment, which claimed 8 GiB matched the largest payload the upload tier allows. The tier allows 20 GB; objects past the spool ceiling are held for manual admin review, which is fail-closed and deliberate.
Co-Authored-By
Claude Opus 5 (1M context) <noreply@anthropic.com>
Author: Max Johnson <me@maxj.phd> · 2026-07-28 01:53 UTC
Signed with PGP, not checked
Commit: d0d38a863950198ea026ca0dcc15a021ba2bde57
Parent: bc97711
5 files changed, +46 insertions, -36 deletions
M server/src/constants.rs +23 -15
@@ -25,22 +25,24 @@
25 25 /// indefinitely, the most common way the pool wedges.
26 26 pub const DB_LOCK_TIMEOUT_SECS: u64 = 30;
27 27
28 - /// Largest object a single presigned `PutObject` can carry. S3 / Ceph (Hetzner
29 - /// Object Storage) reject a single PUT above 5 GiB, larger objects require
30 - /// multipart. The browser upload paths issue exactly one presigned PUT, so this
31 - /// bounds them regardless of the (higher) per-tier `max_file_bytes`; files above
32 - /// it upload through the CLI / desktop clients, which chunk them. Keeping the
33 - /// browser cap here means a too-big browser upload is refused up front with a
34 - /// clear pointer rather than handed a presigned URL that fails at S3.
35 - pub const S3_SINGLE_PUT_MAX_BYTES: u64 = 5 * 1024 * 1024 * 1024;
28 + /// Largest file the browser upload path will accept. The browser issues exactly
29 + /// one presigned `PutObject` and a tab cannot resume it, so a transfer that
30 + /// drops at 90% starts over from zero. That is the binding constraint, not the
31 + /// protocol: S3 / Ceph (Hetzner Object Storage) tolerate a single PUT up to
32 + /// 5 GiB, but 2 GiB is as much as we are willing to ask someone to re-send.
33 + /// Files above it upload through the CLI / desktop clients, which chunk and
34 + /// resume, and the (higher) per-tier `max_file_bytes` still governs there, so
35 + /// the advertised 20 GB tier is unaffected. Refusing up front with a pointer to
36 + /// those clients beats handing out a presigned URL for a doomed transfer.
37 + pub const BROWSER_UPLOAD_MAX_BYTES: u64 = 2 * 1024 * 1024 * 1024;
36 38
37 39 /// Largest source a single server-side `CopyObject` can promote. S3 rejects a
38 40 /// one-shot copy above 5 GiB; a larger source must go through ranged multipart
39 - /// `UploadPartCopy`. Numerically equal to [`S3_SINGLE_PUT_MAX_BYTES`] but a
40 - /// different limit, that one bounds a client PUT, this one bounds a server-side
41 - /// copy, so they are named separately and neither should be reused for the
42 - /// other. Without this branch a large upload succeeds and then fails at promote,
43 - /// the worst position to fail in.
41 + /// `UploadPartCopy`. This is a protocol limit, where
42 + /// [`BROWSER_UPLOAD_MAX_BYTES`] is a product decision about resumability, so the
43 + /// two are separate constants and neither should be reused for the other.
44 + /// Without this branch a large upload succeeds and then fails at promote, the
45 + /// worst position to fail in.
44 46 pub const S3_SINGLE_COPY_MAX_BYTES: u64 = 5 * 1024 * 1024 * 1024;
45 47
46 48 /// Lifetime of an internal-API actor assertion, minted at `ssh-key-lookup` and
@@ -271,8 +273,11 @@
271 273 /// covers process-death.
272 274 pub const SCAN_SPOOL_ORPHAN_AGE_SECS: u64 = 3600;
273 275 /// Hard cap on a single spooled object. Above this, the scanner refuses
274 - /// the job rather than risk filling the volume. 8 GiB matches the largest
275 - /// payload the upload tier currently allows.
276 + /// the job rather than risk filling the volume. It sits below the 20 GB the
277 + /// upload tiers allow, deliberately: objects past this ceiling are held for
278 + /// manual admin review (`scanning::worker`) instead of being auto-scanned,
279 + /// which is fail-closed and cheap at alpha volume. Raising it to cover the full
280 + /// tier would cost spool headroom and add scan latency on every large file.
276 281 pub const SCAN_SPOOL_MAX_BYTES: u64 = 8 * 1024 * 1024 * 1024;
277 282 /// Minimum free space the spool volume must retain after writing the
278 283 /// pending object. The scanner refuses if `statvfs(free) - expected_size`
@@ -513,6 +518,9 @@
513 518 const _: () = assert!(SCAN_SPOOL_FREE_RESERVE_BYTES < SCAN_SPOOL_MAX_BYTES);
514 519 const _: () = assert!(SCAN_SPOOL_MAX_BYTES > SCAN_MAX_MEMORY_BYTES as u64);
515 520 const _: () = assert!(SCAN_JOB_RETENTION_DAYS >= 7); // no same-day purge race
521 + // A browser upload must always be promotable by a single server-side copy, so
522 + // the browser ceiling can never be raised past what `CopyObject` accepts.
523 + const _: () = assert!(BROWSER_UPLOAD_MAX_BYTES <= S3_SINGLE_COPY_MAX_BYTES);
516 524 // The concurrency ceiling must not sit below the worker count, or the memory
517 525 // budget it's meant to enforce is unenforceable (workers would exceed it).
518 526 const _: () = assert!(SCAN_MAX_CONCURRENT >= SCAN_WORKER_COUNT);
M server/src/storage.rs +7 -6
@@ -1710,17 +1710,18 @@
1710 1710 }
1711 1711
1712 1712 #[test]
1713 - fn single_copy_ceiling_matches_s3_and_is_distinct_from_the_put_ceiling() {
1714 - // Same number today, different limits: one bounds a client PUT, the
1715 - // other a server-side copy. Pinned so a future change to one doesn't
1716 - // silently move the other (they are deliberately separate constants).
1713 + fn single_copy_ceiling_matches_s3_and_is_distinct_from_the_browser_ceiling() {
1714 + // Different limits, and since the browser cap dropped to 2 GiB, different
1715 + // numbers too: the copy ceiling is what S3 enforces on a one-shot
1716 + // `CopyObject`, the browser cap is a product call about resumability.
1717 + // Pinned so a future change to one doesn't silently move the other.
1717 1718 assert_eq!(
1718 1719 crate::constants::S3_SINGLE_COPY_MAX_BYTES,
1719 1720 5 * 1024 * 1024 * 1024
1720 1721 );
1721 1722 assert_eq!(
1722 - crate::constants::S3_SINGLE_PUT_MAX_BYTES,
1723 - 5 * 1024 * 1024 * 1024
1723 + crate::constants::BROWSER_UPLOAD_MAX_BYTES,
1724 + 2 * 1024 * 1024 * 1024
1724 1725 );
1725 1726 }
1726 1727
M server/tests/workflows/storage.rs +2 -2
@@ -1907,8 +1907,8 @@
1907 1907 }
1908 1908
1909 1909 #[tokio::test]
1910 - async fn multipart_start_opens_the_band_above_the_single_put_ceiling() {
1911 - // 6 GiB is past S3's 5 GiB single-PUT ceiling and inside the big_files tier's
1910 + async fn multipart_start_opens_the_band_above_the_browser_ceiling() {
1911 + // 6 GiB is past the 2 GiB browser ceiling and inside the big_files tier's
1912 1912 // 20 GB per-file cap. Before multipart this band was unreachable by every
1913 1913 // path; this test is the regression guard that it stays open.
1914 1914 let mut h = cli_harness().await;
M server/src/routes/storage/mod.rs +13 -12
@@ -45,15 +45,16 @@
45 45 return Ok(());
46 46 };
47 47 // Browser transport ceiling: every caller of THIS function issues a single
48 - // presigned PUT, which S3/Ceph rejects above 5 GiB. The tier cap can be
49 - // higher (BigFiles/Everything allow 20 GB), those files upload through the
50 - // CLI/desktop clients, which chunk them, so point a too-big browser upload
51 - // there instead of handing out a presigned URL that would fail at S3.
48 + // presigned PUT, which a tab cannot resume. The tier cap can be much higher
49 + // (BigFiles/Everything allow 20 GB), those files upload through the
50 + // CLI/desktop clients, which chunk and resume, so point a too-big browser
51 + // upload there instead of handing out a presigned URL for a transfer that
52 + // will strand the person if it drops.
52 53 // Deliberately NOT in `validate_declared_upload_size_limits`: it is a
53 54 // property of the one-shot transport, not of the file, and the multipart
54 55 // path is exactly what this message points people toward.
55 - if size as u64 > constants::S3_SINGLE_PUT_MAX_BYTES {
56 - let limit_gb = constants::S3_SINGLE_PUT_MAX_BYTES / (1024 * 1024 * 1024);
56 + if size as u64 > constants::BROWSER_UPLOAD_MAX_BYTES {
57 + let limit_gb = constants::BROWSER_UPLOAD_MAX_BYTES / (1024 * 1024 * 1024);
57 58 return Err(AppError::FileTooLarge(format!(
58 59 "Files larger than {limit_gb} GB must be uploaded with the makenot.work CLI or desktop app."
59 60 )));
@@ -594,11 +595,11 @@
594 595 const GIB: i64 = 1024 * 1024 * 1024;
595 596
596 597 #[test]
597 - fn browser_validator_refuses_above_the_single_put_ceiling() {
598 + fn browser_validator_refuses_above_the_browser_ceiling() {
598 599 // 10 GiB video: within the 20 GB per-type cap, but a browser issues one
599 - // presigned PUT and S3 rejects that above 5 GiB.
600 + // unresumable presigned PUT, so it is refused above 2 GiB.
600 601 let err = validate_declared_upload_size(Some(10 * GIB), FileType::Video, None)
601 - .expect_err("browser upload above 5 GiB must be refused");
602 + .expect_err("browser upload above the browser ceiling must be refused");
602 603 assert!(
603 604 matches!(err, AppError::FileTooLarge(ref m) if m.contains("CLI or desktop app")),
604 605 "expected a pointer to the CLI, got: {err:?}"
@@ -606,12 +607,12 @@
606 607 }
607 608
608 609 #[test]
609 - fn multipart_validator_allows_above_the_single_put_ceiling() {
610 + fn multipart_validator_allows_above_the_browser_ceiling() {
610 611 // The same file over the chunked CLI path is fine, exceeding the
611 612 // one-shot ceiling is the entire point of multipart. If this ever starts
612 - // failing, the 5 GiB-to-20 GB tier band is unreachable again.
613 + // failing, the 2 GiB-to-20 GB tier band is unreachable again.
613 614 validate_declared_upload_size_limits(Some(10 * GIB), FileType::Video, None)
614 - .expect("multipart upload above 5 GiB must be allowed");
615 + .expect("multipart upload above the browser ceiling must be allowed");
615 616 }
616 617
617 618 #[test]
M server/src/routes/api/internal/uploads.rs +1 -1
@@ -98,7 +98,7 @@
98 98 //
99 99 // The chunked counterpart to `presign_upload`, and deliberately only on the
100 100 // internal (CLI/desktop) surface: a browser keeps the one-shot presigned PUT and
101 - // its 5 GiB ceiling, because a tab cannot resume a multi-hour transfer. These
101 + // its 2 GiB ceiling, because a tab cannot resume a multi-hour transfer. These
102 102 // three endpoints replace the *transport* only, the client finishes by calling
103 103 // the existing `/api/internal/upload/confirm`, which reads the authoritative
104 104 // object size from S3 and does all the size/tier/scan/DB work unchanged.