Skip to main content

max / makenotwork

Attribute a bounce or complaint to the send, list and creator behind it 93f23f00, the prerequisite for ede7acec. email_suppressions records (email, reason) and is keyed on the address alone: it says an address complained and cannot say what it complained about. So no complaint rate was computable at any granularity, and complaint rate is what a mail provider actually judges an account on -- the number that predicts reputation damage before the shared Postmark IP pool feels it. Postmark hands back the Metadata a message was sent with, so the id goes out on the mail and comes back on the complaint. One key, `send`: everything else about a fan-out is on the row it names, and a second copy in the metadata is a second thing that can disagree. BOTH HALVES, because a rate has two. Nothing recorded how much mail a list send put on the wire. creator_mail_usage counts what a creator sent, which answers the cap's question and not this one: it is a running counter per billing period, so it cannot be windowed to a fortnight and knows nothing about lists. email_sends is the denominator, written by all four fan-outs -- the two announcements, both broadcast handlers -- beside the reservation that already gates each of them. email_incidents is its own table rather than columns on email_suppressions, and the reason is that table's shape: it is unique on the address and written ON CONFLICT DO NOTHING, so a second complaint from an already-suppressed address records nothing. Right for a suppression list, wrong for a rate. Recorded at the finest grain and the other two derived, settled 2026-08-30 rather than asked: a coarser grain forecloses per-list and per-creator and saves nothing, since the attribution work is identical either way. Bounces ride along for free and are worth keeping -- bounce rate is the other half of what a provider judges on. Three failures, three deliberate directions. An unattributable complaint is still counted, because dropping it would flatter the rate. A send row that fails to write lets the mail go anyway and costs a denominator, which reads the rate high. And the incident write cannot return an error to Postmark: redelivery re-runs the idempotent suppression and then this, which is not, so a retry would count one complaint twice. Fanout groups the two per-send facts the three broadcast senders now take. They were the sixth and seventh parameters of senders that already had five, and a third would have been the eighth.
Author: Max Johnson <me@maxj.phd> · 2026-08-30 23:03 UTC
Signed with PGP, not checked
Commit: 59af29f3eeff72b8ce974755b4829f6ac83ec5d0
Parent: 8d4f291
12 files changed, +621 insertions, -29 deletions
M server/Cargo.lock +8 -8
@@ -10771,6 +10771,14 @@
10771 10771 name = "painhours"
10772 10772 version = "0.1.0"
10773 10773
10774 + [[patch.unused]]
10775 + name = "synckit-client"
10776 + version = "0.10.0"
10777 +
10778 + [[patch.unused]]
10779 + name = "synckit-config"
10780 + version = "0.2.0"
10781 +
10774 10782 [[patch.unused]]
10775 10783 name = "quasi-immediate"
10776 10784 version = "0.91.1"
@@ -10786,11 +10794,3 @@
10786 10794 [[patch.unused]]
10787 10795 name = "quasi-tauri"
10788 10796 version = "0.91.1"
10789 -
10790 - [[patch.unused]]
10791 - name = "synckit-client"
10792 - version = "0.10.0"
10793 -
10794 - [[patch.unused]]
10795 - name = "synckit-config"
10796 - version = "0.2.0"
M server/src/db/id_types.rs +1
@@ -193,6 +193,7 @@
193 193 ModerationActionId,
194 194 MtThreadId,
195 195 PendingAcknowledgementId,
196 + EmailSendId,
196 197 ClaimToken,
197 198 DownloadToken,
198 199 );
M server/src/db/mod.rs +1
@@ -40,6 +40,7 @@
40 40 pub mod items;
41 41 pub mod license_keys;
42 42 pub mod lists;
43 + pub mod mail_attribution; // pub so the integration test crate can drive the rate queries directly
43 44 pub mod mail_caps; // pub so the integration test crate can drive reserve/grant against a live pool
44 45 pub mod mailing_lists;
45 46 pub(crate) mod media_files;
M server/src/email/mod.rs +68 -9
@@ -122,13 +122,19 @@
122 122 unsub_url: Option<&str>,
123 123 ) -> Result<()>;
124 124
125 - /// Send via the broadcast stream with an optional unsubscribe link.
125 + /// Send via the broadcast stream with an optional unsubscribe link, and the
126 + /// fan-out this mail belongs to.
127 + ///
128 + /// On this method alone, because it is the only mail that belongs to one:
129 + /// a transactional send has no list, no fan-out and nothing to attribute a
130 + /// complaint about it to.
126 131 async fn send_email_broadcast_with_unsub(
127 132 &self,
128 133 to: &str,
129 134 subject: &str,
130 135 body: &str,
131 136 unsub_url: Option<&str>,
137 + send: Option<crate::db::EmailSendId>,
132 138 ) -> Result<()>;
133 139 }
134 140
@@ -234,6 +240,30 @@
234 240 pub headers: &'a [(&'a str, String)],
235 241 /// Send on Postmark's broadcast stream rather than the transactional one.
236 242 pub broadcast: bool,
243 + /// The fan-out this mail belongs to, for the provider to hand back when the
244 + /// address bounces or complains.
245 + ///
246 + /// `93f23f00`. Without it a complaint names an address and nothing else,
247 + /// and no complaint rate is computable at any granularity. `None` for every
248 + /// transactional send: those belong to no fan-out, and an id invented for
249 + /// one would attribute a receipt to a list.
250 + pub send: Option<crate::db::EmailSendId>,
251 + }
252 +
253 + /// What a fan-out's mail carries beyond its words.
254 + ///
255 + /// Two facts about the send rather than about the message, and they arrive
256 + /// together at every broadcast sender: where this recipient unsubscribes, and
257 + /// which fan-out this is. Grouped because they were the sixth and seventh
258 + /// parameters of three senders that already took five, and because a third
259 + /// per-send fact would have been the eighth.
260 + #[derive(Debug, Clone, Copy, Default)]
261 + pub struct Fanout<'a> {
262 + /// This recipient's one-click unsubscribe link.
263 + pub unsub_url: Option<&'a str>,
264 + /// The row in `email_sends` this mail belongs to, so a complaint about it
265 + /// can name the list and the creator (`93f23f00`).
266 + pub send: Option<crate::db::EmailSendId>,
237 267 }
238 268
239 269 /// Email client for sending emails
@@ -304,7 +334,13 @@
304 334 let to = audience.email();
305 335 if delivery.broadcast {
306 336 self.transport
307 - .send_email_broadcast_with_unsub(to, subject, body, delivery.unsub_url)
337 + .send_email_broadcast_with_unsub(
338 + to,
339 + subject,
340 + body,
341 + delivery.unsub_url,
342 + delivery.send,
343 + )
308 344 .await
309 345 } else if delivery.headers.is_empty() && delivery.unsub_url.is_none() {
310 346 self.transport.send_email(to, subject, body).await
@@ -381,6 +417,7 @@
381 417 body: &str,
382 418 unsub_url: Option<&str>,
383 419 stream: Option<&str>,
420 + send: Option<crate::db::EmailSendId>,
384 421 ) -> Result<()> {
385 422 match unsub_url {
386 423 Some(url) => {
@@ -392,10 +429,13 @@
392 429 "List-Unsubscribe=One-Click".to_string(),
393 430 ),
394 431 ];
395 - self.send_email_inner(to, subject, &body_with_footer, &headers, stream)
432 + self.send_email_inner(to, subject, &body_with_footer, &headers, stream, send)
433 + .await
434 + }
435 + None => {
436 + self.send_email_inner(to, subject, body, &[], stream, send)
396 437 .await
397 438 }
398 - None => self.send_email_inner(to, subject, body, &[], stream).await,
399 439 }
400 440 }
401 441
@@ -407,6 +447,7 @@
407 447 body: &str,
408 448 extra_headers: &[(&str, String)],
409 449 stream: Option<&str>,
450 + send: Option<crate::db::EmailSendId>,
410 451 ) -> Result<()> {
411 452 // Check suppression list before sending
412 453 if let Some(ref pool) = self.pool {
@@ -424,7 +465,7 @@
424 465 }
425 466
426 467 if let Some(ref token) = self.config.postmark_token {
427 - self.send_via_postmark(token, to, subject, body, extra_headers, stream)
468 + self.send_via_postmark(token, to, subject, body, extra_headers, stream, send)
428 469 .await
429 470 } else {
430 471 tracing::info!(
@@ -436,6 +477,12 @@
436 477 }
437 478
438 479 /// Send email via Postmark API
480 + #[allow(
481 + clippy::too_many_arguments,
482 + reason = "every parameter is a distinct field of one Postmark request, \
483 + and a struct holding exactly the arguments of one private \
484 + call site names nothing"
485 + )]
439 486 async fn send_via_postmark(
440 487 &self,
441 488 token: &str,
@@ -444,6 +491,7 @@
444 491 body: &str,
445 492 extra_headers: &[(&str, String)],
446 493 stream: Option<&str>,
494 + send: Option<crate::db::EmailSendId>,
447 495 ) -> Result<()> {
448 496 let from = format!("{} <{}>", self.config.from_name, self.config.from_address);
449 497
@@ -458,6 +506,15 @@
458 506 payload["MessageStream"] = serde_json::Value::String(stream_id.to_string());
459 507 }
460 508
509 + // Postmark hands `Metadata` back on the bounce and complaint webhooks,
510 + // which is the whole mechanism: the id goes out with the mail and comes
511 + // back with the complaint. One key, because everything else about the
512 + // send -- its list, its creator -- is on the row this names, and a
513 + // second copy in the metadata is a second thing that can disagree.
514 + if let Some(send) = send {
515 + payload["Metadata"] = serde_json::json!({ "send": send.to_string() });
516 + }
517 +
461 518 if !extra_headers.is_empty() {
462 519 let headers: Vec<serde_json::Value> = extra_headers
463 520 .iter()
@@ -526,7 +583,8 @@
526 583 #[async_trait::async_trait]
527 584 impl EmailTransport for PostmarkTransport {
528 585 async fn send_email(&self, to: &str, subject: &str, body: &str) -> Result<()> {
529 - self.send_email_inner(to, subject, body, &[], None).await
586 + self.send_email_inner(to, subject, body, &[], None, None)
587 + .await
530 588 }
531 589
532 590 async fn send_email_with_headers_and_unsub(
@@ -546,11 +604,11 @@
546 604 "List-Unsubscribe-Post",
547 605 "List-Unsubscribe=One-Click".to_string(),
548 606 ));
549 - self.send_email_inner(to, subject, &body_with_footer, &all_headers, None)
607 + self.send_email_inner(to, subject, &body_with_footer, &all_headers, None, None)
550 608 .await
551 609 }
552 610 None => {
553 - self.send_email_inner(to, subject, body, extra_headers, None)
611 + self.send_email_inner(to, subject, body, extra_headers, None, None)
554 612 .await
555 613 }
556 614 }
@@ -562,8 +620,9 @@
562 620 subject: &str,
563 621 body: &str,
564 622 unsub_url: Option<&str>,
623 + send: Option<crate::db::EmailSendId>,
565 624 ) -> Result<()> {
566 - self.send_with_unsub_inner(to, subject, body, unsub_url, Some("broadcast"))
625 + self.send_with_unsub_inner(to, subject, body, unsub_url, Some("broadcast"), send)
567 626 .await
568 627 }
569 628 }
M server/src/scheduler/announcements.rs +43 -3
@@ -5,7 +5,7 @@
5 5 use crate::config::Config;
6 6 use crate::db;
7 7 use crate::db::mail_caps::Verdict;
8 - use crate::db::{DbBlogPost, DbItem, DbUser};
8 + use crate::db::{DbBlogPost, DbItem, DbUser, ListId};
9 9 use crate::email::EmailClient;
10 10
11 11 /// Build the mailing-list unsubscribe URL for one subscriber: user-keyed for an
@@ -61,6 +61,38 @@
61 61 });
62 62 }
63 63
64 + /// Record the fan-out about to happen, so a complaint about it can be traced
65 + /// back to this creator and this list.
66 + ///
67 + /// `93f23f00`. `None` when the row could not be written, and the mail still
68 + /// goes: an announcement is worth more than its attribution, and the failure
69 + /// costs a denominator rather than a send. It shows up as a rate computed over
70 + /// slightly less than was really sent, which reads high -- the safe direction
71 + /// for a number that exists to warn.
72 + async fn attributed(
73 + db: &PgPool,
74 + creator: &DbUser,
75 + list_id: ListId,
76 + recipients: usize,
77 + ) -> Option<db::EmailSendId> {
78 + match db::mail_attribution::record_send(
79 + db,
80 + creator.id,
81 + Some(list_id),
82 + db::mail_attribution::SendKind::Announcement,
83 + i64::try_from(recipients).unwrap_or(i64::MAX),
84 + )
85 + .await
86 + {
87 + Ok(id) => Some(id),
88 + Err(error) => {
89 + tracing::warn!(error = ?error, creator_id = %creator.id,
90 + "could not record the send; this fan-out will be unattributed");
91 + None
92 + }
93 + }
94 + }
95 +
64 96 /// Claim an announcement's recipients against the creator's monthly mail
65 97 /// allowance, or tell them why it did not go.
66 98 ///
@@ -195,6 +227,7 @@
195 227 return;
196 228 }
197 229
230 + let send = attributed(db, &creator, unified, subscribers.len()).await;
198 231 let creator_name = creator
199 232 .display_name
200 233 .as_deref()
@@ -222,7 +255,10 @@
222 255 &creator_name,
223 256 &item_title,
224 257 &item_url,
225 - Some(&unsub_url),
258 + crate::email::Fanout {
259 + unsub_url: Some(&unsub_url),
260 + send,
261 + },
226 262 )
227 263 .await
228 264 {
@@ -294,6 +330,7 @@
294 330 return;
295 331 }
296 332
333 + let send = attributed(db, &creator, unified, subscribers.len()).await;
297 334 let creator_name = creator
298 335 .display_name
299 336 .as_deref()
@@ -321,7 +358,10 @@
321 358 &creator_name,
322 359 &post_title,
323 360 &post_url,
324 - Some(&unsub_url),
361 + crate::email::Fanout {
362 + unsub_url: Some(&unsub_url),
363 + send,
364 + },
325 365 )
326 366 .await
327 367 {
M server/tests/harness/email.rs +11
@@ -17,6 +17,11 @@
17 17 pub body: String,
18 18 pub unsub_url: Option<String>,
19 19 pub stream: Option<String>,
20 + /// The fan-out this mail was attributed to (`93f23f00`). Recorded rather
21 + /// than ignored, because "the id went out with the mail" is the half of the
22 + /// attribution a test can see -- the other half is Postmark handing it
23 + /// back.
24 + pub send: Option<String>,
20 25 }
21 26
22 27 /// In-memory email transport that records all sent emails.
@@ -79,6 +84,8 @@
79 84 body: body.to_string(),
80 85 unsub_url: None,
81 86 stream: None,
87 + // Transactional mail belongs to no fan-out.
88 + send: None,
82 89 });
83 90 Ok(())
84 91 }
@@ -98,6 +105,8 @@
98 105 body: body.to_string(),
99 106 unsub_url: unsub_url.map(std::string::ToString::to_string),
100 107 stream: None,
108 + // Transactional mail belongs to no fan-out.
109 + send: None,
101 110 });
102 111 Ok(())
103 112 }
@@ -108,6 +117,7 @@
108 117 subject: &str,
109 118 body: &str,
110 119 unsub_url: Option<&str>,
120 + send: Option<makenotwork::db::EmailSendId>,
111 121 ) -> Result<()> {
112 122 self.faults.check("send_email")?;
113 123 self.sent.lock().unwrap().push(SentEmail {
@@ -116,6 +126,7 @@
116 126 body: body.to_string(),
117 127 unsub_url: unsub_url.map(std::string::ToString::to_string),
118 128 stream: Some("broadcast".to_string()),
129 + send: send.map(|id| id.to_string()),
119 130 });
120 131 Ok(())
121 132 }
M server/src/email/templates/notifications.rs +10 -6
@@ -90,7 +90,7 @@
90 90 creator_name: &str,
91 91 subject: &str,
92 92 body_text: &str,
93 - unsub_url: Option<&str>,
93 + fanout: crate::email::Fanout<'_>,
94 94 ) -> Result<()> {
95 95 let subject = format!("{subject}, from {creator_name}");
96 96 let body = format!(
@@ -113,8 +113,9 @@
113 113 &subject,
114 114 &body,
115 115 Delivery {
116 - unsub_url,
116 + unsub_url: fanout.unsub_url,
117 117 broadcast: true,
118 + send: fanout.send,
118 119 ..Default::default()
119 120 },
120 121 )
@@ -129,7 +130,7 @@
129 130 creator_name: &str,
130 131 item_title: &str,
131 132 item_url: &str,
132 - unsub_url: Option<&str>,
133 + fanout: crate::email::Fanout<'_>,
133 134 ) -> Result<()> {
134 135 let subject = format!("New release: {item_title} by {creator_name}");
135 136 let body = format!(
@@ -152,8 +153,9 @@
152 153 &subject,
153 154 &body,
154 155 Delivery {
155 - unsub_url,
156 + unsub_url: fanout.unsub_url,
156 157 broadcast: true,
158 + send: fanout.send,
157 159 ..Default::default()
158 160 },
159 161 )
@@ -168,7 +170,7 @@
168 170 creator_name: &str,
169 171 post_title: &str,
170 172 post_url: &str,
171 - unsub_url: Option<&str>,
173 + fanout: crate::email::Fanout<'_>,
172 174 ) -> Result<()> {
173 175 let subject = format!("New post: {post_title} by {creator_name}");
174 176 let body = format!(
@@ -191,8 +193,9 @@
191 193 &subject,
192 194 &body,
193 195 Delivery {
194 - unsub_url,
196 + unsub_url: fanout.unsub_url,
195 197 broadcast: true,
198 + send: fanout.send,
196 199 ..Default::default()
197 200 },
198 201 )
@@ -1014,6 +1017,7 @@
1014 1017 subject: &str,
1015 1018 body: &str,
1016 1019 unsub_url: Option<&str>,
1020 + _send: Option<crate::db::EmailSendId>,
1017 1021 ) -> Result<()> {
1018 1022 self.sent.lock().unwrap().push((
1019 1023 to.to_string(),
M server/src/routes/postmark/mod.rs +103 -1
M server/src/routes/api/internal/cli_features.rs +20 -1
M server/src/routes/api/users/broadcast.rs +23 -1
A server/migrations/200_mail_attribution.sql +70
A server/src/db/mail_attribution.rs +263