max / docengine

Strip historical narrative from documentation Remove what a doc used to say, when it changed, the incidents that justified a rule, finished migration narration, and counts and versions that rot. State the rules in the present tense instead. Keep every instruction, prohibition and threshold, and keep the measurements that make a rule actionable. Public-facing docs keep their explanatory voice.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MptwXZ8k65v19rFmdGAyki

Author: Max Johnson <me@maxj.phd> · 2026-08-31 01:51 UTC

Signed with PGP, not checked

Commit: 871b176785f9ec15a673467ada795f2f4fae77a2

Parent: 810f04c

8 files changed, +53 insertions, -80 deletions

M README.md +12 -15

			@@ -4,10 +4,9 @@
4	4
5	5		Used by MNW (site docs, blog posts, user-generated content), Multithreaded (forum posts), and the desktop apps (descriptions, notes).
6	6
7		-	Extracted from the MNW monorepo (2026-07-30) so it can be consumed from a
8		-	container build, where a cross-repo path dependency is not reachable. Internal:
9		-	not published to crates.io. Consumed by path in-tree, and by git dependency from
10		-	anything built in a container.
	7	+	Internal: not published to crates.io. Consumed by path in-tree, and by git
	8	+	dependency from anything built in a container, where a cross-repo path dependency
	9	+	is not reachable.
11	10
12	11		## Presets
13	12
			@@ -121,12 +120,11 @@
121	120
122	121		## Value substitution
123	122
124		-	`{{ dotted.path \| filter(args) }}` substitution used to be a docengine feature. It moved
125		-	out on 2026-07-25 into two crates: `subst` (the generic engine) and
126		-	`mnw-assumptions` (the MNW business-model layer on top), both in the MNW tree. Nothing
127		-	about the render path changed -- the server builds an `Assumptions` at boot and hands its
128		-	`substitute` to `DocLoaderConfig::pre_process`, which is the same hook any other
129		-	pre-render text transform would use.
	123	+	`{{ dotted.path \| filter(args) }}` substitution is not a docengine feature. It lives in
	124	+	`subst` (the generic engine) and `mnw-assumptions` (the MNW business-model layer on top),
	125	+	both in the MNW tree. The seam is `DocLoaderConfig::pre_process`: MNW's server builds an
	126	+	`Assumptions` at boot and hands over its `substitute`. Any other pre-render text transform
	127	+	plugs into the same hook.
130	128
131	129		## Consumers
132	130
			@@ -158,8 +156,7 @@
158	156
159	157		MIT. See [LICENSE](LICENSE).
160	158
161		-	Permissive on purpose, and not the license the products carry: MNW,
162		-	Multithreaded and SyncKit are PolyForm Noncommercial, while the libraries meant
163		-	for reuse are MIT. That split is what lets a GPL consumer link this crate at all,
164		-	since PolyForm's noncommercial term is an added restriction GPL section 7
165		-	forbids. This file said PolyForm until 2026-07-30, which was wrong.
	159	+	Permissive on purpose. The products carry a different license: MNW and
	160	+	Multithreaded are PolyForm Noncommercial, while the libraries meant for reuse are
	161	+	MIT. That split is what lets a GPL consumer link this crate at all, since
	162	+	PolyForm's noncommercial term is an added restriction GPL section 7 forbids.

M benches/render.rs +1 -1

			@@ -2,7 +2,7 @@
2	2		//!
3	3		//! docengine sits on the MNW per-request render path (creator descriptions,
4	4		//! forum posts) and the startup doc-load path (site-docs). These establish a
5		-	//! baseline to measure any perf refactor against — parser reuse, `Arc<str>`
	5	+	//! baseline to measure a perf refactor against: parser reuse, `Arc<str>`
6	6		//! sharing, or a directive pass that avoids re-scanning the whole HTML.
7	7		//!
8	8		//! Run with: `cargo bench --features full`.

M docs/architecture.md +7 -8

			@@ -55,15 +55,14 @@
55	55
56	56		### Value substitution is a consumer concern, not a docengine feature
57	57
58		-	`{{ dotted.path \| filter(args) }}` substitution shipped as a docengine feature until
59		-	2026-07-25. It was a business-model calculator with a templater attached, and the parts a
60		-	doc engine actually needs from it are zero. It now lives in `../subst` (the generic
61		-	engine) and `../mnw-assumptions` (the MNW-specific toml, derived math, and validation).
	58	+	`{{ dotted.path \| filter(args) }}` substitution lives in `../subst` (the generic engine)
	59	+	and `../mnw-assumptions` (the MNW-specific toml, derived math, and validation). A doc
	60	+	engine needs none of it.
62	61
63		-	Substitution still runs before parsing, for the reason it always did: a regex pre-pass
64		-	sees the raw text, so markers may appear anywhere -- prose, code spans, table cells, link
65		-	text. A markdown-aware pass would either miss code spans (often exactly where a number
66		-	belongs) or mean re-implementing parts of the parser.
	62	+	Substitution runs before parsing. A regex pre-pass sees the raw text, so markers may
	63	+	appear anywhere: prose, code spans, table cells, link text. A markdown-aware pass would
	64	+	either miss code spans (often exactly where a number belongs) or mean re-implementing
	65	+	parts of the parser.
67	66
68	67		The seam is `DocLoaderConfig::pre_process`, a plain `Fn(&str) -> Result<String, String>`
69	68		applied to each file's text before rendering. MNW's server builds an `Assumptions` at boot

M src/doc_loader.rs +3 -4

			@@ -25,10 +25,9 @@
25	25		//! - Anything added to steps 4-5 must emit only HTML it constructs itself, or
26	26		//! sanitize its own input. Do not widen them to interpolate page content.
27	27		//!
28		-	//! The ordering is deliberate, not incidental: directives and UI examples exist
29		-	//! precisely to emit markup ammonia's default policy would strip, so moving
30		-	//! them before step 3 would defeat them. The safety comes from the inputs being
31		-	//! trusted, which is why that constraint is written down here.
	28	+	//! The ordering is deliberate. Directives and UI examples exist to emit markup
	29	+	//! ammonia's default policy would strip, so moving them before step 3 defeats
	30	+	//! them. The safety comes from the inputs being trusted.
32	31
33	32		use std::collections::HashMap;
34	33		use std::path::Path;

M src/lib.rs +17 -31

			@@ -141,21 +141,11 @@
141	141
142	142		/// What every rendering path must be true of, whatever else it does.
143	143		///
144		-	/// The oracle lives here rather than in the fuzz target, and that is the
145		-	/// point of the arrangement rather than a filing preference. `tests/regressions.rs`
	144	+	/// The oracle lives here rather than in the fuzz target. `tests/regressions.rs`
146	145		/// replays the committed corpus against this same function on stable, so a crash
147	146		/// the fuzzer finds becomes a unit test by copying one file into `fuzz/regressions/`,
148		-	/// and neither side can drift into checking less than the other. The pattern is
149		-	/// `MNW/shared/git-command`'s, copied deliberately.
150		-	///
151		-	/// Infra `15991c40`. The doors, counted before the harness was written as that task
152		-	/// instructs: this crate's render entry points are called from **14 files across 6
153		-	/// repos** -- MNW server and multithreaded, goingson, balanced_breakfast,
154		-	/// mnw-assumptions, bb-core -- with `render_permissive` the most used (10 sites),
155		-	/// then `render_standard` (5), `sanitize_html` (3), `render_strict` (3),
156		-	/// `restrict_media_hosts` (2), `render_chat` (1). There is one implementation
157		-	/// behind all of them, so this is a single authority with many consumers rather
158		-	/// than a grammar with two parsers, and a fuzz target is the right shape.
	147	+	/// and neither side can drift into checking less than the other. Keep new assertions
	148	+	/// here rather than in the fuzz target.
159	149		pub mod oracle {
160	150		/// Element names no output of any preset may contain.
161	151		///
			@@ -187,26 +177,22 @@
187	177		/// Assert the safety floor on one rendered output.
188	178		///
189	179		/// **Parses tag and attribute NAMES; never pattern-matches the whole
190		-	/// string.** That distinction is the correctness of this function rather
191		-	/// than an optimisation, and the fuzzer proved it twice within four minutes
192		-	/// of the target first running -- both times against the oracle, not the
193		-	/// sanitizer:
	180	+	/// string.** That distinction is the correctness of this function, not an
	181	+	/// optimisation. Two shapes are the sanitizer working and must not be
	182	+	/// reported as findings:
194	183		///
195		-	/// 1. A whole-string substring scan fired on a markdown image whose title
196		-	/// quote made it fail to parse, so it fell through as literal prose
197		-	/// reading `![img](x" onerror="alert(1))`. ` onerror` was in the output
198		-	/// as escaped TEXT while the real `<img>` beside it had been stripped
199		-	/// clean.
200		-	/// 2. Narrowing the scan to inside `<...>` was not enough either. It then
201		-	/// fired on `<p title="</noscript><img src=x onerror=...>">`,
202		-	/// where the string sits inside an entity-escaped attribute VALUE and a
203		-	/// browser renders it as tooltip text.
	184	+	/// 1. Escaped prose. A markdown image whose title quote makes it fail to
	185	+	/// parse falls through as literal text reading
	186	+	/// `![img](x" onerror="alert(1))`, so ` onerror` is in the output as
	187	+	/// escaped TEXT while the real `<img>` beside it has been stripped clean.
	188	+	/// A whole-string substring scan fires on it.
	189	+	/// 2. Escaped attribute values. `<p title="</noscript><img src=x
	190	+	/// onerror=...>">` puts the string inside an entity-escaped attribute
	191	+	/// VALUE that a browser renders as tooltip text. Narrowing the scan to
	192	+	/// inside `<...>` still fires on it.
204	193		///
205		-	/// Both are the sanitizer working. A check that reports correct behaviour as
206		-	/// a finding is worse than no check: it is the 3am false alarm that trains
207		-	/// people to skim the tier, which is the failure the whole evidence
208		-	/// programme exists to avoid. So the oracle skips quoted values by
209		-	/// construction, and escaped content inside one can no longer reach it.
	194	+	/// So the oracle skips quoted values by construction, and escaped content
	195	+	/// inside one cannot reach it.
210	196		///
211	197		/// Splitting on `<` is sound BECAUSE of what is under test: a sanitizer that
212	198		/// escapes text is the premise, so an unescaped `<` in the output can only

M src/plain.rs +2 -3

			@@ -182,9 +182,8 @@
182	182		/// concatenation of these runs is exactly what it returns.
183	183		///
184	184		/// For a caller whose destination has no markup but is not flat either: a
185		-	/// terminal cell can be bold, an `egui` galley can carry a text format, and
186		-	/// both were previously handed the words with the emphasis already thrown away.
187		-	/// A caller that only wants the words should keep using [`render_plain`].
	185	+	/// terminal cell can be bold, an `egui` galley can carry a text format. A caller
	186	+	/// that only wants the words should use [`render_plain`].
188	187		///
189	188		/// Adjacent text under the same marks arrives as one run, so a paragraph with
190	189		/// no emphasis in it is one run rather than one per parsed event.

M fuzz/fuzz_targets/render.rs +7 -12

			@@ -1,20 +1,15 @@
1	1		//! Structured fuzz over docengine's rendering and sanitization chain.
2	2		//!
3		-	//! Row 3 of `astra-soak-overview`, and the crate with the widest reach of any
4		-	//! soak target so far. The doors were counted before this was written, as infra
5		-	//! `15991c40` instructs: 14 files across 6 repos call these entry points --
6		-	//! MNW server and multithreaded, goingson, balanced_breakfast, mnw-assumptions,
7		-	//! bb-core -- and there is one implementation behind all of them. So this is a
8		-	//! single authority with many consumers rather than a grammar with two parsers,
9		-	//! which is what makes a fuzz target the right shape here and made a shared
10		-	//! crate the right answer for `git_ssh`.
	3	+	//! Row 3 of `astra-soak-overview`. The crate's render entry points are called
	4	+	//! from MNW server and multithreaded, goingson, balanced_breakfast,
	5	+	//! mnw-assumptions and bb-core, with one implementation behind all of them.
11	6		//!
12	7		//! ## The oracle lives in the crate, not here
13	8		//!
14		-	//! Everything asserted is `docengine::oracle::check`. That is deliberate: the
15		-	//! committed regression replay in `tests/regressions.rs` calls the same function
16		-	//! on stable, so a crash found here becomes a unit test by copying one file, and
17		-	//! neither side can drift into checking less than the other.
	9	+	//! Everything asserted is `docengine::oracle::check`. The committed regression
	10	+	//! replay in `tests/regressions.rs` calls the same function on stable, so a crash
	11	+	//! found here becomes a unit test by copying one file, and neither side can drift
	12	+	//! into checking less than the other.
18	13		//!
19	14		//! What it asserts: a safety floor on the output of every preset (no script,
20	15		//! iframe, object, embed, form, svg, math, `javascript:`, `vbscript:` or event

M fuzz/seeds/README.md +4 -6

			@@ -1,10 +1,9 @@
1	1		# Seeds
2	2
3	3		Curated, read-only, and small. These are the inputs a person chose: real site
4		-	docs for the ordinary shapes, plus the classic XSS payloads and the two cases
5		-	that caught the oracle out on its first run (a markdown image that falls through
6		-	as prose carrying ` onerror=`, and an entity-escaped payload inside a `title`
7		-	attribute value).
	4	+	docs for the ordinary shapes, plus the classic XSS payloads and two oracle edge
	5	+	cases (a markdown image that falls through as prose carrying ` onerror=`, and an
	6	+	entity-escaped payload inside a `title` attribute value).
8	7
9	8		They are NOT the corpus. Run the fuzzer with the corpus first and these second:
10	9
			@@ -12,5 +11,4 @@
12	11
13	12		libFuzzer writes new inputs into the FIRST directory only, so this arrangement
14	13		keeps the curated set curated and lets `fuzz/corpus/` grow. Passing seeds as the
15		-	first argument by mistake put 3,042 machine-generated files in here on
16		-	2026-08-23, which is how the note came to be written down.
	14	+	first argument instead fills this directory with machine-generated files.