Skip to main content

max / makenotwork

server: splice notes into a tree, and give the engine a write half The notes engine boundary was read-only. This adds the four write operations P2 needs — write_blob, write_tree, write_commit, update_ref_cas — as a second trait, NoteWrites, so the browsing path keeps compiling against a repository it cannot write to. Both contract rules hold: nothing above the gix adapter computes an object id, and the ref update is a single-ref compare-and-swap with no transaction op to reach for later. Above that sits splice_note, which sets or removes one note and returns the new root tree. The fanout it produces is git's own: a directory is created exactly where two notes collide on a prefix, split as far as their first difference and no further, and a delete prunes a directory it emptied or folds one it left holding a single note. Following git's rule is not required for correctness — every reader handles any shape — but a writer that reshapes the tree makes every other writer rearrange it back. An edit deliberately does not fold, so passing through a one-entry directory leaves it where it was. Entries the notes convention does not explain are carried through untouched at every level, which is what forced EntryKind to grow from three variants to git's five: writing back a symlink or a submodule entry needs its mode, and the coarse enum could not round-trip one. Reporting no change when the content is identical matters more than it looks: blobs are content addressed, so saving an unedited note would otherwise commit an empty diff to the notes ref every time. Thirteen tests over the fanout shapes, the fold, the pruning, junk preservation, and the compare-and-swap losing its race. Nothing calls any of this yet; the retry loop and the routes follow.
Co-Authored-By
Claude Opus 5 (1M context) <noreply@anthropic.com>
Author: Max Johnson <me@maxj.phd> · 2026-08-08 20:29 UTC
Signed with PGP, not checked
Commit: 0369f05ba845ecefd5f006f27d348ffb78c4c13f
Parent: e58daf7
4 files changed, +926 insertions, -39 deletions
M server/src/git/notes/engine.rs +91 -10
@@ -16,9 +16,9 @@
16 16 //! engine and receives an [`Oid`] back. That keeps it out of the hashing
17 17 //! business, and makes SHA-256 repositories a matter of the engine rather
18 18 //! than a port.
19 - //! 2. **Single-ref compare-and-swap only** (arriving with the write half in
20 - //! P2). Multi-ref atomicity is engine-specific; promising it would make the
21 - //! trait unimplementable over a subprocess.
19 + //! 2. **Single-ref compare-and-swap only.** Multi-ref atomicity is
20 + //! engine-specific; promising it would make the trait unimplementable over a
21 + //! subprocess.
22 22 //!
23 23 //! Reads take a visitor or a caller-owned buffer rather than returning owned
24 24 //! collections. A per-object callback boundary otherwise forfeits gix's
@@ -112,14 +112,29 @@
112 112 }
113 113 }
114 114
115 - /// What a tree entry points at. Anything that is neither a subtree nor a blob
116 - /// (commit entries for submodules, in practice) is `Other` — the notes layer
117 - /// only ever descends into trees and reads blobs.
115 + /// What a tree entry points at.
116 + ///
117 + /// These are git's five legal tree entry modes and nothing else. The notes
118 + /// layer only cares whether an entry is a subtree or a blob, but it has to be
119 + /// able to write back entries it did not put there — a notes tree may carry
120 + /// unrelated paths, and dropping a symlink or a submodule while splicing a note
121 + /// would be data loss. A coarser enum could not round-trip them.
118 122 #[derive(Debug, Clone, Copy, PartialEq, Eq)]
119 123 pub enum EntryKind {
120 124 Tree,
121 125 Blob,
122 - Other,
126 + BlobExecutable,
127 + Symlink,
128 + Commit,
129 + }
130 +
131 + impl EntryKind {
132 + /// Whether this entry can hold note content. Git writes notes as plain
133 + /// blobs, but a tree assembled by other means may mark one executable, and
134 + /// a reader that ignored it would report the note missing.
135 + pub fn is_blob(self) -> bool {
136 + matches!(self, Self::Blob | Self::BlobExecutable)
137 + }
123 138 }
124 139
125 140 /// One entry in a tree, borrowed for the duration of the visit.
@@ -132,6 +147,17 @@
132 147 pub oid: Oid,
133 148 }
134 149
150 + /// One entry of a tree about to be written.
151 + ///
152 + /// Owned, unlike [`TreeEntry`]: a splice reads a whole level, edits one entry
153 + /// and writes the level back, so the names have to outlive the read.
154 + #[derive(Debug, Clone, PartialEq, Eq)]
155 + pub struct NewEntry {
156 + pub name: Vec<u8>,
157 + pub kind: EntryKind,
158 + pub oid: Oid,
159 + }
160 +
135 161 /// Whether a tree walk continues after the current entry. Lookups stop as soon
136 162 /// as they have their answer rather than reading the rest of a fanout level.
137 163 #[derive(Debug, Clone, Copy, PartialEq, Eq)]
@@ -173,6 +199,12 @@
173 199 /// The repository holds something the notes convention does not allow.
174 200 #[error("malformed notes data: {0}")]
175 201 Malformed(String),
202 + /// A compare-and-swap ref update lost its race: the ref moved between the
203 + /// read that produced the expected value and the update. The caller reloads
204 + /// and retries; it is the ordinary outcome of two people annotating at
205 + /// once, not a fault.
206 + #[error("the notes ref moved during the update")]
207 + Raced,
176 208 /// The engine failed.
177 209 #[error("git backend error: {0}")]
178 210 Backend(String),
@@ -183,6 +215,7 @@
183 215 match e {
184 216 NotesError::NotFound => Self::RefNotFound,
185 217 NotesError::Malformed(m) => Self::Git(format!("malformed notes data: {m}")),
218 + NotesError::Raced => Self::Git("the notes ref moved during the update".into()),
186 219 NotesError::Backend(m) => Self::Git(m),
187 220 }
188 221 }
@@ -190,9 +223,10 @@
190 223
191 224 /// The read half of the engine boundary.
192 225 ///
193 - /// The write half (`write_blob`, `write_tree`, `write_commit`,
194 - /// `update_ref_cas`) lands with P2 as a separate trait, so that the browsing
195 - /// path — which is read-only — never depends on a repository being writable.
226 + /// The write half is [`NoteWrites`], deliberately a second trait: the browsing
227 + /// path is read-only and must not require a writable repository to compile
228 + /// against, and an engine over a read-only mirror can implement this half
229 + /// alone.
196 230 pub trait NoteObjects {
197 231 /// Resolve a full ref name (`refs/notes/commits`) to the commit it points
198 232 /// at, following annotated tags. `Ok(None)` means the ref does not exist,
@@ -219,6 +253,53 @@
219 253 fn read_blob_into(&self, oid: Oid, out: &mut Vec<u8>) -> Result<(), NotesError>;
220 254 }
221 255
256 + /// The write half of the engine boundary.
257 + ///
258 + /// Every method hands bytes down and gets an [`Oid`] back. Nothing above this
259 + /// trait hashes anything, which is what keeps the notes layer out of the
260 + /// hashing business and makes a SHA-256 repository the engine's problem rather
261 + /// than a port.
262 + pub trait NoteWrites {
263 + /// Store a blob and return its ID. Storing content that is already present
264 + /// is not an error and returns the same ID — object stores are content
265 + /// addressed, so an unchanged note costs nothing.
266 + fn write_blob(&self, content: &[u8]) -> Result<Oid, NotesError>;
267 +
268 + /// Store a tree. `entries` may arrive in any order; putting them in git's
269 + /// canonical order is the engine's job, because the ordering rule (a
270 + /// subtree sorts as though its name ended in `/`) is a property of the
271 + /// object format rather than of notes.
272 + ///
273 + /// An empty slice writes the empty tree, which is what removing the last
274 + /// note in a namespace leaves behind.
275 + fn write_tree(&self, entries: &[NewEntry]) -> Result<Oid, NotesError>;
276 +
277 + /// Store a commit. This writes the object only: moving a ref onto it is
278 + /// [`update_ref_cas`](NoteWrites::update_ref_cas), separately, because the
279 + /// commit has to exist before anything can compare-and-swap onto it.
280 + fn write_commit(
281 + &self,
282 + tree: Oid,
283 + parents: &[Oid],
284 + author: &Signature,
285 + committer: &Signature,
286 + message: &str,
287 + ) -> Result<Oid, NotesError>;
288 +
289 + /// Point `full_name` at `new`, but only if it currently holds `expected`
290 + /// (`None` meaning the ref must not exist yet).
291 + ///
292 + /// [`NotesError::Raced`] when the ref holds something else. Single-ref
293 + /// only, by contract: multi-ref atomicity is engine-specific, and a trait
294 + /// that promised it could not be implemented over a `git` subprocess.
295 + fn update_ref_cas(
296 + &self,
297 + full_name: &str,
298 + expected: Option<Oid>,
299 + new: Oid,
300 + ) -> Result<(), NotesError>;
301 + }
302 +
222 303 #[cfg(test)]
223 304 mod tests {
224 305 use super::*;
M server/src/git/notes/gix_engine.rs +141 -5
@@ -5,9 +5,11 @@
5 5 //! this is the file that gets a sibling and nothing above it changes.
6 6
7 7 use gix::bstr::ByteSlice;
8 + use gix::refs::transaction::{Change, LogChange, PreviousValue, RefEdit, RefLog};
8 9
9 10 use super::engine::{
10 - CommitMeta, EntryKind, NoteObjects, NotesError, Oid, Signature, TreeEntry, Walk,
11 + CommitMeta, EntryKind, NewEntry, NoteObjects, NoteWrites, NotesError, Oid, Signature,
12 + TreeEntry, Walk,
11 13 };
12 14
13 15 /// Adapts a borrowed [`gix::Repository`] to the notes engine boundary.
@@ -53,10 +55,36 @@
53 55 fn to_entry_kind(mode: gix::objs::tree::EntryMode) -> EntryKind {
54 56 match mode.kind() {
55 57 gix::objs::tree::EntryKind::Tree => EntryKind::Tree,
56 - gix::objs::tree::EntryKind::Blob | gix::objs::tree::EntryKind::BlobExecutable => {
57 - EntryKind::Blob
58 - }
59 - _ => EntryKind::Other,
58 + gix::objs::tree::EntryKind::Blob => EntryKind::Blob,
59 + gix::objs::tree::EntryKind::BlobExecutable => EntryKind::BlobExecutable,
60 + gix::objs::tree::EntryKind::Link => EntryKind::Symlink,
61 + gix::objs::tree::EntryKind::Commit => EntryKind::Commit,
62 + }
63 + }
64 +
65 + fn from_entry_kind(kind: EntryKind) -> gix::objs::tree::EntryKind {
66 + match kind {
67 + EntryKind::Tree => gix::objs::tree::EntryKind::Tree,
68 + EntryKind::Blob => gix::objs::tree::EntryKind::Blob,
69 + EntryKind::BlobExecutable => gix::objs::tree::EntryKind::BlobExecutable,
70 + EntryKind::Symlink => gix::objs::tree::EntryKind::Link,
71 + EntryKind::Commit => gix::objs::tree::EntryKind::Commit,
72 + }
73 + }
74 +
75 + /// Ours to gix's, for writing. The inverse of [`to_signature`], which is lossy
76 + /// about time zones by design — the notes layer speaks UTC, so a signature that
77 + /// survives a read-write round trip comes back with a `+0000` offset. Notes
78 + /// commits are written by the server, and the offset carries nothing a reader
79 + /// of them needs.
80 + fn from_signature(sig: &Signature) -> gix::actor::Signature {
81 + gix::actor::Signature {
82 + name: sig.name.as_str().into(),
83 + email: sig.email.as_str().into(),
84 + time: gix::date::Time {
85 + seconds: sig.time.timestamp(),
86 + offset: 0,
87 + },
60 88 }
61 89 }
62 90
@@ -170,3 +198,111 @@
170 198 Ok(())
171 199 }
172 200 }
201 +
202 + impl NoteWrites for GixEngine<'_> {
203 + fn write_blob(&self, content: &[u8]) -> Result<Oid, NotesError> {
204 + let id = self
205 + .repo
206 + .write_blob(content)
207 + .map_err(|e| NotesError::Backend(format!("cannot write note blob: {e}")))?;
208 + to_oid(&id)
209 + }
210 +
211 + fn write_tree(&self, entries: &[NewEntry]) -> Result<Oid, NotesError> {
212 + let mut tree = gix::objs::Tree::empty();
213 + for entry in entries {
214 + tree.entries.push(gix::objs::tree::Entry {
215 + mode: from_entry_kind(entry.kind).into(),
216 + filename: entry.name.as_slice().into(),
217 + oid: to_gix_oid(entry.oid)?,
218 + });
219 + }
220 + // Git's ordering, which sorts a subtree as though its name ended in a
221 + // slash. `Ord` on the gix entry already implements that rule, and
222 + // getting it wrong produces a tree stock git rejects.
223 + tree.entries.sort();
224 + let id = self
225 + .repo
226 + .write_object(&tree)
227 + .map_err(|e| NotesError::Backend(format!("cannot write notes tree: {e}")))?;
228 + to_oid(&id)
229 + }
230 +
231 + fn write_commit(
232 + &self,
233 + tree: Oid,
234 + parents: &[Oid],
235 + author: &Signature,
236 + committer: &Signature,
237 + message: &str,
238 + ) -> Result<Oid, NotesError> {
239 + // Deliberately not `Repository::commit_as`, which writes the commit and
240 + // moves a ref in one call. The ref move is a compare-and-swap that has
241 + // to be able to fail and be retried against a reloaded tree, so the
242 + // object is written first and pointed at afterwards.
243 + let mut gix_parents = Vec::with_capacity(parents.len());
244 + for parent in parents {
245 + gix_parents.push(to_gix_oid(*parent)?);
246 + }
247 + let commit = gix::objs::Commit {
248 + tree: to_gix_oid(tree)?,
249 + parents: gix_parents.into(),
250 + author: from_signature(author),
251 + committer: from_signature(committer),
252 + encoding: None,
253 + message: message.into(),
254 + extra_headers: Vec::new(),
255 + };
256 +
257 + let id = self
258 + .repo
259 + .write_object(&commit)
260 + .map_err(|e| NotesError::Backend(format!("cannot write notes commit: {e}")))?;
261 + to_oid(&id)
262 + }
263 +
264 + fn update_ref_cas(
265 + &self,
266 + full_name: &str,
267 + expected: Option<Oid>,
268 + new: Oid,
269 + ) -> Result<(), NotesError> {
270 + let expected_target = match expected {
271 + Some(oid) => {
272 + PreviousValue::MustExistAndMatch(gix::refs::Target::Object(to_gix_oid(oid)?))
273 + }
274 + None => PreviousValue::MustNotExist,
275 + };
276 + let edit = RefEdit {
277 + change: Change::Update {
278 + log: LogChange {
279 + mode: RefLog::AndReference,
280 + force_create_reflog: false,
281 + message: "notes: update from makenot.work".into(),
282 + },
283 + expected: expected_target,
284 + new: gix::refs::Target::Object(to_gix_oid(new)?),
285 + },
286 + name: full_name.try_into().map_err(|e| {
287 + NotesError::Malformed(format!("{full_name} is not a ref name: {e}"))
288 + })?,
289 + deref: false,
290 + };
291 +
292 + let Err(err) = self.repo.edit_reference(edit) else {
293 + return Ok(());
294 + };
295 + // gix reports a failed precondition inside a transaction error whose
296 + // shape is not worth matching on. Re-reading the ref answers the only
297 + // question the caller has: did we lose the race, or is the repository
298 + // broken. A ref that moved again between the failure and this read
299 + // still reads as a race, which is the right answer.
300 + if self.resolve_ref(full_name)? == expected {
301 + Err(NotesError::Backend(format!(
302 + "cannot update {full_name}: {err}"
303 + )))
304 + } else {
305 + Err(NotesError::Raced)
306 + }
307 + }
308 + }
M server/src/git/notes/mod.rs +286 -24
@@ -14,8 +14,9 @@
14 14 //! this is going (a standalone crate, extracted once the P3 merge strategies
15 15 //! exist and give it something no other implementation has).
16 16 //!
17 - //! Read-only. The write path, its compare-and-swap ref update, and the merge
18 - //! strategies arrive in P2 and P3.
17 + //! The write path splices one note into a tree and hands the result back; the
18 + //! compare-and-swap that publishes it, and the merge strategies, follow. See
19 + //! "Splicing" below for why the shape of the tree we write is not arbitrary.
19 20
20 21 pub mod engine;
21 22 pub mod gix_engine;
@@ -25,7 +26,7 @@
25 26
26 27 use dashmap::DashMap;
27 28
28 - pub use engine::{CommitMeta, NoteObjects, NotesError, Oid, Signature};
29 + pub use engine::{CommitMeta, NewEntry, NoteObjects, NoteWrites, NotesError, Oid, Signature};
29 30 pub use gix_engine::GixEngine;
30 31
31 32 use engine::{EntryKind, TreeEntry, Walk};
@@ -232,15 +233,13 @@
232 233 let mut subtree: Option<Oid> = None;
233 234
234 235 engine.read_tree_with(tree, &mut |entry: TreeEntry<'_>| {
235 - match entry.kind {
236 - EntryKind::Blob if entry.name == remaining => {
237 - leaf = Some(entry.oid);
238 - return Walk::Stop;
239 - }
240 - EntryKind::Tree if remaining.len() > 2 && entry.name == &remaining[..2] => {
241 - subtree = Some(entry.oid);
242 - }
243 - _ => {}
236 + if entry.kind.is_blob() && entry.name == remaining {
237 + leaf = Some(entry.oid);
238 + return Walk::Stop;
239 + }
240 + if entry.kind == EntryKind::Tree && remaining.len() > 2 && entry.name == &remaining[..2]
241 + {
242 + subtree = Some(entry.oid);
244 243 }
245 244 Walk::Continue
246 245 })?;
@@ -286,19 +285,15 @@
286 285 path.extend_from_slice(&prefix);
287 286 path.extend_from_slice(entry.name);
288 287
289 - match entry.kind {
290 - EntryKind::Blob => {
291 - if let Ok(target) = Oid::from_hex(&path) {
292 - visit(target, entry.oid);
293 - }
288 + if entry.kind == EntryKind::Tree {
289 + // Only descend paths that could still become an ID.
290 + if path.len() < 64 && path.iter().all(u8::is_ascii_hexdigit) {
291 + next_level.push((path, entry.oid));
294 292 }
295 - EntryKind::Tree => {
296 - // Only descend paths that could still become an ID.
297 - if path.len() < 64 && path.iter().all(u8::is_ascii_hexdigit) {
298 - next_level.push((path, entry.oid));
299 - }
300 - }
301 - EntryKind::Other => {}
293 + } else if entry.kind.is_blob()
294 + && let Ok(target) = Oid::from_hex(&path)
295 + {
296 + visit(target, entry.oid);
302 297 }
303 298 Walk::Continue
304 299 })?;
@@ -308,6 +303,273 @@
308 303 Ok(())
309 304 }
310 305
306 + // ── Splicing ──
307 +
308 + /// What splicing one level of a notes tree produced.
309 + enum Spliced {
310 + /// The level is exactly as it was. Nothing above it needs rewriting either.
311 + Same,
312 + /// The level was rewritten and now has this ID.
313 + Changed(Oid),
314 + /// A delete emptied the level. The parent drops the directory entry rather
315 + /// than writing an empty tree, which git does not leave lying around inside
316 + /// a notes tree.
317 + Emptied,
318 + /// A delete left the level holding one note and nothing else, so it folds
319 + /// back into the parent as a leaf: the parent replaces its directory entry
320 + /// with a blob named its own name plus this suffix. Git consolidates the
321 + /// same way, and a tree that kept the directory would slowly accumulate
322 + /// one-entry fanout levels that no delete ever cleaned up.
323 + ///
324 + /// Only a delete folds. An edit that happens to pass through a directory
325 + /// holding one note leaves it exactly as it found it — reshaping a tree
326 + /// nobody asked about would make every writer fight over the layout.
327 + Collapsed(Vec<u8>, Oid),
328 + }
329 +
330 + /// Set or remove the note on `target`, returning the new root tree.
331 + ///
332 + /// `root_tree` is `None` for a namespace that does not exist yet. `blob` is the
333 + /// content blob to point at, or `None` to remove the note. `Ok(None)` means the
334 + /// tree already said what was asked for and nothing was written — an unchanged
335 + /// note must not produce a commit.
336 + ///
337 + /// **Shape matters.** Git decides the fanout of a notes tree by insertion: a
338 + /// directory exists exactly where two notes collided on a prefix, and a delete
339 + /// that leaves one behind folds it back. Following the same rule keeps the tree
340 + /// stable across writers, so a `git notes` write and one of ours produce the
341 + /// same shape rather than each rearranging the other's work. Nothing breaks if
342 + /// the shape drifts — every reader, ours included, handles any fanout — but a
343 + /// drifting shape churns the tree for no reason.
344 + ///
345 + /// Entries the notes convention does not explain are carried through untouched,
346 + /// at every level. A notes tree is allowed to hold other paths, and a splice
347 + /// that dropped them would be data loss.
348 + pub fn splice_note<E: NoteObjects + NoteWrites>(
349 + engine: &E,
350 + root_tree: Option<Oid>,
351 + target: Oid,
352 + blob: Option<Oid>,
353 + ) -> Result<Option<Oid>, NotesError> {
354 + let hex = target.to_hex();
355 + let entries = match root_tree {
356 + Some(tree) => read_entries(engine, tree)?,
357 + None => Vec::new(),
358 + };
359 +
360 + match splice_level(engine, entries, hex.as_bytes(), blob, 0)? {
361 + Spliced::Same => Ok(None),
362 + Spliced::Changed(oid) => Ok(Some(oid)),
363 + // Only produced for `depth > 0`; the root is written even when empty,
364 + // because removing the last note still needs a tree to commit.
365 + Spliced::Emptied | Spliced::Collapsed(..) => Err(NotesError::Malformed(
366 + "the notes root tree cannot fold into a parent".into(),
367 + )),
368 + }
369 + }
370 +
371 + /// Read one tree level into owned entries so it can be edited and written back.
372 + fn read_entries<E: NoteObjects>(engine: &E, tree: Oid) -> Result<Vec<NewEntry>, NotesError> {
373 + let mut entries = Vec::new();
374 + engine.read_tree_with(tree, &mut |entry: TreeEntry<'_>| {
375 + entries.push(NewEntry {
376 + name: entry.name.to_vec(),
377 + kind: entry.kind,
378 + oid: entry.oid,
379 + });
380 + Walk::Continue
381 + })?;
382 + Ok(entries)
383 + }
384 +
385 + /// Splice `blob` in at the level whose entries are `entries`, where `remaining`
386 + /// is the part of the target's hex this level and everything below it spells.
387 + fn splice_level<E: NoteObjects + NoteWrites>(
388 + engine: &E,
389 + mut entries: Vec<NewEntry>,
390 + remaining: &[u8],
391 + blob: Option<Oid>,
392 + depth: usize,
393 + ) -> Result<Spliced, NotesError> {
394 + // Deeper than any legitimate fanout. The reader gives up here too, so
395 + // writing below it would put a note where nothing can find it.
396 + if depth >= MAX_FANOUT_DEPTH {
397 + return Ok(Spliced::Same);
398 + }
399 + let deleting = blob.is_none();
400 +
401 + // The note already lives at this level, spelled out in full. Same order as
402 + // the reader: leaf first, then the directory.
403 + if let Some(at) = entries
404 + .iter()
405 + .position(|e| e.kind.is_blob() && e.name == remaining)
406 + {
407 + match blob {
408 + Some(new) if entries[at].oid == new && entries[at].kind == EntryKind::Blob => {
409 + return Ok(Spliced::Same);
410 + }
411 + Some(new) => {
412 + entries[at].oid = new;
413 + // Normalize the mode. Git writes notes as plain blobs, and an
414 + // executable one is a leftover from something else.
415 + entries[at].kind = EntryKind::Blob;
416 + }
417 + None => {
418 + entries.remove(at);
419 + }
420 + }
421 + return finish_level(engine, &entries, remaining.len(), depth, deleting);
422 + }
423 +
424 + // A fanout directory for the next two characters.
425 + if remaining.len() > 2 {
426 + let head = &remaining[..2];
427 + if let Some(at) = entries
428 + .iter()
429 + .position(|e| e.kind == EntryKind::Tree && e.name == head)
430 + {
431 + let below = read_entries(engine, entries[at].oid)?;
432 + match splice_level(engine, below, &remaining[2..], blob, depth + 1)? {
433 + Spliced::Same => return Ok(Spliced::Same),
434 + Spliced::Changed(oid) => entries[at].oid = oid,
435 + Spliced::Emptied => {
436 + entries.remove(at);
437 + }
438 + Spliced::Collapsed(suffix, oid) => {
439 + let mut name = head.to_vec();
440 + name.extend_from_slice(&suffix);
441 + entries[at] = NewEntry {
442 + name,
443 + kind: EntryKind::Blob,
444 + oid,
445 + };
446 + }
447 + }
448 + return finish_level(engine, &entries, remaining.len(), depth, deleting);
449 + }
450 + }
451 +
452 + // The note is not in this tree at all.
453 + let Some(new) = blob else {
454 + // Removing a note that was never there is not an error. Two people
455 + // deleting the same note both succeed, and neither writes a commit.
456 + return Ok(Spliced::Same);
457 + };
458 +
459 + // A note already sitting flat at this level whose ID shares our first two
460 + // characters. Git splits the pair into a directory rather than leaving two
461 + // colliding names side by side; this is the only place fanout is created.
462 + if remaining.len() > 2 {
463 + let head = &remaining[..2];
464 + if let Some(at) = entries.iter().position(|e| {
465 + e.kind.is_blob()
466 + && e.name.len() == remaining.len()
467 + && e.name.starts_with(head)
468 + && e.name.iter().all(u8::is_ascii_hexdigit)
469 + }) {
470 + let neighbour = entries.remove(at);
471 + let subtree = split_pair(
472 + engine,
473 + &neighbour.name[2..],
474 + neighbour.oid,
475 + &remaining[2..],
476 + new,
477 + depth + 1,
478 + )?;
479 + entries.push(NewEntry {
480 + name: head.to_vec(),
481 + kind: EntryKind::Tree,
482 + oid: subtree,
483 + });
484 + return finish_level(engine, &entries, remaining.len(), depth, deleting);
485 + }
486 + }
487 +
488 + entries.push(NewEntry {
489 + name: remaining.to_vec(),
490 + kind: EntryKind::Blob,
491 + oid: new,
492 + });
493 + finish_level(engine, &entries, remaining.len(), depth, deleting)
494 + }
495 +
496 + /// Write a level back, or tell the parent to absorb it.
497 + ///
498 + /// `leaf_len` is how long a note's name is at this level, which is what makes
499 + /// folding safe: a shorter name would not spell a whole object ID once hoisted,
500 + /// so a hand-built tree holding one cannot be collapsed into an unreadable one.
501 + fn finish_level<E: NoteWrites>(
502 + engine: &E,
503 + entries: &[NewEntry],
504 + leaf_len: usize,
505 + depth: usize,
506 + deleting: bool,
507 + ) -> Result<Spliced, NotesError> {
508 + if depth > 0 && deleting {
509 + if entries.is_empty() {
510 + return Ok(Spliced::Emptied);
511 + }
512 + let [only] = entries else {
513 + return Ok(Spliced::Changed(engine.write_tree(entries)?));
514 + };
515 + if only.kind.is_blob()
516 + && only.name.len() == leaf_len
517 + && only.name.iter().all(u8::is_ascii_hexdigit)
518 + {
519 + return Ok(Spliced::Collapsed(only.name.clone(), only.oid));
520 + }
521 + }
522 + Ok(Spliced::Changed(engine.write_tree(entries)?))
523 + }
524 +
525 + /// Build the fanout directory that holds two notes whose IDs collide, adding a
526 + /// level per shared pair of characters until they diverge.
527 + fn split_pair<E: NoteWrites>(
528 + engine: &E,
529 + left_name: &[u8],
530 + left: Oid,
531 + right_name: &[u8],
532 + right: Oid,
533 + depth: usize,
534 + ) -> Result<Oid, NotesError> {
535 + if left_name == right_name {
536 + // Two different targets cannot spell the same ID; a tree that says so
537 + // is corrupt, and writing it would drop one of the two notes.
538 + return Err(NotesError::Malformed(
539 + "two notes claim the same object id".into(),
540 + ));
541 + }
542 +
543 + if depth < MAX_FANOUT_DEPTH && left_name.len() > 2 && left_name[..2] == right_name[..2] {
544 + let inner = split_pair(
545 + engine,
546 + &left_name[2..],
547 + left,
548 + &right_name[2..],
549 + right,
550 + depth + 1,
551 + )?;
552 + return engine.write_tree(&[NewEntry {
553 + name: left_name[..2].to_vec(),
554 + kind: EntryKind::Tree,
555 + oid: inner,
556 + }]);
557 + }
558 +
559 + engine.write_tree(&[
560 + NewEntry {
561 + name: left_name.to_vec(),
562 + kind: EntryKind::Blob,
563 + oid: left,
564 + },
565 + NewEntry {
566 + name: right_name.to_vec(),
567 + kind: EntryKind::Blob,
568 + oid: right,
569 + },
570 + ])
571 + }
572 +
311 573 // ── Attribution ──
312 574
313 575 /// Find the notes commit that set `target`'s note to what it is now.
M server/src/git/notes/tests.rs +408
@@ -65,6 +65,60 @@
65 65 Oid::from_hex(hex.as_bytes()).unwrap()
66 66 }
67 67
68 + fn ours(id: gix::ObjectId) -> Oid {
69 + Oid::from_bytes(id.as_bytes()).unwrap()
70 + }
71 +
72 + /// Every note path in a tree, spelled the way it sits on disk (`aa/bb/ccdd…`).
73 + ///
74 + /// The write tests assert on this rather than on content alone: two trees can
75 + /// hold the same notes in different fanout shapes, and the shape is the part
76 + /// that has to match what git would have produced.
77 + fn paths(engine: &GixEngine<'_>, tree: Oid) -> Vec<String> {
78 + let mut out = Vec::new();
79 + let mut pending = vec![(String::new(), tree)];
80 + while let Some((prefix, tree)) = pending.pop() {
81 + let mut children = Vec::new();
82 + engine
83 + .read_tree_with(tree, &mut |entry: TreeEntry<'_>| {
84 + let name = format!("{prefix}{}", String::from_utf8_lossy(entry.name));
85 + if entry.kind == EntryKind::Tree {
86 + children.push((format!("{name}/"), entry.oid));
87 + } else {
88 + out.push(name);
89 + }
90 + Walk::Continue
91 + })
92 + .unwrap();
93 + pending.extend(children);
94 + }
95 + out.sort();
96 + out
97 + }
98 +
99 + /// The target-to-blob map of a tree, without needing a commit to hang it off.
100 + fn flat(engine: &GixEngine<'_>, tree: Oid) -> Vec<(Oid, Oid)> {
101 + let mut out = Vec::new();
102 + flatten(engine, tree, &mut |target, blob| out.push((target, blob))).unwrap();
103 + out.sort_unstable();
104 + out
105 + }
106 +
107 + fn signature(name: &str) -> Signature {
108 + Signature {
109 + name: name.to_string(),
110 + email: format!("{name}@users.makenot.work"),
111 + time: chrono::DateTime::from_timestamp(1_700_000_000, 0).unwrap(),
112 + }
113 + }
114 +
115 + /// The root tree of a repository's default namespace, or `None` if nobody has
116 + /// annotated anything yet.
117 + fn root_tree(engine: &GixEngine<'_>) -> Option<Oid> {
118 + let ns = resolve_namespace(engine, DEFAULT_NAMESPACE).unwrap()?;
119 + Some(engine.read_commit(ns.tip).unwrap().tree)
120 + }
121 +
68 122 fn namespace(engine: &GixEngine<'_>, name: &str) -> Namespace {
69 123 resolve_namespace(engine, name).unwrap().unwrap()
70 124 }
@@ -413,6 +467,360 @@
413 467 );
414 468 }
415 469
470 + // ── Splicing ──
471 +
472 + #[test]
473 + fn a_first_note_lands_flat_at_the_root() {
474 + let (_tmp, repo) = init_bare();
475 + let engine = GixEngine::new(&repo);
476 + let content = ours(blob(&repo, "first note in the repository"));
477 +
478 + let root = splice_note(&engine, None, oid(T1), Some(content))
479 + .unwrap()
480 + .expect("a new tree");
481 +
482 + // No fanout: one note has nothing to collide with, and git does not create
483 + // a directory speculatively.
484 + assert_eq!(paths(&engine, root), vec![T1.to_string()]);
485 + assert_eq!(flat(&engine, root), vec![(oid(T1), content)]);
486 + }
487 +
488 + #[test]
489 + fn a_colliding_note_splits_the_pair_down_to_where_they_differ() {
490 + let (_tmp, repo) = init_bare();
491 + let engine = GixEngine::new(&repo);
492 + let one = ours(blob(&repo, "one"));
493 + let two = ours(blob(&repo, "two"));
494 +
495 + let first = splice_note(&engine, None, oid(T1), Some(one))
496 + .unwrap()
497 + .unwrap();
498 + let both = splice_note(&engine, Some(first), oid(T2), Some(two))
499 + .unwrap()
500 + .unwrap();
501 +
502 + // T1 and T2 share `aabb`, so one directory is not enough to separate them
503 + // and git adds a second. Splitting only as far as the first difference is
504 + // what keeps the tree from growing a level per byte.
505 + assert_eq!(
506 + paths(&engine, both),
507 + vec![format!("aa/bb/{}", &T2[4..]), format!("aa/bb/{}", &T1[4..]),]
508 + .into_iter()
509 + .collect::<std::collections::BTreeSet<_>>()
510 + .into_iter()
511 + .collect::<Vec<_>>()
512 + );
513 + assert_eq!(flat(&engine, both), {
514 + let mut want = vec![(oid(T1), one), (oid(T2), two)];
515 + want.sort_unstable();
516 + want
517 + });
518 +
519 + // A third note that shares nothing stays flat beside the directory.
520 + let three = ours(blob(&repo, "three"));
521 + let all = splice_note(&engine, Some(both), oid(T3), Some(three))
522 + .unwrap()
523 + .unwrap();
524 + assert!(paths(&engine, all).contains(&T3.to_string()));
525 + assert_eq!(flat(&engine, all).len(), 3);
526 + }
527 +
528 + #[test]
529 + fn an_edit_replaces_the_note_where_it_already_sits() {
530 + let (_tmp, repo) = mixed_fanout_repo();
531 + let engine = GixEngine::new(&repo);
532 + let before = root_tree(&engine).unwrap();
533 + let shape = paths(&engine, before);
534 +
535 + // T2 lives at `aa/bb00…` in this fixture and T3 at `cc/dd/eeff…`. An edit
536 + // moves neither: rewriting a note at a different depth than it was found
537 + // would leave the old path behind and the note doubled.
538 + let mut tree = before;
539 + for target in [T2, T3] {
540 + let replacement = ours(blob(&repo, &format!("rewritten note on {target}")));
541 + tree = splice_note(&engine, Some(tree), oid(target), Some(replacement))
542 + .unwrap()
543 + .expect("the tree changed");
544 + assert_eq!(paths(&engine, tree), shape, "editing {target} moved it");
545 + }
546 +
547 + assert_eq!(flat(&engine, tree).len(), 3);
548 + }
549 +
550 + #[test]
551 + fn writing_the_same_content_back_changes_nothing() {
552 + let (_tmp, repo) = mixed_fanout_repo();
553 + let engine = GixEngine::new(&repo);
554 + let root = root_tree(&engine).unwrap();
555 +
556 + // Blobs are content addressed, so re-submitting the same text produces the
557 + // blob that is already there. Reporting a change would commit an empty diff
558 + // to the notes ref every time somebody pressed save without editing.
559 + let same = ours(blob(&repo, "note on two"));
560 + assert!(
561 + splice_note(&engine, Some(root), oid(T2), Some(same))
562 + .unwrap()
563 + .is_none()
564 + );
565 + }
566 +
567 + #[test]
568 + fn deleting_a_note_prunes_the_directory_it_emptied() {
569 + let (_tmp, repo) = mixed_fanout_repo();
570 + let engine = GixEngine::new(&repo);
571 + let root = root_tree(&engine).unwrap();
572 +
573 + // T3 is the only note under `cc/`, two levels down. Both levels go: an
574 + // empty fanout directory is a path git would never have written.
575 + let after = splice_note(&engine, Some(root), oid(T3), None)
576 + .unwrap()
577 + .expect("the tree changed");
578 +
579 + assert!(
580 + !paths(&engine, after).iter().any(|p| p.starts_with("cc")),
581 + "the emptied directory survived: {:?}",
582 + paths(&engine, after)
583 + );
584 + assert_eq!(flat(&engine, after).len(), 2);
585 + }
586 +
587 + #[test]
588 + fn deleting_a_note_folds_a_directory_left_holding_one() {
589 + let (_tmp, repo) = init_bare();
590 + let engine = GixEngine::new(&repo);
591 + let one = ours(blob(&repo, "one"));
592 + let two = ours(blob(&repo, "two"));
593 +
594 + let tree = splice_note(&engine, None, oid(T1), Some(one))
595 + .unwrap()
596 + .unwrap();
597 + let tree = splice_note(&engine, Some(tree), oid(T2), Some(two))
598 + .unwrap()
599 + .unwrap();
600 + let tree = splice_note(&engine, Some(tree), oid(T1), None)
601 + .unwrap()
602 + .expect("the tree changed");
603 +
604 + // Both levels of `aa/bb/` existed only to separate the two. With one left
605 + // the tree folds all the way back to flat, which is the shape it would have
606 + // had if T2 had been the only note from the start — so an add followed by a
607 + // delete leaves no trace of the pair.
608 + assert_eq!(paths(&engine, tree), vec![T2.to_string()]);
609 + assert_eq!(flat(&engine, tree), vec![(oid(T2), two)]);
610 + }
611 +
612 + #[test]
613 + fn deleting_the_last_note_leaves_an_empty_tree() {
614 + let (_tmp, repo) = init_bare();
615 + let engine = GixEngine::new(&repo);
616 + let content = ours(blob(&repo, "the only note"));
617 +
618 + let tree = splice_note(&engine, None, oid(T1), Some(content))
619 + .unwrap()
620 + .unwrap();
621 + let empty = splice_note(&engine, Some(tree), oid(T1), None)
622 + .unwrap()
623 + .expect("the tree changed");
624 +
625 + // An empty tree, not a missing one: removing the last note still commits,
626 + // so the namespace keeps its history rather than vanishing.
627 + assert!(paths(&engine, empty).is_empty());
628 + assert!(flat(&engine, empty).is_empty());
629 + }
630 +
631 + #[test]
632 + fn deleting_a_note_that_was_never_there_is_not_a_change() {
633 + let (_tmp, repo) = mixed_fanout_repo();
634 + let engine = GixEngine::new(&repo);
635 + let root = root_tree(&engine).unwrap();
636 +
637 + let absent = "1111111111111111111111111111111111111111";
638 + assert!(
639 + splice_note(&engine, Some(root), oid(absent), None)
640 + .unwrap()
641 + .is_none()
642 + );
643 + // And on a namespace that does not exist at all.
644 + assert!(
645 + splice_note(&engine, None, oid(absent), None)
646 + .unwrap()
647 + .is_none()
648 + );
649 + }
650 +
651 + #[test]
652 + fn splicing_carries_unrelated_paths_through() {
653 + let (_tmp, repo) = init_bare();
654 + let note = blob(&repo, "real note");
655 + let stray = blob(&repo, "not a note");
656 +
657 + // The same junk the reader tolerates: git does not promise a notes tree
658 + // holds only notes, so a splice that dropped what it did not recognize
659 + // would quietly delete somebody's file.
660 + let junk_dir = tree(&repo, &[("whatever", stray, GixEntryKind::Blob)]);
661 + let root = tree(
662 + &repo,
663 + &[
664 + (T1, note, GixEntryKind::Blob),
665 + ("README", stray, GixEntryKind::Blob),
666 + ("zz", junk_dir, GixEntryKind::Tree),
667 + ],
668 + );
669 +
670 + let engine = GixEngine::new(&repo);
671 + let added = splice_note(
672 + &engine,
673 + Some(ours(root)),
674 + oid(T3),
675 + Some(ours(blob(&repo, "another"))),
676 + )
677 + .unwrap()
678 + .unwrap();
679 + let removed = splice_note(&engine, Some(added), oid(T1), None)
680 + .unwrap()
681 + .unwrap();
682 +
683 + let surviving = paths(&engine, removed);
684 + assert!(surviving.contains(&"README".to_string()), "{surviving:?}");
685 + assert!(
686 + surviving.contains(&"zz/whatever".to_string()),
687 + "{surviving:?}"
688 + );
689 + assert_eq!(flat(&engine, removed).len(), 1);
690 + }
691 +
692 + #[test]
693 + fn a_spliced_tree_reads_back_through_the_reader() {
694 + let (_tmp, repo) = init_bare();
695 + let engine = GixEngine::new(&repo);
696 +
697 + // The end-to-end contract: whatever shape the writer chose, the reader
698 + // finds every note by target and nothing extra. Written one at a time so
699 + // the tree rebalances underneath, which is when a writer and reader that
700 + // disagree about fanout come apart.
701 + let mut tree: Option<Oid> = None;
702 + let mut expected = Vec::new();
703 + for (target, text) in [(T1, "one"), (T2, "two"), (T3, "three")] {
704 + let content = ours(blob(&repo, text));
705 + tree = splice_note(&engine, tree, oid(target), Some(content)).unwrap();
706 + expected.push((oid(target), content));
707 + }
708 + expected.sort_unstable();
709 +
710 + let tree = tree.unwrap();
711 + assert_eq!(flat(&engine, tree), expected);
712 +
713 + let commit = engine
714 + .write_commit(
715 + tree,
716 + &[],
717 + &signature("Max"),
718 + &signature("Max"),
719 + "notes: fixture\n",
720 + )
721 + .unwrap();
722 + engine
723 + .update_ref_cas("refs/notes/commits", None, commit)
724 + .unwrap();
725 +
726 + let ns = namespace(&engine, DEFAULT_NAMESPACE);
727 + for (target, text) in [(T1, "one"), (T2, "two"), (T3, "three")] {
728 + let note = note_for(&engine, ns.tip, oid(target)).unwrap().unwrap();
729 + assert_eq!(note.content_lossy(), text);
730 + }
731 + }
732 +
733 + // ── The engine's write half ──
734 +
735 + #[test]
736 + fn a_written_commit_carries_its_parent_and_signatures() {
737 + let (_tmp, repo) = init_bare();
738 + let engine = GixEngine::new(&repo);
739 + let empty = engine.write_tree(&[]).unwrap();
740 +
741 + let first = engine
742 + .write_commit(empty, &[], &signature("Author"), &signature("Max"), "one\n")
743 + .unwrap();
744 + let second = engine
745 + .write_commit(
746 + empty,
747 + &[first],
748 + &signature("Author"),
749 + &signature("Max"),
750 + "two\n",
751 + )
752 + .unwrap();
753 +
754 + let meta = engine.read_commit(second).unwrap();
755 + assert_eq!(meta.parents, vec![first]);
756 + assert_eq!(meta.tree, empty);
757 + assert_eq!(meta.committer.name, "Max");
758 + assert_eq!(meta.committer.email, "Max@users.makenot.work");
759 + assert_eq!(meta.author.name, "Author");
760 + assert_eq!(meta.message, "two\n");
761 + }
762 +
763 + #[test]
764 + fn cas_creates_a_ref_only_when_it_does_not_exist() {
765 + let (_tmp, repo) = init_bare();
766 + let engine = GixEngine::new(&repo);
767 + let empty = engine.write_tree(&[]).unwrap();
768 + let who = signature("Max");
769 + let first = engine
770 + .write_commit(empty, &[], &who, &who, "one\n")
771 + .unwrap();
772 + let second = engine
773 + .write_commit(empty, &[first], &who, &who, "two\n")
774 + .unwrap();
775 +
776 + engine
777 + .update_ref_cas("refs/notes/commits", None, first)
778 + .unwrap();
779 + assert_eq!(
780 + engine.resolve_ref("refs/notes/commits").unwrap(),
781 + Some(first)
782 + );
783 +
784 + // Creating a ref that now exists is the same lost race as any other: the
785 + // first writer to a fresh namespace wins and the second retries.
786 + let raced = engine.update_ref_cas("refs/notes/commits", None, second);
787 + assert!(matches!(raced, Err(NotesError::Raced)), "{raced:?}");
788 + }
789 +
790 + #[test]
791 + fn cas_rejects_an_update_from_a_stale_tip() {
792 + let (_tmp, repo) = init_bare();
793 + let engine = GixEngine::new(&repo);
794 + let empty = engine.write_tree(&[]).unwrap();
795 + let who = signature("Max");
796 + let first = engine
797 + .write_commit(empty, &[], &who, &who, "one\n")
798 + .unwrap();
799 + let second = engine
800 + .write_commit(empty, &[first], &who, &who, "two\n")
801 + .unwrap();
802 + let third = engine
803 + .write_commit(empty, &[first], &who, &who, "three\n")
804 + .unwrap();
805 +
806 + engine
807 + .update_ref_cas("refs/notes/commits", None, first)
808 + .unwrap();
809 + engine
810 + .update_ref_cas("refs/notes/commits", Some(first), second)
811 + .unwrap();
812 +
813 + // `third` was built on `first`, which is no longer the tip. Accepting it
814 + // would silently drop whatever `second` added.
815 + let raced = engine.update_ref_cas("refs/notes/commits", Some(first), third);
816 + assert!(matches!(raced, Err(NotesError::Raced)), "{raced:?}");
817 + assert_eq!(
818 + engine.resolve_ref("refs/notes/commits").unwrap(),
819 + Some(second),
820 + "a lost race must not move the ref"
821 + );
822 + }
823 +
416 824 #[test]
417 825 fn cache_agrees_with_an_uncached_walk() {
418 826 let (_tmp, repo) = mixed_fanout_repo();