Skip to main content

max / audiofiles

Retire the bundled official classifier layer Measured against a user's own labels, the layer changed no answers and moved accuracy by no points once that user had tagged about 88 samples. It was a cold-start feature, and browse axes already give a new library structure from zero labels. Reasoning in wiki af-likeness-web. Removes the embedded assets/official.afcl, analysis::official, the afcl generator and its bench mode, the bundled-layer feature in both crates, the first-run seeding, and ConfigKey::OfficialLayerImported. The config key is safe to drop: the feature was never enabled, so no vault carries it and no peer can send it. Kept, deliberately: .afcl import and user-to-user sharing, the exemplar k-NN and its review queue, trained_head, and both bench meters. What was measured to decay is the bundled layer's share of the review queue, and what it decayed into is that queue running on the user's own labels. Renames the settings section to "Imported layers", which is now what every row in it is. Migrates the run tables out of docs/ml_classifier.md into the wiki before trimming it; those numbers existed only in the repo doc. Restates the licence position, reversing the 2026-07-29 reversal that allowed third-party-derived data in the binary. Also fixes two broken rustdoc links to super::official and strips stray XML committed at the end of docs/ml_classifier.md.
Author: Max Johnson <me@maxj.phd> · 2026-08-08 19:29 UTC
Signed with PGP, not checked
Commit: 64f589ea72c0019e2134f2092c3840c4cc4f0690
Parent: ce42999
19 files changed, +116 insertions, -44987 deletions
M docs/ml_classifier.md +25 -218
@@ -144,227 +144,36 @@
144 144
145 145 ## What ships in the binary, and under what licence
146 146
147 - Today: no model and no third-party data. Every number the classifier uses is either
148 - computed from the user's audio at analysis time or written by hand as a threshold.
147 + No model and no third-party data. Every number the classifier uses is either computed
148 + from the user's audio at analysis time or written by hand as a threshold.
149 149
150 - Layer A ships empty and Layer B needs the user's own labels, so a new library starts with
151 - nothing to match against. The intended fix is a bundled official classifier layer: a
152 - `kind = "official"` `.afcl`, imported on first run and weighted below anything the user
153 - labels themselves.
150 + That was the position until 2026-07-29, when it was deliberately reversed to allow a
151 + bundled official classifier layer: a `kind = "official"` `.afcl` built from a CC-BY drum
152 + corpus, embedded in the binary and imported on first run. It was never enabled in a
153 + shipped build, and on 2026-08-08 it was retired outright, so the original position holds
154 + again. Stating that plainly rather than letting it be inferred, because the reversal was
155 + written down here and the return should be too.
154 156
155 - That layer puts data derived from a third-party dataset in the binary. The terms it rests
156 - on:
157 + Why it was retired, in one line: measured against a user's own labels, the layer changed
158 + no answers and moved accuracy by no points once that user had tagged about 88 samples. It
159 + was a cold-start feature, and the app already gives a new library structure from zero
160 + labels through the measured browse axes. The full argument and the measurements are in the
161 + wiki notes `af-likeness-web` and `af-classifier-pipeline`.
157 162
158 - - The corpus is CC-BY 4.0, which permits derivatives.
159 - - Attribution travels with the artifact in `AfclManifest.license_note`, which the UI
160 - surfaces. It credits the dataset the labels came from (`training_source` in the corpus
161 - manifest), not everything the corpus holds.
162 - - An `.afcl` carries no audio. Each exemplar is the 35-number feature vector, 26 of which
163 - are MFCC means and variances, plus its labels. The vector is not reversible to audio.
163 + What that leaves, and it is the whole of it:
164 164
165 - ### Status: built, not shipped
165 + - **Filename rules** name things, and are what the starter pack seeds. Measured at 97.7%
166 + on real packs.
167 + - **The exemplar k-NN** matches against the user's own labels, feeding auto-apply for
168 + their own tagging and a review queue for anything else.
169 + - **Imported `.afcl` layers** still work in both directions. Export yours, import someone
170 + else's; they arrive disabled, weighted below your own labels, and suggest rather than
171 + apply. Nothing is bundled.
172 + - **Browse axes** need no labels at all.
166 173
167 - The whole path exists and is tested. `scripts/corpus.py` lays out the labelled corpus,
168 - `cargo run --release -p audiofiles-bench -- afcl` turns it into a layer,
169 - `crates/audiofiles-core/assets/official.afcl` is the checked-in artifact, and
170 - `analysis::official` embeds it and imports it once per vault.
171 -
172 - It is behind the **`bundled-layer` cargo feature, which is off**, so no shipped build
173 - carries it. Two things have to land before that flips:
174 -
175 - 1. **The layer is drums only.** Built from Reverb Drum Machines one-shots, so it has
176 - exemplars for clap, cymbal, hihat, kick, snare, tom and percussion and none for bass,
177 - vocal, pad, synth, texture, ambience or foley. It would answer for a bass with the
178 - nearest drum it knows. NSynth (CC-BY 4.0, ground-truth `instrument_family_str`) is the
179 - candidate source for the pitched families; it is 16 kHz mono against the drums' 44.1
180 - kHz, and five of the 35 features are Nyquist-bounded, so the two cannot simply be mixed
181 - without the classifier learning sample rate as a proxy for instrument.
182 - 2. **What it can claim is not measured yet, only what it gets right.**
183 - `cargo run --release -p audiofiles-bench -- layer-eval` cross-validates the k-NN
184 - layer over the labelled corpus, five stratified folds, at the runtime `k`, with a
185 - threshold calibrated per class out of fold. The gate lives in `layer_eval.rs` and
186 - was written before the first run. At family resolution both measured families clear
187 - it (below). Accuracy is not the property that decides this, though: the layer's
188 - model is the user's own library, so the question is whether it gives the *same*
189 - answer as that library fills. `layer-stability` measures that half, and at the
190 - weight the app actually builds the answer holds still (1.1% worst consecutive flip
191 - rate). It is measured over two families, so read it as a floor.
192 - 3. **The layer would inherit the global 0.85 regardless.** `include_policy` is `false`
193 - in `afcl_gen`, so the calibrated per-class thresholds are not exported and no
194 - measured number survives the trip into a shipped artifact.
195 -
196 - All three are tracked in the audiofiles GoingsOn project.
197 -
198 - ### What the layer is worth, and when
199 -
200 - It is a cold-start feature. That is a real thing to be, and it is narrower than
201 - "a classifier ships with the app" sounds.
202 -
203 - `layer-stability` compares the index the app builds (the user's own labels at 1.0
204 - beside the bundled layer at 0.5) against the same user labels with no layer at all,
205 - on a held-out probe. Measured 2026-08-07 at `DEFAULT_K` = 5, 1049 drum one-shots,
206 - 178 probe samples of which 69 carry a filename no starter rule fires on. That 69 is
207 - the population the layer exists to serve, since the rest are already answered for
208 - free:
209 -
210 - | user labels | accuracy with the layer | without it | delta |
211 - |---|---|---|---|
212 - | 0 | 98.6% | nothing to answer with | cold start |
213 - | 18 | 98.6% | 91.3% | +7.2 |
214 - | 88 | 100% | 97.1% | +2.9 |
215 - | 352 | 100% | 98.6% | +1.4 |
216 -
217 - The layer answers a library it has never seen at 98.6%, and a user who has tagged a
218 - few dozen samples is getting most of what it has to give. By a few hundred labels
219 - their own tags carry the library and the layer is worth about a point: at 352 labels
220 - it changes one answer in 69.
221 -
222 - Two things not to oversell from this. The cold-start window overlaps something the
223 - app already does without any classifier: browse axes give a new library structure
224 - from zero labels and zero exemplars (`af-browse-axes`). And the measurement is two
225 - families on a drums-only corpus. More classes plausibly widen the window, since the
226 - layer's advantage over a thin user library should grow with the number of things to
227 - be wrong about, so this table is a result at a resolution, not a permanent property.
228 -
229 - ### Retracted: runs 1 and 2 answer the wrong question
230 -
231 - The two tables below grade **seven specific drum instruments**. Both halves of that
232 - were settled against on 2026-07-29, before either run happened:
233 -
234 - - audiofiles classifies at coarse family resolution. Instrument identity is not
235 - recoverable from these features (33.4% for a 7-way specific split with ~40 tuned
236 - thresholds, against 92.4% for families on one unfitted cut), instrument labels are
237 - not perceptually coherent (the best one lands in its top perceptual bin 54% of the
238 - time), and only 13.5% of real users tag a specific instrument at all.
239 - - Drums-only is the defect the scope work exists to fix, not the scope to measure
240 - within.
241 -
242 - They are kept here because the gate change between them is worth auditing and because
243 - deleting a published number is worse than marking it. They are not superseded results,
244 - they are results for a question audiofiles is not asking. Do not build on them, and do
245 - not compare them against the family numbers further down: different label space,
246 - different corpus subset. `AF_BENCH_EVAL_LABELS=instrument` reproduces them.
247 -
248 - ### What runs 1 and 2 said (instrument resolution, retired)
249 -
250 - First run, 1049 files across the seven drum classes, features v5:
251 -
252 - | | top-1 recall | precision @ 0.85 | recall @ 0.85 | recall @ 0.50 |
253 - |---|---|---|---|---|
254 - | kick | 93.7% | 93.7% | 65.4% | 92.7% |
255 - | tom | 89.8% | 100% | 74.4% | 89.0% |
256 - | snare | 76.6% | 97.8% | 47.9% | 69.7% |
257 - | hi-hat | 71.6% | 100% | 5.5% | 56.0% |
258 - | clap | 70.8% | 100% | 6.2% | 47.9% |
259 - | cymbal | 62.8% | 100% | 2.3% | 44.2% |
260 - | percussion | 46.7% | never fires | 0.0% | 28.1% |
261 - | macro | 73.1% | 84.5% | 28.8% | 61.1% |
262 -
263 - Read against the retired threshold tree, this is a different result: 73.1% macro top-1
264 - against that tree's 33.4% strict, with no unreachable class. Instrument identity is
265 - more present in the 35-feature vector under k-NN than it was under thresholds.
266 -
267 - That table grades every class at one global threshold, which turned out to be the
268 - wrong question. A score is the share of the k=15 neighbourhood's kernel weight
269 - carrying a tag, so 0.85 asks for roughly 13 of 15 neighbours to agree, and whether a
270 - class can reach that depends on how many of its own members sit inside a fixed `k`.
271 - Hi-hat (109 files) and snare (188) are equally separable by top-1, 71.6% against
272 - 76.6%, and differ ninefold in recall at 0.85. The score behaves sensibly inside a
273 - class and is not comparable across classes. Nor is 0.85 itself measured: it predates
274 - any evaluation of this layer.
275 -
276 - ### The per-class answer (instrument resolution, retired)
277 -
278 - So the harness calibrates a threshold per class instead: the most permissive one at
279 - which the class is 95%-confident (Wilson lower bound) of holding 95% precision. Those
280 - are `tag_policy` rows, which the layer format can already carry. Thresholds are fitted
281 - on the folds a sample is not in, so nothing picks its operating point from the data it
282 - is graded on.
283 -
284 - | | threshold | held-out precision | held-out recall |
285 - |---|---|---|---|
286 - | tom | 0.759 | 97.5% | 80.9% |
287 - | kick | 0.932 | 99.1% | 50.7% |
288 - | snare | 0.883 | 98.9% | 43.1% |
289 - | hi-hat, clap, cymbal, percussion | none exists | | |
290 -
291 - **Four of the seven classes cannot be auto-applied at 95% precision at any threshold,
292 - and sweeping `k` over 5, 10, 15, 25 and 50 does not rescue them.** Smaller `k` does
293 - improve separability (top-1 macro rises to 80.7% at k=5), so it is worth knowing for
294 - ranking and "more like this", but it does not convert into a shippable auto-apply
295 - policy: the count of uncalibratable classes is four at every `k` from 5 to 15, and
296 - worse above.
297 -
298 - What that leaves is a three-class layer. Kick, snare and tom clear the bar with 43% to
299 - 81% recall, which is a real head start on the classes that dominate a drum library.
300 -
301 - Percussion is a taxonomy problem rather than a tuning one. It is a catch-all (cowbell,
302 - clave, maraca, bongo, conga), and the confusion matrix scatters it across every other
303 - row at 46.7% top-1. No threshold fixes a class that is not one sound.
304 -
305 - ### Run 3, at family resolution
306 -
307 - Same harness, same folds, same calibration. The only change is that corpus labels are
308 - projected onto coarse families as they are read back (`bench/src/families.rs`), so the
309 - index carries family tags and the k-NN aggregates a neighbourhood across every drum in
310 - a family instead of splitting it seven ways.
311 -
312 - Two of the seven families have corpus material. `low` is kick plus tom, `drum-bright`
313 - is snare, clap, hi-hat and cymbal.
314 -
315 - | | n | top-1 recall | calibrated threshold | held-out precision | held-out recall |
316 - |---|---|---|---|---|---|
317 - | low | 451 | 98.2% | 0.652 | 96.7% | 97.6% |
318 - | drum-bright | 431 | 92.8% | 0.194 | 96.8% | 97.2% |
319 - | macro | 882 | 95.5% | | 96.1% | 97.4% |
320 -
321 - Both families clear the gate, and clear it by a distance the instrument runs never
322 - approached: held-out recall at a 95%-confident 95% precision bar goes from 43-81% on
323 - three of seven classes to 97% on both families. The failure the instrument runs
324 - reported was mostly the resolution, not the layer.
325 -
326 - Three things to read carefully rather than celebrate:
327 -
328 - - **`drum-bright` calibrates to 0.194, and the threshold moves 0.13 to 0.34 across
329 - folds.** A class holding 97% precision at a fifth of its neighbourhood is a class the
330 - index rarely confuses, but a threshold that unstable is not yet a property to ship.
331 - - **The shipped global 0.85 costs real recall here and is not the operating point
332 - either family wants.** At 0.85, `drum-bright` recall is 84.0% against 97.2%
333 - calibrated, and 9% of samples get no tag at all. `include_policy` is still `false` in
334 - `afcl_gen`, so a shipped layer today inherits 0.85 and none of the calibrated numbers
335 - hold.
336 - - **`k` is 5, and it no longer selects on the data it reports.** `layer-eval` runs an
337 - outer fold: each fold chooses `k` by inner CV over its own training corpus, then
338 - scores the fold it never saw. All five chose 5, and the pooled outer figure (98.1%
339 - top-1 macro) is exactly what the plain sweep reported at 5, so there was no
340 - selection bias to remove. `DEFAULT_K` was 15 until 2026-08-07, which cost 2.6
341 - points. What the change does not buy is stability: the deployment-weight flip rate
342 - is 1.1% either way, and index composition, index growth and accepted-tag feedback
343 - are all slightly worse at 5 than at 15, having been over their bar at both.
344 -
345 - #### Percussion is excluded, not scored
346 -
347 - 167 files, 16% of the corpus, dropped from training and testing both. It has no honest
348 - family label: it measures 47 low / 120 bright and 13% coherence across 19 perceptual
349 - bins, and the only per-file register signal available is spectral centroid, which is
350 - one of the 35 features the layer scores on. Labelling ground truth with the
351 - classifier's own input would report a class it cannot miss. `texture` is its likely
352 - home once there is material to check that against.
353 -
354 - #### Tom folds into `low`
355 -
356 - The one open split in `af-coarse-families`, and the corpus answers it. Running tom as
357 - its own class (`AF_BENCH_EVAL_LABELS=family-tom-split`) is worse on every axis:
358 -
359 - | | macro top-1 | macro held-out recall | worst class held-out recall |
360 - |---|---|---|---|
361 - | tom folded into `low` | 95.5% | 97.4% | 97.2% |
362 - | tom held out | 92.4% | 78.6% | 57.6% |
363 -
364 - Split out, `low` (kick alone) drops to 57.6% held-out recall and the confusion is
365 - mutual: 20 toms read as `low`, 10 kicks read as tom. Folded in, toms are recovered as
366 - `low` 97.2% of the time against kick's 99.5%, so the family's members behave alike.
367 - Tom is not a family. It is the low register.
174 + `audiofiles-bench` keeps two meters over the k-NN, `layer-eval` for whether an answer is
175 + right and `layer-stability` for whether it is the same answer next week. They outlived the
176 + layer they were built for because they measure the substrate the app still runs on.
368 177
369 178 ## Feature vector
370 179
@@ -417,5 +226,3 @@
417 226 | Spectral features | `crates/audiofiles-core/src/analysis/spectral.rs` |
418 227 | MFCC computation | `crates/audiofiles-core/src/analysis/mfcc.rs` |
419 228 | Crest factor, attack time | `crates/audiofiles-core/src/analysis/basic.rs` |
420 - </content>
421 - </invoke>
M crates/audiofiles-browser/Cargo.toml -3
@@ -7,9 +7,6 @@
7 7 [features]
8 8 default = ["device-profiles"]
9 9 device-profiles = ["dep:audiofiles-rhai", "dep:rayon"]
10 - # Seed a fresh vault with the bundled official classifier layer. Off; see
11 - # audiofiles-core's feature of the same name for why.
12 - bundled-layer = ["audiofiles-core/bundled-layer"]
13 10
14 11 [dependencies]
15 12 audiofiles-core = { workspace = true }
M crates/audiofiles-core/Cargo.toml -7
@@ -7,13 +7,6 @@
7 7 [features]
8 8 default = ["analysis"]
9 9 analysis = ["dep:bs1770", "dep:realfft"]
10 - # The bundled official classifier layer. OFF: the layer is drums-only and
11 - # unmeasured, and a default classifier that is wrong on a user's library is worse
12 - # than none, because it puts wrong tags on their samples with the app's authority
13 - # behind it. Off also means the 1.1 MiB embedded blob is not compiled in.
14 - # Turn back on when the corpus is broadened and the layer measures well enough to
15 - # ship; see the audiofiles GoingsOn project.
16 - bundled-layer = []
17 10
18 11 [dependencies]
19 12 rusqlite = { workspace = true }
M crates/audiofiles-bench/src/labelled.rs +6 -9
@@ -104,12 +104,13 @@
104 104 }
105 105
106 106 /// A scratch vault holding the analysed, tagged corpus.
107 + ///
108 + /// Just the database. It used to carry the per-class file lists and the analysed
109 + /// count as well, for the `afcl` generator's manifest; that generator went with
110 + /// the bundled layer (2026-08-08, wiki `af-likeness-web`) and both meters read
111 + /// the corpus back out of `db` rather than off this struct.
107 112 pub(crate) struct LabelledVault {
108 113 pub(crate) db: Database,
109 - /// Files per class, as found on disk.
110 - pub(crate) by_tag: BTreeMap<&'static str, Vec<PathBuf>>,
111 - /// Files that produced a feature vector.
112 - pub(crate) analyzed: usize,
113 114 }
114 115
115 116 /// Import, analyse and tag every labelled file into a fresh vault at `vault`.
@@ -211,11 +212,7 @@
211 212 }
212 213 );
213 214
214 - Ok(LabelledVault {
215 - db,
216 - by_tag,
217 - analyzed,
218 - })
215 + Ok(LabelledVault { db })
219 216 }
220 217
221 218 #[cfg(test)]
M crates/audiofiles-bench/src/layer_eval.rs +4 -3
@@ -1,8 +1,9 @@
1 1 //! Cross-validated evaluation of the exemplar k-NN layer over the labelled corpus.
2 2 //!
3 - //! This is the meter the bundled `.afcl` never had. `afcl_gen` builds a layer;
4 - //! nothing said whether that layer answers correctly, so "tuned enough to ship"
5 - //! had no number behind it and the layer sits behind an off-by-default feature.
3 + //! This is the meter the bundled `.afcl` never had, and the reason it was
4 + //! eventually retired rather than shipped (2026-08-08, wiki `af-likeness-web`).
5 + //! The generator is gone; this outlived it because what it measures is the
6 + //! exemplar k-NN itself, which the app still runs on.
6 7 //!
7 8 //! What it measures: stratified k-fold cross-validation of [`exemplar`] over the
8 9 //! same corpus the layer is built from, at the same `k` the app uses at runtime.
M crates/audiofiles-bench/src/main.rs +18 -42
@@ -11,31 +11,32 @@
11 11 //! `cargo run --release -p audiofiles-bench -- ingest` vault ingest + queries
12 12 //! `cargo run --release -p audiofiles-bench -- accuracy` bpm/key vs ground truth
13 13 //! `cargo run --release -p audiofiles-bench -- layout` blob layout migration
14 - //! `cargo run --release -p audiofiles-bench -- afcl` build the official layer
15 14 //! `cargo run --release -p audiofiles-bench -- layer-eval` cross-validate that layer
16 15 //! `cargo run --release -p audiofiles-bench -- layer-stability` does it answer twice
17 16 //!
18 - //! Two modes are not measurements. `layout` is a checker: it fabricates flat
19 - //! vaults, sweeps them, and exits non-zero if any scenario fails. `afcl` is a
20 - //! generator: it turns the labelled corpus into the bundled official `.afcl`.
21 - //! Both live here because corpus walking and scratch-vault fabrication do.
17 + //! One mode is not a measurement: `layout` is a checker, fabricating flat
18 + //! vaults, sweeping them, and exiting non-zero if any scenario fails. It lives
19 + //! here because corpus walking and scratch-vault fabrication do.
22 20 //!
23 - //! `layer-eval` and `layer-stability` are the two meters for what `afcl`
24 - //! generates, and they measure independent properties: whether an answer is right,
25 - //! and whether it is the same answer next week. A consistently wrong answer is
21 + //! `layer-eval` and `layer-stability` are the two meters for the exemplar k-NN,
22 + //! and they measure independent properties: whether an answer is right, and
23 + //! whether it is the same answer next week. A consistently wrong answer is
26 24 //! perfectly stable, so neither can be inferred from the other.
27 25 //!
28 - //! `layer-eval` is the meter for what `afcl` generates: stratified k-fold
29 - //! cross-validation of the k-NN layer over the same corpus, per class, against a
30 - //! ship gate written down before the first run. It shares `afcl`'s corpus import
31 - //! (see `labelled`), which is why it lives beside it.
26 + //! `layer-eval` is stratified k-fold cross-validation of the k-NN over the
27 + //! labelled corpus, per class, against a gate written down before the first run.
28 + //! Both modes read the corpus through `labelled`.
29 + //!
30 + //! The `afcl` generator that built the bundled official layer is gone with the
31 + //! layer itself (2026-08-08, wiki `af-likeness-web`). These two meters outlived
32 + //! it deliberately: they measure the substrate the app still runs on, and they
33 + //! are what the retirement decision was made from.
32 34 //!
33 35 //! Env: `AF_BENCH_CORPUS` (corpus root, default `<repo>/samples`),
34 36 //! `AF_BENCH_VAULT` (scratch vault for ingest and layout), `AF_BENCH_FSL10K`
35 37 //! (FSL10K root for accuracy), `AF_BENCH_BATCH`, `AF_BENCH_LIMIT`,
36 38 //! `AF_BENCH_ANALYZE`, `AF_BENCH_LAYOUT_N`, `AF_BENCH_JSON` (machine-readable
37 - //! output path), `AF_AFCL_OUT` (where `afcl` writes the layer),
38 - //! `AF_BENCH_EVAL_FOLDS` and `AF_BENCH_EVAL_K` (folds and the neighbour-count
39 + //! output path), `AF_BENCH_EVAL_FOLDS` and `AF_BENCH_EVAL_K` (folds and the neighbour-count
39 40 //! sweep for `layer-eval`), `AF_BENCH_STABILITY_PROBE` (probe fraction for
40 41 //! `layer-stability`),
41 42 //! `AF_BENCH_STAGES` (files per per-stage probe during `ingest`, 0 = off).
@@ -52,7 +53,6 @@
52 53 //! code; see `storage`.
53 54
54 55 mod accuracy;
55 - mod afcl_gen;
56 56 mod calibration;
57 57 mod families;
58 58 mod ingest;
@@ -323,34 +323,10 @@
323 323 return;
324 324 }
325 325
326 - if args.first().map(String::as_str) == Some("afcl") {
327 - // Its own scratch vault for the same reason `layout` has one: this mode
328 - // deletes and rebuilds the vault, and pointing it at a corpus vault
329 - // someone had just imported would be an expensive surprise.
330 - let vault = std::env::var("AF_BENCH_VAULT")
331 - .map_or_else(|_| std::env::temp_dir().join("af-afcl-gen"), PathBuf::from);
332 - // Defaults into audiofiles-core's own assets/, which is where the layer
333 - // is embedded from. Keeping the artifact inside the crate that
334 - // `include_str!`s it means the path never reaches outside the crate
335 - // directory, and the blob travels with the crate rather than with the
336 - // workspace root.
337 - let out = std::env::var("AF_AFCL_OUT").map_or_else(
338 - |_| {
339 - PathBuf::from(env!("CARGO_MANIFEST_DIR"))
340 - .parent()
341 - .unwrap()
342 - .join("audiofiles-core/assets/official.afcl")
343 - },
344 - PathBuf::from,
345 - );
346 - afcl_gen::run(&samples_dir, &vault, &out, &full_pipeline_config());
347 - return;
348 - }
349 -
350 326 if args.first().map(String::as_str) == Some("layer-eval") {
351 - // Its own scratch vault, distinct from `afcl`'s: the two modes rebuild
352 - // the same corpus and running one must not delete the other's vault out
353 - // from under a comparison.
327 + // Its own scratch vault, distinct from `layer-stability`'s: the two modes
328 + // rebuild the same corpus and running one must not delete the other's
329 + // vault out from under a comparison.
354 330 let vault = std::env::var("AF_BENCH_VAULT").map_or_else(
355 331 |_| std::env::temp_dir().join("af-layer-eval"),
356 332 PathBuf::from,
M crates/audiofiles-bench/src/rows.rs +1 -1
@@ -253,7 +253,7 @@
253 253 manifest: AfclManifest {
254 254 afcl_version: afcl::AFCL_VERSION,
255 255 feat_version: FEATURE_VERSION,
256 - name: "simulated bundled layer".into(),
256 + name: "simulated imported layer".into(),
257 257 description: "built by audiofiles-bench, never written to disk".into(),
258 258 kind: afcl::LayerKind::Official.as_str().to_string(),
259 259 created_at: 0,
M crates/audiofiles-bench/src/stability.rs +7 -5
@@ -9,8 +9,10 @@
9 9 //!
10 10 //! # Why this is an attention gate and not a safety gate
11 11 //!
12 - //! The bundled layer ships **suggest-only** (decided 2026-08-07): it never
13 - //! auto-applies, it populates a review queue. That kills the failure this
12 + //! Exemplars from outside the user's own labelling are **suggest-only** (decided
13 + //! 2026-08-07 for the bundled layer, and inherited by imported layers when that
14 + //! was retired): they never auto-apply, they populate a review queue. That kills
15 + //! the failure this
14 16 //! measurement was originally filed against. The apply path is monotonic
15 17 //! (`apply_policy` skips tags the sample already has, `apply_tag_sourced` is
16 18 //! INSERT OR IGNORE, only `remove_tags_by_source` removes), so a changed answer
@@ -40,7 +42,7 @@
40 42 //! 2. [`size`] — same mix, growing index. A library grows by accretion, so this is
41 43 //! the shape of every real user's second month.
42 44 //! 3. [`deployment_shape`] — the mixed index the app actually builds: the user's
43 - //! own labels at 1.0 beside the bundled layer at `DEFAULT_IMPORT_WEIGHT` 0.5.
45 + //! own labels at 1.0 beside an imported layer at `DEFAULT_IMPORT_WEIGHT` 0.5.
44 46 //! Never measured before this module, and the most decision-relevant of the
45 47 //! four. Every earlier number came off a uniform-weight index, where the weight
46 48 //! is a constant multiplier inside a sum divided by its own total and therefore
@@ -915,7 +917,7 @@
915 917
916 918 /// The mixed index the app actually builds, and the one nothing had measured.
917 919 ///
918 - /// The bundled layer sits at [`rows::IMPORT_WEIGHT`] beneath the user's own labels
920 + /// An imported layer sits at [`rows::IMPORT_WEIGHT`] beneath the user's own labels
919 921 /// at 1.0. Every earlier number came off a uniform index where that weight
920 922 /// cancels; here it does not, and neither does `build_index` fitting the
921 923 /// standardization params over both populations together.
@@ -949,7 +951,7 @@
949 951
950 952 let (layer_rows, user_pool) = halve(pool);
951 953 println!(
952 - " The pool splits stratified into a simulated bundled layer ({} exemplars,\n \
954 + " The pool splits stratified into a simulated imported layer ({} exemplars,\n \
953 955 imported at {}) and a user pool ({} labels, weight 1.0) the user's own\n \
954 956 library is drawn from. Same class mix on both sides.",
955 957 layer_rows.len(),
M crates/audiofiles-core/src/config_key.rs -16
M crates/audiofiles-browser/src/state/tests.rs -39
M crates/audiofiles-browser/src/ui/classifier.rs +21 -40
M crates/audiofiles-browser/src/ui/review_library.rs +8 -10
M crates/audiofiles-core/src/analysis/afcl.rs +17 -13
M crates/audiofiles-core/src/analysis/exemplar.rs +9 -4
M crates/audiofiles-core/src/analysis/mod.rs -2
M crates/audiofiles-browser/src/backend/direct/mod.rs -18
D crates/audiofiles-bench/src/afcl_gen.rs -304
D crates/audiofiles-core/assets/official.afcl -44076
D crates/audiofiles-core/src/analysis/official.rs -177