Drop DEFAULT_K to 5, the value the outer fold chose
All five outer folds selected k=5 by inner CV over their own training
corpus, and the pooled outer figure matched the plain sweep exactly, so
15 was costing 2.6 points of macro top-1 for nothing.
Measured the trade before flipping it. layer-stability at k=5: the
deployment-weight flip rate is 1.1% and the value-add population 1.4%,
both identical to k=15 and both within the bar, while deployment-shape
accuracy rises from 97.8% to 99.4% on the same probe. Composition, size
and feedback each get slightly worse, having been over the bar at 15 too.
The one call site that could have wanted a different k does not exist:
"more like this" is similarity::find_similar with its own limit of 50,
so no constant split is needed.
2 files changed,
+21 insertions,
-3 deletions
| 300 |
300 |
|
calibrated, and 9% of samples get no tag at all. `include_policy` is still `false` in
|
| 301 |
301 |
|
`afcl_gen`, so a shipped layer today inherits 0.85 and none of the calibrated numbers
|
| 302 |
302 |
|
hold.
|
| 303 |
|
- |
- **`k` still selects on the data it reports.** k=5 looks best at every resolution
|
| 304 |
|
- |
(98.1% top-1 macro), and that remains a hyperparameter chosen on the test set.
|
|
303 |
+ |
- **`k` is 5, and it no longer selects on the data it reports.** `layer-eval` runs an
|
|
304 |
+ |
outer fold: each fold chooses `k` by inner CV over its own training corpus, then
|
|
305 |
+ |
scores the fold it never saw. All five chose 5, and the pooled outer figure (98.1%
|
|
306 |
+ |
top-1 macro) is exactly what the plain sweep reported at 5, so there was no
|
|
307 |
+ |
selection bias to remove. `DEFAULT_K` was 15 until 2026-08-07, which cost 2.6
|
|
308 |
+ |
points. What the change does not buy is stability: the deployment-weight flip rate
|
|
309 |
+ |
is 1.1% either way, and index composition, index growth and accepted-tag feedback
|
|
310 |
+ |
are all slightly worse at 5 than at 15, having been over their bar at both.
|
| 305 |
311 |
|
|
| 306 |
312 |
|
#### Percussion is excluded, not scored
|
| 307 |
313 |
|
|
| 19 |
19 |
|
use super::features::{FEATURE_VERSION, NUM_FEATURES};
|
| 20 |
20 |
|
|
| 21 |
21 |
|
/// Default neighbor count.
|
| 22 |
|
- |
pub const DEFAULT_K: usize = 15;
|
|
22 |
+ |
///
|
|
23 |
+ |
/// 5, not the 15 shipped until 2026-08-07. Selected under an outer fold — each fold
|
|
24 |
+ |
/// picks k by inner CV over its own training corpus, then scores the fold it never
|
|
25 |
+ |
/// saw — and all five folds chose 5. Worth 2.6 points of macro top-1 at family
|
|
26 |
+ |
/// resolution, and the queue-stability measurement that decides the ship question
|
|
27 |
+ |
/// (`layer-stability`, deployment weight) is unchanged at 1.1%. The measurements it
|
|
28 |
+ |
/// does worsen (index composition, index growth, accepted-tag feedback) were over
|
|
29 |
+ |
/// their bar at 15 as well; a tighter neighbourhood means each neighbour matters
|
|
30 |
+ |
/// more, so they move in the expected direction.
|
|
31 |
+ |
///
|
|
32 |
+ |
/// Not shared with "more like this": that browse is `similarity::find_similar` with
|
|
33 |
+ |
/// its own limit, so this constant is the classifier's alone.
|
|
34 |
+ |
pub const DEFAULT_K: usize = 5;
|
| 23 |
35 |
|
/// Default score at/above which a tag is queued for review.
|
| 24 |
36 |
|
pub const DEFAULT_REVIEW_THRESHOLD: f64 = 0.5;
|
| 25 |
37 |
|
/// Default score at/above which a tag is auto-applied.
|