max / audiofiles
- Co-Authored-By
- Claude Opus 5 (1M context) <noreply@anthropic.com>
5 files changed,
+16 insertions,
-27 deletions
| @@ -13,8 +13,7 @@ | |||
| 13 | 13 | audiofiles-app/ # App entry point (thin shell) | |
| 14 | 14 | audiofiles-sync/ # SyncKit cloud sync integration | |
| 15 | 15 | audiofiles-rhai/ # Device plugin runtime (TOML manifests + Rhai hooks) | |
| 16 | - | audiofiles-train/ # ML classifier training (dev-only binary) | |
| 17 | - | audiofiles-bench/ # Performance benchmarks | |
| 16 | + | audiofiles-bench/ # Benchmarks (dev-only binary) | |
| 18 | 17 | audiofiles-rhai/plugins/bundled/ # Bundled device profiles (SP-404, MPC, etc.) | |
| 19 | 18 | dist/ # Build scripts for macOS, Windows, Linux | |
| 20 | 19 | ``` | |
| @@ -28,8 +27,7 @@ | |||
| 28 | 27 | | `audiofiles-app` | Entry point | Thin shell, creates backend + launches GUI | | |
| 29 | 28 | | `audiofiles-sync` | Cloud sync | Uses tokio + `spawn_blocking` for rusqlite | | |
| 30 | 29 | | `audiofiles-rhai` | Device plugins | TOML manifests + sandboxed Rhai hooks | | |
| 31 | - | | `audiofiles-train` | ML training | Dev-only, not shipped | | |
| 32 | - | | `audiofiles-bench` | Benchmarks | criterion-based | | |
| 30 | + | | `audiofiles-bench` | Benchmarks | Dev-only, not shipped. Needs a corpus (`scripts/corpus.py`) | | |
| 33 | 31 | ||
| 34 | 32 | **Critical rule:** `audiofiles-core` is entirely synchronous. `rusqlite::Connection` is `!Send`, so all database operations are synchronous. Long-running operations (import, analysis, export) use dedicated worker threads with channel-based message passing. | |
| 35 | 33 |
| @@ -20,8 +20,10 @@ | |||
| 20 | 20 | # Run all workspace tests | |
| 21 | 21 | cargo test --workspace | |
| 22 | 22 | ||
| 23 | - | # Train the ML classifier (developers only) | |
| 24 | - | cargo run -p audiofiles-train -- /path/to/training-data | |
| 23 | + | # Benchmarks (developers only; needs a corpus, see scripts/corpus.py) | |
| 24 | + | cargo run --release -p audiofiles-bench # analysis pipeline | |
| 25 | + | cargo run --release -p audiofiles-bench -- ingest # import and query | |
| 26 | + | cargo run --release -p audiofiles-bench -- accuracy # bpm/key vs ground truth | |
| 25 | 27 | ``` | |
| 26 | 28 | ||
| 27 | 29 | ## Workspace Architecture | |
| @@ -35,9 +37,9 @@ | |||
| 35 | 37 | | `audiofiles-app` | `crates/audiofiles-app/` | Standalone desktop app via eframe. System audio (cpal), drag-and-drop import, native drag-out to Finder/DAWs, system tray, CLI import, OTA updates. | | |
| 36 | 38 | | `audiofiles-sync` | `crates/audiofiles-sync/` | Cloud sync via SyncKit. Pushes/pulls sample metadata, tags, and VFS structure across devices. E2E encrypted. | | |
| 37 | 39 | | `audiofiles-rhai` | `crates/audiofiles-rhai/` | Rhai scripting engine for device export profiles. Transforms sample metadata and file layout for hardware samplers. | | |
| 38 | - | | `audiofiles-train` | `crates/audiofiles-train/` | ML classifier training binary. Builds the random forest model from labeled sample data. Not shipped in the app. | | |
| 40 | + | | `audiofiles-bench` | `crates/audiofiles-bench/` | Benchmark binary. Analysis-pipeline timing, vault ingest and query latency, and BPM/key accuracy against labeled corpora. Not shipped in the app. | | |
| 39 | 41 | ||
| 40 | - | Dependency flow: `audiofiles-core` is the leaf -> `audiofiles-rhai` and `audiofiles-sync` depend on core -> `audiofiles-browser` depends on core, sync, and rhai -> `audiofiles-app` depends on browser and core. `audiofiles-train` depends on core only. | |
| 42 | + | Dependency flow: `audiofiles-core` is the leaf -> `audiofiles-rhai` and `audiofiles-sync` depend on core -> `audiofiles-browser` depends on core, sync, and rhai -> `audiofiles-app` depends on browser and core. `audiofiles-bench` depends on core only. | |
| 41 | 43 | ||
| 42 | 44 | Theme loading comes from [makeover](https://crates.io/crates/makeover), published to crates.io. Shared libraries from `../../MNW/shared/`: [synckit-client](../../MNW/shared/synckit-client/) (cloud sync SDK). | |
| 43 | 45 | ||
| @@ -89,8 +91,8 @@ | |||
| 89 | 91 | | What | Where | | |
| 90 | 92 | |------|-------| | |
| 91 | 93 | | Domain library | `crates/audiofiles-core/src/` | | |
| 92 | - | | ML classifier model | `crates/audiofiles-core/models/layer2_drum.json` | | |
| 93 | - | | Training binary | `crates/audiofiles-train/` | | |
| 94 | + | | Classification rules | `crates/audiofiles-core/src/analysis/classify.rs` | | |
| 95 | + | | Benchmarks + corpus builder | `crates/audiofiles-bench/`, `scripts/corpus.py` | | |
| 94 | 96 | | UI components | `crates/audiofiles-browser/src/` | | |
| 95 | 97 | | Desktop app shell | `crates/audiofiles-app/src/` | | |
| 96 | 98 | | Device export profiles | `crates/audiofiles-rhai/plugins/bundled/` | |
| @@ -6,7 +6,7 @@ | |||
| 6 | 6 | ||
| 7 | 7 | ## Workspace Layout | |
| 8 | 8 | ||
| 9 | - | The project is a 5-crate Rust workspace (plus an optional training binary): | |
| 9 | + | The project is a 5-crate Rust workspace (plus a dev-only benchmark binary): | |
| 10 | 10 | ||
| 11 | 11 | ### audiofiles-core | |
| 12 | 12 | ||
| @@ -81,7 +81,7 @@ | |||
| 81 | 81 | - **BPM**: Onset-based tempo estimation using spectral flux and autocorrelation. | |
| 82 | 82 | - **Key**: Musical key detection from chroma features. | |
| 83 | 83 | - **MFCC**: Mel-Frequency Cepstral Coefficients computed from existing STFT magnitudes (26-band mel filterbank, log energy, DCT-II, 13 coefficients). Aggregated as mean + variance across frames (26 features total). | |
| 84 | - | - **Classification**: Two-layer ML system mapping 35-feature vectors (9 spectral/waveform + 26 MFCC) to 16 sample categories (kick, snare, hihat, cymbal, percussion, bass, vocal, synth, pad, fx, noise, music, ambience, impact, foley, texture). Layer 1: rule-based broad classifier detects drums vs non-drum categories. Layer 2: 200-tree Random Forest (4.0MB, embedded via `include_bytes!`, `OnceLock` lazy init) for drum sub-classification (kick/snare/hihat/cymbal/percussion). Confidence scores from RF vote fraction. 94.4% strict accuracy on 4343 labeled drum samples. Training binary in `audiofiles-train` crate (not built by default). | |
| 84 | + | - **Classification**: Deterministic DSP only. A priority-ordered threshold tree (`analysis/classify.rs`) maps the 35-feature vector (9 spectral/waveform + 26 MFCC) to a `SampleClass`; first rule to match wins. The 35-feature vector is persisted to `sample_features` for the forthcoming rules + k-NN tag pipeline. No trained model ships in the binary, which keeps the classifier free of training-data copyright surface; the Random Forest that previously refined drum sub-classes was removed. See `ml_classifier.md`. Accuracy is measured out-of-tree by `audiofiles-bench` against a labeled corpus. | |
| 85 | 85 | - **Loop detection**: Identifies whether a sample is a seamless loop. | |
| 86 | 86 | - **Fingerprinting**: Computes an amplitude envelope fingerprint for near-duplicate detection across the library. | |
| 87 | 87 | - **Tag suggestion**: Generates tag suggestions from analysis results (classification, BPM range, key, duration bracket) with confidence scores and human-readable reasons. |
| @@ -50,7 +50,7 @@ | |||
| 50 | 50 | ### Classification (16 Categories) | |
| 51 | 51 | Kick, Snare, HiHat, Cymbal, Percussion, Bass, Vocal, Synth, Pad, FX, Noise, Music, Ambience, Impact, Foley, Texture | |
| 52 | 52 | ||
| 53 | - | Two-layer ML system: rule-based broad classifier (Layer 1) + 200-tree Random Forest for drum sub-classification (Layer 2). 94.4% accuracy on labeled drum samples. | |
| 53 | + | Classification runs on deterministic signal measurement: a priority-ordered rule tree over 35 extracted features (spectral shape, crest factor, attack time, MFCCs). No trained model ships in the app, so nothing about your library leaves it and the classifier carries no training-data licence baggage. | |
| 54 | 54 | ||
| 55 | 55 | ### Tag System | |
| 56 | 56 | - Hierarchical dot-notation tags (e.g., `genre.electronic.house`, `instrument.drum.kick`) |
| @@ -249,12 +249,6 @@ | |||
| 249 | 249 | fn main() { | |
| 250 | 250 | let args: Vec<String> = std::env::args().skip(1).collect(); | |
| 251 | 251 | let samples_dir = corpus_dir(); | |
| 252 | - | let project_root = PathBuf::from(env!("CARGO_MANIFEST_DIR")) | |
| 253 | - | .parent() | |
| 254 | - | .unwrap() | |
| 255 | - | .parent() | |
| 256 | - | .unwrap() | |
| 257 | - | .to_path_buf(); | |
| 258 | 252 | ||
| 259 | 253 | if args.first().map(String::as_str) == Some("ingest") { | |
| 260 | 254 | let vault = std::env::var("AF_BENCH_VAULT").map_or_else( | |
| @@ -570,14 +564,9 @@ | |||
| 570 | 564 | println!(" Memory: {:.1} MB", frame_mem as f64 / 1_048_576.0); | |
| 571 | 565 | println!(); | |
| 572 | 566 | ||
| 573 | - | // Model size | |
| 574 | - | let model_path = project_root.join("crates/audiofiles-core/models/layer2_drum.json"); | |
| 575 | - | if let Ok(meta) = std::fs::metadata(&model_path) { | |
| 576 | - | println!( | |
| 577 | - | " RF model (layer2_drum.json): {:.1} MB on disk, embedded at compile time", | |
| 578 | - | meta.len() as f64 / 1_048_576.0 | |
| 579 | - | ); | |
| 580 | - | } | |
| 567 | + | // The random-forest model this used to report on was removed along with the | |
| 568 | + | // trained drum sub-classifier (docs/ml_classifier.md); classification is | |
| 569 | + | // deterministic DSP now, with no model on disk to measure. | |
| 581 | 570 | println!(); | |
| 582 | 571 | ||
| 583 | 572 | // Section 5: Classification Accuracy |