Skip to main content

max / granma

Initial commit: granular sampler design spec
Author: Max J. <87768334+MaxJMath@users.noreply.github.com> · 2026-02-17 18:55 UTC
Commit: ef45cc0a7198cb831bd8348b263590d28df50d39
1 file changed, +236 insertions, -0 deletions
A audiofiles-granular.md +236
@@ -0,0 +1,236 @@
1 + # audiofiles-granular
2 +
3 + A programmable granular sampler CLAP plugin built on the AudioFiles architecture. Designed for experimental sound designers and musicians who find conventional granular instruments too limiting — where the synthesis behavior itself needs to be as composable and scriptable as the music being made.
4 +
5 + ---
6 +
7 + ## The Problem With Existing Granular Tools
8 +
9 + Most granular instruments give you a handful of knobs — position, size, density, scatter — and expect you to twist them manually. That's fine for ambient washes. It's not fine when you want a grain cloud whose density follows the amplitude envelope of a different sample, or position that steps through a Euclidean rhythm, or pitch scatter that narrows as MIDI velocity increases. The parameter space exists; the programmability doesn't.
10 +
11 + audiofiles-granular treats the grain scheduler itself as a scripting target. Every grain spawn is a hook. You write the behavior.
12 +
13 + ---
14 +
15 + ## Architecture
16 +
17 + **New crate: `crates/audiofiles-granular/`**
18 +
19 + Sits alongside `audiofiles-plugin`, depends on `audiofiles-core` and `audiofiles-rhai`. Shares the same SQLite database and flat content-addressed store — no asset duplication, no separate library management. The same samples you've organized in AudioFiles are immediately available as granular source material.
20 +
21 + ```
22 + audiofiles-granular/
23 + src/
24 + lib.rs # nih-plug plugin entry, params
25 + engine.rs # grain scheduler, voice pool
26 + grain.rs # single grain: position, envelope, pitch, pan
27 + sampler.rs # sample buffer cache (Arc<Vec<f32>>, keyed by hash)
28 + modulation.rs # LFO, envelope follower, MIDI CC routing
29 + editor.rs # egui UI: VFS browser, waveform, script editor
30 + state.rs # SharedState bridge (audio/GUI thread sync)
31 + script.rs # Rhai integration for per-grain hooks
32 + ```
33 +
34 + No async runtime. No tokio. The audio thread is entirely synchronous — grain scheduling, buffer reads, output mixing all happen in `process()`. Sample decoding runs on a background `std::thread` with a `crossbeam_channel` handoff, same pattern as the preview engine in `audiofiles-plugin`.
35 +
36 + ---
37 +
38 + ## Grain Engine
39 +
40 + Each grain is a fixed-size struct, allocated upfront in a pool — no heap allocation inside the audio callback:
41 +
42 + ```rust
43 + struct Grain {
44 + sample_hash: [u8; 32], // references content store directly by hash
45 + playhead: f32, // current read position (fractional frames)
46 + duration_frames: usize,
47 + pitch_ratio: f32, // resampling ratio (1.0 = original pitch)
48 + pan: f32,
49 + amplitude: f32,
50 + envelope_phase: f32, // 0.0..1.0, shape set per-grain at spawn
51 + reverse: bool,
52 + active: bool,
53 + }
54 + ```
55 +
56 + Voice pool is fixed at 64 grains (configurable at compile time). The scheduler fires new grains at a rate determined by density, jittered by scatter. Multiple MIDI notes run independent grain clouds with their own position anchors — the instrument is fully polyphonic.
57 +
58 + Sample buffers live in a `HashMap<[u8; 32], Arc<Vec<f32>>>` keyed by content hash. Two VFS nodes pointing to the same underlying sample share one decoded buffer automatically, a free consequence of the content-addressed store. Buffer loads happen off the audio thread; the audio thread receives an `Arc` clone when decoding is done.
59 +
60 + ---
61 +
62 + ## Parameters
63 +
64 + Standard granular parameters exposed as nih-plug `FloatParam`s and automatable from the DAW:
65 +
66 + | Parameter | Range | Notes |
67 + |-----------|-------|-------|
68 + | Position | 0.0–1.0 | Read position in source sample |
69 + | Position Scatter | 0.0–1.0 | Random offset per grain |
70 + | Grain Size | 1–500ms | Duration of each grain |
71 + | Size Scatter | 0.0–1.0 | Random size variation |
72 + | Density | 1–200 grains/sec | Spawn rate |
73 + | Pitch | ±24 semitones | Global transposition |
74 + | Pitch Scatter | 0.0–24 semitones | Random pitch spread per grain |
75 + | Pan Scatter | 0.0–1.0 | Stereo spread |
76 + | Envelope Shape | Sine / Trapezoid / Tukey | Grain amplitude window |
77 + | Reverse Probability | 0.0–1.0 | Per-grain direction flip |
78 +
79 + MIDI pitch maps to playback pitch ratio. MIDI velocity maps to amplitude. All parameters are automatable and exposed to Rhai scripts.
80 +
81 + ---
82 +
83 + ## VFS Integration
84 +
85 + The editor embeds the same `BrowserState` from `audiofiles-plugin`. Users navigate the VFS tree they've already built and select either a single sample or an entire folder.
86 +
87 + **Single sample mode** — granularize one file. Standard use.
88 +
89 + **Folder mode** — each MIDI note randomly or sequentially selects from a VFS folder. This is where the AudioFiles architecture becomes genuinely useful for experimental work: you can treat a folder of field recordings, a folder of drum one-shots, or a folder of vocal fragments as a unified granular source. The VFS abstraction means the instrument doesn't care about filesystem layout — it navigates the virtual tree, tag-filtered and organized however you've set it up.
90 +
91 + Combined with Phase 5's smart folders (saved filter queries), you can point the granular engine at a dynamically defined set of samples: "all samples tagged `texture` with BPM between 60–80" becomes a live instrument.
92 +
93 + ---
94 +
95 + ## Rhai Scripting
96 +
97 + This is the core of what makes the instrument useful for complex pattern-making.
98 +
99 + The `audiofiles-rhai` sandboxed engine (already built for export plugins) is reused here. A script can implement hooks called at grain spawn time, before the grain is committed to the scheduler:
100 +
101 + ```rhai
102 + // Grains step through position in a Euclidean rhythm
103 + // 5 onsets over 8 subdivisions, synced to host BPM
104 +
105 + fn on_grain_spawn(grain, ctx) {
106 + let steps = 8;
107 + let onsets = 5;
108 + let step = (ctx.grain_index % steps);
109 +
110 + // Bjorklund/Euclidean check
111 + if (step * onsets % steps) < onsets {
112 + grain.amplitude = ctx.velocity;
113 + grain.position = step / steps;
114 + } else {
115 + grain.amplitude = 0.0; // silent grain — maintains rhythm without sound
116 + }
117 +
118 + grain
119 + }
120 + ```
121 +
122 + ```rhai
123 + // Pitch scatter narrows as velocity increases (harder hits = more focused pitch)
124 + // Position drifts slowly, creating evolving texture from a static sample
125 +
126 + fn on_grain_spawn(grain, ctx) {
127 + let focus = ctx.velocity; // 0.0..1.0
128 + grain.pitch_scatter = grain.pitch_scatter * (1.0 - focus);
129 +
130 + // Slow position drift using host time
131 + let drift_rate = 0.03;
132 + grain.position = (ctx.time_secs * drift_rate) % 1.0;
133 +
134 + grain
135 + }
136 + ```
137 +
138 + ```rhai
139 + // Density and size respond to a sidechain envelope follower
140 + // (ctx.sidechain_rms is exposed when a sidechain input is active)
141 +
142 + fn on_grain_spawn(grain, ctx) {
143 + let energy = ctx.sidechain_rms;
144 + grain.amplitude = energy * 2.0;
145 + grain.duration_frames = (grain.duration_frames * (1.0 - energy * 0.8)).max(64.0);
146 + grain
147 + }
148 + ```
149 +
150 + ### Context Object
151 +
152 + The `ctx` passed to every hook exposes:
153 +
154 + ```
155 + ctx.time_secs // wall clock time since playback start
156 + ctx.bpm // host BPM (0.0 if not provided)
157 + ctx.beat_position // position in current bar (0.0..1.0)
158 + ctx.note // MIDI note number (0–127)
159 + ctx.velocity // MIDI velocity (0.0–1.0)
160 + ctx.grain_index // count of grains spawned since note-on
161 + ctx.voice_index // which polyphonic voice (for multi-note behavior)
162 + ctx.position // current position param value (before script override)
163 + ctx.density // current density param value
164 + ctx.sidechain_rms // RMS of sidechain input (0.0 if no sidechain)
165 + ctx.sample_hash_hex // hex string of current sample's content hash
166 + ctx.sample_duration // source sample duration in seconds
167 + ```
168 +
169 + Scripts are hot-reloaded — the engine polls file mtime and recompiles on change. Edit a script in your text editor while the DAW is running; the next grain spawn picks up the new behavior. No preset save/load cycle, no plugin restart.
170 +
171 + The sandbox enforces the same limits as the export plugin runtime: 100K ops, 32-level call stack, no `eval`, no filesystem access beyond what the host API exposes.
172 +
173 + ---
174 +
175 + ## Modulation Sources
176 +
177 + Beyond scripting, hardwired modulation sources are available for fast patch-building without writing code:
178 +
179 + - **4 LFOs**: rate (Hz or tempo-synced), shape (sine/tri/square/random), per-voice or shared
180 + - **Envelope follower**: tracks grain output amplitude or sidechain input
181 + - **MIDI CC**: any CC to any parameter
182 + - **Note expression**: per-note pitch bend, pressure, timbre (if host supports MPE)
183 +
184 + Modulation targets are the same parameter set as the knobs, plus `grain.position`, `grain.size`, and `grain.amplitude` as direct per-grain targets.
185 +
186 + ---
187 +
188 + ## Editor UI
189 +
190 + Three panels:
191 +
192 + **Left — VFS Browser**
193 + Reuses the breadcrumb/file-list pattern from `audiofiles-plugin/src/editor.rs`. Navigate to a sample or folder, hit Enter to load. Tag filters exposed as a collapsible sidebar — filter by any tag dimension to narrow the source pool in real time.
194 +
195 + **Center — Waveform + Grain Visualization**
196 + Waveform display using Phase 4's downsampled peak data. Active grains render as short animated lines or dots scrubbing across the waveform — position on the x-axis, amplitude on the y-axis, color-coded by voice. At high densities this becomes a particle system. At low densities you can watch individual grains move.
197 +
198 + The visualization isn't decorative. When you're writing a position script and the grains aren't landing where you expect, watching them move tells you immediately what the math is doing.
199 +
200 + **Right — Script Editor + Parameters**
201 + Monospace code editor using egui's `TextEdit` multiline, with line numbers and inline Rhai error display. The compiler runs on a debounced background thread — errors appear at the offending line within a few hundred milliseconds of a change, without ever blocking the audio thread.
202 +
203 + Below the editor: the standard parameter knobs, laid out as a grid. Knobs with active script overrides are visually distinguished (dimmed ring or indicator dot) so it's clear which parameters the script is controlling versus which are running from the UI value.
204 +
205 + ---
206 +
207 + ## What This Enables
208 +
209 + The combination of per-grain scripting, VFS-integrated multi-sample selection, and host tempo/position context makes patterns possible that simply aren't achievable with conventional granular instruments:
210 +
211 + - **Rhythmic grain clouds** that lock to host tempo without quantizing to a fixed grid — Euclidean patterns, polyrhythmic layering, probabilistic pulse
212 + - **Spectral morphing** across a folder of related samples, where position and sample selection both evolve over time according to the same slow function
213 + - **Velocity-sensitive texture** where playing harder focuses pitch and shortens grains, playing lighter spreads and stretches them
214 + - **Sidechain-reactive granulation** where a kick drum's transients trigger density spikes in a pad texture
215 + - **Tag-driven live composition** where smart folders make the available source material conditional on whatever tags you've applied — changing the "instrument" by re-filtering the library rather than loading a different preset
216 +
217 + ---
218 +
219 + ## Integration With AudioFiles Roadmap
220 +
221 + | Phase | What It Unlocks |
222 + |-------|----------------|
223 + | Phase 4 (Analysis) | BPM-synced density defaults, key detection informs pitch scatter range, waveform data drives center panel |
224 + | Phase 5 (Search) | Smart folders as live instrument definitions, similarity search to find source material related to what's already loaded |
225 + | Phase 6 (Bulk Ops) | Batch-tag a folder of recordings, immediately available as a filtered source pool |
226 + | Phase 9 (Cloud Sync) | Sample buffers cached by hash — cloud samples load on demand without changing any engine code |
227 +
228 + ---
229 +
230 + ## New Work Required
231 +
232 + The grain scheduler, buffer cache, and modulation system are net-new. The LFO and envelope follower are straightforward DSP, a few hundred lines each. The grain scheduler is the interesting part — fixed-size pool, lock-free reads from audio thread, background decode handoff.
233 +
234 + The script editor UI is new egui work, but bounded: it's a styled `TextEdit` with a line number overlay and an error label. No syntax highlighting library needed — Rhai errors include line numbers, which is sufficient for the feedback loop to work.
235 +
236 + Everything else — VFS browser, content-addressed buffer lookup, Rhai sandbox, sample decode via symphonia — is already built.