max / granma

12.0 KB · 237 lines History Blame Raw

1	# audiofiles-granular
2
3	A programmable granular sampler CLAP plugin built on the AudioFiles architecture. Designed for experimental sound designers and musicians who find conventional granular instruments too limiting — where the synthesis behavior itself needs to be as composable and scriptable as the music being made.
4
5	---
6
7	## The Problem With Existing Granular Tools
8
9	Most granular instruments give you a handful of knobs — position, size, density, scatter — and expect you to twist them manually. That's fine for ambient washes. It's not fine when you want a grain cloud whose density follows the amplitude envelope of a different sample, or position that steps through a Euclidean rhythm, or pitch scatter that narrows as MIDI velocity increases. The parameter space exists; the programmability doesn't.
10
11	audiofiles-granular treats the grain scheduler itself as a scripting target. Every grain spawn is a hook. You write the behavior.
12
13	---
14
15	## Architecture
16
17	New crate: `crates/audiofiles-granular/`
18
19	Sits alongside `audiofiles-plugin`, depends on `audiofiles-core` and `audiofiles-rhai`. Shares the same SQLite database and flat content-addressed store — no asset duplication, no separate library management. The same samples you've organized in AudioFiles are immediately available as granular source material.
20
21	```
22	audiofiles-granular/
23	src/
24	lib.rs # nih-plug plugin entry, params
25	engine.rs # grain scheduler, voice pool
26	grain.rs # single grain: position, envelope, pitch, pan
27	sampler.rs # sample buffer cache (Arc<Vec<f32>>, keyed by hash)
28	modulation.rs # LFO, envelope follower, MIDI CC routing
29	editor.rs # egui UI: VFS browser, waveform, script editor
30	state.rs # SharedState bridge (audio/GUI thread sync)
31	script.rs # Rhai integration for per-grain hooks
32	```
33
34	No async runtime. No tokio. The audio thread is entirely synchronous — grain scheduling, buffer reads, output mixing all happen in `process()`. Sample decoding runs on a background `std::thread` with a `crossbeam_channel` handoff, same pattern as the preview engine in `audiofiles-plugin`.
35
36	---
37
38	## Grain Engine
39
40	Each grain is a fixed-size struct, allocated upfront in a pool — no heap allocation inside the audio callback:
41
42	```rust
43	struct Grain {
44	sample_hash: [u8; 32], // references content store directly by hash
45	playhead: f32, // current read position (fractional frames)
46	duration_frames: usize,
47	pitch_ratio: f32, // resampling ratio (1.0 = original pitch)
48	pan: f32,
49	amplitude: f32,
50	envelope_phase: f32, // 0.0..1.0, shape set per-grain at spawn
51	reverse: bool,
52	active: bool,
53	}
54	```
55
56	Voice pool is fixed at 64 grains (configurable at compile time). The scheduler fires new grains at a rate determined by density, jittered by scatter. Multiple MIDI notes run independent grain clouds with their own position anchors — the instrument is fully polyphonic.
57
58	Sample buffers live in a `HashMap<[u8; 32], Arc<Vec<f32>>>` keyed by content hash. Two VFS nodes pointing to the same underlying sample share one decoded buffer automatically, a free consequence of the content-addressed store. Buffer loads happen off the audio thread; the audio thread receives an `Arc` clone when decoding is done.
59
60	---
61
62	## Parameters
63
64	Standard granular parameters exposed as nih-plug `FloatParam`s and automatable from the DAW:
65
66	\| Parameter \| Range \| Notes \|
67	\|-----------\|-------\|-------\|
68	\| Position \| 0.0–1.0 \| Read position in source sample \|
69	\| Position Scatter \| 0.0–1.0 \| Random offset per grain \|
70	\| Grain Size \| 1–500ms \| Duration of each grain \|
71	\| Size Scatter \| 0.0–1.0 \| Random size variation \|
72	\| Density \| 1–200 grains/sec \| Spawn rate \|
73	\| Pitch \| ±24 semitones \| Global transposition \|
74	\| Pitch Scatter \| 0.0–24 semitones \| Random pitch spread per grain \|
75	\| Pan Scatter \| 0.0–1.0 \| Stereo spread \|
76	\| Envelope Shape \| Sine / Trapezoid / Tukey \| Grain amplitude window \|
77	\| Reverse Probability \| 0.0–1.0 \| Per-grain direction flip \|
78
79	MIDI pitch maps to playback pitch ratio. MIDI velocity maps to amplitude. All parameters are automatable and exposed to Rhai scripts.
80
81	---
82
83	## VFS Integration
84
85	The editor embeds the same `BrowserState` from `audiofiles-plugin`. Users navigate the VFS tree they've already built and select either a single sample or an entire folder.
86
87	Single sample mode — granularize one file. Standard use.
88
89	Folder mode — each MIDI note randomly or sequentially selects from a VFS folder. This is where the AudioFiles architecture becomes genuinely useful for experimental work: you can treat a folder of field recordings, a folder of drum one-shots, or a folder of vocal fragments as a unified granular source. The VFS abstraction means the instrument doesn't care about filesystem layout — it navigates the virtual tree, tag-filtered and organized however you've set it up.
90
91	Combined with Phase 5's smart folders (saved filter queries), you can point the granular engine at a dynamically defined set of samples: "all samples tagged `texture` with BPM between 60–80" becomes a live instrument.
92
93	---
94
95	## Rhai Scripting
96
97	This is the core of what makes the instrument useful for complex pattern-making.
98
99	The `audiofiles-rhai` sandboxed engine (already built for export plugins) is reused here. A script can implement hooks called at grain spawn time, before the grain is committed to the scheduler:
100
101	```rhai
102	// Grains step through position in a Euclidean rhythm
103	// 5 onsets over 8 subdivisions, synced to host BPM
104
105	fn on_grain_spawn(grain, ctx) {
106	let steps = 8;
107	let onsets = 5;
108	let step = (ctx.grain_index % steps);
109
110	// Bjorklund/Euclidean check
111	if (step * onsets % steps) < onsets {
112	grain.amplitude = ctx.velocity;
113	grain.position = step / steps;
114	} else {
115	grain.amplitude = 0.0; // silent grain — maintains rhythm without sound
116	}
117
118	grain
119	}
120	```
121
122	```rhai
123	// Pitch scatter narrows as velocity increases (harder hits = more focused pitch)
124	// Position drifts slowly, creating evolving texture from a static sample
125
126	fn on_grain_spawn(grain, ctx) {
127	let focus = ctx.velocity; // 0.0..1.0
128	grain.pitch_scatter = grain.pitch_scatter * (1.0 - focus);
129
130	// Slow position drift using host time
131	let drift_rate = 0.03;
132	grain.position = (ctx.time_secs * drift_rate) % 1.0;
133
134	grain
135	}
136	```
137
138	```rhai
139	// Density and size respond to a sidechain envelope follower
140	// (ctx.sidechain_rms is exposed when a sidechain input is active)
141
142	fn on_grain_spawn(grain, ctx) {
143	let energy = ctx.sidechain_rms;
144	grain.amplitude = energy * 2.0;
145	grain.duration_frames = (grain.duration_frames * (1.0 - energy * 0.8)).max(64.0);
146	grain
147	}
148	```
149
150	### Context Object
151
152	The `ctx` passed to every hook exposes:
153
154	```
155	ctx.time_secs // wall clock time since playback start
156	ctx.bpm // host BPM (0.0 if not provided)
157	ctx.beat_position // position in current bar (0.0..1.0)
158	ctx.note // MIDI note number (0–127)
159	ctx.velocity // MIDI velocity (0.0–1.0)
160	ctx.grain_index // count of grains spawned since note-on
161	ctx.voice_index // which polyphonic voice (for multi-note behavior)
162	ctx.position // current position param value (before script override)
163	ctx.density // current density param value
164	ctx.sidechain_rms // RMS of sidechain input (0.0 if no sidechain)
165	ctx.sample_hash_hex // hex string of current sample's content hash
166	ctx.sample_duration // source sample duration in seconds
167	```
168
169	Scripts are hot-reloaded — the engine polls file mtime and recompiles on change. Edit a script in your text editor while the DAW is running; the next grain spawn picks up the new behavior. No preset save/load cycle, no plugin restart.
170
171	The sandbox enforces the same limits as the export plugin runtime: 100K ops, 32-level call stack, no `eval`, no filesystem access beyond what the host API exposes.
172
173	---
174
175	## Modulation Sources
176
177	Beyond scripting, hardwired modulation sources are available for fast patch-building without writing code:
178
179	- 4 LFOs: rate (Hz or tempo-synced), shape (sine/tri/square/random), per-voice or shared
180	- Envelope follower: tracks grain output amplitude or sidechain input
181	- MIDI CC: any CC to any parameter
182	- Note expression: per-note pitch bend, pressure, timbre (if host supports MPE)
183
184	Modulation targets are the same parameter set as the knobs, plus `grain.position`, `grain.size`, and `grain.amplitude` as direct per-grain targets.
185
186	---
187
188	## Editor UI
189
190	Three panels:
191
192	Left — VFS Browser
193	Reuses the breadcrumb/file-list pattern from `audiofiles-plugin/src/editor.rs`. Navigate to a sample or folder, hit Enter to load. Tag filters exposed as a collapsible sidebar — filter by any tag dimension to narrow the source pool in real time.
194
195	Center — Waveform + Grain Visualization
196	Waveform display using Phase 4's downsampled peak data. Active grains render as short animated lines or dots scrubbing across the waveform — position on the x-axis, amplitude on the y-axis, color-coded by voice. At high densities this becomes a particle system. At low densities you can watch individual grains move.
197
198	The visualization isn't decorative. When you're writing a position script and the grains aren't landing where you expect, watching them move tells you immediately what the math is doing.
199
200	Right — Script Editor + Parameters
201	Monospace code editor using egui's `TextEdit` multiline, with line numbers and inline Rhai error display. The compiler runs on a debounced background thread — errors appear at the offending line within a few hundred milliseconds of a change, without ever blocking the audio thread.
202
203	Below the editor: the standard parameter knobs, laid out as a grid. Knobs with active script overrides are visually distinguished (dimmed ring or indicator dot) so it's clear which parameters the script is controlling versus which are running from the UI value.
204
205	---
206
207	## What This Enables
208
209	The combination of per-grain scripting, VFS-integrated multi-sample selection, and host tempo/position context makes patterns possible that simply aren't achievable with conventional granular instruments:
210
211	- Rhythmic grain clouds that lock to host tempo without quantizing to a fixed grid — Euclidean patterns, polyrhythmic layering, probabilistic pulse
212	- Spectral morphing across a folder of related samples, where position and sample selection both evolve over time according to the same slow function
213	- Velocity-sensitive texture where playing harder focuses pitch and shortens grains, playing lighter spreads and stretches them
214	- Sidechain-reactive granulation where a kick drum's transients trigger density spikes in a pad texture
215	- Tag-driven live composition where smart folders make the available source material conditional on whatever tags you've applied — changing the "instrument" by re-filtering the library rather than loading a different preset
216
217	---
218
219	## Integration With AudioFiles Roadmap
220
221	\| Phase \| What It Unlocks \|
222	\|-------\|----------------\|
223	\| Phase 4 (Analysis) \| BPM-synced density defaults, key detection informs pitch scatter range, waveform data drives center panel \|
224	\| Phase 5 (Search) \| Smart folders as live instrument definitions, similarity search to find source material related to what's already loaded \|
225	\| Phase 6 (Bulk Ops) \| Batch-tag a folder of recordings, immediately available as a filtered source pool \|
226	\| Phase 9 (Cloud Sync) \| Sample buffers cached by hash — cloud samples load on demand without changing any engine code \|
227
228	---
229
230	## New Work Required
231
232	The grain scheduler, buffer cache, and modulation system are net-new. The LFO and envelope follower are straightforward DSP, a few hundred lines each. The grain scheduler is the interesting part — fixed-size pool, lock-free reads from audio thread, background decode handoff.
233
234	The script editor UI is new egui work, but bounded: it's a styled `TextEdit` with a line number overlay and an error label. No syntax highlighting library needed — Rhai errors include line numbers, which is sufficient for the feedback loop to work.
235
236	Everything else — VFS browser, content-addressed buffer lookup, Rhai sandbox, sample decode via symphonia — is already built.
237