Skip to main content

max / everycycle

pin to Ubuntu LTS, drop Mountaineer/musl, import MakeMachine v1 BOM Platform retarget: EveryCycle now pins to Ubuntu LTS 24.04 as the single supported distro (the same target MNW production servers run on). Mountaineer is no longer the dogfood host; the daemon is a standard glibc binary packaged as .deb. Other distros become best-effort with no formal support commitment. Aligns the project with tinygrad's pragmatic stance. Executor isolation (in-process dlopen vs out-of-process sidecar) is now a per-executor stability decision, not a libc-compatibility workaround. CUDA/ROCm sidecars stay justified by SDK crash blast-radius, not packaging constraints. Also imports the MakeMachine v1 BOM from MNW/sando/plans/mm-hardware-bom.md to docs/hardware/mm-v1-bom.md, renamed for v1 specificity so subsequent H2/H3/H4 BOMs can sit alongside it. Cross-repo move, so the file's git history doesn't carry over. Initial commit for docs/roadmap.md and todo.md, which existed locally since the initial workspace commit but were never tracked. README.md, docs/roadmap.md, todo.md: revise platform/distribution language and update BOM path references.
Co-Authored-By
Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Author: Max Johnson <me@maxj.phd> · 2026-06-02 21:14 UTC
Signed with PGP, not checked
Commit: 68634f6192c550e59ac2d380f54ad4ed3d3d4b5b
Parent: 63ceefb
4 files changed, +396 insertions, -19 deletions
M README.md +36 -19
@@ -1,40 +1,57 @@
1 1 # EveryCycle
2 2
3 - Rust inference runtime for heterogeneous and often-cheap GPU hardware.
3 + A Linux system service for heterogeneous GPU compute. `everycycled` owns the GPU fleet on a host, intermediates between the system and the silicon, and exposes a unified API to userspace clients — across vendors, generations, and the long tail of hardware whose vendors have moved on.
4 4
5 - EveryCycle extracts every clock cycle out of whatever silicon is in the box — current datacenter cards, used consumer GPUs, mixed-generation fleets, CPU offload, NVMe kv-cache spillover. It is the software that runs on the inference appliances built by a hardware operation under Make Creative.
5 + EveryCycle makes a box's mix of GPUs behave as one programmable resource. The first reference client is OpenAI-compatible LLM serving; other workloads (vision, scientific compute, batch ML pipelines) follow.
6 6
7 - Status: `0.0.x`, pre-Phase-0. The first concrete artifact is the [architecture sketch](docs/architecture.md).
7 + Status: `0.0.x`, pre-milestone-A1. The first concrete artifact is the [roadmap](docs/roadmap.md).
8 8
9 - ## Crates
9 + ## Two co-equal goals
10 10
11 - | Crate | Role |
12 - |------|------|
13 - | `everycycle-runtime` | Model execution, heterogeneous layer placement, quantization-aware kernel dispatch |
14 - | `everycycle-scheduler` | Continuous batching, paged kv-cache, admission control, speculative-decode coordination |
15 - | `everycycle-orchestrator` | Multi-box control plane: capability map, request routing, fleet management |
16 - | `everycycle-serving` | HTTP API surface (OpenAI-compatible), streaming |
17 - | `everycycle-tui` | Operator dashboard, structured telemetry views |
18 - | `everycycle-hal` | Hardware abstraction: device enumeration, topology, memory pools, telemetry |
19 - | `everycycle-bmc-agent` | Rust agent for the OpenBMC fork on our boxes; structured telemetry, fan curves, power capping |
11 + 1. **The daemon and ecosystem.** A Linux system service (`everycycled`) plus its clients, libraries, and operator tools. MIT-licensed, distributed via standard distro packaging. Open and permissive on principle.
12 + 2. **Reference appliances.** Specific hardware designs that ship pre-configured with EveryCycle — sold and supported as products. Hardware revenue funds the open-source software work.
20 13
21 - The exact factoring is a Phase 0 decision; the names are decided.
14 + ## Wedge
15 +
16 + - **Heterogeneity is the default.** Cheap, mixed-generation, mixed-vendor silicon serves real workloads when the runtime above the kernels is honest about hardware. EveryCycle treats heterogeneity as the normal case, not the edge case.
17 + - **Defensive curation.** When vendors abandon old hardware, EveryCycle catches it. Old hardware that can still serve at all is a first-class supported tier. We watch for support discontinuations, fork upstreams when they drop too aggressively, and partner with OSS driver projects (nouveau, NVK, Mesa, RADV) to extend useful life past vendor end-of-life.
18 +
19 + ## Category
20 +
21 + EveryCycle is the GPU-compute analog of what PipeWire is to audio, NetworkManager is to networking, CUPS is to printing. A Linux system service that owns a device class and provides a stable API to userspace, replacing the current chaos in which every program directly drives the GPU via vendor SDKs.
22 +
23 + ## Components
24 +
25 + | Binary | Role |
26 + |---|---|
27 + | `everycycled` | The system daemon. Audit, supervise, plan, dispatch. |
28 + | `everycycle-cli` | Operator CLI |
29 + | `everycycle-tui` | Operator dashboard |
30 + | `everycycle-serving` | OpenAI-compatible HTTP/SSE server (first-party client) |
31 + | `everycycle-exec-{cuda,rocm,cpu,vulkan}` | Executor modules — sidecars or in-process depending on libc constraints |
32 + | `libeverycycle` / `everycycle-client` | Client SDK (C ABI; Rust crate; other bindings to follow) |
33 + | `everycycle-bmc-agent` | Separate binary for OpenBMC; structured telemetry |
34 +
35 + ## Platform
36 +
37 + **Linux only by design.** macOS-native is not on the roadmap. Apple Silicon may eventually be supported via Asahi Linux as the project's GPU stack matures.
38 +
39 + **Pinned to Ubuntu LTS (24.04).** Ubuntu LTS is the single supported distro — the only target that gets formal support, the same distro MNW production servers run on. The codebase is generic Linux and should work elsewhere, but issues outside Ubuntu LTS are addressed best-effort. Executor isolation (in-process vs sidecar) is a per-executor choice driven by vendor SDK stability and crash blast-radius, not by libc compatibility.
22 40
23 41 ## Building
24 42
25 - Standard Cargo. Rust 1.85+ (edition 2024).
43 + Standard Cargo. Rust 1.85+ (edition 2024). Built and tested on Ubuntu LTS 24.04.
26 44
27 45 ```
28 - cargo build
29 - cargo test
46 + cargo build --release
30 47 ```
31 48
32 49 ## License
33 50
34 51 MIT. See [LICENSE](LICENSE).
35 52
36 - The software is permissively licensed by design. EveryCycle is open because the open-hardware ethos of the boxes is open, and because broad adoption is the wedge against opaque inference appliances. Revenue comes from building great machines and from custom integrations — not from license fees.
53 + The software is permissively licensed by design. EveryCycle is open because the open-hardware ethos of the reference appliances is open, and because broad adoption is the wedge against opaque inference appliances. Revenue comes from building great machines and from custom integrations, not from license fees.
37 54
38 55 ## Status and contact
39 56
40 - Pre-public. The project is unlisted on SourceHut until the Phase 0 demo runs. Documentation will be incomplete until then. If you have somehow found this and want to talk, the founder's contact is in the commit history.
57 + Pre-public. The project is unlisted on SourceHut until milestone A1 runs. Documentation will be incomplete until then. If you have somehow found this and want to talk, the founder's contact is in the commit history.
A docs/hardware/mm-v1-bom.md +91
@@ -1,0 +1,91 @@
1 + # MakeMachine Hardware BOM
2 +
3 + Settled 2026-05-23. Top-of-line host platform; GPUs are fungible and live on the EveryCycle GPU thread (see `~/Code/everycycle/docs/roadmap.md`).
4 +
5 + The substrate is built once and kept stable; GPU experimentation happens above it without revisiting motherboard, CPU, or RAM.
6 +
7 + ## Component list
8 +
9 + | Component | Choice | Approx cost | Notes |
10 + |---|---|---|---|
11 + | CPU | AMD Threadripper Pro 7975WX | $3,900 | 32 cores, 5.3 GHz boost, 8-channel DDR5. Same memory controller and PCIe lanes as bigger SKUs; cores past ~32 starve on memory bandwidth for inference. |
12 + | Motherboard | ASRock Rack WRX90D8-2L/2T | $1,200 | 7× PCIe 5.0 ×16, 8 DIMM slots, **ASPEED AST2600 BMC (OpenBMC-friendly)**, dual 10 GbE, 4× M.2 + 4× SlimSAS. |
13 + | RAM | 8× 64 GB DDR5-5600 ECC RDIMM (512 GB total) | $2,500 | One DIMM per channel — 2 DIMMs per channel forces DDR5 down to ~4400 MT/s on WRX90, costing ~20% memory bandwidth (real impact on CPU-offload layers). Brand: Micron or Hynix off the board QVL. |
14 + | Storage 1 | 4 TB Gen5 NVMe (Samsung 9100 Pro or Crucial T705) | $600 | ZFS root pool: OS, models, Sando state, logs. |
15 + | Storage 2 | 4 TB Gen5 NVMe (same) | $600 | **Raw XFS kv-scratch.** Not ZFS — ZFS caps Gen5 throughput; scratch is by definition disposable. EveryCycle uses this for kv-cache overflow. |
16 + | PSU | 2000 W Titanium-class (Super Flower Leadex Titanium or equivalent) | $600 | Headroom for any GPU combination the loose-parts experimentation hits. |
17 + | Chassis | 4U rackmount with ≥4 dual-slot GPU bays (Sliger CX4712 or similar) | $400 | Matches the eventual EveryCycle reference inference box; rackable from day one. |
18 + | CPU cooler | Silverstone XE360-TR5 (air) or Noctua NH-U14S TR5-SP6 | $200 | Air, not AIO — pump failure on a 24/7 box is worse than fan failure. |
19 + | Case fans | 6× 140 mm Noctua industrial | $200 | Front-to-back airflow; GPUs in a 4U breathe through these. |
20 + | Boot GPU | Nvidia GT 1030 (low-profile) | $80 | WRX90 has no iGPU; need a tiny card for console/boot. Also used as console GPU when datacenter cards (no display output) are installed. GT 710 originally specced but effectively EOL retail in 2026; GT 1030 is the current floor. Alternative: skip entirely if WRX90 BMC serial-over-LAN proves reliable. |
21 + | **Host subtotal** | | **~$10,280** | Before any compute GPU. |
22 +
23 + ## First compute GPU (Thread A1)
24 +
25 + | Component | Choice | Approx cost | Notes |
26 + |---|---|---|---|
27 + | GPU | 1× Nvidia Tesla P40 | $200 | 24 GB GDDR5, Pascal (cc 6.1). On-thesis: Pascal is the next architecture facing CUDA-legacy transition. |
28 + | Cooling adapter | 3D-printed or commercial fan shroud for P40 | $25 | Server card; passive cooling needs chassis airflow OR a strapped-on fan. |
29 + | Power adapter | EPS 8-pin to PCIe (or proper EPS routing) | $10 | P40 uses CPU-style 8-pin, not standard PCIe 8-pin. |
30 + | **GPU subtotal at A1** | | **~$235** | |
31 +
32 + ## Total
33 +
34 + | | |
35 + |---|---|
36 + | Host platform | ~$10,280 |
37 + | First GPU (A1) | ~$235 |
38 + | **MakeMachine v0 (assembled, runnable)** | **~$10,515** |
39 +
40 + Budget originally specced at $14–16K for the previous spec. Net savings: ~$4–5K. The savings are the GPU experimentation budget for advancing the GPU thread through A4–A5 over time.
41 +
42 + ## BIOS settings worth confirming at first boot
43 +
44 + - **Above 4G Decoding: enabled.** Required for datacenter cards with large BARs (Tesla P40, MI50, etc.).
45 + - **Resizable BAR: enabled.** Same reason.
46 + - **IOMMU: enabled.** Required for podman + CDI GPU passthrough.
47 + - **SR-IOV: as needed.** Not critical at A1; revisit at A5.
48 + - **Memory speed: DDR5-5600 (JEDEC).** Not pushed past spec.
49 + - **PCIe lane bifurcation: leave default initially; revisit for multi-GPU configurations.**
50 +
51 + ## Why this spec, summarized
52 +
53 + - **Threadripper Pro WRX90 platform** chosen over consumer TRX50 for the 7× PCIe 5.0 ×16 slots and the AST2600 BMC. EveryCycle's `bmc-agent` crate needs a real BMC to talk to.
54 + - **512 GB RAM** lets a 405B Q4 model fit fully in RAM if ever wanted; the immediate need is comfortable headroom for kv-cache experiments.
55 + - **Two Gen5 NVMes with different filesystems** because ZFS root is the right answer for the OS and model store, but ZFS would cap Gen5 bandwidth on the kv-scratch where bandwidth is the whole point.
56 + - **2000 W PSU** is sized for the worst-case 4-GPU configuration on the GPU thread, not for A1 alone.
57 + - **4U rackmount chassis** because it doubles as a prototype of the eventual EveryCycle reference appliance form factor.
58 +
59 + ## Sourcing notes
60 +
61 + Most of the spine is specialist channels; Microcenter covers the commodity parts.
62 +
63 + | Part | Channel |
64 + |---|---|
65 + | TR Pro 7975WX | Newegg, ShopBLT, Provantage |
66 + | WRX90D8-2L/2T | ASRock Rack direct, ShopBLT (longest lead — order first) |
67 + | DDR5-5600 ECC RDIMM (Micron/Hynix off board QVL) | Nemix, ServerSupply, Newegg |
68 + | Sliger CX4712 | Sliger direct (~6 wk historical lead). Fallback: Rosewill RSV-L4500U from Newegg if waiting is unacceptable. |
69 + | Tesla P40 + EPS-to-PCIe adapter + fan shroud | eBay |
70 + | 2× 4 TB Gen5 NVMe (9100 Pro / T705) | Microcenter |
71 + | PSU | Microcenter if Super Flower Leadex Titanium 2000 W in stock; otherwise Seasonic PRIME TX-1600 is the MC-reliable alternative (sufficient for A1–A3, marginal at A4 4-GPU). |
72 + | Noctua NF-A14 industrialPPC-3000 fans | Microcenter |
73 + | NH-U14S TR5-SP6 cooler | Noctua direct or Newegg; Microcenter rarely stocks |
74 + | GT 1030 boot GPU, paste, cables, M.2 heatsinks | Microcenter |
75 +
76 + Suggested order of operations: board first (longest lead) → CPU + RAM kit together (QVL match) → chassis → Microcenter run for commodity parts → P40 last (cheapest, most fungible).
77 +
78 + ## Open considerations (not blocking purchase)
79 +
80 + - **PSU redundancy.** Single 2000 W Titanium is fine for v0. Dual-redundant CRPS (e.g. FSP Twins) is the server-class move for v1.
81 + - **Cooler headroom.** NH-U14S TR5-SP6 is adequate for the 7975WX's 350 W TDP, not generous. XE360-TR5 (listed alt) gives more headroom for sustained all-core + GPU host load. Both are air.
82 + - **Boot GPU skip.** If WRX90 BMC serial-over-LAN proves reliable, GT 1030 line item is removable and a slot is reclaimed.
83 + - **kv-scratch PLP.** Consumer 9100 Pro / T705 have no power-loss protection. Fine for disposable scratch; revisit if EveryCycle ever checkpoints kv-cache through it.
84 +
85 + ## What this BOM deliberately does not include
86 +
87 + - **Multiple compute GPUs at purchase time.** Each GPU thread milestone is a discrete add; advancing the thread is its own decision moment.
88 + - **Liquid cooling.** More maintenance overhead than a single-operator shop should carry.
89 + - **Optane / SLC drives.** Optane is discontinued; Gen5 TLC is the right answer now.
90 + - **128-core CPU.** Memory-bandwidth-starved for inference; ~$6K extra for no inference gain.
91 + - **A second NIC card.** The 2× 10 GbE onboard is enough for v0.
A docs/roadmap.md +192
@@ -1,0 +1,192 @@
1 + # EveryCycle — Roadmap
2 +
3 + This is the live thread roadmap. Work advances on parallel threads, not numbered phases. The project's "current state" is the snapshot of where every thread stands together.
4 +
5 + When a thread is ready to advance, the next milestone on it is the next bit of work — independent of where other threads are. A resident or intern can take a thread or even a single milestone without having to follow a linear sequence.
6 +
7 + ## Identity
8 +
9 + **EveryCycle is a Linux system service for heterogeneous GPU compute.** The daemon (`everycycled`) owns the GPU fleet on a host and intermediates between the system and the silicon — across vendors, generations, and the long tail of hardware whose vendors have moved on.
10 +
11 + The category EveryCycle is creating is the **GPU-compute analog of PipeWire (audio), NetworkManager (networking), CUPS (printing)**: a system service that owns a device class and provides a stable API to userspace, replacing the chaos in which every program directly drives the GPU via vendor SDKs.
12 +
13 + ## The product and the business
14 +
15 + EveryCycle is software. The daemon (`everycycled` + clients + libraries + operator tools) is the product, MIT-licensed, distributed via standard distro packaging.
16 +
17 + Make Creative, LLC is a for-profit company with good intentions. It funds the software by designing and selling **MakeMachines** — reference appliances pre-configured with EveryCycle. MakeMachine revenue pays for the open-source work. We sell machines, not compute-as-a-service.
18 +
19 + MNW is a MakeMachine customer like any other: it buys MakeMachines from the MakeMachine line at fair internal-transfer pricing and runs its own workloads on them. External sales and internal sales are the same business — both fund EveryCycle.
20 +
21 + This is a clean separation of concerns: software identity lives in EveryCycle, hardware design and revenue live in the MakeMachine line, and MNW is a customer.
22 +
23 + ## Wedge
24 +
25 + 1. **Heterogeneity is the default.** Cheap, mixed-generation, mixed-vendor silicon serves real workloads when the runtime above the kernels is honest about the hardware. We do not assume a homogeneous fleet.
26 + 2. **Defensive curation.** When vendors abandon old hardware, EveryCycle catches it. We watch for support discontinuation, fork upstreams when they drop too aggressively, and partner with OSS driver and kernel projects (nouveau, NVK, Mesa, RADV) to extend useful life past vendor end-of-life. Old hardware that can still serve at all is a first-class supported tier.
27 +
28 + The work is *curatorial and operational*, not greenfield kernel engineering. We bind to existing kernel collections (ggml-cuda for NVIDIA, ggml-rocm for AMD) vendored in-tree so we can fork when needed. The novel work is the layer above: the supervisor, the planner, the audit, the operator UX, the client API.
29 +
30 + ## Platform and distribution
31 +
32 + - **Linux only by design.** macOS-native is not on the roadmap. Apple Silicon support, if it comes, comes via Asahi Linux.
33 + - **Pinned to Ubuntu LTS (24.04).** One supported distro across the whole project — matches the MNW production server target, so one test matrix covers infra and inference. Other distros are best-effort, no support commitment.
34 + - **Glibc build, .deb packaging.** The daemon is a standard glibc binary packaged as a .deb for Ubuntu LTS. Executor isolation (in-process dlopen vs out-of-process sidecar) is decided per executor by vendor SDK stability and crash blast-radius, not by libc compatibility.
35 +
36 + ## Threads
37 +
38 + ### Thread A — GPU complexity
39 +
40 + | Milestone | Cards | Approx cost | What it teaches |
41 + |---|---|---|---|
42 + | **A1** | 1× Tesla P40 (24 GB GDDR5, Pascal) + GT 710 boot card | ~$230 | Audit, plan, serve, ggml-cuda wrap, first defensive-curation regression suite. Pascal is the next NVIDIA shoe to drop, so the wedge is in immediate practice. |
43 + | **A2** | 2× Tesla P40 | ~$400 | Same-executor tensor-split, per-card capability reporting, planner spreads across same-vendor devices. |
44 + | **A3** | 1× P40 + 1× P100 (16 GB HBM2) | ~$500 | Mixed-arch within one executor (cc 6.1 + cc 6.0, different memory bandwidth and fp16 support). Capability-aware placement. |
45 + | **A4** | 1× P40 + 1× MI50 (16 GB HBM2, Vega 20) | ~$550 | Inter-executor activation protocol; AMD executor; cross-vendor supervisor coordination; the wedge demo. **Cross-vendor enters here, not earlier.** |
46 + | **A5** | + 3090 + Arc card | varies | Real heterogeneous fleet; planner makes real tradeoffs. |
47 +
48 + Single-vendor Pascal-era Tesla through A3. Cross-vendor enters at A4. The executor trait must be forward-compatible with cross-vendor handoff from A1 even though no impl uses it until A4.
49 +
50 + ### Thread B — Model complexity
51 +
52 + Smaller models first. Qwen family is the default — open weights, well-quantized in the wild, multiple sizes that step up cleanly. The progression demonstrates the runtime, not the model.
53 +
54 + | Milestone | Model | Why |
55 + |---|---|---|
56 + | **B1** | Qwen 3 0.6B Q8 (or Qwen 2.5 0.5B Q8) | Smoke test. Validates audit → plan → serve end-to-end with a model that loads in seconds. |
57 + | **B2** | Qwen 3 1.7B Q4 (or Qwen 2.5 1.5B Q4) | First "useful" responses. Streaming tokens, OpenAI-compatible chat. |
58 + | **B3** | Qwen 3 8B Q4 (or Qwen 2.5 7B Q4) | The workhorse demo model. Fits a Tesla P40 with comfortable kv-cache headroom. |
59 + | **B4** | Qwen 2.5 32B Q4 | Pushes the P40 harder; still fits 24 GB. |
60 + | **B5** | Qwen 2.5 72B Q4 | Saturates the P40's VRAM; first encounter with kv-cache pressure that wants NVMe overflow. |
61 + | **B6** | Frontier-size with offload | Layer placement across multiple devices + CPU + NVMe. Couples tightly with A4/A5 on the GPU thread. |
62 +
63 + ### Thread C — Supervisor and multi-client arbitration
64 +
65 + The supervisor owns the device fleet on behalf of *concurrent* clients. Single-tenant operation is not a stable resting point — it's a degenerate case of the multi-client design. PipeWire is fundamentally multi-tenant from day one; so is EveryCycle.
66 +
67 + Shape: enumerate-and-dispatch (A1, single client) → **two-client arbitration with priority + fair-share** (A2, the load-bearing milestone for the analogy) → crash isolation (per-executor watchdog + structured panic handling) → hot-plug → preemption and QoS classes → multi-box fleet routing.
68 +
69 + Two clients sharing a GPU under daemon-enforced policy is a required A2 demo, not a late-roadmap luxury. L5 (preemptive multi-tenant scheduling) moves into the early path on this thread.
70 +
71 + ### Thread D — Audit depth
72 +
73 + Open. Shape: enumeration (boot, ms) → calibration cache (first-boot per fingerprint, seconds) → live profiling (background, minutes) → drift detection (continuous; recalibrate when observed perf deviates from cache by N%).
74 +
75 + ### Thread E — Defensive curation
76 +
77 + Open. Shape: regression suite stub (just records what hardware passes today) → watch list of vendors' end-of-life announcements (Pascal, Vega, Polaris are the near-term candidates) → first OSS partnership (likely nouveau/NVK for Pascal post-CUDA, or RADV for Vega post-ROCm) → first real fork-and-maintain event when an upstream drops a generation we promised to keep alive.
78 +
79 + ### Thread F — BMC and hardware product
80 +
81 + Open. Shape: wrap stock Redfish on whatever BMC is on the MakeMachine board (ASRock Rack AST2600) → flash OpenBMC → bmc-agent emits structured telemetry → scheduler consumes thermal data for placement decisions. This thread runs in parallel with the runtime work; it does not block A1.
82 +
83 + ### Thread G — Operator UX
84 +
85 + Open. Shape: audit screen (the brand moment — beautiful, structured, says "this runtime understands your hardware") → live ops dashboard → multi-box view → incident response views. The TUI exists as `everycycle-tui` already; what's missing is the actual screen content.
86 +
87 + ### Thread H — MakeMachine appliance line (the business that funds the software)
88 +
89 + MakeMachines are how Make Creative pays for EveryCycle's development. The first MakeMachine (BOM at `docs/hardware/mm-v1-bom.md`) is in service as Sando host and EveryCycle dev box; MNW will buy MakeMachines from this line for its own workloads. Subsequent designs span a price ladder for external customers. Each design is a coherent BOM + chassis + thermal model + BMC config + sample software config.
90 +
91 + | Milestone | Design | Target | Notes |
92 + |---|---|---|---|
93 + | **H1** | MakeMachine v1 | First unit: Sando host + EveryCycle dev. Subsequent units: MNW (internal-transfer) and small-batch external sales. | Threadripper Pro WRX90 + flexible GPU bays. The dogfood design and the first revenue unit. |
94 + | **H2** | Mid-tier workstation | Solo operators, small labs (~$5K) | Single-CPU EPYC or TR non-Pro, 1-2 GPU bays, tower form factor. |
95 + | **H3** | Entry-level edge | Hobbyists, classrooms, residencies (~$2K) | Mini-ITX or SFF, single GPU, quiet, low-power. |
96 + | **H4** | Rack appliance | Datacenter and colo deployments (~$15K+) | 2U or 4U, 4-8 GPU bays, redundant PSU, dual NIC, OpenBMC. |
97 +
98 + Each milestone produces: published BOM, chassis CAD, thermal model, BMC firmware bundle, EveryCycle config tuned for the platform, assembly guide. See `_meta/docs/hardware_operation.md` for the broader operation framing (chassis, residency, supplier relationships).
99 +
100 + ### Thread I — Client API and adoption
101 +
102 + This is the long-pole project. PipeWire's decade of distro defaults and upstream integrations *was* the work; same applies here. The thread starts at A1, not after A3 — the API surface clients write against has to be designed before the runtime locks shape around its first internal users.
103 +
104 + Shape: native client API sketch (A1, alongside the executor trait) → API v0 published with explicit "unstable" marker (A2) → reference shims and SDKs (A3) → **API v1 freeze** with semver and compat charter (A4 or when the second external client ships against it, whichever first) → distro packaging and default-daemon advocacy (multi-year) → upstream integrations as alternative backends in llama.cpp / vLLM / TGI / ComfyUI / etc.
105 +
106 + The v1 freeze is a gate, not a date. Until it lands, every client is opting into churn; after it lands, breaking the API requires a major version and a migration story. This commitment is what makes EveryCycle a *platform* rather than a tool.
107 +
108 + ### Thread J — Compatibility shims
109 +
110 + The adoption mechanism. PipeWire displaced PulseAudio and JACK not by asking apps to port, but by shipping `pipewire-pulse` and `pipewire-jack` — drop-in replacements that spoke the existing protocols. Existing apps pointed at the new daemon without modification.
111 +
112 + EveryCycle's analogs, in rough priority order:
113 +
114 + | Shim | What it impersonates | Why it matters |
115 + |---|---|---|
116 + | **J1** | OpenAI Chat Completions / Responses API | Largest existing client ecosystem; every LLM tool already speaks it. |
117 + | **J2** | llama.cpp server (`/completion`, `/v1/chat/completions` extensions) | Captures the local-inference tool ecosystem (Open WebUI, LM Studio clients, etc.). |
118 + | **J3** | Ollama API | Captures the hobbyist surface. |
119 + | **J4** | vLLM / TGI HTTP surfaces | Captures production-inference clients without code change. |
120 + | **J5** | ComfyUI / diffusion backend protocols | Extends past text into image/video workloads on the same daemon. |
121 +
122 + J1 ships alongside B2 (first useful responses). The shims are *separate processes* that translate to the native EveryCycle client API — they are clients of Thread I's API, not special-cased daemon code. This keeps the native API honest: if a shim can't express something cleanly, that's a Thread I design problem, not a shim problem.
123 +
124 + ## Settled architectural decisions
125 +
126 + - **Backend approach: wrap vendored kernel collections per vendor.** Don't write kernels from scratch. Vendor the source in-tree so we can fork when needed. NVIDIA executor wraps ggml-cuda; AMD executor wraps ggml-rocm; Apple executor wraps MLX; CPU executor is the always-present fallback.
127 + - **Cross-vendor layer placement: belongs to A4.** The executor trait is forward-compatible from day one (activation-handoff signatures present even if no impl uses them); the inter-executor protocol gets built when A4 is reached.
128 + - **Project structure: parallel threads, not numbered phases.** Each thread has its own pace; project state is the snapshot across all threads.
129 + - **First GPU: Tesla P40.** Walks the defensive-curation talk on day one. Pascal facing imminent CUDA-legacy transition makes this card the canonical example of what EveryCycle exists to support.
130 + - **Host platform: dual-use with the MakeMachine.** See `docs/hardware/mm-v1-bom.md`. Threadripper Pro 7975WX + WRX90D8 + 512 GB ECC + Gen5 NVMe is the substrate; GPUs are fungible.
131 + - **First model family: Qwen, starting small.** B1 is the smallest Qwen variant that loads; subsequent milestones step up. Choice driven by open weights, broad quant availability, and clean size laddering.
132 + - **Client API stability is a sacred commitment.** Once Thread I reaches v1 freeze, the client-facing API follows strict semver: minor versions are additive, patch versions are bug-fix-only, major versions require a migration document and a deprecation window of at least one full release cycle on every supported distro. Before v1 freeze, the API is explicitly marked unstable and may break on any release. The freeze is the line that turns EveryCycle from a tool into a platform; everything downstream (compatibility shims, third-party clients, distro defaults) depends on this commitment being honored.
133 + - **Multi-client is the default, not a feature.** The daemon assumes concurrent clients from A1, even when only one is connected. The internal data model, scheduler, and audit surface are designed around N clients sharing M devices under policy; the N=1 case is just the limit. This is the structural difference between a system service and a runtime library.
134 + - **Compatibility shims are clients, not daemon code.** Each protocol shim (OpenAI, llama.cpp, Ollama, vLLM, ComfyUI) is a separate process that speaks the native EveryCycle client API. The shims do not get privileged access to internals. This forces the native API to be expressive enough that the shims are thin — and keeps the daemon free of protocol-specific code paths.
135 + - **Implementation languages.** Main binary is Rust, built natively for Ubuntu LTS 24.04 (glibc, .deb-packaged). Module boundary is a **stable C ABI** from a reasonable point — early enough that the second executor doesn't lock the trait shape into Rust-internals. Module *implementations* may be in any language that emits a C-ABI shared library (Rust, Zig, C). The BMC agent is a separate binary and is the leading candidate for being written in Zig rather than Rust; that decision deferred until the agent is real.
136 +
137 + ## Model stance
138 +
139 + **Open weights only, on principle.** EveryCycle serves models whose weights are publicly available. We do not wire to closed-weight model APIs and we do not ship closed weights as part of the runtime distribution. Operators are free to load whatever weights they want — we just don't make non-open-weight serving the default path.
140 +
141 + **Models are commodities.** We expect the model market to be fully commoditized: many capable open families, rapid generation turnover, no durable moat at any particular model. EveryCycle's architecture follows this stance:
142 +
143 + - No model-specific kernels. Backend kernels handle architectures generically (attention, RMSNorm, rotary, MoE routing) via the wrapped kernel collections.
144 + - No model-specific loaders. Standard formats (GGUF, safetensors) only; loaders for new architectures land in the wrapped backends, not in EveryCycle.
145 + - No model registry of our own. Operators bring their own models; we serve them.
146 + - The serving API stays OpenAI-compatible — that interface is stable across model churn.
147 + - Marketing and demos focus on **what the runtime does to the hardware**, not on what model is loaded. The model is a workload, not a feature.
148 +
149 + When a new model family emerges (Llama 4, Mistral Next, DeepSeek N+1, etc.), supporting it should be a backend-version bump, not an EveryCycle code change.
150 +
151 + ## Open questions (will be resolved iteratively)
152 +
153 + - Executor isolation: in-process dlopen + watchdog threads + structured panic handling, or out-of-process with local-socket IPC. Per executor, driven by vendor SDK stability and crash blast-radius. In-process is the default for executors that behave; out-of-process when an executor empirically destabilizes the host process. The Ubuntu LTS / glibc pin removes libc compatibility as a forcing factor — CUDA and ROCm SDKs target glibc natively, so sidecar separation is now a stability choice, not a packaging workaround.
154 + - CPU executor placement: statically linked into the base binary (always-present fallback) vs another module
155 + - A1 dispatch planner: greedy heuristic only, or invest in a real optimizer up front
156 + - BMC agent's actual A1-equivalent role: just enumerate Redfish, or already running on flashed OpenBMC
157 + - First-boot online vs offline default
158 +
159 + ## Long-arc vision: the OS-for-compute level decomposition
160 +
161 + EveryCycle's identity grows over time toward "operating system for heterogeneous GPU compute." The progression, in increasing order of difficulty:
162 +
163 + | Level | What | Status |
164 + |---|---|---|
165 + | **L0** | Unified inventory (audit) | Thread D, A1 |
166 + | **L1** | Unified routing (per-request placement) | Thread C, A1 |
167 + | **L2** | Per-model layer placement (across same-vendor devices) | Thread A, A2-A3 |
168 + | **L3** | Virtual VRAM with paging (kv-cache first; weights later) | Thread D, mid-roadmap |
169 + | **L4** | Cross-device kernel sharding (same-vendor first, cross-vendor later) | A4+ |
170 + | **L5** | Preemptive multi-tenant scheduling | Thread C, A2 (fair-share) → mid-roadmap (preemption + QoS classes) |
171 + | **L6** | Transparent failure recovery (request-level early; mid-token much later) | Mid roadmap and beyond |
172 + | **L7** | Unified telemetry surface (operators reason about "the cluster," not individual devices) | Thread G + Thread F |
173 + | **L8** | Full programmability against the abstract device (clients write to EveryCycle's API, never to vendor SDKs) | Thread I; the adoption goal |
174 +
175 + Levels through L4 (same-vendor) are achievable solo + residency. L4 (cross-vendor), full-fidelity L3 (weight paging), L5's preemption+QoS layer (fair-share at L5 is in scope solo), and L6 (mid-token recovery) are team-required and are honest team-scale R&D. Worth naming because the levels frame what each thread is *for* and where each milestone fits in the larger picture.
176 +
177 + What this forbids:
178 + - **Performance parity across topologies.** "One big GPU" means unified abstraction, not magical speed. Cross-vendor will be slower than single-vendor; that's physics.
179 + - **Hiding hardware completely.** Operators still need to see what's there. The abstraction is programmer-facing; the operator surface stays physically honest.
180 + - **Forcing the abstraction.** Workloads that need locality guarantees can pin. "One big GPU" is the opt-in default, not the only mode.
181 +
182 + ## Non-goals
183 +
184 + Restated for emphasis:
185 +
186 + - Not training. Not in v1, not in v3.
187 + - Not writing kernels from scratch. We bind, we wrap, we fork.
188 + - Not Kubernetes-first. Client API; orchestrators can wrap it.
189 + - Not a web UI. TUI + Prometheus/OTel exporters.
190 + - Not a model zoo of our own. We serve the open models that exist.
191 + - Not macOS-native. Linux only; Apple Silicon via Asahi if at all.
192 + - Not a license-paid product. Software is MIT; revenue comes from selling MakeMachine appliances (external and internal-transfer to MNW). Not compute-as-a-service.
A todo.md +77
@@ -1,0 +1,77 @@
1 + # EveryCycle TODO
2 +
3 + Open work only. Completed items move to `todo_done.md` (sibling) when one exists. Design notes go in `docs/<name>.md`, not folded into checkboxes.
4 +
5 + Format rule: every actionable line is a `- [ ]` checkbox. Headings group phases and themes; do not put status updates in them.
6 +
7 + Live roadmap: `docs/roadmap.md`. This file is the execution queue against that roadmap.
8 +
9 + ## Key Paths
10 +
11 + Read these to orient before working on EveryCycle:
12 +
13 + - `docs/roadmap.md` — threads, milestones, settled decisions
14 + - `docs/architecture.md` — component layout, executor model
15 + - `Cargo.toml` — workspace members
16 + - `crates/hal/src/lib.rs` — device enumeration entry point (Thread D1)
17 + - `crates/runtime/src/lib.rs` — executor trait host (Thread A1)
18 + - `crates/tui/src/lib.rs` — operator UX, audit screen target (Thread G)
19 + - `docs/hardware/mm-v1-bom.md` — MakeMachine v1 BOM (substrate)
20 +
21 + ---
22 +
23 + ## Phase 0 — First code, no GPU required
24 +
25 + Hardware-free work to land before the A1 hardware arrives.
26 +
27 + - [ ] Sketch `hal::DeviceInventory` shape (PCI, DRM nodes, NVML when present, hwmon). Land minimal enumeration that runs on any Linux box.
28 + - [ ] Define the executor trait in `runtime/` with cross-vendor activation-handoff signatures present from day one (forward-compat for A4 per roadmap line 43).
29 + - [ ] Sketch the native client API surface in `crates/api/` (or `crates/serving/` — see Phase 1 rename decision). Just types and the IPC shape; no implementation. This is Thread I starting at A1.
30 + - [ ] Write `docs/api-stability.md` capturing the v1-freeze gate, semver policy, and deprecation window from the new "Client API stability" settled decision.
31 + - [ ] Wire `everycycle-tui`'s audit screen against real `hal` data — even on a GT-710-only box. Thread G brand moment.
32 +
33 + ## Phase 1 — Open roadmap decisions
34 +
35 + - [ ] Decide on rename `serving` → `api` (or `client-api`) and reframe Thread B from "serve models" to "first workload class" (item E from 2026-05-23 roadmap review). If yes, do the crate rename + B-thread heading rewrite in one pass.
36 + - [ ] Decide executor isolation final shape for the CPU executor (in-process dlopen vs sidecar; CUDA/ROCm are already forced sidecar).
37 + - [ ] Decide A1 dispatch planner sophistication (greedy heuristic vs real optimizer up front).
38 + - [ ] Decide first-boot online vs offline default.
39 + - [ ] Decide BMC agent A1-equivalent scope (just enumerate Redfish, or already running flashed OpenBMC).
40 +
41 + ## Phase 2 — A1 hardware
42 +
43 + - [ ] Order Tesla P40 (24 GB GDDR5, Pascal) + GT 710 boot card. ~$230 per roadmap A1.
44 + - [ ] Install P40 in MakeMachine v1; verify thermals and PCIe link state via hal enumeration.
45 + - [ ] Vendor ggml-cuda in-tree at a known good commit; build glibc sidecar that wraps it.
46 + - [ ] First Qwen 0.6B Q8 smoke test (B1) end-to-end: audit → plan → serve.
47 +
48 + ## Phase 3 — Multi-client (Thread C, A2)
49 +
50 + - [ ] Stand up two-client arbitration demo on a single P40 with fair-share scheduling. This is the load-bearing PipeWire-analogy demo per the revised Thread C.
51 +
52 + ## Phase 4 — Compatibility shims (Thread J)
53 +
54 + - [ ] Ship J1: OpenAI Chat Completions shim as a separate process that consumes the native client API. Land alongside B2 (first useful responses).
55 + - [ ] Ship J2: llama.cpp server protocol shim.
56 + - [ ] Defer J3 (Ollama), J4 (vLLM/TGI), J5 (ComfyUI) until J1/J2 are stable.
57 +
58 + ---
59 +
60 + ## Business and governance (Make Creative, LLC)
61 +
62 + Captured from the 2026-05-23 Fellow-lead framework discussion. Not coding work, but blocks scaling the project beyond solo operation.
63 +
64 + - [ ] File "EveryCycle" trademark under Make Creative, LLC. Mark ownership controls brand identity through any future structural change.
65 + - [ ] Draft `docs/governance.md` charter: who decides what at Phase 1 (Fellow inside Make Creative). Includes technical-direction authority, brand/trademark gate, profit-share structure on MakeMachine sales attributable to EveryCycle.
66 + - [ ] Draft CLA / IP assignment template for contributors (Fellow, residents, interns, contractors). Standard work-for-hire; code stays MIT outward.
67 + - [ ] Decide profit-share parameters for the eventual Fellow lead (% of MakeMachine gross margin, duration, vesting).
68 + - [ ] Document Phase 2 trigger conditions (when to spin out `EveryCycle, LLC` as a subsidiary of Make Creative). Don't act on this until triggered.
69 +
70 + ---
71 +
72 + ## Backlog (not yet ordered)
73 +
74 + - [ ] Ship .deb packaging for Ubuntu LTS 24.04 (the single supported target). Other distros are best-effort, no formal packaging commitment.
75 + - [ ] Thread E first deliverable: regression suite stub that records what hardware passes today.
76 + - [ ] Thread F: enumerate Redfish on MakeMachine v1's AST2600; defer OpenBMC flash.
77 + - [ ] Long-arc: when v1 API freeze approaches, write the migration document and the deprecation window policy as their own published doc.