Skip to main content

max / makenotwork

18.2 KB · 469 lines History Blame Raw
1 # Contributing to MNW Server
2
3 Patterns, conventions, and rules for working on the MNW server codebase. Read this before making changes.
4
5 ## Project Structure
6
7 ```
8 server/
9 src/
10 main.rs # Entry point
11 lib.rs # Library root, AppState definition
12 config.rs # Configuration from environment
13 error.rs # AppError enum, HTTP error responses
14 auth.rs # Session auth extractors (AuthUser, MaybeUser, AdminUser)
15 csrf.rs # CSRF middleware + token management
16 helpers.rs # Shared utilities (format_price, get_initials, etc.)
17 db/ # Database queries, one file per domain
18 types/ # View types + Db→View conversions
19 routes/ # HTTP handlers, grouped by domain
20 templates/ # Askama template structs (Rust side)
21 payments/ # Stripe integration
22 email/ # Postmark transactional email
23 scanning/ # File scanning (ClamAV, YARA, hash lookup)
24 storage.rs # S3 storage abstraction
25 synckit_auth.rs # SyncKit JWT auth
26 migrations/ # PostgreSQL migrations (numbered, auto-applied on boot)
27 templates/ # Askama HTML templates (Jinja2-like)
28 static/ # CSS, JS, fonts, images
29 tests/ # Integration tests
30 deploy/ # Deployment scripts and configs
31 ```
32
33 ## Error Handling
34
35 All handlers return `Result<T, AppError>`. The `AppError` enum (`src/error.rs`) maps each variant to an HTTP status code, a Sentry tag, and a user-facing message. Internal details (DB errors, storage errors) are logged but never exposed to users.
36
37 ```rust
38 #[derive(Debug, thiserror::Error)]
39 pub enum AppError {
40 #[error("Not found")]
41 NotFound, // 404
42 #[error("Bad request: {0}")]
43 BadRequest(String), // 400 — message shown to user
44 #[error("Database error: {0}")]
45 Database(#[from] sqlx::Error), // 500 — generic message to user
46 #[error("Internal server error")]
47 Internal(#[from] anyhow::Error), // 500 — generic message to user
48 // ... see error.rs for full list
49 }
50 ```
51
52 **Rules:**
53 - Use `?` for error propagation. Never `.unwrap()` in production code.
54 - Use `.ok_or(AppError::NotFound)?` when an optional DB result must exist.
55 - Use `AppError::BadRequest("message")` for user-caused errors — the string is shown directly.
56 - Use `AppError::Validation("message")` for form validation failures (returns 422).
57 - Never expose internal error details to users. `Database` and `Internal` variants always show "Something went wrong."
58 - Convert external errors with `From` impls, not string formatting. Add `#[from]` to AppError variants for automatic conversion.
59
60 On API routes (`/api/*`), a middleware layer (`json_error_layer`) automatically converts HTML error responses to `{"error": "message"}` JSON. Handlers don't need to handle this — it's transparent.
61
62 ## Route Handlers
63
64 ### Signature Pattern
65
66 Every handler follows this structure:
67
68 ```rust
69 #[tracing::instrument(skip_all, name = "module::handler_name")]
70 async fn handler_name(
71 State(state): State<AppState>, // App state (DB pool, config, etc.)
72 session: Session, // Session store
73 AuthUser(user): AuthUser, // Or MaybeUser(maybe_user), or AdminUser
74 Path(slug): Path<String>, // URL parameters
75 Query(params): Query<FilterQuery>, // Query string
76 Form(form): Form<CreateForm>, // POST body (form-encoded)
77 ) -> Result<impl IntoResponse> {
78 let csrf_token = get_csrf_token(&session).await;
79 // ... business logic ...
80 Ok(MyTemplate { csrf_token, session_user: Some(user.into()), /* ... */ })
81 }
82 ```
83
84 **Rules:**
85 - Every handler gets `#[tracing::instrument(skip_all, name = "...")]` for structured logging.
86 - Return type is always `Result<impl IntoResponse>`.
87 - Extract auth requirements via the type system: `AuthUser` (login required), `MaybeUser` (optional), `AdminUser` (admin only, returns 404 to hide admin routes from non-admins).
88 - Askama templates implement `IntoResponse` — return the struct directly.
89 - CSRF token goes into every template that renders forms.
90
91 ### HTMX Responses
92
93 Full-page handlers extend `base.html` and include `session_user`, `csrf_token`, navigation, etc. HTMX handlers return partial templates — HTML fragments without the base layout.
94
95 ```rust
96 // Full page — extends base.html
97 Ok(FullPageTemplate { csrf_token, session_user: maybe_user, /* ... */ })
98
99 // HTMX fragment — standalone partial, no base layout
100 Ok(FilteredEntriesTemplate { items, current_page, total_pages })
101 ```
102
103 In templates, HTMX attributes trigger fragment requests:
104 ```html
105 <button hx-get="/dashboard/items?page=2"
106 hx-target="#item-list"
107 hx-swap="innerHTML">Next</button>
108 ```
109
110 The CSRF token is included in HTMX requests via the `X-CSRF-Token` header, set globally from the `<meta name="csrf-token">` tag.
111
112 ### Route Module Organization
113
114 Routes are grouped by domain under `src/routes/`. Each domain is either a single file (for small domains) or a directory module (when it exceeds ~500 lines).
115
116 ```
117 routes/
118 mod.rs # Declares modules + re-exports *_routes() functions
119 auth.rs # Login, signup, logout
120 admin.rs # Admin panel (or admin/ directory)
121 pages/ # All public HTML pages (directory module)
122 mod.rs # Composes sub-routers
123 public/ # Public-facing pages
124 dashboard/ # Creator dashboard + HTMX tabs
125 api/ # JSON API endpoints
126 stripe/ # Stripe webhooks + checkout
127 synckit/ # SyncKit API
128 ```
129
130 Each module exposes a `*_routes()` function that returns an `axum::Router`:
131 ```rust
132 pub fn page_routes() -> Router<AppState> {
133 Router::new()
134 .route("/", get(home))
135 .route("/:username", get(profile))
136 // ...
137 }
138 ```
139
140 These are composed in `main.rs` via `.merge()` or `.nest()`.
141
142 ## Database Layer
143
144 ### Query Pattern
145
146 All queries use sqlx with compile-time checking. Queries live in `src/db/`, one file per domain (e.g., `db/users.rs`, `db/items.rs`, `db/synckit.rs`).
147
148 ```rust
149 pub async fn get_user_by_id(pool: &PgPool, id: UserId) -> Result<Option<DbUser>> {
150 let user = sqlx::query_as::<_, DbUser>(
151 "SELECT * FROM users WHERE id = $1"
152 )
153 .bind(id)
154 .fetch_optional(pool)
155 .await?;
156 Ok(user)
157 }
158 ```
159
160 **Rules:**
161 - Always use positional parameters (`$1`, `$2`, ...). Never interpolate values into SQL strings.
162 - Use `sqlx::query_as::<_, DbRow>` for typed results. Use `sqlx::query!` only when the macro's compile-time checking is needed.
163 - `.fetch_one()` when exactly one row expected (errors on zero), `.fetch_optional()` when zero or one, `.fetch_all()` for lists.
164 - Newtype ID wrappers (`UserId`, `ProjectId`, etc.) work directly with `.bind()` — they implement sqlx's `Encode`/`Decode`.
165 - Multi-line SQL uses `r#"..."#` raw strings.
166
167 ### DB Row Types vs View Types
168
169 Database rows are `Db*` structs (e.g., `DbUser`, `DbProject`) in `src/db/`. View types for templates are in `src/types/` (e.g., `User`, `Project`). Conversions between them use `From` trait impls in `src/types/conversions.rs`.
170
171 ```rust
172 // In src/types/conversions.rs
173 impl From<&db::DbUser> for User {
174 fn from(u: &db::DbUser) -> Self {
175 User {
176 username: u.username.to_string(),
177 avatar_initials: get_initials(u.display_name.as_deref().unwrap_or(&u.username)),
178 stripe_connected: u.stripe_account_id.is_some(),
179 // ... computed fields
180 }
181 }
182 }
183 ```
184
185 This separation means:
186 - `Db*` types mirror the database schema exactly (derive `sqlx::FromRow`).
187 - View types have display-ready fields: formatted dates, computed booleans, pre-rendered HTML.
188 - Handlers call `let user: User = (&db_user).into();` to convert.
189
190 ## Type Safety
191
192 ### ID Newtypes
193
194 All database IDs are newtype wrappers defined via the `define_pg_uuid_id!` macro in `src/db/id_types.rs`:
195
196 ```rust
197 define_pg_uuid_id!(UserId, ProjectId, ItemId, VersionId, /* ... */);
198 ```
199
200 This generates `UserId(Uuid)` with `Display`, `FromStr`, `sqlx::Type`, `Encode`, `Decode`, `Serialize`, `Deserialize`, and `Default` (generates new v4 UUID). Use these everywhere — never pass raw `Uuid` or `String` for IDs.
201
202 ### String Enums
203
204 Domain enums that map to database TEXT columns use the `impl_str_enum!` macro in `src/db/enums.rs`:
205
206 ```rust
207 #[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
208 pub enum ItemType { Audio, Video, Text, Software, /* ... */ }
209
210 impl_str_enum!(ItemType {
211 Audio => "audio",
212 Video => "video",
213 Text => "text",
214 Software => "software",
215 });
216 ```
217
218 This generates `Display`, `FromStr`, and sqlx `Type`/`Encode`/`Decode`. Add `#[serde(rename_all = "lowercase")]` if the JSON representation should match the database strings.
219
220 ## Migrations
221
222 Migrations live in `server/migrations/` and are numbered sequentially (e.g., `001_initial_schema.sql`, `053_add_video_support.sql`). They run automatically on application boot via sqlx.
223
224 **Rules:**
225 - Use `IF NOT EXISTS` guards wherever possible (tables, indexes, extensions).
226 - Use `TIMESTAMPTZ` for all timestamps (UTC-aware).
227 - Use `gen_random_uuid()` for UUID primary keys.
228 - Always add `DEFAULT` values for `NOT NULL` columns.
229 - Create indexes after the table definition, in the same migration.
230 - Prefer additive migrations (add columns, add tables). Destructive changes (drop columns, rename tables) need careful planning.
231 - Name migrations descriptively: `NNN_what_it_does.sql`.
232
233 ```sql
234 -- Example: 004_file_scan_status.sql
235 ALTER TABLE items ADD COLUMN scan_status TEXT NOT NULL DEFAULT 'pending';
236
237 CREATE TABLE file_scan_results (
238 id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
239 s3_key TEXT NOT NULL,
240 scan_status TEXT NOT NULL,
241 scanned_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
242 );
243
244 CREATE INDEX idx_file_scan_results_s3_key ON file_scan_results(s3_key);
245 ```
246
247 ## Templates
248
249 Askama templates (Jinja2-like syntax) live in `server/templates/`. Template structs (Rust side) live in `server/src/templates/`.
250
251 ### Layout Inheritance
252
253 All full pages extend `base.html`:
254 ```html
255 {% extends "base.html" %}
256 {% block title %}Page Title{% endblock %}
257 {% block content %}
258 {% include "partials/site_header.html" %}
259 <!-- page content -->
260 {% endblock %}
261 ```
262
263 Blocks available in `base.html`: `title`, `meta_description`, `head` (extra CSS/meta), `body_attrs`, `content`, `scripts`.
264
265 ### HTMX Partials
266
267 HTMX fragment templates do NOT extend `base.html`. They render standalone HTML fragments:
268 ```html
269 {# No extends — this is a partial #}
270 {% for item in items %}
271 <div class="item-row">{{ item.title }}</div>
272 {% endfor %}
273 ```
274
275 ### Template Variables
276
277 Every full-page template needs at minimum:
278 - `csrf_token: Option<String>` — for the CSRF meta tag
279 - `session_user: Option<SessionUser>` — for the header (login state, avatar)
280
281 ## Frontend Performance
282
283 These rules apply to all HTML templates. The goal is zero layout shift, no wasted pixels, and instant-feeling interactions.
284
285 ### Images
286
287 Every `<img>` must have explicit dimensions to prevent layout shift:
288 ```html
289 <!-- Inline style with both width and height -->
290 <img src="{{ url }}" alt="{{ title }}"
291 style="width: 120px; height: 120px; object-fit: cover;">
292
293 <!-- Or use aspect-ratio for responsive images -->
294 <img src="{{ url }}" alt="{{ title }}"
295 style="width: 100%; aspect-ratio: 1 / 1; object-fit: cover;">
296 ```
297
298 Never use `loading="lazy"` for images that may appear above the fold (the first visible screen before scrolling).
299
300 ### Navigation and Reloads
301
302 Never use `window.location.reload()` when an HTMX partial swap can do the job. Full-page reloads discard all client state, re-parse all JS/CSS, and feel slow.
303
304 ```html
305 <!-- Bad: full-page reload after action -->
306 <button hx-put="/api/items/{{ id }}"
307 hx-on::after-request="if(event.detail.successful) window.location.reload()">
308
309 <!-- Good: re-fetch the relevant tab/section -->
310 <button hx-put="/api/items/{{ id }}"
311 hx-on::after-request="if(event.detail.successful) document.getElementById('tab-settings').click()">
312 ```
313
314 Legitimate uses of `window.location.href`:
315 - Navigating to a different page entirely (login, checkout redirect, file download)
316 - After destructive account actions (delete account → login page)
317
318 ### Loading States
319
320 Prefer server-complete responses over client-side loading placeholders. The server should wait for data and send a complete fragment in one paint, rather than sending a skeleton and filling it in later.
321
322 Use `hx-trigger="revealed"` with a loading placeholder only when the data is expensive to fetch AND hidden behind a `<details>` element (e.g., 2FA status, passkey list). Never use loading placeholders for content that's visible on initial tab load.
323
324 ### Static JavaScript
325
326 JavaScript lives in `server/static/`, one file per feature area:
327
328 ```
329 static/
330 mnw.js — core utilities (CSRF, toasts, tabs, shortcuts) — loaded globally
331 upload.js — S3 upload (S3Uploader, initDropzone) — loaded globally
332 passkey.js — WebAuthn registration/login
333 insertions.js — clip management
334 wizard.js — wizard navigation
335 docs-search.js — doc search index
336 item-details.js — bundle, section, tag management
337 item-upload.js — audio + version upload flows
338 blog-editor.js — blog save/autosave/publish
339 style.css — main stylesheet
340 wizard.css — wizard-specific styles
341 ```
342
343 Only `mnw.js`, `upload.js`, and `htmx.min.js` are loaded globally (in `base.html` / `_head_assets.html`). All other JS files are loaded via `{% block scripts %}` in the page that needs them.
344
345 **Passing server data to static JS:** Use `data-*` attributes on the feature's container element. The JS file reads them on init.
346
347 ```html
348 <!-- In template -->
349 <div id="audio-upload" data-item-id="{{ item.id }}">
350 <!-- upload UI -->
351 </div>
352
353 <!-- In {% block scripts %} -->
354 <script src="/static/item-upload.js"></script>
355 ```
356
357 ```javascript
358 // static/item-upload.js
359 (function() {
360 var el = document.getElementById('audio-upload');
361 if (!el) return;
362 var itemId = el.dataset.itemId;
363 // ...
364 })();
365 ```
366
367 For complex structured data (JSON arrays/objects that can't fit in an attribute), use a minimal inline script:
368
369 ```html
370 <script>window.MNW = window.MNW || {}; window.MNW.pageData = { segments: {{ segments_json|safe }} };</script>
371 <script src="/static/audio-player.js"></script>
372 ```
373
374 **HTMX partial re-initialization:** Static JS loaded in `{% block scripts %}` runs once on page load. For HTMX partials (tab content swapped dynamically), use `htmx:afterSwap` to re-initialize:
375
376 ```javascript
377 document.body.addEventListener('htmx:afterSwap', function(e) {
378 if (e.detail.target.id === 'tab-content') {
379 initMyFeature();
380 }
381 });
382 ```
383
384 **When inline is OK:** Under 20 lines, no template variables that could be data attributes, and tightly coupled to a single template's DOM structure (e.g., tab overflow close handler, single-use form validation).
385
386 ### Inline CSS
387
388 Page-specific `<style>` blocks in `{% block head %}` are acceptable when the styles are truly unique to that page. Shared patterns (form layouts, tables, status badges, cards) belong in `style.css`.
389
390 ### Information Density
391
392 Show more data in less space. Prefer tight rows over hero cards. Let users compare by scanning, not by clicking into detail pages. A page that shows 15 items is more useful than one that shows 3.
393
394 Use typography (weight, size) and whitespace for hierarchy instead of borders and background colors. A 16px gap groups elements as well as a 1px border, with less visual noise.
395
396 ## CSRF Protection
397
398 The CSRF middleware (`src/csrf.rs`) validates all state-changing requests (POST, PUT, PATCH, DELETE) except exempted paths (webhooks, auth endpoints, OAuth).
399
400 **For HTMX requests:** The CSRF token is sent via the `X-CSRF-Token` header, read from the `<meta name="csrf-token">` tag by client-side JS.
401
402 **For vanilla forms:** Include a hidden `_csrf` field:
403 ```html
404 <input type="hidden" name="_csrf" value="{{ csrf_token.as_deref().unwrap_or_default() }}">
405 ```
406
407 Token comparison uses constant-time comparison to prevent timing attacks.
408
409 ## Tracing
410
411 Every handler and significant function gets `#[tracing::instrument(skip_all, name = "...")]`. The `name` parameter uses the pattern `module::function_name` (e.g., `"pages::blog_post_page"`, `"admin::approve_user"`).
412
413 `skip_all` prevents large structs (State, Session, request bodies) from being serialized into spans. Add specific fields if needed:
414 ```rust
415 #[tracing::instrument(skip_all, fields(user_id = %user.id))]
416 ```
417
418 Internal errors are logged at `error!` level. Security events (failed CSRF, malware detection) at `warn!`.
419
420 ## Testing
421
422 ### Unit Tests
423
424 In-file `#[cfg(test)]` modules. No database needed.
425
426 ```rust
427 #[cfg(test)]
428 mod tests {
429 use super::*;
430
431 #[test]
432 fn status_code_not_found() {
433 assert_eq!(AppError::NotFound.status_code(), StatusCode::NOT_FOUND);
434 }
435 }
436 ```
437
438 ### Integration Tests
439
440 Each integration test creates and drops its own PostgreSQL database. Requires `TEST_DATABASE_URL` pointing to a PostgreSQL instance with `CREATE DATABASE` permission.
441
442 ```bash
443 TEST_DATABASE_URL="postgres:///postgres" cargo test --test integration
444 ```
445
446 On Astra, use `--test-threads=8` (or the `RUST_TEST_THREADS=8` env var) to avoid overwhelming PostgreSQL with concurrent database creation.
447
448 ## Rust Edition and Style
449
450 - **Rust 2024 edition** (Rust 1.85+). Uses `gen` keyword restrictions and other 2024 features.
451 - No `.unwrap()` in production code. Use `?`, `.ok_or()`, or `unwrap_or_default()`.
452 - Prefer `Option::and_then`/`map` over `if let Some`/`match` for simple transforms.
453 - File size guideline per root `CONTRIBUTING.md`: 500-line limit on branching logic, flat lists exempt. Route files follow the same rule — split into directory modules when they grow beyond 500 lines.
454
455 ## Dependencies
456
457 Always use the latest stable release of every dependency. When upgrading introduces breaking API changes, update the code — never pin old versions to avoid migration work.
458
459 ## Deployment
460
461 Deploy from the `server/` directory:
462 ```bash
463 ./deploy/deploy.sh # Full: cross-compile + config + binary + restart
464 ./deploy/deploy.sh --quick # Binary only + restart
465 ./deploy/deploy.sh --config # Config files only
466 ```
467
468 Cross-compiles to x86_64-unknown-linux-gnu via `cargo zigbuild`. Version is read from `Cargo.toml` and compiled into the binary via `env!("CARGO_PKG_VERSION")` for Sentry release strings. Bump the version in `Cargo.toml` before every production deploy.
469