Skip to main content

max / shop

Treat a combining mark as an amendment, not a character A mark took a cell of its own, because a cell holds a char and there was nowhere else to put one. That left shop a column AHEAD of the application on any line carrying NFD text — macOS filenames, mostly — the mirror of the wide-character bug fixed one commit ago. Zero width is not a width. A mark always follows its base, so it does not need placing, looking ahead at, or segmenting out of a stream: it amends the character already on screen and leaves the cursor alone. place_char gains one case and all three now answer the same question, which is what a character does to the columns. The base is found from the cursor rather than remembered, so nothing has to survive a scroll between a base arriving and its mark. The cell keeps its base character inline in c_raw's low 21 bits and refers to its marks by an id in the remaining 11. Interning the MARKS rather than whole clusters is what makes this small: real text draws on a handful of distinct sequences however much goes past, since it is the bases that vary. So the table stays in the dozens, Cell::c stays exact and free, and everything reading it — the glyph lookup, the fills scan, word boundaries — is untouched. A cell stays Copy and twelve bytes, so resize_buf, the ring origin and the rewrap keep moving cells by memcpy knowing none of this. Rejected: a boxed vec of marks per cell, which is alacritty's shape. Same feature, worse structurally — Cell stops being Copy and an allocation lands in the path of every scroll. Rejected: dropping marks to make the columns add up, which buys a clean number by losing bytes; a pasted path that no longer names its file is worse than a misdrawn accent. The renderer shapes a cell that carries marks instead of looking one glyph up in the charmap, so the mark lands where the font's GPOS puts it, and caches the result by glyph id like any other. Ordinary cells keep the charmap path. Hot path unchanged: 187-189 Mcell/s against a 183.6 baseline.
Co-Authored-By
Claude Opus 5 (1M context) <noreply@anthropic.com>
Author: Max Johnson <me@maxj.phd> · 2026-08-09 14:59 UTC
Signed with PGP, not checked
Commit: 46d3630d2aaaa7e6a8c4e18655250b6860e0544a
Parent: 6b330de
7 files changed, +458 insertions, -67 deletions
M README.md +17 -3
@@ -135,9 +135,23 @@
135 135 with one column left at the right edge moves to the next row whole rather than
136 136 being split, and the column it could not use is left blank.
137 137
138 - Combining marks are still one cell each, which is wrong in the other direction:
139 - a cell holds a character rather than a grapheme, so a mark has nowhere to go but
140 - its own cell. Fixing that means changing what a cell holds.
138 + A combining mark takes no column at all. It is not placed anywhere: it amends
139 + the character already on screen and leaves the cursor where it was, which is
140 + what the program that printed it did too. Marks always follow their base, so
141 + nothing has to look ahead or segment a stream.
142 +
143 + A cell keeps its base character inline and refers to its marks by a small id
144 + into a table the grid owns. Only the marks are interned, not whole clusters, and
145 + that is what keeps this cheap: real text draws on a handful of distinct mark
146 + sequences however much of it goes past, so the table stays small, a cell stays
147 + twelve bytes, and everything that only wants to know what a cell looks like
148 + never learns the table exists. Copying gets the whole cluster, so a path off a
149 + Mac-formatted volume pastes back as the path it came from.
150 +
151 + Ligatures are a different thing and shop does not do them. `>=` drawn as one
152 + connected glyph is the font's business, not the buffer's — the two characters
153 + still occupy two cells — and a ligature makes a cursor sitting between them land
154 + visually inside a glyph. Column-accurate editing wins that trade.
141 155
142 156 ## Emit
143 157
M crates/shop-grid/src/lib.rs +326 -9
@@ -12,7 +12,7 @@
12 12 pub use selection::{Point, Selection, SelectionMode, SelectionSpan};
13 13
14 14 use shop_vt::{Params, Perform};
15 - use std::collections::VecDeque;
15 + use std::collections::{HashMap, VecDeque};
16 16 use tracing::trace;
17 17
18 18 /// A terminal color.
@@ -49,7 +49,19 @@
49 49 /// Layout of `fg_word` / `bg_word`:
50 50 /// - bits 0–23: color payload — RGB if `src == Rgb`, low byte = palette index if Named/Indexed
51 51 /// - bits 24–25: source tag (`SRC_DEFAULT`/`NAMED`/`INDEXED`/`RGB`)
52 - /// - bits 26–31: attribute flags (fg_word only; bg_word spare)
52 + /// - bits 26–31: attribute flags (fg_word only; bg_word width flags + spare)
53 + ///
54 + /// Layout of `c_raw`:
55 + /// - bits 0–20: the base character. A `char` is 21 bits, so this is the whole
56 + /// of one and [`Cell::c`] is a mask away.
57 + /// - bits 21–31: id into the grid's mark table, 0 meaning none.
58 + ///
59 + /// Keeping the base character INLINE and interning only the marks after it is
60 + /// what makes combining marks cost nothing to everything that does not care:
61 + /// the renderer's glyph lookup, the fills scan and word boundaries all read
62 + /// `c()` and never learn the table exists. It also keeps the cell `Copy` and
63 + /// memcpy-able, which is what lets `resize_buf`, the ring's origin advance and
64 + /// the scrollback rewrap move cells around without knowing any of this.
53 65 #[derive(Copy, Clone, Debug, PartialEq, Eq, Hash)]
54 66 #[repr(C)]
55 67 pub struct Cell {
@@ -58,6 +70,22 @@
58 70 bg_word: u32,
59 71 }
60 72
73 + /// A `char` is 21 bits; `c_raw`'s low 21 hold one whole.
74 + const CHAR_MASK: u32 = (1 << 21) - 1;
75 + const MARKS_SHIFT: u32 = 21;
76 + /// Eleven bits of mark-sequence id, 0 reserved for "no marks". Distinct mark
77 + /// SEQUENCES in real text run to dozens — it is distinct clusters that run to
78 + /// thousands, and those are not what is interned here — so 2047 is not a
79 + /// ceiling anything is expected to reach.
80 + const MARKS_LIMIT: u32 = (1 << 11) - 1;
81 +
82 + /// How many combining marks one cell will accumulate.
83 + ///
84 + /// A stream of combining marks is unbounded input, and a cluster long enough to
85 + /// need more than this is not text anyone is reading. Past the cap the mark is
86 + /// dropped rather than the cell rewritten.
87 + const MAX_MARKS: usize = 8;
88 +
61 89 const SRC_DEFAULT: u32 = 0;
62 90 const SRC_NAMED: u32 = 1;
63 91 const SRC_INDEXED: u32 = 2;
@@ -139,10 +167,29 @@
139 167 }
140 168 }
141 169
170 + /// The cell's base character, with any combining marks left behind.
171 + ///
172 + /// This is what the renderer draws and what word boundaries read. The marks
173 + /// change how a cluster looks, not what it is or how wide it is, so nothing
174 + /// laying out columns needs them.
142 175 #[inline]
143 176 pub fn c(&self) -> char {
144 177 // Grid only ever writes valid `char` values into `c_raw`.
145 - char::from_u32(self.c_raw).unwrap_or('\u{FFFD}')
178 + char::from_u32(self.c_raw & CHAR_MASK).unwrap_or('\u{FFFD}')
179 + }
180 +
181 + /// Id of this cell's combining marks in the grid's table, or 0 for none.
182 + #[inline]
183 + pub fn marks_id(&self) -> u16 {
184 + ((self.c_raw >> MARKS_SHIFT) & MARKS_LIMIT) as u16
185 + }
186 +
187 + #[inline]
188 + fn with_marks(self, id: u16) -> Self {
189 + Self {
190 + c_raw: (self.c_raw & CHAR_MASK) | (u32::from(id) << MARKS_SHIFT),
191 + ..self
192 + }
146 193 }
147 194
148 195 #[inline]
@@ -239,18 +286,64 @@
239 286 }
240 287 }
241 288
289 + /// The combining-mark sequences the cells refer to by id.
290 + ///
291 + /// Append-only and interned. Real text draws from a handful of distinct
292 + /// sequences however much of it goes past — one `U+0301` is every acute accent
293 + /// on screen — so this stays in the dozens and never needs freeing. Interning
294 + /// the marks rather than whole clusters is what keeps it that small: the bases
295 + /// they attach to are what vary.
296 + #[derive(Debug, Default)]
297 + struct MarkTable {
298 + /// Id `n` is at index `n - 1`; id 0 means a cell has no marks and is never
299 + /// stored.
300 + seqs: Vec<Box<[char]>>,
301 + ids: HashMap<Box<[char]>, u16>,
302 + }
303 +
304 + impl MarkTable {
305 + /// The marks for `id`, or empty for id 0.
306 + fn get(&self, id: u16) -> &[char] {
307 + match id.checked_sub(1) {
308 + Some(i) => self.seqs.get(i as usize).map_or(&[], |s| s),
309 + None => &[],
310 + }
311 + }
312 +
313 + /// The id for a sequence, interning it if it is new.
314 + ///
315 + /// `None` once the id space is exhausted, which is the caller's cue to drop
316 + /// the mark. Refusing to store a mark keeps the cell as it was; there is no
317 + /// id it could be given that would not mean somebody else's marks.
318 + fn intern(&mut self, seq: &[char]) -> Option<u16> {
319 + if let Some(id) = self.ids.get(seq) {
320 + return Some(*id);
321 + }
322 + let id = u16::try_from(self.seqs.len() + 1).ok()?;
323 + if u32::from(id) > MARKS_LIMIT {
324 + return None;
325 + }
326 + let seq: Box<[char]> = seq.into();
327 + self.seqs.push(seq.clone());
328 + self.ids.insert(seq, id);
329 + Some(id)
330 + }
331 + }
332 +
242 333 /// How many columns a character occupies, as the application computing its own
243 334 /// cursor moves will have counted it.
244 335 ///
245 - /// Zero-width characters (combining marks) are counted as one column, which is
246 - /// what shop does with them today: a cell holds a `char`, so a mark has nowhere
247 - /// to go but its own cell. That is wrong in the other direction — shop ends up
248 - /// a column ahead of the application rather than behind it — and fixing it
249 - /// means a cell holding a grapheme rather than a `char`. Tracked separately;
250 - /// counting it as zero here without somewhere to put it would drop the accent.
336 + /// Zero is a real answer here rather than a degenerate one: a combining mark
337 + /// occupies no column of its own, it modifies the one before it. `place_char`
338 + /// reads it as an instruction to amend rather than to place.
339 + ///
340 + /// A character with no width at all (a control) counts as one. `execute`
341 + /// handles C0 so one should not reach a print, and one column is the answer
342 + /// that leaves the cursor where the application put it if one does.
251 343 fn char_cols(c: char) -> u16 {
252 344 match unicode_width::UnicodeWidthChar::width(c) {
253 345 Some(2) => 2,
346 + Some(0) => 0,
254 347 _ => 1,
255 348 }
256 349 }
@@ -412,6 +505,13 @@
412 505 history: VecDeque<HistoryRow>,
413 506 /// How many rows `history` keeps before dropping its oldest.
414 507 history_limit: usize,
508 + /// The combining marks that cells refer to by id.
509 + ///
510 + /// Grid-wide rather than per-screen, so a cell can be copied between the
511 + /// live screen, the alt screen and history without its marks needing to
512 + /// travel or be rewritten. That is the point of storing an id: a cell stays
513 + /// a value, and every structural move of cells in this file stays a memcpy.
514 + marks: MarkTable,
415 515 /// How far back the viewport sits, in rows. Zero is live. Never exceeds
416 516 /// `history.len()`, and forced to zero on the alt screen.
417 517 view_offset: u16,
@@ -497,6 +597,7 @@
497 597 alt_wrapped: vec![false; rows as usize],
498 598 history: VecDeque::new(),
499 599 history_limit: DEFAULT_HISTORY_LIMIT,
600 + marks: MarkTable::default(),
500 601 view_offset: 0,
501 602 view_dirty: false,
502 603 on_alt: false,
@@ -700,6 +801,32 @@
700 801 &self.active_cells()[start..end]
701 802 }
702 803
804 + /// Append a cell's text — base character then any combining marks — to
805 + /// `out`.
806 + ///
807 + /// The seam between "what a cell looks like" and "what a cell is". The
808 + /// renderer and word boundaries want the base and use [`Cell::c`]; anything
809 + /// producing text a human or another program will read wants the cluster
810 + /// and comes here, because an accent dropped on the way to the clipboard is
811 + /// a path that no longer names the file it came from.
812 + pub(crate) fn push_cell_text(&self, cell: &Cell, out: &mut String) {
813 + out.push(match cell.c() {
814 + '\0' => ' ',
815 + c => c,
816 + });
817 + out.extend(self.marks.get(cell.marks_id()));
818 + }
819 +
820 + /// A cell's combining marks, in the order they arrived, empty for almost
821 + /// every cell.
822 + ///
823 + /// The renderer needs these to draw a cluster; nothing else outside this
824 + /// crate does, because [`Cell::c`] already answers what the cell is and how
825 + /// wide it is.
826 + pub fn marks(&self, cell: &Cell) -> &[char] {
827 + self.marks.get(cell.marks_id())
828 + }
829 +
703 830 /// The column a pointer at `col` was aiming at.
704 831 ///
705 832 /// Half a wide character is not a thing anyone can mean to click on, so a
@@ -1235,6 +1362,13 @@
1235 1362 // A wide character needs two columns, so on a one-column grid there is
1236 1363 // no such thing and the pair never forms.
1237 1364 let width = if self.cols < 2 { 1 } else { char_cols(c) };
1365 + if width == 0 {
1366 + // A combining mark is not placed anywhere. It amends the character
1367 + // already on screen and leaves the cursor where it was, which is
1368 + // what the application counting its own columns did too.
1369 + self.amend_with_mark(c);
1370 + return;
1371 + }
1238 1372 // Deferred wrap: if the previous print landed on the rightmost cell,
1239 1373 // the next visible char starts a new line.
1240 1374 if self.cursor.wrap_next {
@@ -1338,6 +1472,48 @@
1338 1472 }
1339 1473 }
1340 1474
1475 + /// Attach a combining mark to the character the cursor last passed over.
1476 + ///
1477 + /// The base is found from the cursor rather than remembered, so nothing has
1478 + /// to survive a scroll between the base arriving and its mark: it is the
1479 + /// cell the cursor is about to move past, stepped back once more when that
1480 + /// lands on a wide character's second column.
1481 + fn amend_with_mark(&mut self, mark: char) {
1482 + // With the deferred wrap pending, the cursor is still ON the last
1483 + // character it wrote rather than after it.
1484 + let Some(col) = (if self.cursor.wrap_next {
1485 + Some(self.cursor.col)
1486 + } else {
1487 + self.cursor.col.checked_sub(1)
1488 + }) else {
1489 + // A mark with nothing before it on this row. Malformed — there is
1490 + // no base for it to change, and inventing a cell for it would be
1491 + // the column error this whole thing exists to remove.
1492 + return;
1493 + };
1494 + let start = self.row_start(self.cursor.row);
1495 + let cells = self.active_cells();
1496 + let col = if cells[start + col as usize].is_spacer() && col > 0 {
1497 + col - 1
1498 + } else {
1499 + col
1500 + };
1501 + let base = cells[start + col as usize];
1502 + let mut seq: Vec<char> = self.marks.get(base.marks_id()).to_vec();
1503 + if seq.len() >= MAX_MARKS {
1504 + return;
1505 + }
1506 + seq.push(mark);
1507 + let Some(id) = self.marks.intern(&seq) else {
1508 + // The id space is spent. Dropping the mark leaves the cell holding
1509 + // what it held, which is the only outcome here that is not a lie.
1510 + return;
1511 + };
1512 + let cells = self.active_cells_mut();
1513 + cells[start + col as usize] = base.with_marks(id);
1514 + self.mark_row_dirty(self.cursor.row);
1515 + }
1516 +
1341 1517 /// Leave column `col` of the cursor's row as a blank the width of one cell
1342 1518 /// that is not a character: the column a wide character could not fit into.
1343 1519 fn write_pad(&mut self, col: u16) {
@@ -3216,4 +3392,145 @@
3216 3392 sel.drag_to(Point::new(0, 2));
3217 3393 assert_eq!(g.selection_text(&sel), "日本");
3218 3394 }
3395 +
3396 + // -- combining marks ---------------------------------------------------
3397 + //
3398 + // A mark modifies the character before it and occupies no column. The cell
3399 + // keeps its base inline and refers to the marks by id, so everything that
3400 + // only wants to know what a cell LOOKS like is unchanged and only the paths
3401 + // producing text have to reassemble the cluster.
3402 +
3403 + #[test]
3404 + fn a_combining_mark_takes_no_column_of_its_own() {
3405 + let mut g = Grid::new(8, 2);
3406 + feed(&mut g, "e\u{301}x".as_bytes());
3407 + assert_eq!(g.cursor().col, 2, "the mark consumed a column");
3408 + assert_eq!(g.row(0)[0].c(), 'e', "the base is not inline any more");
3409 + assert_eq!(
3410 + g.row(0)[1].c(),
3411 + 'x',
3412 + "the mark displaced the next character"
3413 + );
3414 + }
3415 +
3416 + #[test]
3417 + fn a_combining_mark_copies_back_with_its_base() {
3418 + // The whole point: a path off a Mac-formatted volume has to paste back
3419 + // as the path it came from, accents and all.
3420 + let mut g = Grid::new(20, 2);
3421 + feed(&mut g, "Jose\u{301}/".as_bytes());
3422 + assert_eq!(g.text_range(0, 1), "Jose\u{301}/\n");
3423 + }
3424 +
3425 + #[test]
3426 + fn several_marks_stack_on_one_base() {
3427 + let mut g = Grid::new(8, 2);
3428 + feed(&mut g, "o\u{323}\u{302}".as_bytes());
3429 + assert_eq!(g.cursor().col, 1);
3430 + assert_eq!(g.text_range(0, 1), "o\u{323}\u{302}\n");
3431 + }
3432 +
3433 + #[test]
3434 + fn a_mark_after_a_wide_character_lands_on_the_character_not_its_spacer() {
3435 + let mut g = Grid::new(8, 2);
3436 + feed(&mut g, "日\u{301}".as_bytes());
3437 + assert_eq!(g.row(0)[0].marks_id(), 1, "the mark missed the lead");
3438 + assert_eq!(g.row(0)[1].marks_id(), 0, "the spacer took the mark");
3439 + assert_eq!(g.text_range(0, 1), "日\u{301}\n");
3440 + }
3441 +
3442 + #[test]
3443 + fn a_mark_at_the_right_edge_amends_the_character_still_under_the_cursor() {
3444 + // The deferred wrap leaves the cursor ON the last character it wrote
3445 + // rather than after it, so the base is found differently there.
3446 + let mut g = Grid::new(4, 2);
3447 + feed(&mut g, "abcd\u{301}".as_bytes());
3448 + assert_eq!(g.text_range(0, 1), "abcd\u{301}\n");
3449 + assert_eq!(g.cursor().col, 3, "the mark moved the cursor off the edge");
3450 + }
3451 +
3452 + #[test]
3453 + fn a_mark_with_nothing_before_it_is_dropped() {
3454 + // Malformed. Giving it a cell would be exactly the column error this
3455 + // is here to remove.
3456 + let mut g = Grid::new(8, 2);
3457 + feed(&mut g, "\u{301}x".as_bytes());
3458 + assert_eq!(g.cursor().col, 1);
3459 + assert_eq!(g.text_range(0, 1), "x\n");
3460 + }
3461 +
3462 + #[test]
3463 + fn marks_survive_a_rewrap() {
3464 + // The id rides in the cell, so every structural move of cells in this
3465 + // file carries the marks with no help. This is the assertion that says
3466 + // so out loud.
3467 + let mut g = Grid::new(10, 2);
3468 + feed(&mut g, "abcde\u{301}fghij\r\nx\r\ny".as_bytes());
3469 + let before = history_text(&g);
3470 + assert!(before.contains('\u{301}'));
3471 + g.resize(4, 2);
3472 + g.resize(10, 2);
3473 + assert_eq!(history_text(&g), before);
3474 + }
3475 +
3476 + #[test]
3477 + fn overwriting_a_base_drops_the_marks_that_were_on_it() {
3478 + // The marks belonged to the character that was there, not to the cell.
3479 + let mut g = Grid::new(8, 2);
3480 + feed(&mut g, "e\u{301}".as_bytes());
3481 + feed(&mut g, b"\x1b[1;1Hx");
3482 + assert_eq!(g.text_range(0, 1), "x\n");
3483 + assert_eq!(g.row(0)[0].marks_id(), 0);
3484 + }
3485 +
3486 + #[test]
3487 + fn the_same_mark_sequence_is_interned_once() {
3488 + // What keeps the table in the dozens however much text goes past: it
3489 + // is the bases that vary, not the sequences attached to them.
3490 + let mut g = Grid::new(20, 2);
3491 + feed(
3492 + &mut g,
3493 + "a\u{301}e\u{301}i\u{301}o\u{301}u\u{301}".as_bytes(),
3494 + );
3495 + let ids: Vec<u16> = (0..5).map(|c| g.row(0)[c].marks_id()).collect();
3496 + assert_eq!(ids, vec![1; 5], "one sequence took five ids");
3497 + }
3498 +
3499 + #[test]
3500 + fn a_cell_stops_taking_marks_at_the_cap() {
3501 + // Unbounded input. Past the cap the mark is dropped and the cell keeps
3502 + // what it had, rather than the cell being rewritten or the id space
3503 + // being spent on a cluster nobody is reading.
3504 + let mut g = Grid::new(8, 2);
3505 + let mut bytes = String::from("e");
3506 + for _ in 0..MAX_MARKS + 5 {
3507 + bytes.push('\u{301}');
3508 + }
3509 + feed(&mut g, bytes.as_bytes());
3510 + let text = g.text_range(0, 1);
3511 + assert_eq!(
3512 + text.chars().filter(|c| *c == '\u{301}').count(),
3513 + MAX_MARKS,
3514 + "the cap did not hold"
3515 + );
3516 + }
3517 +
3518 + #[test]
3519 + fn a_marked_character_is_one_word_for_a_double_click() {
3520 + // Word boundaries read the base and never learn the table exists.
3521 + let mut g = Grid::new(20, 2);
3522 + feed(&mut g, "Jose\u{301} x".as_bytes());
3523 + let sel = Selection::new(SelectionMode::Word, Point::new(0, 0));
3524 + assert_eq!(g.selection_text(&sel), "Jose\u{301}");
3525 + }
3526 +
3527 + #[test]
3528 + fn a_cell_is_still_twelve_bytes_and_still_copy() {
3529 + // The property the whole layout choice exists to protect: cells move by
3530 + // memcpy through resize, the ring origin and history.
3531 + assert_eq!(std::mem::size_of::<Cell>(), 12);
3532 + let a = Cell::default();
3533 + let b = a;
3534 + assert_eq!(a, b);
3535 + }
3219 3536 }
M crates/shop-grid/src/selection.rs +8 -8
@@ -267,16 +267,16 @@
267 267 } else {
268 268 hi
269 269 };
270 - // One character per character, not per column: a wide character's
271 - // second column is not its own character and copies as nothing.
272 - let line: String = cells[lo as usize..=hi as usize]
270 + // One entry per character, not per column: a wide character's
271 + // second column is not its own character and copies as nothing,
272 + // and a character carrying combining marks copies with them.
273 + let mut line = String::new();
274 + for cell in cells[lo as usize..=hi as usize]
273 275 .iter()
274 276 .filter(|cell| !cell.is_spacer())
275 - .map(|cell| match cell.c() {
276 - '\0' => ' ',
277 - c => c,
278 - })
279 - .collect();
277 + {
278 + self.push_cell_text(cell, &mut line);
279 + }
280 280 // Trailing blanks on a wrapped row are real cells the text ran
281 281 // through, not padding — stripping them would eat the space
282 282 // between two words that happened to straddle the edge.
M crates/shop-grid/src/text.rs +14 -15
@@ -57,9 +57,10 @@
57 57 // Trailing blanks on a wrapped row are real cells the text ran
58 58 // through. Stripping them would eat the space between two
59 59 // words that happened to straddle the edge.
60 - out.extend(chars(cells));
60 + self.push_row_text(cells, &mut out);
61 61 } else {
62 - let line: String = chars(cells).collect();
62 + let mut line = String::new();
63 + self.push_row_text(cells, &mut line);
63 64 out.push_str(line.trim_end());
64 65 out.push('\n');
65 66 }
@@ -95,19 +96,17 @@
95 96 }
96 97 }
97 98
98 - /// The characters a row of cells reads as, one per character rather than one
99 - /// per column: a wide character's second column is not a character of its own,
100 - /// so it is skipped rather than emitted as the blank it holds.
101 - fn chars(cells: &[Cell]) -> impl Iterator<Item = char> + '_ {
102 - cells.iter().filter(|c| !c.is_spacer()).map(cell_char)
103 - }
104 -
105 - /// The character a cell reads as. Unwritten cells hold `\0` and look like
106 - /// blanks on screen, so they emit as blanks.
107 - fn cell_char(cell: &Cell) -> char {
108 - match cell.c() {
109 - '\0' => ' ',
110 - c => c,
99 + impl Grid {
100 + /// A row of cells as text: one entry per CHARACTER rather than per column,
101 + /// and each of those the whole cluster.
102 + ///
103 + /// A wide character's second column is not a character of its own and is
104 + /// skipped rather than emitted as the blank it holds; a character carrying
105 + /// combining marks comes out with them.
106 + fn push_row_text(&self, cells: &[Cell], out: &mut String) {
107 + for cell in cells.iter().filter(|c| !c.is_spacer()) {
108 + self.push_cell_text(cell, out);
109 + }
111 110 }
112 111 }
113 112
M crates/shop-render/src/lib.rs +1 -1
@@ -14,4 +14,4 @@
14 14 mod shaper;
15 15
16 16 pub use image::{ImagePlacement, ImageRenderer};
17 - pub use pipeline::{BgFill, CellDraw, TextRenderer};
17 + pub use pipeline::{BgFill, CellDraw, CellText, TextRenderer};
M crates/shop-render/src/pipeline.rs +78 -28
@@ -131,9 +131,25 @@
131 131 slot: AtlasSlot,
132 132 left: i32,
133 133 top: i32,
134 + /// Offset within the cell, in pixels. Zero for the one glyph an ordinary
135 + /// cell draws; non-zero only for the marks of a cluster, which the shaper
136 + /// positions against their base rather than against the cell.
137 + dx: f32,
138 + dy: f32,
134 139 color: [f32; 4],
135 140 }
136 141
142 + /// What one cell draws.
143 + ///
144 + /// Almost always a single character looked up in the font's charmap, which is
145 + /// the path worth keeping cheap. A cell carrying combining marks is shaped
146 + /// instead, because where a mark sits over its base is the font's business and
147 + /// GPOS is how it says so.
148 + pub enum CellText {
149 + Char(char),
150 + Cluster(String),
151 + }
152 +
137 153 impl TextRenderer {
138 154 pub fn new(
139 155 device: &wgpu::Device,
@@ -328,7 +344,7 @@
328 344 &mut self,
329 345 queue: &wgpu::Queue,
330 346 row: u16,
331 - cells: impl IntoIterator<Item = (u16, char, [f32; 4])>,
347 + cells: impl IntoIterator<Item = (u16, CellText, [f32; 4])>,
332 348 ) {
333 349 let Some(slot) = self.row_cache.get_mut(row as usize) else {
334 350 return;
@@ -337,35 +353,66 @@
337 353 let atlas = &mut self.atlas;
338 354 let cache = &mut self.cache;
339 355 let shaper = &mut self.shaper;
340 - for (col, c, color) in cells {
341 - let glyph_id = shaper.glyph_id_for(c);
342 - let cached = match cache.get(&glyph_id).copied() {
343 - Some(v) => v,
344 - None => {
345 - let entry = shaper.rasterize(glyph_id).and_then(|r| {
346 - atlas
347 - .upload(queue, r.width, r.height, &r.bitmap)
348 - .map(|s| CachedGlyph {
349 - slot: s,
350 - left: r.placement_left,
351 - top: r.placement_top,
352 - })
356 + // One glyph id to an atlas slot, memoized. The id is what the cache is
357 + // keyed on either way, so a mark rasterized for one base is reused for
358 + // every other base it ever sits on.
359 + let mut push =
360 + |slot: &mut Vec<CachedCell>, shaper: &mut Shaper, id, col, dx: f32, dy: f32, color| {
361 + let cached = match cache.get(&id).copied() {
362 + Some(v) => v,
363 + None => {
364 + let entry = shaper.rasterize(id).and_then(|r| {
365 + atlas
366 + .upload(queue, r.width, r.height, &r.bitmap)
367 + .map(|s| CachedGlyph {
368 + slot: s,
369 + left: r.placement_left,
370 + top: r.placement_top,
371 + })
372 + });
373 + cache.insert(id, entry);
374 + entry
375 + }
376 + };
377 + if let Some(g) = cached
378 + && g.slot.px[0] > 0
379 + && g.slot.px[1] > 0
380 + {
381 + slot.push(CachedCell {
382 + col,
383 + slot: g.slot,
384 + left: g.left,
385 + top: g.top,
386 + dx,
387 + dy,
388 + color,
353 389 });
354 - cache.insert(glyph_id, entry);
355 - entry
356 390 }
357 391 };
358 - if let Some(g) = cached
359 - && g.slot.px[0] > 0
360 - && g.slot.px[1] > 0
361 - {
362 - slot.push(CachedCell {
363 - col,
364 - slot: g.slot,
365 - left: g.left,
366 - top: g.top,
367 - color,
368 - });
392 + for (col, text, color) in cells {
393 + match text {
394 + CellText::Char(c) => {
395 + let id = shaper.glyph_id_for(c);
396 + push(slot, shaper, id, col, 0.0, 0.0, color);
397 + }
398 + CellText::Cluster(s) => {
399 + // The pen walks the cluster so a mark lands wherever the
400 + // font puts it: a combining mark carries no advance, so it
401 + // stacks on the base rather than following it.
402 + let mut pen_x = 0.0;
403 + for g in shaper.shape(&s) {
404 + push(
405 + slot,
406 + shaper,
407 + g.id,
408 + col,
409 + pen_x + g.x_offset,
410 + g.y_offset,
411 + color,
412 + );
413 + pen_x += g.advance;
414 + }
415 + }
369 416 }
370 417 }
371 418 }
@@ -407,7 +454,10 @@
407 454 for cell in row {
408 455 let cell_x = pad_x + cell.col as f32 * cell_w;
409 456 instances.push(Instance {
410 - pos: [cell_x + cell.left as f32, row_y + ascent - cell.top as f32],
457 + pos: [
458 + cell_x + cell.left as f32 + cell.dx,
459 + row_y + ascent - cell.top as f32 - cell.dy,
460 + ],
411 461 size: [cell.slot.px[0] as f32, cell.slot.px[1] as f32],
412 462 uv_min: cell.slot.uv_min,
413 463 uv_max: cell.slot.uv_max,
M crates/shop/src/main.rs +14 -3
@@ -23,7 +23,7 @@
23 23 use kittygfx as kgp;
24 24 use shop_grid::{Color as GridColor, CursorShape, Grid, Point, Selection, SelectionMode};
25 25 use shop_pty::{Pty, PtySize};
26 - use shop_render::{BgFill, ImagePlacement, ImageRenderer, TextRenderer};
26 + use shop_render::{BgFill, CellText, ImagePlacement, ImageRenderer, TextRenderer};
27 27 use shop_wayland::{
28 28 Capability, CompositorHandler, CompositorState, Connection, OutputHandler, OutputState,
29 29 Pending, ProvidesRegistryState, QueueHandle, RegistryState, SeatHandler, SeatState,
@@ -866,7 +866,8 @@
866 866 .enumerate()
867 867 .filter_map(|(col, cell)| {
868 868 let c = cell.c();
869 - if c == ' ' || c == '\0' {
869 + let marks = app.grid.marks(cell);
870 + if (c == ' ' || c == '\0') && marks.is_empty() {
870 871 return None;
871 872 }
872 873 let mut fg = resolve_color(cell.fg(), app.palette.fg, &app.palette);
@@ -875,7 +876,17 @@
875 876 std::mem::swap(&mut fg, &mut bg);
876 877 }
877 878 let _ = bg;
878 - Some((col as u16, c, fg))
879 + // Almost every cell is one character and takes the charmap
880 + // path; only a cell carrying marks is worth shaping.
881 + let text = if marks.is_empty() {
882 + CellText::Char(c)
883 + } else {
884 + let mut s = String::with_capacity(1 + marks.len());
885 + s.push(c);
886 + s.extend(marks);
887 + CellText::Cluster(s)
888 + };
889 + Some((col as u16, text, fg))
879 890 }),
880 891 );
881 892 }