| 1 | # Handover — `highlight` (syntax highlighting in the blob view) |
| 2 | |
| 3 | Branch `feat/highlighting`. One commit of code, plus this file. |
| 4 | |
| 5 | ## What shipped |
| 6 | |
| 7 | `/{handle}/repos/{name}/tree/{rev}/-/{path}` renders a text file with syntax |
| 8 | highlighting. Nothing else changed: no new route, no new `GitQuery` method, **no extra |
| 9 | `git` subprocess** — the bytes were already being read for the plain-text view. |
| 10 | |
| 11 | - `src/infrastructure/highlight.rs` — the whole adapter. `source_lines(file_name, text)` |
| 12 | returns `Source { lines: Vec<SourceLine>, too_large: bool }`, one entry per line of |
| 13 | `text.lines()`, each either `Plain(String)` or `Classed(String)` of HTML. |
| 14 | - `src/infrastructure/web/browse.rs` — only the `source` component: it takes a |
| 15 | `file_name` as well as the text, and the code cell renders `Classed` through |
| 16 | `Unescaped`. The row, the line-number cell and the scroll container are untouched, so |
| 17 | the blob-header toggle and the `id="L<n>"` anchors being added by sibling agents do |
| 18 | not collide with this. |
| 19 | - `styles.css` — a `--syntax-*` family on `:root` and `.dark`, and the mapping onto the |
| 20 | `hl-` classes at the bottom of the file. |
| 21 | - `syntect 5.3` with `default-features = false, features = ["default-fancy"]`: the pure |
| 22 | Rust regex engine, so the Debian release build gains no C dependency. |
| 23 | - `two-face 0.5` with `default-features = false, features = ["syntect-fancy"]`: the |
| 24 | grammars syntect's defaults lack. Its default feature is `syntect-onig`, so |
| 25 | `default-features = false` is not tidiness — it is what keeps `onig`, and therefore a |
| 26 | C dependency, out of the release build. |
| 27 | |
| 28 | ## progress.md material — decisions worth not rediscovering |
| 29 | |
| 30 | - **Classes, not inline styles.** `syntect` will happily emit `style="color:#..."`. |
| 31 | That would hardcode one palette into generated markup and follow neither a palette |
| 32 | change nor the colour scheme, which is the one rule `styles.css` states outright. So |
| 33 | the output is `ClassStyle::SpacedPrefixed { prefix: "hl-" }` and the colours live in |
| 34 | `styles.css` with every other token. |
| 35 | - **`ClassedHTMLGenerator` cannot be used here.** It produces a single string whose |
| 36 | `<span>`s cross line boundaries — its own docs say the output must go in one `<pre>` — |
| 37 | and the blob view is a table with one row per line, so those spans cannot be split |
| 38 | across cells. `highlight.rs` therefore drives `ParseState` + `ScopeStack` + |
| 39 | `line_tokens_to_classed_spans` itself: the parse state carries across lines (a block |
| 40 | comment opens on one and closes on another) but the *markup* does not — whatever is |
| 41 | open at the end of a line is closed there and reopened at the start of the next. That |
| 42 | is the whole reason the module is 150 lines rather than 20. |
| 43 | - **The line ending is stripped by character, not trimmed.** The `_newlines` syntaxes |
| 44 | need the `\n` handed to the parser, but it comes back *escaped inside the output* and |
| 45 | can land before or after a closing `</span>` depending on where a scope pops. Left in, |
| 46 | `white-space: pre` renders a second, empty line inside every row and the whole file |
| 47 | reads double-spaced. It is dropped with a `filter`, not a `trim_end`. |
| 48 | - **`hl-` classes are scope *atoms*, and atoms are not positional.** `keyword.control` |
| 49 | becomes `class="hl-keyword hl-control"`, and `meta.function` — which wraps a whole |
| 50 | function *body* — becomes `class="hl-meta hl-function"`. A bare `.hl-function` rule |
| 51 | would therefore colour every function body. Single-atom selectors are only safe for an |
| 52 | atom that leads its scope; the rest are spelled as compounds |
| 53 | (`.hl-entity.hl-name.hl-function`). **Rule order is load-bearing** for the same |
| 54 | reason: `punctuation.definition.string` carries both `hl-punctuation` and `hl-string`, |
| 55 | equal specificity, so `.hl-string` is written later and a quote reads as its string. |
| 56 | The comment rule is last so a comment is a comment whatever it contains. |
| 57 | - **Plain text is not "highlighted with a plain-text syntax".** When the resolved syntax |
| 58 | is syntect's `Plain Text`, `source_lines` returns `Plain` lines, so a `LICENSE` is not |
| 59 | wrapped in spans that colour nothing. |
| 60 | - **The caps are 512 KiB and 10,000 lines**, and they are two caps because the cost is |
| 61 | both per byte (one enormous minified line) and per line. The page states it — |
| 62 | otherwise a large Rust file silently looks like an unsupported language. Neither |
| 63 | number was derived from a measurement; see below. |
| 64 | - **The syntax set is loaded once into a `OnceLock`** on the first blob viewed. It is a |
| 65 | few MB of resident memory for the life of the process. |
| 66 | |
| 67 | ## current.md material — holes opened and shortcuts taken |
| 68 | |
| 69 | - **`.jsx` is the one extension still unhighlighted.** `two_face` closed the gap that |
| 70 | mattered — TOML, Dockerfile and TypeScript all highlight now, and the first two are in |
| 71 | this repository — but its JavaScript grammar claims only `js` and `htc`, while React |
| 72 | is covered on the TypeScript side by `TypeScriptReact` for `tsx`. So a `.jsx` file |
| 73 | renders plain. Nothing here uses one; it is recorded rather than fixed, and the fix if |
| 74 | it is ever wanted is to alias `jsx` onto the JavaScript syntax in `syntax_for`, not |
| 75 | another crate. `the_languages_this_repository_uses_are_all_highlighted` asserts the |
| 76 | sixteen that do work, so a future syntax-set swap cannot quietly reopen the hole. |
| 77 | |
| 78 | - **Highlighting costs ~87 ms per 1,255-line file in a release build** — measured on |
| 79 | `browse.rs` with `cargo test --release`, and it is paid on every view because nothing |
| 80 | caches. In the `topcoat dev` (debug) build the same page takes ~0.7 s to serve versus |
| 81 | ~0.06 s for a file past the cap, so debug is roughly eight times slower and is not the |
| 82 | number to plan against. 87 ms is still the most expensive thing on a blob page: it is |
| 83 | a Rust regex engine over every line, not a subprocess. If it needs to go, the answers |
| 84 | in order are cache per blob object id (the id is already in `FileView`), then lower the |
| 85 | caps. |
| 86 | - **Nothing caches.** The same file is re-highlighted on every view. |
| 87 | - **Diffs, READMEs and markdown code fences are unhighlighted**, as scoped. A README |
| 88 | rendered on the repository page goes through `web/markdown.rs`, which writes its own |
| 89 | `<pre><code>`; wiring this module into it is a small, obvious follow-up and was left |
| 90 | alone to keep the branches apart. |
| 91 | - **`\r\n` files keep the `\r` inside the highlighted markup** (it is stripped from the |
| 92 | plain path by `str::lines`). Invisible in `white-space: pre`; noted rather than fixed. |
| 93 | - **The blob view's own header has no "this is Rust" indicator.** The language is |
| 94 | detected and then only ever visible as colour. |
| 95 | |
| 96 | ## The `two-face` decision, and what it cost |
| 97 | |
| 98 | Taken by the user after the first pass reported the gap. |
| 99 | |
| 100 | - **The syntax set is now `two_face::syntax::extra_newlines()`**, which is syntect's |
| 101 | defaults plus the grammars `bat` curates. It is a drop-in for |
| 102 | `SyntaxSet::load_defaults_newlines` — one line in the `OnceLock`. Nothing else moved: |
| 103 | the `hl-` class output and every rule in `styles.css` are unchanged, because the extra |
| 104 | grammars emit the same TextMate scopes the mapping was already written against. |
| 105 | - **The release binary grew 0.59 MiB, from 13.65 MB to 14.27 MB** (`cargo build |
| 106 | --release`, measured on this branch with and without the dependency). That is the |
| 107 | embedded grammar dump; it is data, not code. Worth knowing because the artifact is |
| 108 | downloaded by `install.sh` over whatever connection an operator has, and because |
| 109 | 5b's release notes quote a size. |
| 110 | - **No new C dependency.** `syntect-fancy` resolves to `syntect/regex-fancy`, the same |
| 111 | pure-Rust engine already in use, so the musl/glibc reasoning recorded in `current.md` |
| 112 | is untouched. |
| 113 | |
| 114 | ## ui.md material |
| 115 | |
| 116 | - The blob's code cell is the only place colour carries information rather than |
| 117 | position. The palette is deliberately narrow — comments below muted-foreground, |
| 118 | strings and keywords carrying the contrast, punctuation just under foreground, |
| 119 | everything else close to it — so a file reads as text with structure, not as a parade. |
| 120 | It is one more `--syntax-*` block in `styles.css`, so it follows the palette. |
| 121 | - **The over-cap notice is a row of the blob panel, not a banner**: one line of muted |
| 122 | mono between the file header and the table, sharing the header's hairline. A page |
| 123 | state, not an error. |
| 124 | - Entry point: none. Highlighting has no control and no URL of its own — it is what the |
| 125 | blob view now does. The "Where the next features go" table needs no new row. |