| | @@ -0,0 +1,105 @@ |
| 1 | +# Handover — `highlight` (syntax highlighting in the blob view) |
| 2 | + |
| 3 | +Branch `feat/highlighting`. One commit of code, plus this file. |
| 4 | + |
| 5 | +## What shipped |
| 6 | + |
| 7 | +`/{handle}/repos/{name}/tree/{rev}/-/{path}` renders a text file with syntax |
| 8 | +highlighting. Nothing else changed: no new route, no new `GitQuery` method, **no extra |
| 9 | +`git` subprocess** — the bytes were already being read for the plain-text view. |
| 10 | + |
| 11 | +- `src/infrastructure/highlight.rs` — the whole adapter. `source_lines(file_name, text)` |
| 12 | + returns `Source { lines: Vec<SourceLine>, too_large: bool }`, one entry per line of |
| 13 | + `text.lines()`, each either `Plain(String)` or `Classed(String)` of HTML. |
| 14 | +- `src/infrastructure/web/browse.rs` — only the `source` component: it takes a |
| 15 | + `file_name` as well as the text, and the code cell renders `Classed` through |
| 16 | + `Unescaped`. The row, the line-number cell and the scroll container are untouched, so |
| 17 | + the blob-header toggle and the `id="L<n>"` anchors being added by sibling agents do |
| 18 | + not collide with this. |
| 19 | +- `styles.css` — a `--syntax-*` family on `:root` and `.dark`, and the mapping onto the |
| 20 | + `hl-` classes at the bottom of the file. |
| 21 | +- `syntect 5.3` with `default-features = false, features = ["default-fancy"]`: the pure |
| 22 | + Rust regex engine, so the Debian release build gains no C dependency. |
| 23 | + |
| 24 | +## progress.md material — decisions worth not rediscovering |
| 25 | + |
| 26 | +- **Classes, not inline styles.** `syntect` will happily emit `style="color:#..."`. |
| 27 | + That would hardcode one palette into generated markup and follow neither a palette |
| 28 | + change nor the colour scheme, which is the one rule `styles.css` states outright. So |
| 29 | + the output is `ClassStyle::SpacedPrefixed { prefix: "hl-" }` and the colours live in |
| 30 | + `styles.css` with every other token. |
| 31 | +- **`ClassedHTMLGenerator` cannot be used here.** It produces a single string whose |
| 32 | + `<span>`s cross line boundaries — its own docs say the output must go in one `<pre>` — |
| 33 | + and the blob view is a table with one row per line, so those spans cannot be split |
| 34 | + across cells. `highlight.rs` therefore drives `ParseState` + `ScopeStack` + |
| 35 | + `line_tokens_to_classed_spans` itself: the parse state carries across lines (a block |
| 36 | + comment opens on one and closes on another) but the *markup* does not — whatever is |
| 37 | + open at the end of a line is closed there and reopened at the start of the next. That |
| 38 | + is the whole reason the module is 150 lines rather than 20. |
| 39 | +- **The line ending is stripped by character, not trimmed.** The `_newlines` syntaxes |
| 40 | + need the `\n` handed to the parser, but it comes back *escaped inside the output* and |
| 41 | + can land before or after a closing `</span>` depending on where a scope pops. Left in, |
| 42 | + `white-space: pre` renders a second, empty line inside every row and the whole file |
| 43 | + reads double-spaced. It is dropped with a `filter`, not a `trim_end`. |
| 44 | +- **`hl-` classes are scope *atoms*, and atoms are not positional.** `keyword.control` |
| 45 | + becomes `class="hl-keyword hl-control"`, and `meta.function` — which wraps a whole |
| 46 | + function *body* — becomes `class="hl-meta hl-function"`. A bare `.hl-function` rule |
| 47 | + would therefore colour every function body. Single-atom selectors are only safe for an |
| 48 | + atom that leads its scope; the rest are spelled as compounds |
| 49 | + (`.hl-entity.hl-name.hl-function`). **Rule order is load-bearing** for the same |
| 50 | + reason: `punctuation.definition.string` carries both `hl-punctuation` and `hl-string`, |
| 51 | + equal specificity, so `.hl-string` is written later and a quote reads as its string. |
| 52 | + The comment rule is last so a comment is a comment whatever it contains. |
| 53 | +- **Plain text is not "highlighted with a plain-text syntax".** When the resolved syntax |
| 54 | + is syntect's `Plain Text`, `source_lines` returns `Plain` lines, so a `LICENSE` is not |
| 55 | + wrapped in spans that colour nothing. |
| 56 | +- **The caps are 512 KiB and 10,000 lines**, and they are two caps because the cost is |
| 57 | + both per byte (one enormous minified line) and per line. The page states it — |
| 58 | + otherwise a large Rust file silently looks like an unsupported language. Neither |
| 59 | + number was derived from a measurement; see below. |
| 60 | +- **The syntax set is loaded once into a `OnceLock`** on the first blob viewed. It is a |
| 61 | + few MB of resident memory for the life of the process. |
| 62 | + |
| 63 | +## current.md material — holes opened and shortcuts taken |
| 64 | + |
| 65 | +- **The default syntax set is missing four languages this project itself uses.** Checked |
| 66 | + by test (`which_common_languages_the_default_set_knows`, prints its findings): |
| 67 | + |
| 68 | + | Highlighted | **Plain** | |
| 69 | + |---|---| |
| 70 | + | Rust, shell, Markdown, Python, JavaScript, Go, YAML, JSON, CSS, SQL, Makefile, C | **TOML, Dockerfile, TypeScript, JSX/TSX** | |
| 71 | + |
| 72 | + `Cargo.toml` and `Dockerfile` render unhighlighted in this repository today — the |
| 73 | + `highlight-toml-dark.png` screenshot is exactly that. `two-face` bundles the missing |
| 74 | + grammars and is the fix, **deliberately not added here**: it is another dependency and |
| 75 | + a decision for the orchestrator. As briefed, it was not attempted. |
| 76 | +- **Highlighting costs ~87 ms per 1,255-line file in a release build** — measured on |
| 77 | + `browse.rs` with `cargo test --release`, and it is paid on every view because nothing |
| 78 | + caches. In the `topcoat dev` (debug) build the same page takes ~0.7 s to serve versus |
| 79 | + ~0.06 s for a file past the cap, so debug is roughly eight times slower and is not the |
| 80 | + number to plan against. 87 ms is still the most expensive thing on a blob page: it is |
| 81 | + a Rust regex engine over every line, not a subprocess. If it needs to go, the answers |
| 82 | + in order are cache per blob object id (the id is already in `FileView`), then lower the |
| 83 | + caps. |
| 84 | +- **Nothing caches.** The same file is re-highlighted on every view. |
| 85 | +- **Diffs, READMEs and markdown code fences are unhighlighted**, as scoped. A README |
| 86 | + rendered on the repository page goes through `web/markdown.rs`, which writes its own |
| 87 | + `<pre><code>`; wiring this module into it is a small, obvious follow-up and was left |
| 88 | + alone to keep the branches apart. |
| 89 | +- **`\r\n` files keep the `\r` inside the highlighted markup** (it is stripped from the |
| 90 | + plain path by `str::lines`). Invisible in `white-space: pre`; noted rather than fixed. |
| 91 | +- **The blob view's own header has no "this is Rust" indicator.** The language is |
| 92 | + detected and then only ever visible as colour. |
| 93 | + |
| 94 | +## ui.md material |
| 95 | + |
| 96 | +- The blob's code cell is the only place colour carries information rather than |
| 97 | + position. The palette is deliberately narrow — comments below muted-foreground, |
| 98 | + strings and keywords carrying the contrast, punctuation just under foreground, |
| 99 | + everything else close to it — so a file reads as text with structure, not as a parade. |
| 100 | + It is one more `--syntax-*` block in `styles.css`, so it follows the palette. |
| 101 | +- **The over-cap notice is a row of the blob panel, not a banner**: one line of muted |
| 102 | + mono between the file header and the table, sharing the header's hairline. A page |
| 103 | + state, not an error. |
| 104 | +- Entry point: none. Highlighting has no control and no URL of its own — it is what the |
| 105 | + blob view now does. The "Where the next features go" table needs no new row. |