steid

@jamesgill /

steid/plans/current.md
20.1 KBCode·Blame·Raw
ab7fea9chore: plans setup1mo
1# Current
2
3> Keep this file short. One active step, one ordered backlog. Completed work moves to
4> [progress.md]progress.md. If this file starts reading like a changelog, it has
5> drifted — that's exactly what went wrong last time.
6
3dd7293feat: ship Steid as an installable binary8d
7## Active: Milestone 5b — Deployable by anyone
a814db5feat: push and clone private repositories with a token8d
8
3dd7293feat: ship Steid as an installable binary8d
9**Goal:** a stranger can install Steid on a fresh Linux box in one command and reach it
10over HTTPS, and the author can publish an instance and push Steid's own source to it.
11Distributed as **a binary plus its assets**, not a container.
a814db5feat: push and clone private repositories with a token8d
12
3dd7293feat: ship Steid as an installable binary8d
13**Out of scope:** push-to-release and any CI/CD (8+ — see
14[ROADMAP.md]ROADMAP.md#milestone-ladder), multi-user registration (7), and hosting
15release artefacts on Steid itself, which it has no feature for.
16
17### Steps
18
19- [x] Release build producing `steid-<version>-<target>.tar.gz` — binary, `assets/`,
20 README — and verified to boot from a clean extraction
21- [x] `install.sh` — download, systemd unit, Caddy with automatic HTTPS, idempotent
22- [x] `README.md` — the repo has none, and it is the front door of a portfolio project
23- [x] Operability: `/healthz`, graceful shutdown on SIGTERM
24- [x] `STEID_SETUP_TOKEN` as an optional override, so claiming is not a race with
25 `journalctl`
26- [x] Rate limiting on `/auth/login` and `/auth/setup`
27- [x] Backup and restore, documented — two paths, rsync is enough
28
29### Done when
30
31`curl … | sh -s -- --domain git.example.com` on a fresh VPS yields a working HTTPS
32instance, and Steid's own source is pushed to it and browsable there.
33
34### Settled
35
36- **A binary plus `assets/`, not a container.** Topcoat can embed the asset *manifest*
37 (`Manifest::parse` + `include_str!`, for WASM) but not the asset *bytes* — that path
38 pairs with `AssetConfig::hosted_at`, which expects a CDN. A single self-contained file
39 would mean fighting the framework, so the artifact is a directory. The Dockerfile stays
40 as a secondary path.
41- **A TLS proxy is mandatory, not conventional.** Topcoat 0.5 has no TLS: no rustls, no
42 ACME, no HTTPS listener — checked, not assumed. Since git authenticates over HTTP
43 Basic, without TLS a personal access token crosses the network in cleartext on every
44 push. Caddy is chosen because automatic certificates are its headline feature and the
45 config is two lines.
46- **glibc, built on Debian bullseye — not static musl.** Two reasons, the second
47 decisive. musl failed on `ring` (Debian's `musl-gcc` wrapper rejects `-m64`) and would
48 need a real cross toolchain. But **musl's whole point is a binary with no runtime
49 dependencies, and Steid hard-requires `git` on `PATH`** — anyone installing it already
50 has a package manager, so the portability musl buys cannot be used. Building on
51 bullseye pins the glibc floor at 2.31, covering Debian 11+ and Ubuntu 20.04+; building
52 on bookworm would need 2.36 and silently exclude Ubuntu 22.04, which is still
53 everywhere. Verified: a release build in `rust:1.97-slim-bullseye` succeeds, 11.5 MB.
54- **The runtime binary links a TLS stack it never uses.** `ring``rustls``ureq`
55 Topcoat's `icon-iconify`/`tailwind` features, whose `ureq` exists to download the
56 Tailwind CLI *at build time*. Those features are enabled on the normal dependency as
57 well as the build one, so the crypto comes along for the ride. Moving them to
58 build-dependencies only would shrink the binary and drop an unused dependency from the
59 attack surface — worth trying, not yet attempted, and it is what made the musl attempt
60 fail where it did.
61- **Rsync is the deployment mechanism for now**, and that is enough while the artifact is
62 two paths. Push-to-release is the eventual answer and is parked at 8+.
8608574feat: personal access tokens, as a domain type8d
63
be4fac5docs: correct the Topcoat guidance in CLAUDE.md24d
64### Open
65
3dd7293feat: ship Steid as an installable binary8d
66- **Whether to add `STEID_TRUSTED_PROXY`.** The rate limiter keys on `X-Forwarded-For`
67 because **Topcoat 0.5 discards the peer address at accept time and never exposes it**
68 a handler sees headers and nothing else. Behind a proxy that is fine. Exposed directly,
69 a caller can vary the header for a fresh budget, leaving only the global cap. An
70 explicit "trust forwarded headers" flag defaulting to off would close it, at the cost
71 of one more thing an operator must get right. Not picked.
4ef3945feat: repository settings, README rendering, branch switcher, raw files7d
72- **`MAX_RAW_BYTES` is 10 MiB**, chosen rather than derived. It bounds what one raw
73 request can hold in memory, because `GitQuery` reads bytes rather than streaming them.
74 Raising it means a handful of concurrent requests can hold that much each.
75- **Renaming a repository** is still impossible, and now needs its own use case plus an
76 answer for moving the directory under every existing clone.
ef23868feat: rebuild the profile page on flat navigation7d
77- **The profile has no links.** `Organization` carries a display name and a bio and
78 nothing else, so the design's links row is not backed by data and was left out rather
79 than faked. A `links` field plus a settings control is the small feature that fixes it.
80- **`create_repo` and `serve_git` read the clock internally** rather than taking a `now`,
81 which is what `issue_token` does. Two call-site edits, or a `Clock` port if a third
82 case appears.
83- **A push touches `updated_at` twice** — once for the advertisement, once for the RPC.
84 Harmless, same second, but two writes per push.
3dd7293feat: ship Steid as an installable binary8d
85- **A licence.** A public portfolio repository probably wants one, and the README
86 deliberately says nothing about licensing rather than guessing.
0f19654feat: security headers on every response1d
87- ~~Blocked on a domain transfer~~**transferred 2026-08-29**, so Phases 1–6 of the
88 deployment runbook are unblocked.
7da29e0fix: the domain is jpgill.dev, not jpgilldev.com1d
89- **(historic) Blocked on a domain transfer** (noted 2026-08-29). `jpgill.dev` is the
4ef3945feat: repository settings, README rendering, branch switcher, raw files7d
90 intended host for both the instance and the release downloads; the transfer is in
91 flight. Phases 1–6 of the deployment runbook cannot start until DNS resolves, because
92 Caddy requests a certificate on startup. Nothing else is blocked by it — the artifact
93 is built and verified, and `install.sh` still needs its container dry-run.
3dd7293feat: ship Steid as an installable binary8d
94- **A domain.** Caddy needs a real hostname to obtain a certificate. This is the one
95 blocker that is DNS rather than code.
be4fac5docs: correct the Topcoat guidance in CLAUDE.md24d
96
4414bbadocs: record the first real deployment22h
97### Open
98
99- **Whether an absent *public* repository can 404 while private ones still 401.** The
100 uniform 401 from [0007]decisions/0007-tokens-over-http-basic.md is what makes a
101 private repository indistinguishable from one that does not exist — but it also turns
102 every mistyped URL into "Authentication failed", which cost real time on the first
103 deployment. The leak being prevented concerns *private* names only, so answering 404 for
104 a repository that would be public if it existed may cost nothing. Needs thought about
105 whether that is actually true.
607da17docs: licence Steid under AGPL-3.022h
106- **Zero-downtime deploys — deferred, deliberately.** `deploy.sh` currently stops the
107 service, swaps the binary and starts it: about two seconds. The plan, when it is worth
108 building: a systemd **template unit** `steid@.service` where the instance name *is* the
109 port, with `/opt/steid/3000` and `/opt/steid/3001` as separate install directories.
110 Deploy alternates — install to the idle port, start it, health-check it **directly on
111 loopback**, rewrite Caddy's upstream and `caddy reload` (graceful; existing connections
112 finish), then stop and disable the old one and enable the new so a reboot brings back
113 the right one.
114
115 Two things make this more than a shell script, and they are the reason it is deferred
116 rather than half-built:
117
118 - **Both processes share one SQLite database during the overlap.** WAL allows that, and
119 Steid's writes are short, so the overlap itself is fine. **Migrations are not.** The
120 new binary migrates on boot while the old one is still serving, so any schema change
121 that the old binary cannot tolerate — a new `NOT NULL` column, a dropped column —
122 breaks the still-live version. That is a constraint on how migrations are written
123 (**expand, deploy, contract**), not something the deploy script can solve.
124 - **An in-flight `git push` is cut when the old process stops.** `topcoat::start`
125 already drains on SIGTERM, so the fix is to stop the old one *after* Caddy has
126 repointed and let it finish what it has.
127
128 Worth building when instances belong to other people. At two seconds on a personal site,
129 it buys nothing today.
4414bbadocs: record the first real deployment22h
130- **No scheduled backups.** `/var/lib/steid` is the whole of the state and copies of it
131 exist only because they were taken by hand. This is the largest gap now that the
132 instance is live.
133- **Releases are not served from the instance yet.** The Caddy blocks are written and
134 commented in `deploy/Caddyfile`, and `install.sh`'s `RELEASE_BASE_URL` points at
135 `/jamesgill/repos/steid/releases`, but nothing has been rsynced there — so the
136 one-command install a stranger would run does not work yet.
137
6f0a608docs: fold the highlighting handover into plans17h
138### Opened by the section 1 wave
139
140- **Highlighting caches nothing.** The same file is re-highlighted on every view at
141 ~87 ms per thousand lines in release. Cache per blob object id when it shows.
142- **Diffs, READMEs and markdown code fences are unhighlighted.** The README goes through
143 `web/markdown.rs`, which writes its own `<pre><code>`; wiring `highlight.rs` into it is
144 a small follow-up.
145- **`.jsx` is plain**: two-face's JavaScript grammar claims only `js`/`htc`; `tsx` is
146 covered by TypeScriptReact. An alias is one line.
147- **`\r\n` files keep the `\r` inside highlighted markup.** Invisible under
148 `white-space: pre`; noted rather than fixed.
149- **The blob header does not name the detected language.** It is only visible as colour.
150
9b1feb6docs: fold the branches and tags handover into plans17h
151- **No ahead/behind counts on the branches page**, by decision: a `rev-list` per branch
152 is twenty forks for twenty branches. Waits for a kept-alive git.
153- **Neither refs page is paginated.** A thousand branches render a thousand rows.
154- **A repository with tags but no branches** says "No branches"; real but exotic, not
155 fixture-tested.
156- **`web::browse::{encode, timestamp}` are now `pub(super)`**, and `repo_stats` takes
157 `handle` and `name` so the sidebar counts can link. Watch for the next page that wants
158 `timed_out` and move it to `web::context` then.
159
e0856ebfeat: clone a public repository over HTTP8d
160### Carried over — small, unblocked
d7b99d9docs: record milestone 0 progress and routing findings1mo
161
47db238feat: serve the git protocol through http-backend, behind a port8d
162- **A client that disappears mid-request leaves the body-copy task waiting.** The copy
e0856ebfeat: clone a public repository over HTTP8d
163 into git's stdin runs in its own task and nothing cancels it if the connection drops.
47db238feat: serve the git protocol through http-backend, behind a port8d
164 Bounded by the backend exiting and closing the pipe, but not by anything deliberate.
a814db5feat: push and clone private repositories with a token8d
165- **`REMOTE_USER` is not set on the backend**, so a push is recorded in the repository's
166 reflog without naming who made it. Steid knows the actor by then; it simply is not
167 passed through. Small, and worth doing before anything reads reflogs.
0f19654feat: security headers on every response1d
168- **No automated test asserts the security headers.** There is no HTTP-level test
169 harness, so losing the CSP would be silent. The likeliest regression is someone using
170 `insert()` instead of `or_insert()` and flattening the raw endpoint's stricter policy.
171- **`cargo audit` has never been run**, and the dependency tree has not been reviewed.
a814db5feat: push and clone private repositories with a token8d
172- **No rate limiting on token authentication.** A token is 256 bits so guessing is not
173 the worry; unbounded hashing on an open endpoint is.
174- **Tokens have no expiry and no last-used timestamp.** Both deliberate omissions for
175 now — see [0007]decisions/0007-tokens-over-http-basic.md — but a token list with no
176 "last used" makes it hard to know which are safe to revoke.
e0856ebfeat: clone a public repository over HTTP8d
177- **A subprocess per git request.** Unlike Milestone 3's once-per-creation, this is on a
178 hot path and has not been measured. Milestone 5 is where that bill comes due.
179- **Streaming is by construction, not by measurement.** The response body is never
180 collected, but no clone large enough to prove it has been run.
6a2a925docs: close Milestone 3, plan Milestone 4a8d
181- **An orphaned repo directory is possible** if the process dies between the record
182 write and the filesystem write, and it then blocks re-creating that name. The durable
183 fix is a reconciliation sweep on boot
184 ([architecture.md]architecture.md#db-plus-filesystem-writes); clearing one is a
185 manual `rm` today, since repo deletion does not exist.
186- **The duplicate-name check races.** The loser is caught by `init_bare` or the unique
187 constraint, but surfaces as an opaque storage error rather than "name taken".
188- **Bare repos created on macOS carry `ignorecase = true`.** A migration gotcha if the
189 data directory ever moves to Linux.
0aca94efeat: a repository is one place, and its landing page says what it is17h
190- ~~**Light mode is still untested**~~**looked at 2026-09-05** on the repository
191 landing page, a tree and a blob, by temporarily flipping the layout's `class="dark"`
192 and screenshotting. Nothing was wrong: every colour on these pages already comes from
193 a token. The profile, the settings pages and the log have still not been checked, and
194 **there is still no way for a visitor to choose** — the layout hardcodes `dark`.
dce0bf3feat: browse a repository's files and history8d
195- **Submodule rendering was never seen**, only compiled: no fixture contained one.
4ef3945feat: repository settings, README rendering, branch switcher, raw files7d
196- **The `/log` page's switcher opens with nothing marked current** when no revision is
197 in the URL, because `repo_log` still does not report the revision it resolved. Now more
198 visible than before, since there is a switcher to look wrong.
199- **Task-list items keep their bullet** and footnotes render in place rather than
200 collected at the end. Both cosmetic.
0aca94efeat: a repository is one place, and its landing page says what it is17h
201- **The landing page now makes 15 `git` processes** for a repository with a README and
202 a licence (10 without, 1 for an empty one), up from 7. Five are the About sidebar's
203 and run concurrently, so the *latency* is roughly one call — but it is fifteen forks
204 per view, and this is now the most expensive page in Steid. The fix is 0006's
205 kept-alive `cat-file --batch`, not trimming the sidebar.
206- **`count_commits` costs two processes, not one.** It resolves the revision before
207 running `rev-list --count`, because `rev-list` is fatal on an empty repository and on
208 an unknown branch, and this module's rule is that a non-zero exit from git is always a
209 real fault. `--ignore-missing` was tried: it covers a bad object id, not a bad
210 revision *name*.
211- **`bg-muted` is not a token and never was.** Three `<pre>` blocks used it, so they
212 have been rendering with no background at all — silently, exactly as `ui.md` warns
213 about classes Tailwind never ships. Fixed on the clone block and the empty-repository
214 push snippet; **`token.rs`'s new-token block still has it** and was left alone as
215 outside this step.
216- **The licence sniffer reads 1 KiB, not the 200 bytes first sketched.** BSD-2 and
217 BSD-3 differ only by a third clause about 900 bytes in. `LICENSE-MIT` and other
218 suffixed spellings are not detected: the filename list is deliberately short, because
219 every extra candidate is another speculative blob read.
220- **Nothing links to a commit by its sha yet.** The latest-commit bar's sha points at
221 the log, which is the only page that shows the commit in context. It becomes a link
222 to the commit page the moment there is one.
dce0bf3feat: browse a repository's files and history8d
223- **A per-file last-commit column is still absent**, deliberately — see
224 [0006]decisions/0006-git-binary-behind-narrow-ports.md#amendment--20260829-the-milestone-5-read-path.
225 Wanting it is the trigger to move to a kept-alive `cat-file --batch`, not to reopen
226 `gix`.
2eb8681docs: put git next, plan the repo model24d
227- **Fonts are not loaded.** The theme names Geist and IBM Plex Mono; both fall back
228 today. Topcoat's `font-fontsource` feature handles it.
5d3dfa5feat: a global top bar, and pages choose their own width22h
229- **The auth pages are unstyled.** `/auth/setup` and `/auth/login` are still bare
230 milestone-1 HTML, and the styled top bar now sits right above them making it obvious.
231- **The top bar's wordmark says "steid"**, not the owner's identity — see the open
232 question at the end of [ui.md]ui.md#the-shell.
2eb8681docs: put git next, plan the repo model24d
233- **Light mode is untested.** The palette defines it; nobody has looked at it.
f0444b7docs: plan milestone 2 in two phases1mo
234- **No rate limiting** on `/auth/login` or `/auth/setup`.
076dbc9docs: close milestone 1, open milestone 21mo
235- **`sweep_expired` is never called**, so expired session rows accumulate. Expiry is
236 enforced on read, so this is tidiness, not a hole.
2eb8681docs: put git next, plan the repo model24d
237- **CSRF.** `SameSite=Lax` covers the common case. Forms now exist, so this is decidable
238 rather than hypothetical.
ab7fea9chore: plans setup1mo
239
240## Backlog
241
242Ordered. Pull from the top.
243
3dd7293feat: ship Steid as an installable binary8d
2441. **Milestone 6 — Writing.** Posts, markdown, `/{handle}/posts/{slug}`. Open when it
245 starts: which markdown crate, and whether raw HTML in markdown is trusted — safe for a
246 single author, a stored-XSS hole the moment Milestone 7 adds a second user. A
247 repository's README rendering on its page falls out of the same pipeline.
ab7fea9chore: plans setup1mo
248
31a6a7edocs: record the forge feature survey and what can run in parallel18h
249### Forge feature candidates (surveyed 2026-09-05)
250
251Not yet ordered against Milestone 6. Grouped by what they touch, because the plan is to
252run several agents at once and the grouping is what decides what can run together.
253**Section 1 is being tackled first.** Shared hotspots: `GitQuery` in `port.rs` (every
254browsing feature appends a method), `git_query.rs`, the repo sub-nav, and `plans/`
255itself. New tables touch `sqlite.rs` + `in_memory.rs` and should merge one at a time.
256
0aca94efeat: a repository is one place, and its landing page says what it is17h
257**Done from section 1:** the **two-column repository landing page** — a shared header
258and tab strip on every repository page, an About sidebar, a latest-commit bar, and
259branch/tag counts. See [ui.md]ui.md#the-repository-page for the layout and the
260entry-point map, and [progress.md]progress.md for what it cost.
261
31a6a7edocs: record the forge feature survey and what can run in parallel18h
2621. **Read-only browsing** (no DB; parallel-safe with each other): commit page with
263 diff · branches and tags pages · compare view (merge-base) · archive download ·
264 syntax highlighting · blame · per-file last-commit column (blocked on the
265 [0006]decisions/0006-git-binary-behind-narrow-ports.md amendment: it is the trigger
266 for a kept-alive `cat-file --batch`) · code search via `git grep`.
2672. **Repo model** (a column or use case each; merge serially): rename · default branch
268 setting · archived flag · topics and pinned repos on the profile · orphan-directory
269 reconciliation sweep on boot.
2703. **Collaboration** (8+): issues · pull requests (needs the commit and compare pages,
271 plus a merge on the bare repo) · labels and milestones.
2724. **Transport and git ops**: import from URL (mirror clone) · post-receive events and
273 webhooks (first brick of CI) · `REMOTE_USER` passthrough · LFS (not yet) · SSH
274 (would reverse [0001]decisions/0001-git-over-http-not-ssh.md).
2755. **Identity and auth**: token expiry and last-used · rate limiting on token auth ·
276 CSRF (touches every form — run alone) · Milestone 7 multi-user (run alone).
2776. **Portfolio**: posts (Milestone 6) · profile links · Atom feeds for posts and commits.
2787. **Platform and quality**: scheduled backups and served releases (closes 5b) ·
279 HTTP-level test harness plus security-header tests and `cargo audit` · `/api`
280 coverage for existing use cases · styled auth pages, light mode, fonts.
281
282Process decisions still open for parallel work: whether agents get worktrees and
283short-lived branches (bending the commit-to-main rule), and that agents do **not** edit
284`plans/` — it is updated once per wave at merge time.
285
ab7fea9chore: plans setup1mo
286## Open questions
287
bd48b4bdocs: serve git over smart HTTP, reorder roadmap portfolio-first1mo
288- **Topcoat is early** (v0.5.0, first released 2026-07-22, breaking changes expected
289 by its own authors). Expect churn that isn't feature work.
290- Topcoat ships Tailwind without Node, which reopens the design system attempt #1
291 dropped purely to avoid an npm build step — see [ui.md]ui.md.
292
293## Routing findings (Milestone 0)
294
295- **Topcoat 0.5 requires rustc ≥ 1.95.** On an older toolchain `cargo add topcoat`
296 silently resolves to an empty `topcoat v0.0.0` placeholder instead of failing. Local
297 stable is now 1.97.1.
298- `Router::builder().discover()` collects `#[page]`-annotated items **at link time**,
299 so pages can live in any module. Layering is our choice, not the framework's.
300- `module_router!` derives each URL from the module tree rather than a path string.
aaefaabfeat: root handles, grouped routes, reserved-handle denylist1mo
301 Still deferred. Application routes now group cleanly (`auth/login`, `api/me`), but
302 handles sit at the root ([0004]decisions/0004-root-handles-grouped-routes.md), so a
303 parameterised root segment still has to coexist with static ones. Worth checking how
304 `module_router!` handles that before committing to it.
bd48b4bdocs: serve git over smart HTTP, reorder roadmap portfolio-first1mo
305- Path and query params are read from `Cx` via `path_param!` / `#[query_params]`, not
306 injected as handler arguments. Parses are memoized per request.
307- Layouts wrap by path prefix and nest outermost-first, and a layout can catch a page's
308 `NotFoundError` to render a branded 404.
309- `HOST` / `PORT` configure the bind address, so `STEID_LISTEN_ADDR` is gone.
310- `Body` is a boxed `http_body::Body` used for both requests and responses, with
311 `into_data_stream()` to read and `Body::new()` to wrap a stream — pack data can
ca76e1bdocs: fix milestone cross-references after the reorder24d
312 stream both directions without buffering. This is what makes Milestone 4 viable.