steid

docs: close Milestone 3, plan Milestone 4a

Milestone 3 is done: repositories exist as records and bare repos on disk,
appear on the profile, and are readable over /api with visibility enforced in
one place.

Milestone 4 is split before starting it, acting on the ROADMAP's own warning
about protocol work sprawling rather than waiting to be surprised. 4a is an
anonymous clone of a public repository with no authentication code at all — a
working `git clone` in one small milestone. 4b adds tokens, push and private
clone on top, and if it sprawls, 4a has already banked the capability.

Two things were measured before being written down. A real client gzips its
POST body once a repository has a nontrivial number of refs (201 refs, 5KB
body), on protocol v0 and v2 alike — which is what decides `git http-backend`
over spawning `upload-pack --stateless-rpc` directly, since the direct route
passes against a one-ref test repo and fails on the first real one. And
`--advertise-refs` emits no service header, so the direct route would own that
framing too.

0006 gets an amendment rather than a rewrite: it sketched the protocol ports as
`upload_pack`/`receive_pack`, mirroring attempt #2's SSH shape, and http-backend
is a CGI. What survives is the part that mattered — narrow ports, one invoker,
authorization decided before the subprocess exists.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JZwc7URWKVhkAuRTWiDmjA

JamesPatrickGill authored 8 days agoparent8622a05Browse files6a2a925b4612724df18c963e4890e11071df181f

4 files changed+127 −117

plans/ROADMAP.md+10 −2View file

		@@ -54,8 +54,9 @@ a baseline.
54	54	\| 0 \| Skeleton — Topcoat app boots, config, one page, SQLite wired \| done \|
55	55	\| 1 \| Identity, thin — claim on first run, login, session \| done \|
56	56	\| 2 \| Profile page — `/{handle}` as the real profile \| done \|
57		−\| 3 \| Repo model — records + bare repos on disk \| active \|
58		−\| 4 \| Git over HTTP — `git http-backend`, PATs for auth \| not started \|
	57	+\| 3 \| Repo model — records + bare repos on disk \| done \|
	58	+\| 4a \| Clone over HTTP — `git http-backend`, public repos, no auth \| active \|
	59	+\| 4b \| Push and tokens — PATs over HTTP Basic, push, private clone \| not started \|
59	60	\| 5 \| Repo browsing — tree, blob, commit log \| not started \|
60	61	\| 6 \| Writing — posts, markdown \| not started \|
61	62	\| 7 \| Identity, full — multi-user, orgs, invites, registration policy \| not started \|
		@@ -92,6 +93,13 @@ previous attempts. Milestone 3 is deliberately small and ships something visible
92	93	profile; if milestone 4 starts to sprawl, that is the signal to bank it and go back to
93	94	writing.
94	95
	96	+Milestone 4 was split before it started, acting on that warning rather than waiting
	97	+to be surprised by it. It bundled two subsystems — personal access tokens and the smart
	98	+HTTP protocol — and neither shipped anything until both were done. Split, 4a is
	99	+anonymous clone of a public repository with no authentication code at all, which is a
	100	+working `git clone` in one small milestone; 4b adds tokens, push, and private clone on
	101	+top. If 4b sprawls, 4a has already banked the capability.
	102	+
95	103	One numbering scheme. The previous attempt kept two (a roadmap ladder and a
96	104	separate build log with conflicting numbers) and they drifted apart within three
97	105	milestones. If the plan changes, edit this table — don't start a second list.

plans/current.md+74 −112View file

		@@ -4,132 +4,96 @@
4	4	> [progress.md](progress.md). If this file starts reading like a changelog, it has
5	5	> drifted — that's exactly what went wrong last time.
6	6
7		−## Active: Milestone 3 — Repo model
	7	+## Active: Milestone 4a — Clone over HTTP
8	8
9		−Goal: repositories exist as records and as bare git repos on disk, and they appear
10		−on the profile. No git protocol yet — that is milestone 4. This milestone fills the
11		−Repositories section and gets the storage layout right before anything serves it.
	9	+Goal: `git clone https://host/{handle}/repos/{name}.git` works against a public
	10	+repository, for anyone, with no credentials. The protocol is delegated to `git
	11	+http-backend` per [0001](decisions/0001-git-over-http-not-ssh.md).
12	12
13		−Out of scope: clone, push, browsing a tree, README rendering, forks, stars.
14		−Deleting a repo — worth having, but it makes the filesystem/database consistency
15		−problem twice as interesting, so not in the first pass.
	13	+Out of scope: personal access tokens, HTTP Basic, push, cloning a private repo —
	14	+all of that is 4b. Also out: browsing a tree in the UI (Milestone 5), and any repo
	15	+statistic the clone path could tempt us into computing.
16	16
17	17	### Steps
18	18
19		−- [x] Domain: `RepoId`, `RepoName`, `Visibility` (Public/Private), `Repository`
20		−- [x] Domain: `RepoRepository` port — `find_by_id`, `find_by_org_and_name`,
21		− `list_by_org`, `save`
22		−- [x] Infrastructure: in-memory + SQLite implementations, migration
23		−- [x] Application: `GitStorage` port — `init_bare`, `remove`, `repo_path`
24		−- [x] Infrastructure: `DiskGitStorage`, shelling out to `git init --bare`
25		−- [x] Application: `create_repo` use case — owner only, validates, creates record and
26		− bare repo
27		−- [x] Application: `view_repo` read model — visibility-aware
28		−- [x] Web: `/{handle}/repos/new` form, `/{handle}/repos/{name}` page
29		−- [x] Application: `list_repos` + the profile's Repositories section listing what the
30		− viewer may see
31		−- [x] `/api/users/{handle}/repos`
	19	+- [ ] Probe `git http-backend`'s actual contract — which CGI variables it reads, how it
	20	+ reports failure, what it does with an unauthorised path. Findings to
	21	+ `progress.md`; no application code in this step.
	22	+- [ ] Application: `GitProtocolServer` port — a CGI-shaped request/response pair, plus
	23	+ `GitOperation` (Read/Write) as the thing authorization is decided on
	24	+- [ ] Infrastructure: `GitHttpBackend` adapter, spawning through the existing `run_git`
	25	+ invoker; streams stdin in and stdout out, parsing CGI headers off the front
	26	+- [ ] Application: `serve_git` use case — resolves the repository, enforces visibility,
	27	+ refuses writes outright, and only then delegates
	28	+- [ ] Web: the three git routes under `/{handle}/repos/{name}.git/`, with `body_limit`
	29	+ raised
	30	+- [ ] Verify with a real `git clone` of a repo with enough refs to trigger a gzipped
	31	+ request body
32	32
33	33	### Done when
34	34
35		−The owner creates a repo through the UI, a bare repo appears at
36		−`{data_dir}/{handle}/{name}.git`, and it is listed on the profile. A private repo is
37		−invisible to a signed-out visitor. `git clone` does not work yet — that is
38		−milestone 4.
	35	+`git clone http://127.0.0.1:3000/{handle}/repos/{name}.git` produces a working
	36	+checkout of a public repository, with no credentials, and the cloned history matches
	37	+the origin. A private repository is not clonable by anyone yet — not even its owner.
	38	+`git push` is refused.
39	39
40	40	### Settled
41	41
42		−- `list_repos` returns `Option<Vec<_>>`. `None` is an unknown handle,
43		− `Some(vec![])` a handle whose repositories the viewer cannot see. `/api` needs that
44		− difference to answer 404 rather than `[]` for a user who does not exist.
45		−- `RepoSummary` is separate from `RepoView`. The owning handle and
46		− `viewer_is_owner` are constant across a listing and already known to the page, so a
47		− listing type carries neither.
48		−- `list_repos` was split from `view_repo` and moved to the listing step. `view_repo`
49		− had a consumer immediately; `list_repos` would have been a third read model with no
50		− caller, which is what the previous three steps already were.
51		−- An invisible private repo is `None`, not `Forbidden`. `view_repo` answers the same
52		− way for "does not exist" and "not allowed to see", and the page 404s identically. A
53		− 403 would confirm the repository exists and leak its name.
54		−- Private repos are visible to any member, not only the owner — seeing is weaker
55		− than changing, matching attempt #2's clone rule.
56		−
57		−- Repo name rules: `OrgName`'s, plus `.` and `_` for names like `.github` and
58		− `foo.js`. Lowercased, max 100. Also rejects a name of nothing but dots and any name
59		− ending `.git` — the first is traversal, the second would live at `foo.git.git`.
60		−- Reserved repo names: `import`, `new`, `search`. Only names directly under
61		− `/{handle}/repos/` can collide.
62		−- Visibility defaults to public, matching a portfolio-first product.
63		−- Repositories carry an optional description, capped at 300 characters — a sentence
64		− for the profile listing, not a README. Kept deliberately: portfolio-first is the
65		− tie-break, and this milestone's own "Done when" puts repositories on the profile, so
66		− the consumer is inside the milestone rather than hypothetical. The per-repo analogue
67		− of `Organization::bio`.
68		−- `/api/users/{handle}/repos` answers with a bare array, not an
69		− `{"repositories": […]}` envelope, matching `/api/users/{handle}` returning a bare
70		− object. The cost is knowingly taken: adding pagination later means a wrapper and a
71		− breaking change. Personal-first means a handful of repositories, and the API has no
72		− consumers to break yet.
73		−- `/api` resolves the handle without loading a profile. `handle_param` was split
74		− out of `profile_for` so the listing route can 404 on `list_repos`' own `None` rather
75		− than on a profile lookup it does not otherwise need — which is what that `Option` was
76		− given a meaning for.
77		−
78		−- `list_by_org` returns every repository regardless of visibility. Filtering is an
79		− authorization decision and belongs to the use case, so the page and `/api` cannot end
80		− up applying different rules. The cost is that a private repo is briefly in memory
81		− before being filtered, which is fine in-process.
	42	+- `git http-backend`, not direct `--stateless-rpc`. Both put identical bytes on the
	43	+ wire for a modern clone of a small repo; the difference is entirely in the tail, which
	44	+ is what wide adoption means. Measured before choosing: a client cloning a repo with
	45	+ 201 refs gzip-compresses the POST body (5KB here), on protocol v0 and v2. A
	46	+ direct implementation must therefore inflate request bodies and forward
	47	+ `Git-Protocol` itself, and gets neither the dumb-protocol fallback nor the header set.
	48	+ The failure mode decided it — a direct implementation passes against a one-ref test
	49	+ repo and breaks on the first real one.
	50	+- The clone URL is `/{handle}/repos/{name}.git`, matching the page at
	51	+ `/{handle}/repos/{name}`. Scoped rather than root-level, per
	52	+ [0003](decisions/0003-scoped-urls.md); the `.git` suffix separates protocol from page.
	53	+- Only the three known endpoints are routed — `info/refs`, `git-upload-pack`,
	54	+ `git-receive-pack`. `http-backend` will otherwise serve dumb-protocol object files
	55	+ under any path handed to it, which would be a read of a repository nothing
	56	+ authorized. The router is the allowlist.
	57	+- Authorization is decided before the subprocess is spawned, from the service name
	58	+ in the request, not from anything `http-backend` reports back. By the time git is
	59	+ running it is too late to refuse.
	60	+- Milestone 4 was split. See [ROADMAP.md](ROADMAP.md#why-this-order).
82	61
83	62	### Open
84	63
85		−Nothing open. `GitStorage`'s shape and how git is invoked are recorded in
86		−[0006](decisions/0006-git-binary-behind-narrow-ports.md).
	64	+- What a private repository answers to an anonymous clone. 4a has no credentials at
	65	+ all, so 404 is the only honest answer and matches `view_repo`'s "absent, not
	66	+ forbidden" rule. But git only sends credentials after a 401, so 4b will need a 401
	67	+ with `WWW-Authenticate` on exactly the case that 404s today — which leaks that the
	68	+ repository exists. Gitea and GitHub both accept that leak. Decide it in 4b, with the
	69	+ tension recorded rather than rediscovered.
87	70
88	71	### Watch for
89	72
90		−- `redirect()` is a 307 and re-POSTs. Post/redirect/get needs a 303. `see_other()`
91		− is the right status but is a response type, and `#[page]` must return a view so the
92		− layout can wrap a failure re-render. The way to get both from one handler is a
93		− `StatusCode` and a `Location` pair inside `view!` — wrapped as
94		− `context::location`. This shipped broken in Milestone 2's settings form and was
95		− caught by browser verification, not by any test.
96		−
97		−- An orphaned directory is indistinguishable from a duplicate to the visitor.
98		− `create_repo` maps `GitStorageError::AlreadyExists` to "that name is taken", which is
99		− true from outside but hides the inconsistency from the operator. There is no logging
100		− story yet for it to surface in. The durable fix is the reconciliation sweep in
101		− [architecture.md](architecture.md#db-plus-filesystem-writes).
102		−- The duplicate check races. Two concurrent creates of the same name can both pass
103		− `find_by_org_and_name`; the loser is then stopped by `init_bare` or, failing that, by
104		− the `unique (org_id, name)` constraint — which surfaces as an opaque storage error
105		− rather than "name taken". Correct, just ugly, and single-user for now.
106		−
107		−- The database and the filesystem cannot share a transaction. Creating a repo
108		− writes a row and a directory. Neither previous attempt solved this properly — see
109		− [architecture.md](architecture.md#db-plus-filesystem-writes). A compensating delete is
110		− good enough to ship, but write down that an orphaned directory is possible if the
111		− process dies between the two, rather than rediscovering it.
112		−- Path traversal. `{data_dir}/{handle}/{name}.git` is built from user input. A name
113		− containing `..` or `/` must be impossible before it reaches the filesystem, and
114		− `RepoName` is the place to make it impossible rather than sanitising at the call site.
115		−- Visibility is an authorization decision, so it belongs in the use case. A private
116		− repo must be absent from listings, not merely unlinked — and `/api` must agree with
117		− the page.
118		−- `git` is a dependency of the test suite too, not only of the runtime —
119		− `DiskGitStorage`'s tests run real `git init`. A machine without `git` fails
120		− `cargo test`, not just the app.
121		−- A handle rename is a directory move. The layout is keyed by handle for
122		− legibility, so whenever renaming arrives it has to move the tree; it cannot be a row
123		− update. Nothing renames handles today.
124		−- Bare repos created on macOS carry `ignorecase = true` in their config, because
125		− git probes the filesystem at init. Correct where it was created, wrong if the data
126		− directory is ever moved to Linux. A migration gotcha, not a bug.
127		−- An orphaned directory blocks re-creating that name. `init_bare` refuses rather
128		− than adopting what is already there, and repo deletion is out of scope this
129		− milestone, so clearing one is a manual `rm` for now.
	73	+- `body_limit` will reject pushes and large fetches. `topcoat-router` caps request
	74	+ bodies; the git routes need it raised. Expect a confusing failure rather than a clear
	75	+ one — noted since [0001](decisions/0001-git-over-http-not-ssh.md).
	76	+- CGI header parsing sits in front of a stream. `http-backend` writes headers, a
	77	+ blank line, then the body. Reading the headers must not buffer the body — that is the
	78	+ whole reason this transport was judged viable on `Body::into_data_stream`.
	79	+- A subprocess per request, unlike Milestone 3's once-per-creation. Fork/exec cost
	80	+ now sits on a hot path; measure before assuming it is fine.
	81	+- `http-backend` reports failure through CGI status lines, not exit codes alone. A
	82	+ non-zero exit and a `404 Not Found` on stdout mean different things.
	83	+- The advertisement must not be cached. `Cache-Control: no-cache` on `info/refs`, or
	84	+ clients fetch a stale ref list and fail to find commits that exist.
130	85
131	86	### Carried over — small, unblocked
132	87
	88	+- An orphaned repo directory is possible if the process dies between the record
	89	+ write and the filesystem write, and it then blocks re-creating that name. The durable
	90	+ fix is a reconciliation sweep on boot
	91	+ ([architecture.md](architecture.md#db-plus-filesystem-writes)); clearing one is a
	92	+ manual `rm` today, since repo deletion does not exist.
	93	+- The duplicate-name check races. The loser is caught by `init_bare` or the unique
	94	+ constraint, but surfaces as an opaque storage error rather than "name taken".
	95	+- Bare repos created on macOS carry `ignorecase = true`. A migration gotcha if the
	96	+ data directory ever moves to Linux.
133	97	- Fonts are not loaded. The theme names Geist and IBM Plex Mono; both fall back
134	98	today. Topcoat's `font-fontsource` feature handles it.
135	99	- Light mode is untested. The palette defines it; nobody has looked at it.
		@@ -143,9 +107,10 @@ Nothing open. `GitStorage`'s shape and how git is invoked are recorded in
143	107
144	108	Ordered. Pull from the top.
145	109
146		−1. Milestone 4 — Git over HTTP. `git http-backend` subprocess, PATs over HTTP
147		− Basic. See [0001](decisions/0001-git-over-http-not-ssh.md). The `body_limit` cap will
148		− reject large pushes until raised.
	110	+1. Milestone 4b — Push and tokens. Personal access tokens over HTTP Basic, `git
	111	+ push`, private clone. Open decisions when it starts: how tokens are hashed (session
	112	+ token hashing already exists to copy), whether tokens carry scopes, and the 401-vs-404
	113	+ tension above.
149	114	2. Milestone 5 — Repo browsing. Tree, blob, commit log. **Start with domain value
150	115	objects** — `ObjectId`, `RefName`, `TreeEntry` — before any adapter. A query port
151	116	returning `String`s is an anaemic pass-through that pushes validation into the page.
		@@ -158,9 +123,6 @@ Ordered. Pull from the top.
158	123
159	124	- Topcoat is early (v0.5.0, first released 2026-07-22, breaking changes expected
160	125	by its own authors). Expect churn that isn't feature work.
161		−- Body size limits will reject large pushes at Milestone 4 — `topcoat-router` has a
162		− `body_limit` layer that needs raising on the git routes. Recorded here because it
163		− will surface as a confusing failure rather than a clear one.
164	126	- Topcoat ships Tailwind without Node, which reopens the design system attempt #1
165	127	dropped purely to avoid an npm build step — see [ui.md](ui.md).
166	128

plans/decisions/0006-git-binary-behind-narrow-ports.md+23 −0View file

		@@ -84,3 +84,26 @@ through `run_git`, which sets the isolation once.
84	84	one caller. Swapping the binary for `gix` behind an unchanged `GitStorage` is
85	85	contained; doing it after `GitProtocolServer` exists is not, because the protocol
86	86	adapter is the one that genuinely cannot be rewritten in `gix` today.
	87	+
	88	+## Amendment — 2026-08-28, at the start of Milestone 4a
	89	+
	90	+The table above sketched the protocol family as `upload_pack` / `receive_pack`, mirroring
	91	+attempt #2's SSH ports. That shape is wrong for HTTP and is not what gets built.
	92	+
	93	+[0001](0001-git-over-http-not-ssh.md) delegates the protocol to `git http-backend`,
	94	+which is a CGI: it takes a request method, a path, a query string and an environment,
	95	+and writes headers followed by a body. Semantic per-operation methods would mean
	96	+re-deriving that CGI shape at the call site, so `GitProtocolServer` is CGI-shaped
	97	+instead — one request in, one streaming response out.
	98	+
	99	+The direct alternative — spawning `git upload-pack --stateless-rpc` per route, which
	100	+would give the semantic ports named above — was measured and rejected before this was
	101	+written. A real client gzip-compresses its POST body once a repository has a nontrivial
	102	+number of refs, on protocol v0 and v2 alike, so the direct route means owning request
	103	+inflation and `Git-Protocol` forwarding. It passes against a one-ref test repository and
	104	+fails on the first real one. Recorded in `current.md` under Milestone 4a.
	105	+
	106	+What survives unchanged is the part that mattered: several narrow ports rather than one
	107	+git service, and one infrastructure-side invoker owning how git is actually run.
	108	+Authorization still happens in the use case, before the subprocess exists — the CGI
	109	+shape moves how git is called, not who decides it may be.

plans/progress.md+20 −3View file

		@@ -97,12 +97,20 @@ URLs settled as root handles with grouped application routes
97	97	in rather than depended on ([0005](decisions/0005-tailwind-and-copied-components.md)).
98	98	Components reference theme tokens, never raw colours.
99	99
100		−### Milestone 3 — Repo model · in progress
	100	+### Milestone 3 — Repo model · done
101	101
102		−Domain, both persistence adapters, and now `GitStorage` with `DiskGitStorage` behind
103		−it. How git is invoked is recorded in
	102	+Repositories exist as records and as bare repos on disk, and they appear on the
	103	+profile. Domain, both persistence adapters, `GitStorage` with `DiskGitStorage` behind
	104	+it, `create_repo` / `view_repo` / `list_repos`, the `/{handle}/repos/new` form and
	105	+`/{handle}/repos/{name}` page, the profile's Repositories section, and
	106	+`/api/users/{handle}/repos`. How git is invoked is recorded in
104	107	[0006](decisions/0006-git-binary-behind-narrow-ports.md).
105	108
	109	+Verified in a browser and by curl: the owner creates a repo through the form, a
	110	+bare repo appears at `{data_dir}/{handle}/{name}.git`, and it lists on the profile; a
	111	+private repo is absent for a signed-out visitor on both the page and `/api`; an unknown
	112	+handle 404s rather than answering `[]`. `git clone` does not work yet — Milestone 4.
	113	+
106	114	#### Decisions worth remembering
107	115
108	116	- `git init` on an existing repository exits 0 and re-initialises in silence.
		@@ -171,6 +179,15 @@ it. How git is invoked is recorded in
171	179	- `InMemoryGitStorage` enforces `AlreadyExists` too. A permissive fake would let
172	180	`create_repo` pass while the real adapter refused. The fake mirroring the rule is the
173	181	point of having two implementations.
	182	+- `/api` resolves the handle without loading a profile. `handle_param` split out
	183	+ of `profile_for` so the repo listing route 404s on `list_repos`' own `None`. Routing
	184	+ it through the profile would have made that `None` unreachable and left the page and
	185	+ `/api` disagreeing about what an empty portfolio means — the exact duplication
	186	+ `architecture.md` says to watch for.
	187	+- The `/api` listing is a bare array, not an envelope. Matches `/api/users/{handle}`
	188	+ returning a bare object. Pagination later means a wrapper and a breaking change; taken
	189	+ knowingly, since personal-first means a handful of repositories and there are no
	190	+ consumers yet.
174	191	- `Repository::description` stays. Added unrequested and flagged; kept on review
175	192	because this milestone's own "Done when" puts repositories on the profile, which
176	193	makes it a consumer inside the milestone rather than speculation. Worth noting the