steid

steid/plans/decisions/0006-git-binary-behind-narrow-ports.md

10.3 KBRaw

1	# 0006 — Drive git through the binary, behind narrow ports
2
3	Status: accepted · Date: 2026-08-13
4
5	## Context
6
7	Milestone 3 is the first time Steid has to touch git at all: creating a repository
8	means a bare repo on disk as well as a row in SQLite. It will not be the last. The
9	ladder needs at least three families of git operation, and they arrive one milestone
10	apart:
11
12	\| Family \| Shape \| Consumer \|
13	\|---\|---\|---\|
14	\| Lifecycle — `init_bare`, `remove`, `repo_path` \| fallible, fire-and-forget \| `create_repo` (M3) \|
15	\| Protocol — `upload_pack`, `receive_pack` \| bidirectional streaming \| `serve_clone` / `serve_push` (M4) \|
16	\| Query — refs, tree, blob, log \| returns domain objects \| read models (M5) \|
17
18	Two questions fall out, and answering only the first is what left attempt #2 with git
19	invocation details spread across the code.
20
21	What runs git? [0001](0001-git-over-http-not-ssh.md) already commits to delegating
22	the smart HTTP protocol to `git http-backend`, so the `git` binary is a runtime
23	dependency from Milestone 4 whatever happens at Milestone 3.
24
25	What shape does the application see? `architecture.md` names `GitStorage` and
26	`GitProtocolServer` as separate ports, but nothing had been built, so whether that
27	survived contact was untested.
28
29	There is also a quieter pressure. Invoking git safely is not one decision but half a
30	dozen — ambient configuration, redirected object storage, inherited stdin, argument
31	injection, blocking the async runtime, what a non-zero exit means. Each is invisible
32	when wrong. Getting them right in `init_bare` and forgetting them in the Milestone 4
33	`http-backend` spawn would produce no failure, just an inconsistent posture.
34
35	## Decision
36
37	Steid shells out to the `git` binary, exposed to the application layer as **several
38	narrow ports rather than one git service, with a single infrastructure-side
39	invoker** that owns how git is actually run.
40
41	`GitStorage` (`init_bare`, `remove`, `repo_path`) lands now, in
42	`application/port.rs`. `GitProtocolServer` and a query port follow when their use
43	cases exist, not before. All of their adapters live in `infrastructure/git.rs` and go
44	through `run_git`, which sets the isolation once.
45
46	## Alternatives considered
47
48	- `gix` (pure-Rust git). Faster — microseconds against the measured ~13ms of a
49	`git init --bare` fork/exec — and no `PATH` dependency. Rejected on consistency, not
50	speed: [0001](0001-git-over-http-not-ssh.md) already requires the binary for the
51	protocol, so this would mean two implementations holding assumptions about the same
52	on-disk format, and the faster one is on the path that runs once per repository
53	creation rather than once per request. Worth revisiting at Milestone 5, where
54	browsing is read-only, hot, and the place `gix` is strongest.
55	- One `GitService` port. A single trait covering all three families. Rejected: the
56	shapes do not unify — one streams, one queries — and every test fake would have to
57	implement the whole surface to exercise a use case that needs two methods. Ports
58	live where they are consumed.
59	- No shared invoker; each adapter spawns its own `Command`. Rejected because the
60	isolation flags are exactly the kind of thing that drifts silently between call
61	sites. There is one caller today, which makes this look premature; the recipe is one
62	private function rather than a public abstraction precisely so it is not.
63	- A domain abstraction over git objects now. Rejected as premature. Neither
64	creation nor the protocol needs the domain to understand a commit — pack data is
65	opaque bytes in transit. Milestone 5 is where `ObjectId`, `RefName` and `TreeEntry`
66	become real domain value objects, and it should start with them rather than an
67	adapter returning `String`s.
68
69	## Consequences
70
71	- Makes easy: adding the Milestone 4 and 5 ports without re-deciding how git is
72	invoked; testing `create_repo` against a fake `GitStorage` that never touches disk.
73	- Makes hard: anything wanting git operations not expressible as a subprocess.
74	Also every operation pays a fork/exec — fine at once per repository creation,
75	something Milestone 5 should measure before browsing does it per page view.
76	- New dependencies: `git` on `PATH` is now required by the test suite as well
77	as at runtime, since `DiskGitStorage`'s tests run real `git init`. `tokio` gains the
78	`process` and `fs` features.
79	- Known hole: the repository row and its directory cannot share a transaction. The
80	compensating transaction in `architecture.md` is what `GitStorage::remove` exists
81	for, and it still leaves an orphaned directory if the process dies between the two
82	writes.
83	- Reversibility: the port shape is cheap to change while there is one adapter and
84	one caller. Swapping the binary for `gix` behind an unchanged `GitStorage` is
85	contained; doing it after `GitProtocolServer` exists is not, because the protocol
86	adapter is the one that genuinely cannot be rewritten in `gix` today.
87
88	## Amendment — 2026-08-28, at the start of Milestone 4a
89
90	The table above sketched the protocol family as `upload_pack` / `receive_pack`, mirroring
91	attempt #2's SSH ports. That shape is wrong for HTTP and is not what gets built.
92
93	[0001](0001-git-over-http-not-ssh.md) delegates the protocol to `git http-backend`,
94	which is a CGI: it takes a request method, a path, a query string and an environment,
95	and writes headers followed by a body. Semantic per-operation methods would mean
96	re-deriving that CGI shape at the call site, so `GitProtocolServer` is CGI-shaped
97	instead — one request in, one streaming response out.
98
99	The direct alternative — spawning `git upload-pack --stateless-rpc` per route, which
100	would give the semantic ports named above — was measured and rejected before this was
101	written. A real client gzip-compresses its POST body once a repository has a nontrivial
102	number of refs, on protocol v0 and v2 alike, so the direct route means owning request
103	inflation and `Git-Protocol` forwarding. It passes against a one-ref test repository and
104	fails on the first real one. Recorded in `current.md` under Milestone 4a.
105
106	What survives unchanged is the part that mattered: several narrow ports rather than one
107	git service, and one infrastructure-side invoker owning how git is actually run.
108	Authorization still happens in the use case, before the subprocess exists — the CGI
109	shape moves how git is called, not who decides it may be.
110
111	## Amendment — 2026-08-29, the Milestone 5 read path
112
113	0006 deferred one question to this point: whether `gix` should take over the read path,
114	"where browsing is read-only, hot, and the place `gix` is strongest". Answered here, with
115	the measurement it asked for.
116
117	The read path stays on the `git` binary, one process per query, for now.
118
119	### What was measured
120
121	On a 201-commit repository, 50 runs averaged per command: `rev-parse` 11.2ms, `ls-tree`
122	11.7ms, `cat-file` 11.6ms, `log -20` 11.8ms, `for-each-ref` 14.4ms. **The cost is
123	starting the process, not answering the question** — every command lands in the same
124	band regardless of the work it does, matching the ~13ms `git init --bare` from Milestone
125	3. A three-call tree page is therefore ~35ms of pure overhead.
126
127	### Why not `gix`
128
129	Checked against gitoxide's own crate status rather than recalled. The read path is
130	comprehensively covered — `gix-ref`, `gix-odb`, `gix-pack`, `gix-object`, `gix-traverse`
131	and `gix-revwalk` all do what browsing needs. Capability is not the obstacle.
132
133	Server-side protocol support is. `gix-protocol` offers `upload-pack` /
134	`receive-pack` plumbing for in-process transports only — not a network server. Since
135	[0001](0001-git-over-http-not-ssh.md) commits to `git http-backend`, the binary is a
136	permanent runtime dependency, so adopting `gix` would not replace an implementation. It
137	would add one: two libraries reading the same bytes on disk, forever, needing to
138	agree. That is the cost this ADR priced originally, now confirmed rather than assumed —
139	and the price of avoiding it is 35ms a page for one user.
140
141	### Why not fork gitoxide and write the server side
142
143	Considered and rejected. A fork only pays off if it lets Steid delete the binary, and
144	that requires server-side `receive-pack`: pack verification and fsck, quarantine, atomic
145	ref updates, hooks, report-status — executed against untrusted packs from the internet,
146	a surface with a real CVE history. Months of work, no user-visible capability, and
147	[ROADMAP.md](../ROADMAP.md#why-this-order) names disappearing into protocol work as what
148	ended both previous attempts. If the work is wanted for its own sake it belongs upstream
149	in gitoxide, whose stated goal already includes server support, on its own timeline —
150	not on Steid's critical path.
151
152	### The upgrade path, which is the point
153
154	Staged, and each stage keeps `GitQuery` unchanged:
155
156	1. One-shot commands. ~35ms a page. Where Milestone 5 starts.
157	2. A `cat-file --batch` process held open per repository. Deletes the fork cost for
158	object reads; what Gitaly and Gitea both do. Projected at ~1–3ms a page — projected,
159	not measured, and to be benchmarked before it is claimed.
160	3. `gix-object` decoding bytes fetched by stage 2, if tree and commit parsing ever
161	need to happen in-process. Worth distinguishing from adopting `gix`: a decoder for a
162	documented format is not a second implementation of repository access — no ref
163	resolution, no pack access, no protocol. Full `gix` and the fork stay rejected;
164	neither is on this ladder.
165
166	One thing must be got right now for stage 2 to stay cheap. A batch adapter owns live
167	subprocesses, so it has to be a shared handle in app context rather than constructed per
168	request as the SQLite adapters are. Built per request, the pool would spawn and kill a
169	process per request — the fork cost back, plus a pool that never pools. The port gives us
170	the seam; it does not give us the lifetime, and the lifetime is a composition-root
171	decision that has to be made before the first adapter is wired.
172
173	### Reopening conditions
174
175	Evidence, not intuition:
176
177	- A per-file last-commit column becomes a must-have — ~230ms for a twenty-file directory
178	at one fork per entry. Go to stage 2 first, not to `gix`.
179	- Browsing takes real public traffic, making the read path hot rather than occasional.
180	- gitoxide ships usable server-side `upload-pack` and `receive-pack`, at which point it
181	stops being additive and becomes a candidate to replace the binary outright.