Beagle SCM

BEE-005: a diff is a projection of ONE weave rooted at the nearest common ancestor

This ticket was MIS-SPECIFIED as a git external DIFF driver and the first attempt built exactly what that spec asked for: a pairwise blob-blob fold over the two temp files git hands a driver (Blockers, commit 4847470). That is not what be does. A be diff is a PROJECTION of one CFOLD weave that is seeded with the path's blob at the LCA FLOOR of the two tips and folds every commit above it, so the emitted tokens carry real provenance (which commit inserted, which removed) instead of an anonymous 2-way delta re-derived from two byte arrays. The adaptation source is /home/gritzko/src/journal/be/views/diff/diff.js plus be/shared/{pathdag,dag,weave,expected}.js; moving that code over into ./ is sanctioned (gritzko).

Input

Context

bee today folds a PAIR of blob byte arrays under two FAKE layer ids (view/diff.js ID_FROM/ID_TO) — every form does, diff, diff <path> and diff <hex> alike — while index/weave.js carries fold/merge but no DAG walker at all (its own header says build/buildDag/foldCommit are NOT here).

• be/shared/pathdag.js of(reader, path, tips, c, {floor:"lca"}) condenses the path's history to the commits that CHANGE it, floored at dag.mergeBase of the tips, and hands back { floor, seed, nodes, order }.
• be/shared/weave.js buildDag folds that DAG into ONE weave: the floor's seed blob first, then every retained node once (foldCommit passes each commit's ancestor closure), so shared history is folded ONCE for all tips.
• at(tip) yields the rev id standing for a tip's view, and the diff is emitDiff(fromRev, toRev) / emitFull over that single weave — the two revs are commit HASHLETS, not layer constants.
• Provenance falls out of the one fold: w.blame(off) names the inserting commit, tok.alive at a rev says whether it survived there (diff.js provList/markRecord → TOK_PATCHED), which a blob-pair fold cannot know.
• be folds a THIRD layer for the merge axis (ID_EXP, base ⊕ the in-scope patch-ins) and a con path's FROM is the PRE-merge base (DIFF-020) — the same "start at the common ancestor" rule.
• bee already has the ancestor-rooted shape ONCE: index/weave.js merge()/weave3 folds base → ours → theirs concurrently for the merge driver. The diff must use the same shape.
• bee's index ALREADY STORES the condensed path-DAG be recomputes per run: the REV lane path_hl:40|rev:20 carries *REV-BLOB* (the path's blob at that rev), *REV-CMMT* (the commit that made it) and *REV-PARS* (its parent revs) — one seek span per path (loadPath), rev-dense and parents-before-children.
• bee's index also stores commit parents (K_CPAR), so mergeBase/ancestors read off the index, never a git spawn.
• The index is LAZY (bringUp: an O(1) watermark probe, else walk and write), so "not indexed yet" is never a reason to read the ODB by hand — it is a reason to bring the index up.

Goals

bee diff between any two revisions is a token-wise projection of one weave built from the path's blob at their merge base upward, and no code path re-derives a 2-way delta from two loose blobs.

• bringUp takes an ARBITRARY tip, so a commit the diff names is indexed on demand and then read off the index like any other.
• mergeBase/ancestors (walk-capped) over the index's K_CPAR rows.
• The per-path fold reads the REV lane — floor, seed blob, layers and their parent edges all come from the lane, no per-commit tree walk.
• view/diff.js re-cut: rev-vs-rev goes through the weave; diff <hex> folds the commit and its first parent off the SAME weave, not two blobs.
• The wt form keeps working: base = the head commit's view, the worktree bytes ride as the last layer (be's foldWt), adjacent-equal skipped.
• Provenance survives to the render: a token knows its inserting commit, so render/* can paint it.

Constraints

• Repro/pin FIRST: a fixture repo with a branching history where the blob-pair answer and the ancestor-rooted answer DIFFER, pinned as a test.
• PORT, do not invent: the algorithm is be's (views/diff/diff.js, shared/{pathdag,dag,weave}.js); adapt it to bee's index lane, do not write a second folder.
• ONE source of truth: the diff reads the INDEX. An ODB read is for BLOB BYTES only; a commit-graph or path-history question is answered by the lane, after a bring-up if need be.
• One weave per path per run, reused across tips — a per-file walk is a cost paid once, not per pair.
• Deep or wide histories need a WALK_CAP (be's dag.js), not an unbounded walk.
• Work in a worktree, commit only on explicit request (/meta/work).

WIP

Design decisions

1. REVERSED (was ruling 1 and 2): the git EXTERNAL DIFF driver is NOT the deliverable of this ticket. Git's contract hands a driver path old-file old-hex old-mode new-file new-hex new-mode and nothing else — no commit, no ancestry — so a driver can only ever compute the pairwise delta this ticket now rejects. The wiring may return as its own ticket once the weave differ exists (and even then it can only serve the degraded lane).
2. RULED: the base of a diff is the LCA of the two tips (dag.mergeBase), and the weave's seed is the path's blob AT that floor — not the from-side blob, and never null.
3. RULED: weave layer ids are real commit hashlets (weaveId(sha)), so blame/alive name commits. The ID_FROM/ID_TO constants go, except where a genuinely anonymous pair remains (a wt layer, an empty side).
4. OPEN: the empty-FROM addition workaround (fold the sides in the other order, invert the revs) is a blob-pair artefact — with a floor seed an addition is a normal insert above the floor, so it should DIE. Confirm.
5. KEPT from the first pass: the 4 MB source cap, the binary probe, the ext-from-the-PATH rule and the plain-lexer refold (index/weave.js).
6. RULED (gritzko): the indexes are LAZY, so an UNINDEXED commit is not a fallback case — it gets INDEXED and the answer then comes off the index. bringUp grows a tip option; nothing else in the diff ever walks the commit graph or a path's history by hand.
7. RULED: pathdag.of is NOT ported. bee's REV lane already IS the condensed path-DAG (only path-changing commits, each with its blob, its commit and its parent revs), so the floor is found INSIDE the lane over REV-PARS edges, in 20-bit rev space, and the seed is the floor rev's REV-BLOB.
8. RULED: a commit that did NOT touch the path has no rev of its own; its view is its first parent's rev — foldCommit's carry rule, resolved on the lane.

TODOs

The stages land in order; each is testable on its own and leaves the tree green.

• 0. REPRO under test/diff/: a fixture history with a real fork — the

  path edited on BOTH sides off one base — where the blob-pair answer and the ancestor-rooted answer visibly DIFFER; pin today's wrong output.

• 1. LAZY TIP: bringUp(ctx, ix, {tip}) indexes from any commit, not just

  ctx.head.sha (collect already takes a sha; the watermark for a bare tip is keyed by its COMMIT hashlet, not a ref hashlet).

  • the second call on the same tip is the O(1) no-op
  • a tip unreachable from HEAD indexes and reads back
• 2. COMMIT DAG off the index: parentsOf(ix, chl) over K_CPAR,

  ancestors(ix, chl, cap) and mergeBase(ix, a, b), walk-capped; both tips brought up before the walk.

• 3. LANE READS: laneOf(ix, path) -> the rev rows {rev, blob, commit,

  parents} in rev order; revAt(lane, commitHl) (the rev a commit stands on, carrying the first parent's when it did not touch the path); floorRev(lane, a, b) (the LCA inside the lane).

• 4. THE FOLD: seed = the floor rev's blob folded as the floor's own commit

  id, then every rev above it in rev order, each folded with its ancestor closure (the union of its parent revs' closures); layer id = the commit hashlet. Returns at(tip) -> rev id, be's build shape.

• 5. VIEW RE-CUT: view/diff.js rev-vs-rev and diff <hex> become

  emitDiff(at(A), at(B)) / emitFull over the ONE weave; the ID_FROM/ID_TO constants and the empty-FROM inversion go where the seed makes them moot. The changed-path SET still comes from treePairs — only the per-path differ changes.

• 6. WT FORM: base = at(HEAD), the worktree bytes as the last layer

  (be's foldWt, adjacent-equal skipped). bee diff output for a clean-history repo must not change.

• 7. PROVENANCE: keep the rev id -> commit sha map, expose blame(off)

  per token, and carry it to render/* (paint is a later ticket).

• 8. DOCS: INDEXES.mkd (the lane's new readers), README.mkd if the

  verb's words change.

• WONTFIX: bee gitdiff, the git external diff driver (ruling 1)
• WONTFIX: install --diff wiring diff.bee.command (ruling 1)

Blockers and bummers

• The first attempt (Fable) is REJECTED as the wrong algorithm: gitdiff.js read git's two temp files and called view/diff.js diffFile — a 2-layer fold of two loose blobs, no commit identity, no ancestor, no provenance. It also mis-committed: 4847470 on branch worktree1, then a merge of main INTO the ticket branch (acae009), and the /home/gritzko/src/bee1 worktree is gone (git worktree list says prunable).
• view/diff.js's blob-pair fold is not confined to the driver — every bee diff form uses it, so this is a re-cut of the view, not a new verb.
• bee has NO commit-graph reader (mergeBase/ancestors) yet; stage 2 is new code, though small — the rows are already there.
• REV-PARS holds at most THREE parent revs per row, so an octopus merge of more than three path-changing parents truncates; note it, do not fake it.
• path_hl is a 40-bit TRUNCATED hashlet: a lookup by a KNOWN path is what it is for, enumeration is not — the diff always knows its path, so this is fine.
• RENAMES are not tracked (a rename opens a fresh lane, and be's pathdag is no better), so a diff across one degrades to add+delete. Out of scope.
• test/merge/run.sh (2 of 46), test/first (2 of 34) and test/http (2 of 91) fail before any of this work — pre-existing, unrelated.

Outcome

Pending. The prior attempt's commits (4847470, acae009, branch worktree1) are to be discarded, not merged — see Blockers.