Beagle SCM

GRAF-002: lazy commit indexer for first-touch/mtime (p1/p2/p3/p4)

A lazy, watermark-driven commit indexer that extracts each commit ONCE into the graf .graf.idx LSM family, so first-touch/mtime queries run index-only (no keeper inflate). Record set: p1 parentage, p2 commit->tree, p3 LAZY tree-child edges (/todo/BLAME/BLAME-002 shape, new-node-only), p4 commit-ts. First-touch = a p1 walk with p2/p3 descent-by-name, ts read from p4 (NOT a seek). Unifies today's three independent DAG walkers: /todo/LIST/LIST-001 (shared/lastcommit.js), log:path file-history (views/log/log.js fileHistory), and the wanted mtime-sorted todo board.

Input

Context

Three consumers re-derive the same per-commit tree-diff on demand, none cached; graf.js (GRAF-001) only caches ahead/behind pairs.

• shared/lastcommit.js (LIST-001) walks tip->parent, changedpaths. changedCommits per commit — O(history x tree-diff), every call.
• views/log/log.js fileHistory dag.ancestors(tip) over the WHOLE reachable set + a tree descent per commit — quadratic across paths.
• C graf (BLAME-002) materialises tree-child edges EAGERLY, one per distinct tree at ingest — the p3 blowup this ticket avoids.
• No path->last-touch posting exists on either the C or JS side.

Goals

An index that answers "newest commit that changed path P, and its ts" with ONE seek, built incrementally, sized by diff volume not tree count.

• Lazy watermark catch-up: on open walk tip -> last-indexed, extract only new commits, append records, advance the mark.
• Change postings: pathhash60 -> (ts, commit_h60, blob_h60) emitted only where a leaf differs from the mainline parent (add/mod), plus a del tombstone (blob_h60 = 0) where it disappears.
• Mtime query = seek pathhash, newest ts wins; a del newest = gone.
• Serve the mtime todo board, list:, and log:path off one family.

Constraints

Reuse the existing LSM/idxmaint plumbing and graf's 60-bit hashlet identity; keep the change unrelated to the ahead/behind pair cache.

• Persist via shared/idxmaint.js under .lock.graf, a NEW ext (e.g. touch.idx) or a disjoint type nibble in .graf.idx.
• Record halves stay wh128 hashlet[60] | type[4]; new types must not collide with native 1..5 (DAG_T_*) or graf.js 0xA/0xB.
• Trunk-scoped (.graf.idx is trunk): board still blends the wtlog for uncommitted edits on top — the index gives COMMITTED mtime only.
• Read-only store degrades to mem-only (idxmaint norm), never crashes.

WIP

Design decisions

Record set: p1 parentage, p2 commit->tree, p4 commit-ts (sentinel+DAG), p3 = TREE-CHILD edges (content-addressed, NOT change postings — AMENDED 2026-07-23, see below). Rulings pinned below.

• p3 = combine(parent_tree_h60, name) -> child_h60 (the C BLAME-002 DAGChildStep shape), materialised ONCE per DISTINCT tree/blob node, LAZILY — only for a node NEWLY appearing at ingest (dedup by node hash). Supports descent BY KNOWN NAME, NOT enumeration.
• ts lives in p4 ONLY (commit_h60 -> author-ts ron60, dag.commitTs); p3 carries NO ts. First-touch of a KNOWN path = a p1 walk newest- first, descending p2->p3 by name at C vs its mainline parent, comparing the leaf child_h60; the commit where it changes/first-appears is the touch, ts read from p4[C]. NOT a single seek.
• No tombstone: deletion is STRUCTURAL — the parent^name edge is ABSENT in the newer tree. A p3 miss on a COVERED tree = genuine absence; a miss on an UNCOVERED tree => keeper-inflate fallback, so the indexer tracks the set of materialised tree hashes.
• Coverage sentinel = p2/p4 (every commit has one); a covered commit with no p1 is a root. Sentinel absent => reindex that commit.
• Root commit PARENTS ITSELF (p1 key.hl == val.hl == root_h60), so "p1 always exists" is universal — walker guards the self-loop (the seen-set halts it; parent==self IS the root/stop test).
• Merge basis = mainline first parent (argmax commit-ts, the lastcommit.mainlineParent rule): the first-touch walk descends the mainline p1 line. log fileHistory's differs-from-EVERY-parent is a distinct simplification — note it at that call site.
• Prune: descent stops where the subtree child_h60 is equal between C and its parent (subtree-hashlet equality), so a walk touches only the path segments that actually changed.

TODOs

• Core scaffold + hermetic test landed (in-process, WRONG p3 = change postings). Reworking to tree-child p3 per the amendment.
• Rework shared/commitindex.js p3 => tree-child edges combine(parent_tree_h60,name)->child_h60, lazy new-node-only via a store.readTree walk; strip ts from p3; drop the blob-0 tombstone.
• firstTouch(path) = p1 walk + p2/p3 descent-by-name, ts from p4, keeper-inflate fallback on a p3 miss over a covered tree set.
• Watermark/coverage is per-COMMIT (p2 sentinel); rebase/reset re- walks tip->first-covered by construction (no forward-range rebuild).
• changedpaths integ: an INDEX-BACKED reader (resolve child by (tree_h60,name) via p3, inflate on miss) fed to changedpaths' existing readTreeRecursive reader slot — the ONE shared descent. Indexer materialises p3 by its own tree walk, NOT changedCommits.
• Wire the mtime todo board (todo recent, lowercase arg route in views/todo/todo.js) onto firstTouch, wtlog blend on top.
• Migrate lastcommit.js (LIST-001) + log:path fileHistory onto the index reader with keeper-inflate fallback on any coverage miss.
• Parity test vs the on-demand walkers on a real repo (journal).

Blockers and bummers

• p3 blowup bounded by DISTINCT tree-node count (new-node-only, deduped by hash) — accepted; scope to queried prefixes (todo/) if it bites.
• Persistence deferred: p3 is a clean wh128 (parent^name->child) so it FITS the .graf.idx family; a later pass lands it via idxmaint.
• Rename-follow is OUT (LIST-001 precedent): a path appears/disappears only; a move reads as del+add.

Outcome

Nothing landed yet. Suggested first commit once the indexer + board land: GRAF-002: lazy commit indexer, change postings, mtime todo board.