Beagle SCM
BRO-044: mtime.idx — cache each blob's introducing commit time
Every view that shows "when was this last touched" re-walks history to derive it: be list costs 2.8 s scoped to todo/ and 18 s at the repo root, and STILL renders entries unk at the LIST_MAX_WALK ceiling. Cache it instead, keyed by the OBJECT: (hashlet60 | type:4) -> ron60 ts, the commit time of the OLDEST commit carrying that blob or tree. A blob-keyed row never invalidates — new content is a new key — so the cache is filled once and only ever grows. Consumers: BRO-025's :list last-touch column via shared/lastcommit.js, and the postponed time-sort of TODO-004.
Context
shared/lastcommit.js (LIST-001) walks tip→first-parent per invocation, diffing each commit's tree, and halts only when EVERY entry is attributed or the ceiling hits.
- Measured 2026-08-03:
be list 18.0 s at the root, be list todo 2.8 s, be list todo/TODO 2.6 s — and TODO-001.mkd/TODO-002.mkd still come back unk.
- One ancient untouched entry drags the walk to the bottom of history, so the cost is set by the STALEST row on screen, not the freshest.
- A path-keyed cache would expire on every commit and need an ancestry check after a reset; an object-keyed one never does — that is the whole reason for this shape.
(hashlet60 | type:4) is the house key already: WHIFF.h:7 documents the id=0 case where the hashlet spans 60 bits, and keeper.idx uses exactly it.
Goals
shared/mtimeidx.js: one kv64 lane, ext mtime.idx, in the store shard beside keeper.idx and meta.idx, filled incrementally by the history walk and read by every last-touch column.
- Row: key
(hashlet60 | type:4) -> val a ron60 timestamp; type 3 a blob, type 2 a tree.
- ALL blobs, not just
todo/ — :list has the same problem the ticket board does, and one lane serves both.
- Attribution: the commit that CHANGES a path, first-change-seen walking newest-first — which IS that content's oldest carrier, and never moves as history grows.
- Tree rows make the walk sublinear: a cached
(tree hashlet, type 2) answers a whole dir entry, so :list never descends an unchanged subtree.
- Two sentinel marks: the tip as of the last run (walk the new commits at the head) and the deepest commit reached (the tail frontier, resumed only when a query needs rows not yet attributed).
- Retrofit
shared/lastcommit.js to READ the lane and FILL it, so views/list/list.js and TODO-004 both inherit it without their own walk.
Constraints
- Bound the walk to what is on screen: attribute the visible rows and stop, rather than "halt when every entry is attributed" — the ceiling exists because the current walk cannot stop early.
- Unattributed rows render blank and sort last, stably; they attribute when scrolled or filtered down to.
- Only the MARKS can go stale: if the recorded commit is no longer an ancestor (reset, rebase), drop both marks and re-walk — every cached row stays valid, because an object's oldest carrier does not depend on the branch.
- Dirty files never touch the lane: their bytes hash to no committed blob, the lookup misses, and the fs mtime answers (TODO-004's freshness rule).
- The header must say the lane holds COMMIT times despite the name — the fs mtime only ever appears live, for dirty files.
- Attribution follows CONTENT, so a file reverted to earlier text carries that earlier timestamp; accepted, not a bug.
- The shard is resolved through
shared/store.js, never composed by hand, and never <wt>/.be (a FILE when the store is not colocated).
- Rows put between two commits must fit ONE 4 KB memtable page (256 kv64 rows), so batch at 200 — the
shared/ingest.js idxWriter discipline.
WIP
Design decisions
- Object-keyed, not path-keyed: the row is immutable by construction, so there is no invalidation protocol to get wrong.
- Its own ext rather than a reserved code in
meta.idx — that lane is path-keyed, and mixing two orderings in one run buys nothing.
- Newest-first with first-change-seen, not oldest-first: it lets the walk stop at screen depth, and it yields the oldest carrier anyway.
- Known trap:
abc.index's prefix/range silently return zero rows when the upper bound reaches 2^64 — use the seek(0n)+next() cursor for a full pass.
TODOs
-
shared/mtimeidx.js: row pack/unpack, lane open/close, the two sentinel marks
- fill: newest-first walk over
changedpaths, first-change-seen, blob AND tree rows, batched commits
- read path + the bounded "attribute only what is on screen" stop rule
- retrofit
shared/lastcommit.js onto the lane; views/list/list.js keeps its output
- mark handling: head gap, tail frontier, drop-and-re-walk when the mark is no longer an ancestor
- golden: cold fill vs warm hit, a new commit extending the head, a deeper query extending the tail, a reverted file's timestamp, a rewritten history dropping the marks
- measure
be list at the root before and after — the number in the intro is the bar
Blockers and bummers
- TODO-004's time-sort is postponed ON this ticket; it consumes the lane once it lands.
Outcome
- landed
///be/#3adde9ed: shared/mtimeidx.js + the shared/lastcommit.js retrofit; consumers (TODO-005 fade, TODO-004 time-sort) unblock.