Beagle SCM

BEE-024: the kv lane — keyed kv64 rows in .git/be/, first kind: file headers

A second family beside *.lite2.idx in <gitdir>/be/: abc.index("kv64", {dir, ext: ".kv.idx"}), the KEYED lane — a re-put OVERWRITES, so a row is a mutable cell where a lite2 row is a fact. As /INDEXES: derived, the kind nibble in the LOW 4 bits, one lane for every cell-like record kind. The FIRST kind is FILE HEADERS: the Key: value meta pairs under a StrictMark title (/meta/todo) and the keys of a Markdown YAML preamble — UPDATEABLE (Now: flips OPEN to DONE in place), which lite2 cannot say. TODO-003 is the model.

Input

Context

• Have: index/index.js:316:cn openIndex opens the wh128 family with {dir, ext}; sweep unlinks any file in be/ whose ext is not the current one (BEE-002) — the sweep must learn a SECOND legal ext or it eats the kv runs.
• Have: quickjab pup.c serves kv64/kv64Z lanes (quickjab/test/hash.js:8); be/test/metaidx.js is the golden of the row logic this ports.
• Have: view/status.js (BEE-022) walks the worktree under igno and compares wt bytes to the stage — the dirty-file set a lazy sweep wants.
• Have: hlOfText, hashlet60 helpers in index/index.js; ron.decode("Now") is the verbatim 18-bit key code.
• Missing: any bee record that can change value; every lite2 row is a fact about a blob or a commit.
• Consumer: the bee todo board (BEE-025) — Now: OPEN presence, Sev:, Who:, Due: filters, over one repo or ALL registered repos.

Goals

• index/kv.js: open/close of the .kv.idx family in indexDir(gitdir) (the common dir, so linked worktrees SHARE it as they share lite2, BEE-009); bee index prints its count on the one summary line.
• Row, kind HEAD (1): key path_hl:40 | key_code:20 | 1, val vkind:4 | payload:60 — the wh64 field split; path-major, so one file's pairs are ONE contiguous prefix scan.
• key_code for a META PAIR is the 3-char key VERBATIM, ron.decode("Now"), 18 of the 20 bits; for a YAML key it is the TOP 20 bits of the key-text hashlet, kind YAML (2), so Now: and now: never meet.
• vkind: LIT the payload is the ron60 of the normalized value (despaced, decased) when it fits 60 bits — it ranges and compares exactly; HASH the hashlet60 of the value, equals only; TOMB the pair is gone; HEAD the block header (the file exists, owning-wt code); MARK the per-wt mark.
• Per-key normalizers where a range is worth it: Due: 2026-08-19 -> the ron60 date, so Due:<26901 is a prefix range; free text hashes.
• The SWEEP is lazy and owns staleness: the first query brings the lane up — candidate files are every .mkd/.md the stage lists plus the untracked ones the status walk finds; re-lex every file whose mtime is AT OR PAST the mark; reconcile its block (put what was lexed, TOMB what vanished); TOMB the whole block of a file gone from disk; write the mark row LAST.
• The answer API: find(repo, [{key, value | prefix | any}]) -> file paths, intersecting on path_hl across keys; the path comes back through the sweep's own path list, never decoded from a hash.
• path_hl collision detection: two live paths on one hashlet -> both are dropped from the index for the run and read directly.

Constraints

• The lane decides WHICH files to open, never what to show: titles and bodies come off the file (TODO-003 rule).
• Extraction is the line-local regex ^([A-Z][a-z][a-z]): (.*)$ for meta pairs under the title, and the ---…--- block for YAML; the ragel rule of /meta/abc is about C, this is JS — no hand parser beyond the regex that IS the grammar.
• The mark is the MAX mtime OBSERVED in the scan, a row per worktree (path_hl hashes the wt-qualified path; the HEAD row carries the wt code); a clock is never consulted; intermediate seals never advance it.
• Rows put between two commits fit ONE 4 KB memtable page (256 kv64 rows): batch at 200.
• sweep in index/index.js is the one place that knows both exts; no second sweep function.
• Worktree-scope ONLY for HEAD rows: the committed blob's pairs are a different fact (blob-keyed) and NOT this ticket; BEE-025 needs what is on disk.

WIP

Design decisions

• RULED (gritzko 2026-08-19): one kv64 lane, many kinds, same as lite2 — not a per-purpose file; .kv.idx is the ext, kv the name.
• RULED: meta pair keys verbatim, YAML keys hashed, two kinds.
• Carried from TODO-003: path-major (a whole file in one prefix), literal-or-hash values, mtime mark, tombstones over delete; dropped from TODO-003: the store shard (bee has a gitdir), the wt-code-in-key (the gitdir is the sharing unit, the wt code rides the HEAD row).
• A Now: query is a full lane scan under path-major order — cheaper than the stat sweep before it; a key-major twin kind is the fallback if it shows.
• OPEN (gritzko): the YAML status key name for Markdown tickets (status:?) — the board needs ONE spelling per syntax.
• OPEN: does bee index run the sweep eagerly (a full wt stat pass on every index run) or only the lazy path? Default: lazy + bee index runs it so the summary line is honest.

TODOs

• index/index.js: KV_EXT = ".kv.idx", sweep spares both exts, openKv(gitdir, ro)
• index/kv.js: pack/unpack, key codes (verbatim + hashed), value normalizers, put/tomb/mark
• the sweep: candidate set (stage .md/.mkd + untracked via the status walk), mtime-vs-mark, block reconcile, mark last, batch 200
• the lexer: meta pairs under a StrictMark title; YAML preamble at line 1
• find(repo, filters): scan, match LIT/HASH/prefix, intersect on path_hl, map back to paths; collision detector
• bee index summary line: kv: N files, M pairs; bee lindex-style bare query verb? (no — todo is the query door)
• test/kv/run.sh: cold fill, warm skip, edited file re-lex, vanished pair tomb, deleted file, YAML file, two worktrees of one repo, crash mid-sweep re-lex, sweep does not eat lite2 and vice versa
• INDEXES.mkd: a kv section — kinds HEAD/YAML, the vkind table, the mark

Blockers and bummers

• sweep's "EXTENSION IS THE FORMAT" rule (BEE-002) must be generalized FIRST or the first bee index after this lands unlinks the kv runs.

Outcome