Beagle SCM
JS-121 DONT: un-defer the kv64 abc.index lane — keyed compaction (newest-wins per key), get/range/seek
kv64 (dog/abc/KV.h: (u64 key, u64 val), KEY-ONLY Z/hashEq) is
already registered in jab's container bindings — HEAP_REG/
HASH_REG/HIT_REG(kv64) (heap.hpp, hash.hpp, hit.hpp) — but the
LSM surface abc.index serves only u64 and wh128: kv64 is
explicitly DEFERRED (index.hpp 28, API.md 396) because compaction
and .seek dedup are FULL-ELEMENT newest-wins, which a key-only
comparator breaks. Build the keyed variant: one live val per key,
newest wins, older same-key rows collapse. This is the true-map twin
of wh128's multimap — the LWW lane ANNO-001's status atoms want.
Context
- kv64Z compares keys only, so same-key rows are Z-equal ties; the
HIT ladder's "newest in the stream wins on a tie" already picks a
winner — it just doesn't DROP the losers (full-element dedup).
- Query today:
get scans runs newest-first (shadowing at query
time); .seek merges via a min-heap with full-element dedup —
both are one keyed-equality rule away from map semantics.
- v1 deletes (API.md): a newer run shadows an older at query time,
compaction never drops — the tombstone mechanism is being ruled
in ANNO-001 TODO-1 (negative runs vs sidecar mask).
- ANNO-001's multimap rides wh128 (full-pair Z); single-val-per-
key annos (a ticket's ONE status atom, LWW) fit a keyed lane.
- TODO-003 is the second driver: its board query
Sta: OPEN streams a prefix block, and with overwrites never
collapsed it returns every ticket EVER opened — i.e. all of them.
- kv64 and wh128 are byte-identical 16-byte pairs (KV.h:19,
WHIFF.h:53) differing only in Z/hashEq, so an existing wh128 run
re-reads as kv64 — adopting the lane costs no data migration.
Goals
abc.index kv64 lane: put/get/range/seek returning ONE live val
per key (newest wins); keyed compaction collapses shadowed rows.
- Same run/mmap machinery: ron60 run names, alignment check,
idxmaint-style ladder — a lane, not a fork.
- Query-time overwrite collapse on
.range/.prefix too, not just
get/seek: the streaming fast-drain is the board's query shape
and today it deliberately skips cross-run dedup.
Constraints
- wh128 and u64 lane behavior stays byte-identical; keyed dedup is
a per-lane property, never a global switch.
- Rule #4: all state in JS, native leaves are marshalling only;
extend HITx.h with a keyed-collapse variant, no second LSM impl.
- Don't paint the tombstone ruling into a corner: a keyed lane
makes key-wide kills natural (newest tombstone per key wins,
evaporates at the bottom run) — keep the merge shaped for that.
WIP
Design decisions
- Keyed collapse = dedup on Z-equality instead of full-element
equality; kv64Z's key-only ordering makes same-key rows adjacent
in the merge, so the collapse is a comparator swap, not new flow.
- Memtable put replaces the same key (HASHkv64 exists);
get =
per-run sFindGE newest-first, first hit wins — today's
shadowing shape, now also the compaction rule.
TODOs
- 1. Table-driven C tests FIRST for the keyed HITkv64Compact:
same-key rows across runs (newest wins), ties inside one run,
torn/misaligned run rejection.
- 2. HITx.h keyed-collapse variant instantiated for kv64; prove
wh128/u64 ladders unchanged (existing tests stay green).
- 3. index.hpp: wire the kv64 lane — put crosses (key,val) as two
BigInts, get/needles/lo/hi as the key BigInt, val returned;
per-run alignment check (16-byte slot, same as wh128).
- 4. JS surface:
abc.index({lane:'kv64'}) put/get/range/seek;
seek exposes .key/.val/.entry like the wh128 lane.
- 4b. Query-time collapse for
.range/.prefix on the keyed
lane: keyed dedup in the seekrange drain, newest run wins;
test a key overwritten in a younger run AND within one run.
- 5. Maintenance: generalize idxmaint.js (EXT/SLOT/lane param) or
add the kv64 twin; ladder-compaction test through JS.
- 6. Docs: API.md kv64 section (deferral note removed), jab
INDEX.md; note the lane choice rule (map=kv64, multimap=wh128).
- 7. Deletes: adopt the tombstone mechanism once ANNO-001
TODO-1 is ruled; key-wide kill = newest-tombstone-per-key,
dropped when the merge reaches the bottom run.
Blockers and bummers
- Only TODO-7 is gated (the tombstone ruling); 1–6 are unblocked.
- Run-age ordering is load-bearing for keyed ties: the ladder must
keep the oldest-first stack invariant or LWW silently inverts.
Outcome
- Open. Filed 2026-07-24, split from the ANNO-001 design chat
(kv64 deferral found while checking jab index lanes).