Beagle SCM

JS-022 MED: generic mmap LSM index — abc.index (point / range / prefix)

The lane containers (HASH/HEAP/HIT) expose the LSM primitives but not a reusable index. Add abc.index(lane, opts) — a stack of mmapped sorted runs + a memtable, with point/range/prefix query and ladder compaction. This is the substrate the pack-log index (JS-024 git.pack) builds on. Method: Issues.

Input

Context

• js/cont.cpp binds HASH (sets/maps), HEAP (sorted runs + sort), and

  HIT abc.merge/abc.intersect — the LSM building blocks — but NO index control structure or query verbs over a stack of persisted runs.

• Grounded in the real indexes (same LSM stack, differ only by lane): keeper

  wh128 puppy registry (KEEPLookup point/prefix, KEEPCompact 1/8 ladder) and spot wh64 trigram (CAPO SeekRange). Design pass produced this scope.

Goals

• abc.index(lane, opts) → object owning runs + memtable, with .put /

  .flush / .compact / .get (point) / .range / .prefix / .seek.

• IN-MEMORY vs ON-DISK by opts.dir: *present* → persisted file runs

  (abc.book, re-opened on open); *absent* → ephemeral io.ram runs (no files, gone at exit). Same leaves/memtable/compaction; only run backing + persistence differ (JS-side branch).

• Push queries stream hits through an IN-FRAME callback (.range/.prefix,

  rule #4). PULL via .seek(needle) → a cursor (.next / .key / .val / .entry) to read several entries at your own pace, stop anytime.

• Prefix is HALF-OPEN [p, p+2^lowBits) (spot-style). Hits are a BigInt,

  [k,v] pair, or blob per lane.

Constraints

• Write path PURE JS: flush = memtable.sort() → abc.merge into a fresh

  run → push onto .runs. ON-DISK run = abc.book file (close msync+trim, rename, re-open RO); IN-MEMORY run = io.ram (kept as-is, no file). Runs are immutable (growth = new runs).

• .seek(needle) is a PURE-JS k-way merge cursor: per-run positions seeked via

  _findge_<lane>, a tiny JS heap + newest-wins dedup, reading lane elements off the run typed-array views; JS owns ALL cursor state (rule #4 — no held native state, like the ULOG/HUNK cursors). .range/.prefix may be sugar over it or keep the native _seekrange_ fast-drain — implementer's call.

• 3 new hit.hpp-sized leaves in a new js/index.hpp (registered in

  JABCContInstall): _findge_<lane> (binary-search point, mirrors KEEPLookup), _seekrange_<lane> (binds HIT<lane>SeekRange), _compact_<lane> (binds HIT<lane>Compact — the 1/8 ladder, keeper/spot parity, full-element dedup + newest-run-wins). NO keyed/tombstone compaction.

• u64 keys cross as BigInt; needles + lo/hi are BigInt; mind lane

  alignment on re-mmap; validate byteLength % (w*BPE) === 0 on open.

WIP

Design decisions

• (open) abc.index lives in the JABC_CONT_JS bundle (like Buf/pol);

  js/index.hpp holds the 3 leaves. v1 deletes = newest-run shadows older at query time (no tombstone drop) — DOCUMENT it.

TODOs

• First be get --nosub onto current trunk (JS-021 landed); build.
• Repro-first: js/test/index.js (model on hit.js) — build wh128 + u64

  runs, flush, compact to the 1/8 ladder, assert POINT/RANGE/PREFIX vs a brute-force JS oracle. Red before, green after.

• Add the 3 leaves + the abc.index object; doc INDEX.md / API.md.

Blockers

• None. Mass-indexing (keeper/pack-log) is out of scope → JS-024.

Outcome

• LANDED 5d162989 (2026-06-21): abc.index(lane, opts) — generic mmap LSM

  over u64/wh128: put/flush/compact (HIT<lane>Compact 1/8 ladder)/ get/range/prefix (half-open)/seek. ON-DISK (opts.dir → abc.book files, re-opened) vs IN-MEMORY (no dir → io.ram, ephemeral). .seek(needle) = pure-JS pull cursor (per-source _findge_ + JS min-heap + newest-wins dedup; rule #4, memtable snapshot on seek). 3 leaves in js/index.hpp; flush path pure JS; added io.rename/io.mkdir. js/test/index.js (JABCindex) vs a brute-force oracle red→green; ctest -R JABC 20/20, ASAN/LSan clean.

• NOTE: .range/.prefix (native _seekrange_ drain) do NOT dedup across

  runs; .seek does. kv64 + keyed/tombstone compaction deferred.