Beagle SCM

JS-055 BUG: git.pack scratch sized to pack length → resolve/scan NOROOM on an object bigger than the pack

git.pack's _scratch() sizes the resolve scratch to the pack's OWN byte length on the assumption it bounds any single object — false for a compressible blob. One historical ~46 MB tree-sitter parser.c (sha 7ff79400, inflated 46 386 339 B > the 40 220 837 B pack) overflows the 40 MB _bsc → PACKResolveOfs NOROOM → that ONE object makes the WHOLE PIDXScan fail, mis-reported as "ref-delta? out full?". This forces store.js off its fast index path into the O(all-objects) JS walk (~11 s / 2.5 GB on a fresh all-OFS journal pack). Found tracing a jab status "hang". Method: Issues.

Input

Context

• js/cont.cpp PACK_PROTO._scratch(): `n = this.byteLength; _bsc = u8n;

  _dsc = u82n` — comment claims byteLength is "an upper bound for any single object/delta"; an inflated >pack-sized blob breaks that.

• js/pack.hpp:281 JABCpackScan: ANY PIDXScan != OK throws the lumped

  "pack.scan: scan (ref-delta? out full?)" — hides NOROOM-on-scratch behind a ref-delta guess (the pack here has ZERO ref-delta: commit/tree/blob/ofs only).

• Repro store /home/gritzko/beagle-journal/.be/beagle/0000000001.keeper: 37 591

  objects, all 37 590 resolve, the lone 46 MB blob fails; a bigger out buffer does not help (the limiter is _bsc, not out).

Goals

• pk.resolve/pk.scan succeed on any object whose inflated size exceeds the

  pack length (size scratch off the record, not the pack).

• Disambiguate the scan error: NOROOM-grow vs a true REF_DELTA (no sha base).
• Repro-first: a property test resolving an object larger than its pack.

Constraints

• js/cont.cpp (_scratch) + js/pack.hpp (JABCpackScan error) + maybe

  dog/git/PIDX.c/PACK if NOROOM must surface distinctly. Repro-first (CLAUDE.md §17); keep the OFS-only invariant; ASAN/ctest green.

WIP

Design decisions

• Size _bsc to the record's DECLARED resolved size (PACK header carries it;

  pk.size) and grow-on-NOROOM, rather than to this.byteLength; _dsc likewise. Mirror store.js readRecord's grow-and-retry, but at the scratch.

• Surface a distinct NOROOM code from PIDXScan/PACKResolveOfs so the binding

  can report (and the caller can grow) instead of guessing ref-delta.

TODOs

• Repro: property test — feed/resolve an object whose inflated size > pack

  byteLength; assert resolve+scan succeed (currently NOROOM/scan-fail).

• Fix _scratch() to size off the record's declared size + grow on NOROOM.
• Disambiguate the js/pack.hpp scan error (NOROOM vs REF_DELTA).
• Verify pk.scan/pk.resolve green on the journal pack (the 46 MB blob).

Blockers and bummers

• Independent of the lazy-index rework (the store.js perf discussion); this

  bug bites ANY >pack-sized resolve, lazy lookup just stops hitting it per-run.

Outcome

• LANDED 1c70e9d8: _scratch(need) sizes off pk.size (record's declared

  resolved size), a _grow(need, op) wrapper re-runs doubling the scratch on the binding's NOROOM throw (capped 40 tries); resolve/scan/feed all route through it. js/pack.hpp surfaces NOROOM vs REF_DELTA as DISTINCT throws (PACKResolveOfs/PIDXScan codes), no more "ref-delta? out full?" guess.

• Repro js/test/pack.js (over-pack resolve, 2 MB inflated into a 256 KB pack) +

  js/test/packidx.js (over-pack scan + REF_DELTA-not-misreported). Full JABC suite 29/29 + PACK C tests green under ASAN.