Beagle SCM

JS-040 MED: bin/lib/ingest.js — pack ingest (keeper write side)

Land a received packfile into the shard: resolve every object (incl. REF_DELTA against the existing store / in-pack bases), re-encode OFS-only into a fresh NNNNN.keeper pack-log, and index it into the shard's wh128 LSM. Mirrors keeper/UNPK.c + KEEPIngestFile (unlinkable). Part of JS-038.

Input

Context

• The keeper store is OFS-only pack-logs + an abc.index wh128 lane.

  keeper.js READS them but indexPackByWalk SKIPS REF_DELTA ("unresolvable in pure JS") — false: _pack_inflate the delta + getObject(baseSha) + git.delta.apply resolves it. Received git packs ARE thin (REF_DELTA).

Goals

• ingest(repo, packBytes): walk records (_pack_next/_pack_type), resolve

  raw/OFS via _pack_resolve, REF via inflate+apply against in-pack map or keeper.getObject; re-feed full bytes through pack.feed (OFS-only log).

• Index the new log into the shard (pack.scan → abc.index.put/flush), so a

  follow-up keeper.getObject of any landed sha succeeds.

Constraints

• libdog+abc ONLY. Write a new NNNNN.keeper beside existing shard packs;

  never mutate a landed pack. Re-encode deflates via pack.feed (or JS-035).

WIP

Design decisions

• Two-pass: pass 1 builds the received pack's sha→offset map (raw+OFS first),

  pass 2 resolves REF bases (map ∪ store) — the UNPK dep-order, in JS.

• Un-stub keeper.js REF_DELTA on the read path too (shared resolver helper).

TODOs

• Repro-first test/js/get/ingest.js (helpers test/js/lib/): ingest a real

  thin pack (fixture from test/get/), then keeper.getObject each tip object re-hashes to its sha; OFS-only log re-scans clean (no REF left).

• Implement bin/lib/ingest.js + REF resolver; reindex the shard.

Blockers

• ← JS-035 (deflate, re-encode robustness), JS-039 (supplies real packs).

Outcome

• (pending)