jab get of a big remote holds the entire packfile in process memory: wire.js fetch reads the raw pack to EOF into a chunks[] array and then CONCATENATES it into one Uint8Array — peak RSS 2× pack size — and 3.5 GB) peaks at ~7 GB and OOMs modest boxes. Fix = git's own model: stream the pack into a ingest writes that single buffer out afterwards. Observed live this session: jab get ssh://git@github.com/torvalds/linux.git (pid 14199) at RSS 2.78 GB after 2.7 GB received, fd table shows NO store file open (only the ssh pipe) — a Linux-scale pack (tmp_pack_* file INSIDE the destination shard, hash as it streams, index/ingest off mmap, atomically rename into place on a verified trailer. Method: work.
jab get ssh://git@github.com/torvalds/linux.git from ~/tmp/try; ssh child moving ~2.8 MB/s; jab RSS tracked received bytes 1:1 (2 785 424 kB RSS vs rchar 2.7 GB); /proc/14199/fd = tty, urandom, pipe, timerfd — no store fd, nothing durable until EOF.be/shared/wire.js "read the raw pack to EOF" (fetch chunks.push(...) then out.set(c, off) — chunks + out live simultaneously, so peak = 2× pack.be/shared/ingest.js (clone/add → writeBytes, e.g. 0000000001.keeper) takes the whole buffer; verbs/get/get.js L208/L218/L230 passes f.pack around by value.JS exception: RangeError: length too large / Uint8Array@[native code] / readToEof@ / fetch@ / seedRemote@ / handleSeed@ / get@ — the pack exceeds the engine's max Uint8Array length, so ANY large-repo clone deterministically crashes after the full download (~50 min discarded). Not just an OOM risk; bumped MED→HIGH.wire.fetch streams the incoming pack to a tmp file (e.g. tmp_pack_*) in the DESTINATION shard dir, constant memory, hashing the trailer as it streams.ingest.clone/ingest.add accept the tmp-file path (read/mmap), and land it by ATOMIC rename once the pack sha trailer verifies; abort = unlink the tmp file, store untouched./tmp (often tmpfs = RAM again, and cross-device rename isn't atomic).be/, run via jab); keep the fetch() -> {pack, refs, ...} callers working — the small local/be:// paths may keep the in-memory shape if that simplifies, but the ssh/git wire path must stream.GET-044 comments; ticket-coded ≤64-char commits.tmp_pack_<rand> in the shard, verify the 20-byte sha1 trailer, then rename into the keeper log name (or hand the verified file to ingest).test/wire_stream.js (be-js-unit-wire_stream): RED on pristine wire/ingest, GREEN on fix; covers stream+head reassembly, verified-flag skip, corrupt-trailer abort, sha1s-vs-native.wire.js drainToFile streams the spawn-path pack to tmp_pack_* in the destination shard; incremental pure-JS SHA-1 (shared/util/sha1s.js, 183 MB/s) trails 20 bytes behind, trailer verified at EOF.ingest.js clone/add/land take {packFile, packLen, verified}; atomic rename + trailer strip; abort unlinks; 2^31-1 mmap-cap guards refuse cleanly. get.js seedRemote resolves the shard BEFORE fetching; submount.js streams child fetches. HTTP/small-local paths keep the in-memory shape.|0 int32 cursors cap io.mmap at 2^31-1 — the landed 6.4 GB keeper can't be indexed/read) and GIT-021 (git.tree: bad tree entry mid-checkout at medium scale, stateful read bug). After JAB-007 lands, lift the MMAP_CAP guard in ingest.js..beagle-ext moved 73fc0c91 → 46af906f across its posts (its rows all carry GET-044).GET-044 (wt /home/gritzko/src/GET-044, store ~/.be): parent commits f9a9c781 + 68d3b802, sub committed post-order, trunk untouched. PENDING user inspect/merge; fixtures (linux-mirror.git, medium.git, ~17 GB hardlinked) still in the wt.