Beagle SCM

JAB-033: mmap/Buf objects must tell the collector what they pin — GC never feels a leaked mapping

Now

DONE

A jab mapping is a Uint8Array over foreign pages, and the collector is told NOTHING about them: 62749 live RO maps of a 183-byte file survived two hours in one process without a single collection (BRO-046). JSC lets an object declare that it pins a native resource; jab's map/Buf objects must do it, so a program that maps in a loop gets collected instead of dying at vm.max_map_count. The unit that matters is the MAPPING COUNT, not the byte size — 62749 VMAs is fatal while the 250 MB they carry is nothing.

Input

Context

Found while diagnosing BRO-046: an idle be todo pager leaked 19.2 RO ULOG mappings/sec and wedged at 65531 VMAs, 92% of its CPU in a kernel mmap that could no longer succeed.

• io.cpp:632 hands a mapping to JS as JSObjectMakeTypedArrayWithBytesNoCopy with a JABCMapROFree/JABCMapFree deallocator — the ONLY release path for an RO map.
• io.cpp:642 io._munmap is a documented SILENT NO-OP for a read map (ABC-023: no fd, no slot) — so RO mappings have NO explicit release at all, GC is the whole story.
• GC does not run: 62749 live 4 KiB maps = ~250 MB of pages plus a few MB of JS wrappers, far under any heap trigger, so the allocator never feels pressure.
• JABC.hpp:12-17 includes only the six PUBLIC JSC C headers; the sole collector control in them is JSGarbageCollect (JSBase.h:140).
• Unknown to be MEASURED first: whether …BytesNoCopy reports its external size to the collector at all under this JSC build — the observation says it is not enough either way.
• Same exposure for every other pinning binding: abc.mmap/io.mmap, git.pack.mmap, abc.ram/io.ram, and any Buf holding a booked fd.

Goals

A JS program that maps in a loop and drops the handles must be collected before it exhausts VMAs or fds — the collector must know a mapping is expensive.

• Red-first probe: map one small file N times in a loop, dropping each handle; count /proc/self/maps lines. Grows without bound today; must plateau.
• Pressure is counted in MAPPINGS and FDS, not only bytes — a 4 KiB map costs one irreplaceable VMA slot and (for RW) one fd.
• The same accounting covers abc.ram and git.pack.mmap, not just io.mmap.

Constraints

jab/ is read-only reference for everything else in the fleet; this ticket is the exception that edits it, and only the binding layer.

• Public JSC C API only unless gritzko rules otherwise — if the pin has to be declared through an internal JSC::Heap entry point, STOP and report the tradeoff before writing it.
• A JSGarbageCollect call driven by a jab-side pinned-resource counter is an acceptable shape; a collection on EVERY map is not.
• Reuse the existing FILE slot book (FILEBookedFD, JABCMapCtx) for the count — no parallel registry of live mappings.
• This is the ENGINE half only: the JS-side "do not open it in the first place" work is BRO-046 and BE-050, the explicit-release half is ABC-020.
• Clone ///jab/ from the main tree per work; stop and report on any clone failure, no improvised fallback.

WIP

Design decisions

TODOs

• measure first: does a …BytesNoCopy array report ANY external cost to this JSC? probe with a map loop + heap stats
• red repro test: N × (map a small file, drop it) → /proc/self/maps line count, unbounded today
• declare the pin for io.mmap/abc.mmap RO and RW maps; count mappings and fds, not just bytes
• extend to git.pack.mmap and abc.ram/io.ram
• threshold-driven JSGarbageCollect if the pin declaration alone does not move the collector
• re-run the BRO-046 idle-pager probe on the rebuilt jab; report the VMA slope
• update jab/INDEX.md + jab/API.md if the binding surface changes; full suite green

Blockers and bummers

Outcome