Beagle SCM

JAB-035: embed a default jsrc/ pack in the binary — the permanent floor of the jsrc stack

A prebuilt jab binary is useless alone: every bareword dies with "cannot find jsrc/main.js" until the user clones a jsrc/ checkout — the install is a two-step. Let the binary CARRY a default pack of .js files and append it as the LAST entry of the frozen jsrc stack, so it backfills per-file under any user jsrc/. That both collapses the install to "download, run" and makes incremental extension the norm: a user's jsrc/ holding a single verbs/foo.js overrides that one file, every other bareword falls through to the pack.

Input

Context

require.cpp resolves barewords by the once-frozen upward climb (pinJsrcStack, GET-041); resolveJsrc already falls through the stack PER SPEC, so overlay semantics come for free — the floor just has to be on the stack.

• jab/require.cpp pinJsrcStack() — the only integration point; the floor is appended after the climb, unconditionally.
• Interior requires are barewords against the same stack, so a pack file requiring core/x.js picks up a user override too — overrides compose.
• A full checkout shadows the pack completely (it has every file); dev setups behave exactly as today.
• Payload is small: the whole Beagle jsrc is 106 files / 1.5 MB raw, ~350 KB deflated; zip.cpp (libz) already links.
• Embedded-JS-in-binary is house style (require.cpp, buf.cpp string literals); this adds a generated byte array, not a new mechanism.

Goals

A release jab that runs be status out of the box with no checkout, while any jsrc/ up the chain overlays it file-by-file.

• Build mode: a cmake option, e.g. -DJAB_JSRC=<dir>, packs the given tree (deflated file table + bytes) and attaches it to the binary (mechanism: see Design decisions). REQUIRED: without the option the build has no floor and behaves exactly as current jab.
• Runtime: after the climb, always push the floor; on first resolve, extract the pack to $HOME/.cache/jsrcs/<contenthash>/ ($XDG_CACHE_HOME respected; temp dir + rename, atomic; skip if present) and push that real dir.
• Extraction yields a real directory so io.mmap loads, argv[1] pinning, _here derivation and beside-the-verb asset reads all work UNMODIFIED — no virtual fs.
• jab/API.md + jab/INDEX.md + both READMEs updated; the two-step install text collapses.

Constraints

• The stack stays frozen at startup (GET-041); the floor is part of that one-time pin, never re-scanned.
• Cache dir keyed by the pack CONTENT hash (it is in the preamble): same jsrc rebuilt → same dir reused, changed jsrc → fresh dir; .jsrcs/ is thus a content-addressed store, stale hashes GC-able. The leaf dir is <hash>, not jsrc — the climb can never pick it up. Never extract into ~/jsrc (user territory, would shadow-by-existence forever).
• No new dependencies: libz deflate for the pack, existing io/buf leaves for extraction.
• Release CI decides where the Beagle snapshot comes from (pinned rev passed as -DJAB_JSRC); jab's tree grows no beagle submodule.

WIP

Design decisions

Always-append, per-file fallthrough — overlay is the extension mechanism, not a hazard.

• Empty-only arming was REJECTED: the point is extending jab/be incrementally from a partial jsrc/, no full checkout needed.
• Embedding: .incbin in a 3-line .S — uniform gcc/clang, ELF+Mach-O, codesign-safe, no generated .c. One tiny C accessor exposes the start/end symbols as a global Buf; everything else runs in the require bootstrap JS.
• The appended-trailer variant was REJECTED: arm64 macOS requires a signature (ld ad-hoc-signs automatically, no dev account), the signature must end with the file, codesign refuses trailing data. Dies with it: the "re-pack any binary sans toolchain" perk.
• Pack format: ustar + zip.deflate (ZLIB wrapper — ZINF.c is inflateInit, NOT gzip) behind a 16-byte preamble: magic, version, raw length, content hash. Preamble raw length pre-sizes the inflate buffer, no NOROOM/grow dance.
• Pack step is SELF-HOSTED: cmake post-build runs the freshly built bin/jab with a JS pack script — sorted walk, ustar written in JS (~30 lines, zeroed mtime/uid), deterministic bytes so the content hash is stable. Zero new build deps.
• Compression stays jab's own deflate: level 9 vs default 6 is ~3-5% on 1.5 MB of JS; a real win needs zstd/xz — a NEW dependency, rejected.
• Dev builds (no -DJAB_JSRC) have no floor — zero behavior change, the pack only matters for release binaries.
• Residual skew: an overlay pinned against pack N meets N+1 siblings after a binary update — inherent to overlays, same as stacked jsrc dirs today; at most warn later.

TODOs

• cmake: -DJAB_JSRC=<dir> → bootstrap jab_boot builds the pack, .incbin .S links into jab
• JS pack script jsrcpack.js: sorted walk → deterministic ustar → zip.deflate → preamble
• C accessor jsrcpack.cpp: symbols published as global jsrcpack, a no-copy Uint8Array (absent → no floor)
• JS unpack in require bootstrap: preamble-sized inflate, ustar parse, .tmp-<pid> + atomic rename
• pinJsrcStack: probe $XDG_CACHE_HOME||$HOME/.cache + /jsrcs/<hash>/, extract, append the floor
• tests: test/jsrcpack.js + test/run-035.sh + test/packtoy/ — no-floor, cold/warm floor, overlay
• jab/API.md, jab/INDEX.md, jab/README.md install section
• be/README.mkd two-step install text — other repo, left for a follow-up
• release CI: be's release.yml builds jab from source with -DJAB_JSRC=<git archive of the release rev>, smoke-tests the floor from a fresh HOME, a binaries matrix mirroring jab's release (be-linux-x86_64, be-linux-arm64, be-macos-arm64; ninja, clang on linux), gated by the suite job, each smoke-testing the floor from a fresh HOME; RUNS GREEN ONLY AFTER JAB-035 LANDS in ///jab (the smoke step catches an unpacked jab); jab already CIs on macOS — only the .S APPLE branch is unproven (no jab workflow passes -DJAB_JSRC)

Blockers and bummers

• First run on a read-only $HOME/no-$HOME box cannot extract — needs a fallback dir decision ($TMPDIR?) or a clean error.
• Concurrent first runs race on the cache dir — temp+rename makes the winner atomic, losers must tolerate rename EEXIST/ENOTEMPTY.

Outcome

Implemented in //JAB-035 (base ///jab/#c7c0c495), 16 files staged, NOT committed.

• Suggested commit: JAB-035: embed a default jsrc pack as the stack floor
• Verified: 115/115 tests both with and without -DJAB_JSRC (113 baseline + 2 new); cold start from a scratch $HOME extracts to .cache/jsrcs/<16hex>/ and resolves; warm start skips extraction; overlay: one local jsrc/ file wins, siblings from the floor; 8 concurrent cold starts → one dir, no temp litter; orchestrator re-ran cold+overlay by hand, both good.
• Deviations: global is a plain Uint8Array jsrcpack (typed arrays are the boundary currency), magic is "JSR"+1-byte version, hash dir is 16 hex chars; unwritable/absent cache is non-fatal — floor skipped, reason appended in plain words to the resolve error, no $TMPDIR fallback; packer skips non-regular files (fifos/sockets in a real be tree killed io.mmap).
• Real-tree numbers: packing be (677 files, 6.0 MB) → 2.0 MB pack in 0.22 s.
• STATIC be (2026-08-11): musl + static JSCOnly (webkitgtk 2.52.5, -DPORT=JSCOnly -DENABLE_STATIC_JSC=ON) + embedded pack = 43.7 MB stripped, zero deps — but ICU: distro libicudata.a is a 64-BYTE STUB, Intl silently reads /usr/share/icu/*/icudt*.dat when present and SIGABRTS on Intl.DateTimeFormat without it.
• Fix: slim the host .dat with icupkg -r (drop locale-ID bundles only — nuking res_index/zoneinfo64/supplemental SEGFAULTS), genccode -e icudt<V> → linked as icudt<V>_dat; root+en = 13 MB, total 56.8 MB, ICU_DATA=/nonexistent probes all pass (de falls back to root).
• be/.github/static-be.sh + release.yml static job (alpine:3.22 container, actions/cache'd JSC archives+headers, asset be-linux-x86_64-static-<tag>); NO static mac. Verified on the journal repo: status/log/work/todo views render.
• CI job replayed in qemu VM (alpine:3.22 via podman, v0.1.2 tag): cold JSC ~50 min at RAM-capped -j, warm cache → minutes; produced 54.6 MB binary runs on BARE UBUNTU (statically linked, floor extracts to the same deterministic hash, Intl works with ICU_DATA=/nonexistent). Bare-image lessons baked into the script: apk needs xz, file, curl-dev; ninja jobs capped by MemAvailable/2.5 GB (FTLLowerDFGToB3.cpp OOMs otherwise).