Beagle SCM

JAB-032: fsw drops the wd, swallows queue overflow

FSWDir throws away inotify_add_watch's return and FSWDrain never reports ev->wd, so every event reaches JS as {wd: 0, name} and the watcher fd is the only dir identity left — JAB-031 shipped one wfd + one pol handler + one drain Buf PER DIR, which its own Goals said it would not do. FSWDrain also skips every ev->len == 0 record, and that is exactly IN_Q_OVERFLOW: a burst past max_queued_events drops events silently, so a consumer cannot tell "nothing changed" from "I lost your changes". Both block the per-repo cache, BRO-043.

Input

Context

The consumer is the per-repo cache in the long-lived pager/watcher process: a bucket per repo dir (jab/dog/abc, jab/dog, jab), dropped with its ancestors on any event under it.

• dog/abc/FSW.c:36-40 — int wd = inotify_add_watch(...), checked for < 0, then discarded; FSWDir returns bare ok64.
• dog/abc/FSW.c:59 — if (cb && ev->len > 0): IN_Q_OVERFLOW (wd -1, len 0) and IN_IGNORED (watch died) are both dropped on the floor.
• jab/fsw.cpp packs a wd slot into the drain Buf and fsw.records parses it, but the value is a hardcoded 0 — the framing is right, the source is missing.
• jab/INDEX.md already carries the admission as an owed fix-up: "one wfd per dir is the only way to tell dirs apart".
• inotify's own model is one fd for a whole tree with wd as the dir key; the current shape burns one fd + one JS handler + one 64 KiB Buf per watched dir against RLIMIT_NOFILE, on top of the 8192 max_user_watches cap.
• The mask (FSW.c:37-39) has no IN_DELETE_SELF/IN_MOVE_SELF, so a watched dir being removed or renamed away is invisible to the cache.

Goals

One watcher fd for many dirs, every event attributable to the dir it fell in, and a loud signal when the kernel dropped events.

• FSWDir(wfd, path, i32 *wd) returns the watch descriptor; FSWcb takes (i32 wd, u8cs name, void *ctx).
• FSWDrain reports len == 0 records instead of skipping them: overflow surfaces as wd == -1 + empty name, a dead watch as its own wd + empty name.
• fsw.dir(wfd, path) returns the wd to JS; fsw.watch keeps ONE wfd, a wd -> dir map and one drain Buf, arming further dirs on the same watcher.
• Repro-first: two dirs on one wfd, a file created in each, the two records carry the two distinct wds fsw.dir handed back (red today — both 0).
• dog/abc/FSW.{h,md}, dog/abc/INDEX.md, jab/API.md, jab/INDEX.md updated; the INDEX.md fix-up note goes away.

Constraints

Two repos: dog/abc/FSW.{c,h,md} + dog/abc/INDEX.md in dog, the binding and docs in jab. Every native op keeps its call() (JAB-008).

• FSWDir's signature change is source-breaking; FSW.c has no consumer outside jab/fsw.cpp, so the blast radius is the binding.
• kqueue has no watch descriptor: report the pinned dir fd as the wd (it is already the dir's identity there) and keep the empty-name rescan contract.
• No macOS/BSD box here — the kqueue arm ships read-only-reviewed, flagged untested, exactly as JAB-031 left it.

WIP

Design decisions

• Consumer contract (gritzko 2026-08-02): the watcher starts off the event loop once the project root is known, and the cache object exists only under a live watcher — a failed start leaves it null and everything is recomputed, so a missed event can never be read back as truth.
• wd is i32, not u32: -1 is the overflow sentinel and must not wrap into a plausible descriptor.
• Overflow is a record, not an error return — the drain must deliver the events it already read before the loss marker, and the JS consumer decides (drop everything) rather than the leaf.
• IN_DELETE_SELF/IN_MOVE_SELF added to the mask: a vanished repo dir is a cache drop, and the resulting IN_IGNORED rides the same len-0 path.

TODOs

• repro test/fsw.js §4: two dirs, one wfd — red pre-fix ("fsw.dir returned no wd: 0,0"); §5: 16384 creates — red pre-fix ("16384 records, no overflow marker"), re-verified by toggling the ev->len > 0 skip back
• FSW.h/FSW.c inotify: FSWDir out-param wd, FSWcb wd arg, len-0 records delivered, FSWOVERFLOW const, mask += DELETE_SELF/MOVE_SELF
• FSW.c kqueue arm: the pinned dir fd IS the wd (ev.ident hands it back); untested, no BSD box
• jab/fsw.cpp: fsw.dir returns wd, collector packs the real wd, record parsed as getInt32
• JS bundle: fsw.watch on ONE lazily-armed wfd + wd -> dir map + subs table; fsw.OVERFLOW, fsw.onoverflow, fsw.stop
• docs: FSW.h, FSW.md, dog/abc/INDEX.md, jab/API.md, jab/INDEX.md (fix-up note dropped)
• ctest 108/108 green (was 108/108 at base) incl. JABCfsw; test/fsw.js runs in 0.25s

Blockers and bummers

• Arming is LAZY (gritzko 2026-08-02): a dir nobody cached has nothing to invalidate, so a cache MISS is what subscribes — no eager tree walk, no max_user_watches wall on dirs no view ever touched.
• Ordering the consumer owes: arm the dir BEFORE computing the value, then store. Arm-after-compute leaves a window where a write lands between the read and fsw.dir and the entry is born stale.
• Overflow is now reported but not prevented: max_queued_events (16384) is a sysctl, and a ninja build under a watched tree will hit it.
• pol.init() wipes the fd table and silently strands the shared watcher — the bundle still thinks it is registered, so events stop arriving with no error. fsw.stop() must precede any pol.init(); found by the §6 test, which failed exactly that way.
• A Buf too small for the burst still throws out of the raw fsw.drain (JAB-031's contract, unchanged); the fsw.watch sugar catches it and routes it to the same lost-events path, since "my Buf overflowed" and "the kernel's queue overflowed" are one fact to a cache.
• fsw.unwatch(wd) forgets a dir but cannot remove the kernel watch (FSWUndir deleted under ABC-013), so its events keep arriving unclaimed — a long-lived watcher over a churning tree leaks kernel watches until fsw.stop().

Outcome

Implemented in wt //JAB-032 (base ///jab/#2980a754), NOT staged and NOT committed — awaiting inspection.

• //JAB-032: fsw.cpp, API.md, INDEX.md, test/fsw.js.
• //JAB-032/dog/abc: FSW.c, FSW.h, FSW.md, INDEX.md.
• Suggested messages, one per repo: JAB-032: one wfd per tree, report queue overflow.
• //JAB-032/dog/abc/Bx.h bAllocate: typed malloc replaces the Balloc u8** laundering — strict aliasing hoisted the head load in C++ TUs and memset(NULL)'d (JABCindex/JABCpackidx segfaults; 107/107 after). Suggested message: JAB-032: bAllocate stores the block in its own type. May also cure the optimized-build listTopics segfault — same miscompile class.