ULOG-002 MED: ULOG open must back-scan the zero pad, not forward-scan the whole log

ULOGOpenBooked (dog/ULOG.c:494) forward-scans the ENTIRE log on every open via ulog_scan_log (:196, called :537) only to position the buffer's DATA/IDLE division past the last complete row — an O(content) row-by-row walk paid on every be/jab command that touches refs/wtlog. The index that scan builds is then discarded (ulog_idx_free_anon, :538); the real idx is loaded/rebuilt separately by ULOGOpenIdx (:555) against a freshness-checked .idx sidecar. The tail only ever needs finding at the zero pad: a booked file is grown to a page boundary and abnormal termination leaves trailing NULs, so the last row's end is O(page) back from the mapped end. Replace the forward tail-scan with a BACKWARD scan over the zero pad. Surfaced while reviewing JS-073 (JS ingest.add crash-safe refs append) and relates to the ULOG-001 torn-log / initial-NUL work. Method: work.

Input

Context

ULOGOpenBooked (dog/ULOG.c:494) always runs ulog_scan_log (:537) to set the byte size, then frees its scratch idx (:538) — the forward walk's only surviving outputs are the DATA/IDLE level and a strict-ts re-verify.
ulog_scan_log (:196-244) walks [base, first-NUL) row by row via ULOGu8sDrain — O(content), not O(zero-tail); it has no last-byte or backward fast path.
RW open already grows the file to a page and maps MAP_SHARED (FILEBookFD FILEResize→map_size, abc/FILE.c:877-879); the trailing [content_end, map_size) is the page-aligned zero pad.
Clean close trims via FILETrimBook (abc/FILE.c:1045) → FILEResize(fd, u8bDataLen) + b[3]=b[2]; abnormal exit skips it, leaving the zero pad the next open must re-level.
The .idx sidecar already lets ULOGOpenIdx skip the idx rebuild when fresh (pre_mtime), so the forward tail-scan is redundant work even on the sidecar-hit path.

Goals

ULOG open finds the DATA end by scanning BACKWARD over the trailing zero pad (make the NULs IDLE), not by a forward walk of the whole log — O(zero-tail) ≈ O(page), not O(content).
RW open: file expanded to page len, mmapped MAP_SHARED, back-scanned so the zero pad is IDLE (writable append room) and DATA = the real rows.
RO open: mmapped PROT_READ (NO expand — never grow the file behind a concurrent writer), back-scanned to set the DATA end; zeros beyond are ignored (no IDLE, no appends).
Close trims the file to PAST+DATA length (drop IDLE) — the real content end, dropping the zero pad.

Constraints

A partial trailing row (interrupted append: the last non-zero byte is not a row terminator) SELF-HEALS: back up to the previous \n, drop the incomplete row, open with the complete rows (the original graceful NODATA-drop behavior) — do NOT fail(ULOGTORN) on it (that would brick a log on every crash-mid-append). Genuine mid-content corruption (a NUL with real bytes after it) stays caught by the stale-sidecar forward ulog_scan_log rebuild; do NOT add an O(content) embedded-NUL scan to the fast path — the goal is O(page) opens.
Trim target is PAST+DATA (b[2]-b[0]), dropping only IDLE ([b[2],b[3])); current FILETrimBook uses DATA len (b[2]-b[1]), correct only while PAST is empty (ULOG maps with b[0]==b[1]) — keep that invariant or trim PAST+DATA explicitly.
Reader-visible result must be identical to the forward scan on a well-formed log: DATA ends exactly after the last complete row's terminator.
C / libdog only (dog/ULOG.c, abc/FILE.c); repro-first per abc; keep the ULOG ctest suite green.

WIP

Design decisions

Back-scan: from the mapped end (b[3]/map_size) skip trailing NULs to the last non-zero byte; a \n there → DATA ends after it, IDLE = the skipped pad; a non-\n (partial trailing row) → continue back to the previous \n and drop the partial row (self-heal). No O(content) embedded-NUL scan — opens stay O(pad).
A non-NUL last on-disk byte means a cleanly-trimmed file (no pad): DATA end is content_end with a zero-length back-scan — the common clean-close case, O(1).
Keep the .idx sidecar freshness path as the index source; the back-scan only sets the buffer level, it need not rebuild the idx.

TODOs

Repro-first: T_backscan_zero_pad (dog/test/ULOG.c:1208) — zero-pad open, DATA = last complete row, IDLE = the pad — landed df3b5885.
Repro: partial trailing row SELF-HEALS — T_backscan_torn_partial_row (dog/test/ULOG.c:1286), incomplete append dropped, not ULOGTORN — landed df3b5885.
Forward ulog_scan_log tail-find replaced by ulog_backscan_tail (dog/ULOG.c:262, called from booked :574 and RO :617 opens); the pre-scan throw-away anon idx alloc removed — landed df3b5885.
Close trims to PAST+DATA: FILETrimBook now resizes to b[2]-b[0] (abc/FILE.c:1045-1054, ULOG-002 comment) — landed abc e8d7c59d.

Blockers and bummers

The initial-NUL trigger (what writes a LEADING NUL) flagged in the ULOG README is still unpinned — a leading NUL is the torn case, not a clean pad, so the back-scan must not mask it (ties to ULOG-001).

Outcome

LANDED dog df3b5885 ("expose ulog append in the JS API, self-heal", 2026-07-04): ulog_backscan_tail back-scan on both RW-booked and RO opens, partial-row self-heal, O(page) not O(content); tests T_backscan_zero_pad + T_backscan_torn_partial_row.
LANDED abc e8d7c59d (same session): FILETrimBook trims to PAST+DATA (b[2]-b[0]), dropping only IDLE.
The commit message carries no ULOG-002 tag — grep df3b5885/"self-heal", not the ticket code; the code comments cite ULOG-002 (dog/ULOG.c:567, abc/FILE.c:1049).