Beagle SCM

LITE-030: vendor commonmark.js 0.31.2 into lite/mark/ — CommonJS, zero npm deps

Normalizing GitHub Markdown into StrictMark needs a real parser, not regexes over source bytes: the input dialect has containers, lazy continuation and reference links, and only an AST survives that. The parser is commonmark.js — John MacFarlane's reference JS implementation, same author as the spec, BSD-2-Clause, 3.3K lines of JS of which only the parse half is wanted. This ticket VENDORS it: lib/ minus the renderers, ESM rewritten to lite's CommonJS, the three npm deps removed, plus the regeneration script that makes re-vendoring a re-run rather than a hand-edit. No StrictMark logic here — see LITE-032; no GitHub extensions — see LITE-031.

Input

Context

Upstream checkout is ~/src/commonmark.js at v0.31.2 (package.json), lib/ is ESM ("type": "module"): blocks.js 1019, inlines.js 1074, node.js 328, common.js 114, from-code-point.js 65 lines. render/, bin/, dingus/, bench/ are out of scope — the emitter is ours and the HTML oracle is the cmark-gfm binary.

• lite's runtime is CommonJS: require.c builds each module with new Function("module","exports","require", src) — import/export will not parse.
• Specifiers are ROOT-relative to the ONE frozen require base (require.c, QJAB-002), so ./node.js would resolve to the pack root; vendored files must say mark/node.js.
• Three npm deps: entities.decodeHTMLStrict (inlines.js:794, common.js:61), mdurl/encode.js (via normalizeURI, in the PARSE path — inlines.js:215,223,593,596), minimist (bin/ only, dropped with bin/).
• JAB_JSRC packs the tree with file(GLOB_RECURSE) (quickjab/CMakeLists.txt:92) — every byte dropped under lite/ ships inside the binary.
• Licenses split: lib/ is BSD-2-Clause; test/spec.txt is CC-BY-SA 4.0 (its front matter, line 6), as is GitHub's GFM spec.

Goals

require("mark/blocks.js") parses Markdown inside a lite worktree under quickjab, producing the same AST as upstream node for the ported examples, with nothing reaching out to node_modules.

• lite/mark/{node,common,inlines,blocks,from-code-point}.js — CJS, root-rooted specifiers, otherwise byte-faithful to upstream.
• lite/mark/vendor.sh regenerates those five from a commonmark.js checkout: the ESM→CJS rewrite, the specifier rooting, the dep stubs. Re-vendoring 0.31.3 must be a re-run.
• lite/mark/LICENSE — upstream's BSD-2-Clause text verbatim, including its from-code-point.js polyfill note.
• lite/test/mark/run.sh — the conformance leg, upstream spec examples through the vendored parser.

Constraints

• Ported tests: 500 lines MAX (gritzko, 2026-08-14). Pick examples that cover the block stack and the inline precedence, not the long tail.
• test/spec.txt is NOT vendored: it is CC-BY-SA 4.0 and 8.5K lines of pack bloat. The examples the test leg embeds are a derivative — carry the attribution and the license name in the file's header comment.
• ZERO quickjab edits; the parser is pure JS over the existing runtime.
• Test convention is lite's: test/mark/run.sh under dash, $LITEJAB picks the runtime, fixtures under $HOME/tmp (see test/now/run.sh).

WIP

Design decisions

• normalizeURI becomes identity rather than vendoring mdurl: it percent-encodes link destinations INSIDE the parse path, and the emitter must re-emit the source URL bytes, not a rewritten form.
• Entity table, INTERIM until gritzko rules: decode numeric (&, &) plus the named entities the ported tests need; leave every other &name; verbatim. Full entities is ~2.1K entries of table in the binary. Note the divergence in the test header.
• Vendored files stay byte-faithful apart from the mechanical rewrite — no reformatting, no ABC-ification. Drift must stay reviewable against upstream.

TODOs

• vendor.sh: copy the five files, ESM→CJS, root the specifiers, stub the deps
• lite/mark/LICENSE + the per-file upstream headers kept intact
• test/mark/run.sh: ported examples, ≤500 lines, attribution + CC-BY-SA note
• jab --eval smoke inside the wt: parse, walk, dump node types
• report to gritzko; entity-table ruling still owed

Blockers and bummers

• Open: the entity table — full ~2.1K-entry map for conformance, or the interim numeric+few set above. gritzko's call.

Outcome

Staged in work/LITE-030, 8 files, nothing posted. Verified by the orchestrator: test/mark/run.sh PASS at 66 checks under jab, mark/vendor.sh re-run reproduces all five files byte-identically, no ESM keyword / mdurl / entities / minimist / require("./…") survives, mark/LICENSE in place.

• mark/ 2649 lines vendored + vendor.sh 158 + test/mark/run.sh 330 (cap 500), 64 spec examples compared as one-line AST sexps.
• Two DIVERGENCES pinned by tests, both labelled as such: normalizeURI is the identity (EX 489 keeps /my uri unencoded), and the interim entity table decodes all numeric refs plus ~90 named ones, leaving the rest verbatim.
• decodeHTMLStrict lives in common.js and is re-exported to inlines.js, keeping the vendored file count at exactly five.
• Noted for future lite tests: io.log writes to stderr; test drivers need io.writeAll(io.stdout, …).
• Suggested commit message: LITE-030: vendor commonmark.js 0.31.2 parse half into mark/