Beagle SCM

DOG-036 DONE: nav lexer tweaks — key min 2+2, rooted paths, anchor as evidence

gritzko's ruling 2026-08-14, from real ~/src/linux text: GPL-2 must not lex as a ticket key while /README:10 and path/README:10 must lex as file refs. The fused-F rules in dog/tok are REBUILT around one component grammar, extracted into dog/tok/LINK.rl and included by every prose machine — .mkd, .md and comment bodies alike (see Design decisions) — the known-basenames dictionary was NOT taken, so a bare extensionless README:10 stays unfused.

Input

Context

Today the key rule is [A-Z][A-Z0-9_]* "-" dgt+ (fires on GPL-2, C-1, BUG-1); pathslash requires a dotted final ((seg "/")+ seg("."seg)+), so abc/Makefile:20 never fuses; no rooted or ./ head exists. DOG-034's pathanchor rides both tiers.

• All real ticket codes are >=2-letter topics + zero-padded 3-digit numbers; corpus-checked — every sub-minimum shape (C-1, BUG-1..8, ANY-3??) is a non-ticket.
• The DOG-033 dot ban (no ../x, no dotfiles) STAYS; ./ and / heads are new, distinct provisions.
• Bare-tier pins KEPT: localhost:8080, example.com:443, google.com:80, 12:30 never fuse; base.ext still needs a known ext.

Goals

Every prose machine rebuilt around ONE shared component grammar (below), regenerated twins, fixtures. The whole rule is: a fused F is a ticket code, or a name that carries its own EVIDENCE — a known extension, or an anchor past a slash.

• 1. code = >=2 leading LETTERS + >=2 digits, plus a trailing guard: not a code when followed by -digit, .digit or a LETTER. TICKET-12, BE-030 fuse; GPL-2, GPL-2.0, ISO-8859-1, C-1, BUG-1, AB12-34CD, KEY-12abc never.
• 2. rooted — a / or ./ head fuses ONLY with an anchor: /README:10, ./README:10, /abc/S.h:12, ./run.sh:3 fuse; /abc/S.h, /etc/passwd, ./configure do not, which is what keeps a URL path (https://foo.io/bar.c) prose without any scheme rule. ../x and dotfiles stay banned (.. needs its own punct rule once ./ is a head).
• 3. An anchor is evidence only PAST A SLASH, and then it vouches for any name at all — extensionless or unknown-extension alike: path/README:10, abc/Makefile:20, src/weird.xyz:12 fuse; anchor-less and/or, abc/Makefile, src/weird.xyz stay prose, and slashless example.com:443, weird.xyz:12, 12:30 stay prose too.
• 4. Ticket-code nav targets: ticket = code + optional anchor (DOG-032:18, permalink DOG-032:k4:d8K3); keyed = the code HEADS a path, its slash counting as the slash (DOG-032/README.mkd:18, DOG-032/log.txt, DOG-032/repro:3, DOG-032/weird.xyz:12; DOG-032/repro does not fuse). Bare codes unchanged. Accepted noise, pinned as positives: UTC-12:30, WIDTH-16:0, LEN-16, AB12-3456 DO fuse — resolution answers the quiet miss.
• 4a. anchor = line | commit — :line[:col] (digits) or a :off:hashlet permalink; DOG-034's generic ron60 word is OVERTURNED (file.c:100:a7 no longer fuses whole), the 10-char cap stands on both shapes.
• 5. LITE-side leg (Rep: ///lite, SEPARATE worktree, after the lexer lands): the LITE-026 hook mints an added KEY:line exactly like path:line — key -> ticket file (thin, else fat README.mkd), mint against its staged/HEAD blob, rewrite to KEY:off:hashlet; ticket files churn fastest, so this is where permalinks pay most.

Constraints

• ///quickjab worktree, its dog/tok (+ fixtures) only; jab/ and beagle/ untouched; no VC ops.
• Ragel only — the rules live in the grammars, the components in the shared include; regenerate every touched .rl.c twin with the documented ragel line; ctest green.
• Fixtures pin EVERY shape above, positive and negative; then the lite refline + perma suites re-run under the rebuilt binary (they ride this lexer) — failure counts vs baseline identical.

WIP

Design decisions

gritzko's component spec (2026-08-14, the RESTRUCTURE ruling) — the accreted pathslash/pathbare/pathfinal defines are retired, both grammars read as this, one named define per component:

code    = [A-Z]{2} [A-Z0-9_]* "-" [0-9]{2,}     # + the trailing guard (-d/.d/letter)
seg     = [A-Za-z_] [A-Za-z0-9_]*
ext     = "." KNOWN                             # known extension
extany  = "." seg                               # any dotted tail
line    = ":" [0-9]+ (":" [0-9]+)?
commit  = ":" ron64{1,4} ":" (ron64 ron64){2,5}
anchor  = line | commit
dirs    = (seg "/")+
relfile = dirs? seg ext anchor?                 # known ext: slash optional, anchor optional

rooted = ("/" | "./") dirs? seg extany? anchor # anchor always required ticket = code anchor? | code "/" relfile ACCEPT = ticket | relfile | rooted

Three renderings the .rl files make of it, each forced by the machine:

• ext and extany have the same SHAPE; KNOWN is a dictionary, not a grammar — it stays the TOKPathRefKnown -> TOKKnownExt gate on the emit side, so dog/tok/TOK.h's table remains the one source of truth. extany = ("." seg)+ repeats, or dog/tok/FREE.c.rl:122 (two dotted tails) would not compose.
• The two branches of relfile need different EVIDENCE, so they are two scanner rules with two actions: relknown = dirs? seg ext anchor? faces the gate, relanch = dirs? seg extany? anchor is vouched for. relfile of the spec is relknown | dirs relanch; keyed is code "/" relknown and code "/" relanch. The anchored rule is listed FIRST so it wins the tie on src/weird.xyz:12.
• The spec is EXTRACTED, not copied: dog/tok/LINK.rl is a definitions-only machine (no actions, no main, no write statements) whose header comment is the table above; every prose machine pulls it in with include LINK "LINK.rl"; and binds the components to its own actions. It defines its char classes inline ([0-9], [A-Z], ron64) so an includer's own dgt/idalpha never collide, and declares the same alphtype unsigned char all includers use. Three includers, because the comment machine IS the free-text machine: MKDT.c.rl (.mkd), MDT.c.rl (.md), FREE.c.rl (comment bodies + free text — 56 language tokenizers re-lex their comments through it via FREECommentFeed, and dog/INDEX.md names it as the comment-body scanner). TXTT.c.rl, the .txt/.rst and unknown-ext fallback, has no link rules and was NOT wired — that would newly fuse refs in plain text, which nobody ruled.
• line/commit carry DOG-034's 10-char cap (dgt{1,10}, ron64{1,4} + (ron64 ron64){2,5}). When the cap cuts an anchor short, a vouched-for ref has nothing left to stand on and retreats to crumbs (TOKUNCUT), while a known-ext ref or a code just drops the tail.

TODOs

• both grammars rebuilt on the component spec, one define per component
• code min 2+2 + trailing guard (-d, .d, letter), both machines
• rooted / and ./ heads, anchor mandatory
• anchor as evidence past a slash: ext-less AND unknown-ext both fuse
• ticket anchors + keyed code-headed paths, both machines
• anchor = line | commit, DOG-034's generic ron60 word retired
• extract dog/tok/LINK.rl, defs only; MKDT/MDT/FREE include it
• .md catches up: the same five rules, divergence closed
• TOK fixtures: all positives/negatives incl. kept pins
• rebuild, lite refline + perma legs vs baseline
• LITE leg: hook mints KEY:line -> KEY:off:hashlet (separate wt, after the lexer)

Blockers and bummers

Outcome

Landed in //DOG-036 (staged, not committed): the component spec above is EXTRACTED into dog/tok/LINK.rl and included by every prose machine, twins regenerated with ragel -C <X>.c.rl -o <X>.rl.c -L.

• Defines now live ONCE, in dog/tok/LINK.rl, spec order: code, seg, keyvoid, extany, ext, ron64, line, commit, anchor, dirs, relknown, relanch, rooted, ticket, dotdot. Retired everywhere: pathseg, pathbare, pathslash, pathfinal, pathanchor, issuekey. Three includers bind the same five rules to their own actions: ticket => on_key, code keyvoid => on_nokey, dirs relanch | rooted | code "/" relanch => on_anchored, relknown | code "/" relknown => on_pathref, dotdot => on_dotdot.
• .md caught up (dog/tok/MDT.c.rl): it had drifted with a key rule of its own and no path rules at all. It gains the five rules, the four new actions and two callbacks (MDTonPathRef, MDTonCrumbs) modelled on the MKDT pair. All 58 nav cases now answer IDENTICALLY through .mkd, .md and comments — the divergence flagged in the first cut is closed.
• Actions: on_key fuses (a code is self-evident), on_anchored fuses unless the cap cut its anchor (then crumbs), on_pathref faces the KNOWN gate, on_nokey/on_dotdot retreat to crumbs via TOKSplitText ('S' / 'P'). TOKUNCUT is TOKUNTAIL with its answer carried out (TOKUNTAIL now wraps it). TOKPathRefKnown in dog/tok/TOK.c is the shared ext gate — it REPLACES the copy-pasted TOKPathRefBase + PATHu8sExt + TOKKnownExt block in both emit paths. One new callback, MKDTonCrumbs(tag, tok, state). No manual byte scanning, no new tag, no tok32 change.
• The .. ban needs the explicit ("../")+ punct rule: with a ./ head admitted, ../x.c:12 would otherwise open a ref at offset 1. It keeps ../x.c lexing exactly as DOG-033 pinned it (PPPF).
• Fixtures: FREENavTweakTest + MKDTNavTweakTest + MDTNavTweakTest in dog/test/TOK01.c, ONE 58-case list run through all three grammars (174 assertions); 95 of the 174 were RED against the pre-DOG-036 lexers, all green after.
• .md fixture flips (MDTIssueKeyTest, same rule, same overturn as the .mkd twin): PROJ-1, X-99, A1B2-7, A_B-9 all F -> SPL; AB1C-23 and AB_C-90 added as positives. Nothing else in the MDT suite moved — MDTBasicTest, MDTEmphTest, MDTCodeFenceTest, MDTFenceIsolationTest and the MDTFileTest corpus pass untouched.
• Fixtures OVERTURNED and amended in place (each pins a rule this ticket rewrites): DOG-033's dog/tok/FREE.c.rl and quickjab/lsan.supp no longer fuse bare (unknown ext, no anchor — they fuse the moment an anchor is added); DOG-034's file.c:100:a7 -> FPS and TCP.c:9~_Z -> the commit shape TCP.c:9:a7f3, since a bare ron60 word is no longer an anchor; MKDTIssueKeyTest's PROJ-1/X-99/A1B2-7/A_B-9 flip F -> SPL (new AB1C-23/AB_C-90 positives); MKDTMetaPairTest's header fixture takes ABC-123 for ABC-1 (tags unchanged).
• ctest 121/121 green. lite test/refline 38 checks 0 bad and test/perma 51 checks 0 bad under the rebuilt binary — identical to the pre-tweak baseline ~/tmp/l25/qjb/bin/quickjab.
• The flag from the first cut is GONE: path/README:12345678901 now yields SSSPL, not FPL — with the anchor over the cap there is no evidence left, so the ref retreats to crumbs. A known-ext ref still keeps its name (file.c:12345678901 -> FPL), which is DOG-034's cap ruling intact.
• The comment machine is dog/tok/FREE.c.rl — the same file as the free-text one, so the include sites are three, not four: 56 language tokenizers (CT.c, PYT.c, SHT.c, ...) re-lex their comment bodies through it via FREECommentFeed, dog/INDEX.md names it the comment-body scanner, and no other machine in dog/tok carries comment-body rules. TXTT.c.rl (.txt/.rst, and the unknown-ext fallback) is the only other candidate and was deliberately left alone — wiring it would newly fuse refs inside plain text. Say the word if that is wanted.

Suggested commit: DOG-036: LINK.rl — one link grammar, mkd/md/comments