Skip to content
Back to search

claude-skills

MarketplaceWorks with Claude CodeGitHub

Benjamin Vial's skills: own skills plus vetted copies of external ones.

Safety scan

A static text scan, not a guarantee. Select the badge to see what was scanned and every finding.

Checked today

Check it deeper on your machine

It has changed since its scan on 19 Nov 2026. Its scripts were only partly scanned.

As a Claude Code plugin, install it and then ask Claude Code:

/plugin marketplace add keithmackay/mackayi/plugin install scanskillsafety@mackayiDo a deep scan of https://github.com/benvial/claude-skills with scanskillsafety

Or from a terminal, clone it and run it:

git clone https://github.com/keithmackay/scanskillsafety-skill ~/.claude/skills/scanskillsafetynode ~/.claude/skills/scanskillsafety/cli.cjs --deep 'https://github.com/benvial/claude-skills'

It checks the exact commit: every file, its packages against OSV (api.osv.dev), how it compares with this page's scan (findsafeskills.com receives the repository's address), and, if you agree, Claude reading every file with your own Claude subscription. It never runs the code.

Badge for your README
findsafeskills safety rating

Markdown

[![findsafeskills safety rating](/api/listings/benvial--claude-skills/badge.svg)](/listings/benvial--claude-skills)

HTML

<a href="/listings/benvial--claude-skills"><img src="/api/listings/benvial--claude-skills/badge.svg" alt="findsafeskills safety rating" /></a>

Install

Install instructions from the README

As written by the repo's author. findsafeskills has not verified these steps.

Claude Code, on each machine:

claude plugin marketplace add benvial/claude-skills
claude plugin install skills@claude-skills

Then enable auto-update: `/plugin` → Marketplaces → claude-skills → Enable auto-update.

claude.ai: Customize → Plugins → Add → Add marketplace → Add from a repository → `benvial/claude-skills`, then install `skills`.

What was scanned

The files the safety scan read, as written by the repository's author and shown as plain text. Lines with a finding are marked.

.claude-plugin/marketplace.json · Manifest · 15 lines
  1. {
  2. "$schema": "https://json.schemastore.org/claude-code-marketplace.json",
  3. "name": "claude-skills",
  4. "description": "Benjamin Vial's skills: own skills plus vetted copies of external ones.",
  5. "owner": {
  6. "name": "Benjamin Vial"
  7. },
  8. "plugins": [
  9. {
  10. "name": "skills",
  11. "source": "./plugins/skills",
  12. "description": "All skills and subagents in one plugin."
  13. }
  14. ]
  15. }
README.md · README · 40 lines
  1. # claude-skills
  2. One Claude Code plugin marketplace holding all my skills: my own, plus pinned copies of external ones.
  3. ## Layout
  4. - `plugins/skills/skills/<name>/` — every skill. Own skills are edited here directly.
  5. - `plugins/skills/agents/` — subagents that come with external skills.
  6. - `vendor.json` — external skills: upstream repo, pinned commit, and which paths to copy (or `plugin`: take every skill an upstream plugin.json lists).
  7. - `LICENSES/<upstream>/` — each upstream's license and the exact commit copied.
  8. - `scripts/sync.py` — copies the pinned external skills into the plugin.
  9. ## Install
  10. Claude Code, on each machine:
  11. ```bash
  12. claude plugin marketplace add benvial/claude-skills
  13. claude plugin install skills@claude-skills
  14. ```
  15. Then enable auto-update: `/plugin` → Marketplaces → claude-skills → Enable auto-update.
  16. claude.ai: Customize → Plugins → Add → Add marketplace → Add from a repository → `benvial/claude-skills`, then install `skills`.
  17. ## Updating
  18. - Own skill: edit it under `plugins/skills/skills/`, commit, push.
  19. - External skills: a weekly workflow moves every pin to upstream HEAD and opens a PR. Review and merge it. To run it by hand: `python scripts/sync.py --bump`.
  20. - Add or drop an external skill: edit `vendor.json`, run `python scripts/sync.py`, commit.
  21. ## External sources
  22. | Upstream | Skills |
  23. | --- | --- |
  24. | [JuliusBrussee/caveman](https://github.com/JuliusBrussee/caveman) (Apache-2.0) | caveman, caveman-commit, caveman-review, caveman-compress |
  25. | [mattpocock/skills](https://github.com/mattpocock/skills) (MIT) | every skill in the upstream plugin (27), read from its `plugin.json` |
  26. | [pbakaus/impeccable](https://github.com/pbakaus/impeccable) (Apache-2.0) | impeccable + 5 subagents |
  27. | [wdm0006/python-skills](https://github.com/wdm0006/python-skills) (MIT) | designing-python-apis |
  28. | [conorbronsdon/avoid-ai-writing](https://github.com/conorbronsdon/avoid-ai-writing) (MIT) | avoid-ai-writing |
plugins/skills/skills/avoid-ai-writing/detector/patterns.js · Bundled script · 3,762 lines
  1. /**
  2. * Avoid AI Writing — detection engine (canonical source of truth)
  3. * Implements regex, structural, and stylometric pattern detection. This repo's SKILL.md
  4. * catalogs the human-editable pattern rules; this engine is the executable
  5. * expression of the regex-detectable subset and extends it with stylometric and
  6. * AI-tool-fingerprint detectors that don't make sense as skill prose
  7. * (cross-paragraph burstiness, smart-punct signatures, function-word
  8. * trigram entropy, low type-token ratio, AI-tool URL parameters,
  9. * chatbot citation markup leaks, unfilled placeholders).
  10. *
  11. * Scoring model:
  12. * Each category has a weight in the ISSUE_WEIGHTS table. Detection runs
  13. * produce raw (possibly duplicate) issues which are then deduplicated by
  14. * (type, text) pair. rawScore is the sum of category weights across the
  15. * deduped list — so the number reflects the same distinct signals the
  16. * user sees in the issue list.
  17. *
  18. * Weights are deliberately non-flat across severity tags. Cutoff
  19. * disclaimers (10) and chatbot artifacts (8) weigh more than vague
  20. * attributions (5), even though all three are tagged `critical`, because
  21. * the skill treats them as stronger or weaker AI-origin signals.
  22. *
  23. * rawScore is then normalized to 0-100 via `log2(wordCount/50)` so longer
  24. * texts don't accumulate unboundedly on the same density of patterns.
  25. */
  26. const AIDetector = (() => {
  27. // ═══ Tier 1 pre-pass: normalize bypass tricks ══════════════════════
  28. //
  29. // Humanizer tools and prompt-injection bypass techniques insert
  30. // invisible / lookalike chars to defeat exact-string detectors. Strip
  31. // them BEFORE pattern matching so "delve" with a Cyrillic 'е' still
  32. // hits the Tier 1 list. Unicode ranges sourced from
  33. // It-s-AI/llm-detection/detection/attacks/.
  34. //
  35. // Tracks what was stripped so the trinary classifier can use
  36. // "normalization triggered" as a corroborating AI signal — humans don't
  37. // paste ZWSPs into their own writing.
  38. const CYRILLIC_LOOKALIKES = {
  39. 'а': 'a', 'е': 'e', 'о': 'o', 'р': 'p', 'с': 'c', 'х': 'x',
  40. 'у': 'y', 'к': 'k', 'м': 'm', 'н': 'h', 'в': 'b', 'т': 't',
  41. 'А': 'A', 'Е': 'E', 'О': 'O', 'Р': 'P', 'С': 'C', 'Х': 'X',
  42. 'У': 'Y', 'К': 'K', 'М': 'M', 'Н': 'H', 'В': 'B', 'Т': 'T',
  43. };
  44. const GREEK_LOOKALIKES = { 'ο': 'o', 'Ο': 'O', 'α': 'a', 'Α': 'A', 'ρ': 'p', 'Ρ': 'P' };
  45. // ─── Source-coordinate mapping (issue #189) ─────────────────────────
  46. //
  47. // Each entry maps one code unit in the working string to the matching
  48. // code unit in the caller's source. Deletion passes copy the entries for
  49. // retained characters once, so interleaved or overlapping removals cannot
  50. // double-count offsets. Masking and homoglyph replacement keep their input
  51. // length and therefore keep the current map unchanged.
  52. function identitySourceMap(length) {
  53. return Array.from({ length }, (_, index) => index);
  54. }
  55. function appendMapRange(target, source, start, end) {
  56. for (let index = start; index < end; index += 1) target.push(source[index]);
  57. }
  58. function remapFindingsToSource(issues, regions, sourceMap) {
  59. for (const issue of issues) {
  60. if (Number.isInteger(issue.index)) issue.index = sourceMap[issue.index];
  61. }
  62. for (const region of regions) {
  63. region.start = sourceMap[region.start];
  64. region.end = sourceMap[region.end - 1] + 1;
  65. }
  66. }
  67. const ZERO_WIDTH_RE = /[​-‍⁠]/u;
  68. const ZERO_WIDTH_GLOBAL_RE = /[​-‍⁠]/gu;
  69. const HOMOGLYPH_RE = /[Ѐ-ӿͰ-Ͽ]/u;
  70. const HOMOGLYPH_GLOBAL_RE = /[Ѐ-ӿͰ-Ͽ]/gu;
  71. const LATIN_LETTER_RE = /\p{Script=Latin}/u;
  72. // Dates, versions and ASCII filenames keep internal dots; prose may hard-wrap.
  73. // Other sentence punctuation and Markdown/HTML block starts separate units, even
  74. // without a space, so unrelated Russian text cannot hide an English word.
  75. const SENTENCE_UNIT_GLOBAL_RE = /(?:[^.!?\r\n]|(?<=[A-Za-z0-9_])\.(?=[A-Za-z0-9_])|\r?\n(?![ \t]*(?:\r?\n|(?:[-*+]|#{1,6}|\d+[.)])\s|[><|]|`{3}|~{3}|(?:(?:-+|=+)[\t ]*|(?:\*[ \t]*){3,}|(?:_[ \t]*){3,}|(?:-[ \t]*){3,})(?:\r?\n|$)|:?-+:?[\t ]*(?:\|[\t ]*:?-+:?[\t ]*)+(?:\|[\t ]*)?(?:\r?\n|$))))+/g;
  76. // Words are letter runs; a hyphen splits them, so "API-сервис" stays two
  77. // words and its Russian half is not swapped.
  78. const LETTER_RUN_GLOBAL_RE = /[\p{L}\p{M}]+/gu;
  79. const ROLEPLAY_VERBS_RE = /^(?:nods|sighs|laughs|smiles|frowns|shrugs|grins|winks|chuckles|gasps|pauses|thinks|wonders|whispers|shouts|gestures|raises|leans|turns|looks|glances|smirks|blinks|nodding|sighing|laughing|smiling|thinking|gesturing)\b/i;
  80. const ROLEPLAY_MARKER_RE = /(?<!\*)\*([^*\n]{1,80}?)\*(?!\*)/gu;
  81. // A word joiner (U+2060) only counts as a bypass character when it splits a
  82. // word: letters on both sides, the shape a humanizer uses to break "delve"
  83. // apart. Show-notes and CMS editors insert word joiners next to URLs and
  84. // punctuation to control line breaking (#351); those are still stripped so
  85. // matching sees clean text, but they are typesetting, not AI evidence.
  86. // Neighbours are read past a short adjacent zero-width run, so a doubled
  87. // joiner inside a word still counts. The bound keeps a long run linear.
  88. const LETTER_END_RE = /\p{L}$/u;
  89. const LETTER_START_RE = /^\p{L}/u;
  90. const MAX_ZERO_WIDTH_SKIP = 8;
  91. function countsAsBypass(chars, i) {
  92. if (chars[i] !== '\u2060') return true;
  93. let before = i - 1;
  94. while (before >= 0 && i - before <= MAX_ZERO_WIDTH_SKIP && ZERO_WIDTH_RE.test(chars[before])) before -= 1;
  95. let after = i + 1;
  96. while (after < chars.length && after - i <= MAX_ZERO_WIDTH_SKIP && ZERO_WIDTH_RE.test(chars[after])) after += 1;
  97. // A two-unit slice lets the Unicode regex see a supplementary-plane letter.
  98. return LETTER_END_RE.test(chars.slice(Math.max(0, before - 1), before + 1))
  99. && LETTER_START_RE.test(chars.slice(after, after + 2));
  100. }
  101. function normalizeText(text, sourceMap) {
  102. const flags = { zeroWidth: 0, homoglyph: 0, roleplay: 0 };
  103. let out = text;
  104. let map = Array.isArray(sourceMap) ? sourceMap : null;
  105. // 1. Strip zero-width chars (ZWSP U+200B, ZWNJ U+200C, ZWJ U+200D,
  106. // BOM U+FEFF, word joiner U+2060). Word joiners outside a word are
  107. // stripped without being counted; see countsAsBypass.
  108. if (map) {
  109. const chars = [];
  110. const nextMap = [];
  111. for (let i = 0; i < out.length; i += 1) {
  112. if (ZERO_WIDTH_RE.test(out[i])) {
  113. if (countsAsBypass(out, i)) flags.zeroWidth += 1;
  114. continue;
  115. }
  116. chars.push(out[i]);
  117. nextMap.push(map[i]);
  118. }
  119. out = chars.join('');
  120. map = nextMap;
  121. } else {
  122. out = out.replace(ZERO_WIDTH_GLOBAL_RE, (_, offset, whole) => {
  123. if (countsAsBypass(whole, offset)) flags.zeroWidth += 1;
  124. return '';
  125. });
  126. }
  127. // 2. Swap Cyrillic / Greek Latin-lookalike chars back to Latin so
  128. // pattern matching catches obfuscated tokens. Swapping every а, е, о
  129. // reported thousands of "homoglyph swaps" on plain Russian text, and
  130. // swapping inside ordinary Russian words flagged bilingual technical
  131. // text, so only two word shapes are swapped:
  132. // - mixed-script words ("pаypal", "dеlve"), anywhere;
  133. // - words of two or more letters spelled entirely in lookalike
  134. // letters ("аст" for "act"), when the sentence around them has no
  135. // Russian or Greek in it: every Cyrillic and Greek letter in the
  136. // sentence is a lookalike. Deciding per sentence, not per document,
  137. // keeps Russian padding from hiding such a word in an English
  138. // sentence.
  139. // Sentence punctuation, blank lines and Markdown block starts separate units.
  140. // Internal ASCII-token dots ("07.10", "v1.2", "report.pdf") and a prose
  141. // hard wrap do not end it: cutting there left pieces such as "поста в
  142. // r/StableDiffusion от 07" that looked like Latin text.
  143. // Ordinary Russian words contain non-lookalike letters (з, п, и, л),
  144. // so short Russian words in Russian technical prose ("на Enum",
  145. // "от 07.10") stay put, and one-letter prepositions (с, о, у) are too
  146. // short.
  147. // Known limits, because no letter-level rule separates these from
  148. // real Russian: a fully substituted word inside or next to Russian
  149. // words ("МЕТА поможет", "пароль: аст"), including across prose hard wraps,
  150. // is left alone; a one-letter
  151. // lookalike split off by a hyphen ("а-ct") is left alone; and a short
  152. // Russian sentence spelled only in lookalike letters ("Он сам.") is
  153. // swapped.
  154. const lookalikeFor = (m) => CYRILLIC_LOOKALIKES[m] ?? GREEK_LOOKALIKES[m];
  155. const swapLookalike = (m) => {
  156. const swap = lookalikeFor(m);
  157. if (swap) { flags.homoglyph++; return swap; }
  158. return m;
  159. };
  160. const letterCount = (word) => (word.match(/\p{L}/gu) || []).length;
  161. const allLookalike = (word) => letterCount(word) >= 2
  162. && [...word].every((ch) => lookalikeFor(ch) || !/\p{L}/u.test(ch));
  163. out = out.replace(SENTENCE_UNIT_GLOBAL_RE, (unit) => {
  164. const scriptLetters = unit.match(HOMOGLYPH_GLOBAL_RE) || [];
  165. const onlyLookalikes = scriptLetters.length > 0 && scriptLetters.every((ch) => lookalikeFor(ch));
  166. const words = [...unit.matchAll(LETTER_RUN_GLOBAL_RE)];
  167. let result = '';
  168. let last = 0;
  169. words.forEach((match) => {
  170. const word = match[0];
  171. let next = word;
  172. if (HOMOGLYPH_RE.test(word)) {
  173. if (LATIN_LETTER_RE.test(word) || (allLookalike(word) && onlyLookalikes)) {
  174. next = word.replace(HOMOGLYPH_GLOBAL_RE, swapLookalike);
  175. }
  176. }
  177. result += unit.slice(last, match.index) + next;
  178. last = match.index + word.length;
  179. });
  180. return result + unit.slice(last);
  181. });
  182. // 3. Strip *roleplay-action* markers — paired *...* containing an
  183. // action verb (nods, sighs, laughs, smiles, etc.) anchored to
  184. // the start of the inner phrase. This is the actual chat-model
  185. // artifact shape. Markdown `**bold**` is rejected by the
  186. // lookbehind/lookahead; legitimate multi-word `*italic*` is
  187. // preserved because the verb whitelist is narrow.
  188. if (map) {
  189. const chars = [];
  190. const nextMap = [];
  191. const matcher = new RegExp(ROLEPLAY_MARKER_RE.source, ROLEPLAY_MARKER_RE.flags);
  192. let cursor = 0;
  193. let match;
  194. while ((match = matcher.exec(out)) !== null) {
  195. if (!ROLEPLAY_VERBS_RE.test(match[1])) continue;
  196. chars.push(out.slice(cursor, match.index));
  197. appendMapRange(nextMap, map, cursor, match.index);
  198. flags.roleplay += 1;
  199. cursor = match.index + match[0].length;
  200. }
  201. chars.push(out.slice(cursor));
  202. appendMapRange(nextMap, map, cursor, map.length);
  203. out = chars.join('');
  204. map = nextMap;
  205. } else {
  206. out = out.replace(ROLEPLAY_MARKER_RE, (m, inner) => {
  207. if (ROLEPLAY_VERBS_RE.test(inner)) {
  208. flags.roleplay += 1;
  209. return '';
  210. }
  211. return m;
  212. });
  213. }
  214. return map ? { text: out, flags, sourceMap: map } : { text: out, flags };
  215. }
  216. // Terms with legitimate technical meaning that are suppressed when contextMode === 'technical'.
  217. // See references/patterns.md and issue #237.
  218. const TECHNICAL_EXEMPT = new Set([
  219. 'robust',
  220. 'comprehensive',
  221. 'seamless',
  222. 'seamlessly',
  223. 'ecosystem',
  224. 'leverage',
  225. 'leverages',
  226. 'leveraging',
  227. 'leveraged',
  228. 'facilitate',
  229. 'facilitates',
  230. 'underpin',
  231. 'underpinning',
  232. 'underpinnings',
  233. 'streamline',
  234. ]);
  235. // ─── Tier 1: Always flag ───────────────────────────────────────────
  236. const TIER1 = {
  237. 'delve': 'explore, dig into, look at',
  238. 'tapestry': 'complex structure, rich history',
  239. 'paradigm': 'model, approach, framework',
  240. 'beacon': 'rewrite entirely',
  241. 'robust': 'strong, reliable, solid',
  242. 'comprehensive': 'thorough, complete, full',
  243. 'cutting-edge': 'latest, newest, advanced',
  244. 'pivotal': 'important, key, critical',
  245. 'meticulous': 'careful, detailed, precise',
  246. 'meticulously': 'carefully, precisely',
  247. 'seamless': 'smooth, easy, without friction',
  248. 'seamlessly': 'smoothly, easily',
  249. 'game-changer': 'changes how [X] works',
  250. 'game-changing': 'changes how [X] works',
  251. 'nestled': 'is located, sits',
  252. 'vibrant': 'lively, active',
  253. 'thriving': 'growing, active',
  254. 'bustling': 'busy, active',
  255. 'intricate': 'complex, detailed',
  256. 'intricacies': 'complexities, details',
  257. 'ever-evolving': 'changing, growing',
  258. 'enduring': 'lasting, long-running',
  259. 'daunting': 'hard, difficult',
  260. 'holistic': 'complete, full, whole',
  261. 'holistically': 'completely, fully',
  262. 'actionable': 'practical, useful, concrete',
  263. 'impactful': 'effective, significant',
  264. 'learnings': 'lessons, findings, takeaways',
  265. 'synergy': 'combined effect, benefit of working together',
  266. 'synergies': 'combined effect, benefit of working together',
  267. 'interplay': 'relationship, connection',
  268. 'symphony': 'coordination, balance',
  269. 'embrace': 'adopt, accept, use',
  270. };
  271. // Multi-word tier 1 phrases
  272. // "features" is copula avoidance only as a verb: "the app features a
  273. // dashboard". The plural noun ("three new features", a "Features" heading,
  274. // "the features we shipped") is ordinary product and technical writing, and
  275. // it is most of what the word does in corpus/ (#351). A verb needs a subject
  276. // before it and an object after it, so the word counts as a noun when it
  277. // opens a sentence, line or list item; follows a determiner, number,
  278. // possessive, quantifier, "of", "or" or a compound modifier; or comes
  279. // before punctuation, a line break, a preposition, a conjunction, a pronoun
  280. // or a verb.
  281. // Singular "this", "that" and "each" cannot determine the plural noun:
  282. // here they are subjects ("each features a...") or a relative pronoun.
  283. const FEATURES_NOUN_BEFORE_RE = /(?:^|[.!?:;(]\s*|(?:^|\n)[ \t]*(?:[-*+>#]+[ \t]*)?|\b(?:the|a|an|these|those|its|their|our|your|my|his|her|of|or|new|key|main|core|top|other|more|most|many|several|some|all|any|few|no|both|only|about|over|just|free|similar|common|specific|distinct|additional|extra|premium|advanced|basic|best|unique|missing|upcoming|existing|latest|major|minor|useful|important|various|different|certain|such|\d+|two|three|four|five|six|seven|eight|nine|ten|whose)\s+|\w['’]s\s+|\w[-/]\w+\s+)$/i;
  284. // "on-device" is an object modifier, not the preposition "on".
  285. const FEATURES_NOUN_AFTER_RE = /^(?:[ \t]*(?:[.,;:!?()[\]"'’”\n]|$)|\s+(?:of|for|in|on|at|into|and|or|but|nor|are|were|is|was|be|been|being|that|which|who|whose|like|such|to|from|with|without|than|help|helps|helped|let|allow|make|keep|give|provide|enable|offer|save|protect|ensure|improve|reduce|support|remain|vary|need|unlock|bring|include|includes|included|work|works|worked|will|can|could|should|would|may|might|must|do|did|does|have|has|had|we|you|they|i|it)(?![\w-]))/i;
  286. // Bare product objects: "we ship features", "teams build features".
  287. // A subject is required so noun subjects such as "the ship"/"the build"
  288. // still flag. One optional modifier preserves "we ship GPT-5 features".
  289. const FEATURES_NOUN_OBJECT_RE = /\b(?:i|we|you|they|teams?|developers?|engineers?|users?|companies|vendors|the\s+(?:release|update))\s+(?:(?:can|could|will|would|should|must)\s+)?(?:ship|ships|shipped|shipping|build|builds|built|building|add|adds|added|adding)\s+(?:[\w./-]+\s+)?$/i;
  290. // Leads that make "which/what features" an indirect question (#384): a question
  291. // verb that opens its clause ("Decide which...", "we should check which..."),
  292. // or which/what at the start of a sentence or list item. A lead word inside a
  293. // noun phrase ("a security check which features...") does not count.
  294. const FEATURES_QUESTION_LEAD_RE = /(?:^|[.!?:;]\s*|(?:^|\n)[ \t]*(?:[-*+>#]+[ \t]*)?|(?:^|[.!?:;,]\s*|(?:^|\n)[ \t]*(?:[-*+>#]+[ \t]*)?|\b(?:i|we|you|they|he|she|let['’]?s|to|can|could|should|must|will|would|please|just|then|first|now|and|or|but|so)\s+)(?:decide|decides|deciding|choose|choosing|pick|check|checking|see|know|knowing|learn|find\s+out|figure\s+out|work\s+out|identify|determine|ask|asking|asked|tell\s+(?:me|us|them|you)|show\s+(?:me|us|them|you)|understand|explain|wonder|discover|confirm|consider|evaluate|prioritize|about)\s+)(?:which|what)\s+$/i;
  295. // A lead word after a determiner is a noun ("a track which features...",
  296. // "the review which features..."), not a question verb.
  297. const FEATURES_LEAD_AS_NOUN_RE = /\b(?:a|an|the|this|that|these|those|my|our|your|their|its|his|her|each|every)\s+[\w-]+\s+(?:which|what)\s+$/i;
  298. // Subjects that definitively make "features" a verb, avoiding ambiguity with
  299. // plural-noun subjects like "The experimental features" or "Security features".
  300. const FEATURES_VERB_SUBJECTS = /\b(?:library|tool|app|application|platform|system|language|service|product|update|release|version|site|website|game|device|model|framework|package|plugin|extension|which|what|that|who|it|he|she)\s+$/i;
  301. function featuresIsNoun(text, index) {
  302. const end = index + 'features'.length;
  303. const before = text.slice(Math.max(0, index - 40), index);
  304. const after = text.slice(end, end + 40);
  305. if (FEATURES_NOUN_OBJECT_RE.test(before)) return true;
  306. // A recognizable model/version followed by an object article is a
  307. // subject: "GPT-5 features a...", "Windows 11 features a...".
  308. // Determiner-led counts ("These 3 features a customer requested") and
  309. // the explicit object contexts above remain nouns. This is a bounded
  310. // context heuristic, not a grammatical parse of every use of the word.
  311. const product = /\b([A-Z][\w]*)(?:[-/]\d[\w.-]*|[ \t]+\d+(?:\.\d+)*)[ \t]+$/.exec(before);
  312. if (product && !FEATURES_NOUN_BEFORE_RE.test(`${product[1]} `)
  313. && /^\s+(?:a|an|the)(?![\w-])/i.test(after)) return false;
  314. // A short comma-delimited aside can precede the verb's article-led
  315. // object. Strong noun contexts still take priority over punctuation.
  316. if (!FEATURES_NOUN_BEFORE_RE.test(before)
  317. && /^[ \t]*,[^,\n]{1,80},\s*(?:a|an|the)(?![\w-])/i.test(text.slice(end, end + 110))) return false;
  318. // "which/what features" is a noun in an indirect question: right after a
  319. // question verb ("Decide which features matter") or at the start of a
  320. // sentence ("Which features matter most..."). Every other context falls
  321. // through to the checks below unchanged, so relative clauses ("a library
  322. // which features dashboards") keep their verb finding (#384).
  323. if (FEATURES_QUESTION_LEAD_RE.test(before) && !FEATURES_LEAD_AS_NOUN_RE.test(before)) {
  324. // If there is no trailing verb (e.g. end of clause), it's the verb case ("Check what features a dashboard and export tools.").
  325. // Look past a compound subject and optional appositive for a trailing verb before returning the verb classification.
  326. const match = /^\s+(?:a|an|the)\s+[\w-]+\s*(?:(?:and\b|or\b)\s+(?:a\s+|an\s+|the\s+)?(?:[\w-]+\s+){0,2}[\w-]+\s*)?(?:([,.!?;])|$)/i.exec(after);
  327. if (match) {
  328. const predicateRe = /\b(?:can|could|should|will|would|may|might|must|is|are|was|were|has|have|had|do|does|did|include|includes|disable|disables|reject|accept|allow|deny|provide|require|use|make|work|help|give|take|need|become|seem|look|show|offer|support|supports|fail|pass|lack|prefer|choose|prevent|stop)\b/i;
  329. // If the matched subject ends with a predicate right before the punctuation,
  330. // the predicate was consumed as part of the compound subject.
  331. const subjectText = match[0].slice(0, match[1] ? -match[1].length : undefined).trim();
  332. const lastWord = subjectText.split(/\s+/).pop();
  333. if (predicateRe.test(lastWord)) {
  334. return true;
  335. }
  336. if (match[1] === ',') {
  337. // If the comma leads into a phrase, limit noun classification to a predicate
  338. // belonging to the article-led subject (after the next comma), excluding predicates inside subordinate clauses.
  339. const remainder = text.slice(end + match[0].length, end + 120);
  340. // An adverb can sit between the closing comma and the predicate.
  341. const remainderPredicateRe = new RegExp(`^[^,]+,\\s*(?:(?:[\\w-]+ly|also|not|never)\\s+){0,2}${predicateRe.source}`, 'i');
  342. if (remainderPredicateRe.test(remainder)) {
  343. return true;
  344. }
  345. }
  346. return false;
  347. }
  348. return true;
  349. }
  350. // "The library features support for..." is a verb.
  351. // "Security features support for..." is a plural noun subject + verb.
  352. if (FEATURES_VERB_SUBJECTS.test(before) && /^\s+support\s+for\b/i.test(after)) {
  353. // A nearby quantifier in an earlier clause cannot make a singular pronoun plural.
  354. if (/\b(?:it|he|she)\s+$/i.test(before)) return false;
  355. const hasPluralDeterminer = /\b(?:these|those|all|some|many|few|various|multiple|several|both)\s+(?:[\w-]+\s+){1,2}$/i.test(before);
  356. // Count words modify "features" only when they follow its determiner.
  357. // A digit inside a product name ("This Windows 11 app") or a date before
  358. // the subject ("In 2024 the library") is not a plural count.
  359. const hasPluralCount = /\b(?:the|a|an|this|that|my|our|your|their|its|his|her|each|every)\s+(?:(?:\d{1,2}|two|three|four|five|six|seven|eight|nine|ten|eleven|twelve|thirteen|fourteen|fifteen|sixteen|seventeen|eighteen|nineteen|twenty|dozen)\s+(?:[\w-]+\s+){1,2}|(?!(?:19|20)\d{2}\s)\d{3,}\s+(?!(?:[\w-]+\s+)?(?:app|release|model|version|update)\s+$)(?:[\w-]+\s+){1,2})$/i.test(before);
  360. const extendedAfter = text.slice(end, end + 100);
  361. // A predicate inside a relative clause modifying the object (e.g. "teams that connect and remain online")
  362. // is not coordinated with the main subject. Limit predicate search to the direct prepositional object.
  363. const hasPartitivePlural = /\b(?:all|some|many|most|few|several|both|\d+|two|three|four|five|six|seven|eight|nine|ten)\s+of\s+(?:the|my|our|your|their|its|his|her)\s+(?:[\w-]+\s+){1,3}$/i.test(before);
  364. const hasPluralPredicate = /^\s+support\s+for(?:(?!\b(?:that|which|who|whom|whose|where|when)\b)[^.?!;]){0,80}?\b(?:and|but|or|nor)\s+(?:are|were|have|do|reject|accept|allow|deny|provide|require|use|make|work|help|give|take|need|become|seem|look|show|include|offer|support|remain|fail|pass|lack|prefer|choose|prevent|stop)\b/i.test(extendedAfter);
  365. const hasForObject = /^\s+support\s+for(?:-|\s+)(?:profit|loops?)\b/i.test(after);
  366. // Numeric modifiers and plural cues exclude a singular-subject reading:
  367. // "The two platform features..." has plural noun "features".
  368. // Do not borrow a determiner across a preposition or conjunction:
  369. // "a suite of system features" still has plural noun "features".
  370. // "that" can introduce a clause and "which" can determine plural nouns.
  371. // A relative pronoun needs its own determiner-led subject evidence.
  372. const hasUnambiguousSingular = /\b(?:a|an|this|that|every|each|one)\s+(?:(?!(?:of|for|in|with|and|or|which|what|that|who|two|three|four|five|six|seven|eight|nine|ten|multiple|several|many|few|both|various)\b)[\w-]+\s+){1,3}$/i.test(before)
  373. || /\b(?:it|he|she)\s+$/i.test(before)
  374. || /\b(?:a|an|this|that|each|every|one)\s+[\w-]+\s+(?:which|that|who)\s+$/i.test(before);
  375. const hasSingularEvidence = hasUnambiguousSingular
  376. || /\b(?:the|my|our|your|their|its|his|her)\s+(?:(?!(?:of|for|in|with|and|or|which|what|that|who|two|three|four|five|six|seven|eight|nine|ten|multiple|several|many|few|both|various)\b)[\w-]+\s+){1,3}$/i.test(before)
  377. || /\b(?:the|my|our|your|their|its|his|her)\s+[\w-]+\s+(?:which|that|who)\s+$/i.test(before);
  378. // Unambiguous singular subjects ("It", "This app") cannot be plural-noun phrases,
  379. // so hasForObject ("support for loops") only suppresses ambiguous determiners ("The library").
  380. if (!hasPluralDeterminer && !hasPluralCount && !hasPartitivePlural && !hasPluralPredicate && (hasUnambiguousSingular || (!hasForObject && hasSingularEvidence))) {
  381. return false;
  382. }
  383. }
  384. return FEATURES_NOUN_BEFORE_RE.test(before) || FEATURES_NOUN_AFTER_RE.test(after);
  385. }
  386. const TIER1_PHRASES = [
  387. { pattern: /\bdelve\s+into\b/gi, replace: 'explore, dig into' },
  388. { pattern: /\blandscape\b/gi, replace: 'field, space, industry', filter: true },
  389. { pattern: /\brealm\b/gi, replace: 'area, field, domain' },
  390. { pattern: /\btestament\s+to\b/gi, replace: 'shows, proves' },
  391. { pattern: /\bleverag(?:e|es|ing|ed)\b/gi, replace: 'use' },
  392. { pattern: /\bwatershed\s+moment\b/gi, replace: 'turning point, shift' },
  393. { pattern: /\bmarking\s+a\s+pivotal\s+moment\b/gi, replace: 'changing [X], leading to [Y]' },
  394. { pattern: /\bthe\s+future\s+looks\s+bright\b/gi, replace: 'cut or say something specific' },
  395. { pattern: /\bonly\s+time\s+will\s+tell\b/gi, replace: 'cut or say something specific' },
  396. { pattern: /\bdespite\s+challenges[^.]*continues?\s+to\s+thrive\b/gi, replace: 'keeps growing despite [named difficulty], continues to succeed despite [named difficulty]' },
  397. { pattern: /\bdeep\s+dive\b/gi, replace: 'look at, examine' },
  398. { pattern: /\bdive\s+into\b/gi, replace: 'look at, examine' },
  399. { pattern: /\bunpack(?:ing)?\b/gi, replace: 'explain, break down' },
  400. { pattern: /\bcomplexities\b/gi, replace: 'problems, details' },
  401. { pattern: /\bthought\s+leader(?:ship)?\b/gi, replace: 'expert, authority' },
  402. { pattern: /\bbest\s+practices\b/gi, replace: 'what works, proven methods' },
  403. { pattern: /\bat\s+its\s+core\b/gi, replace: 'cut, just state it' },
  404. { pattern: /\bin\s+order\s+to\b/gi, replace: 'to', clarity: true },
  405. { pattern: /\bdue\s+to\s+the\s+fact\s+that\b/gi, replace: 'because', clarity: true },
  406. { pattern: /\bserves\s+as\b/gi, replace: 'is', clarity: true },
  407. { pattern: /\bfeatures\b/gi, replace: 'has, includes', filter: true, clarity: true, skip: featuresIsNoun },
  408. { pattern: /\bboasts\b/gi, replace: 'has', clarity: true },
  409. { pattern: /\butiliz(?:e|es|ing|ed)\b/gi, replace: 'use', clarity: true },
  410. { pattern: /\bshowcas(?:e|es|ing|ed)\b/gi, replace: 'show, demonstrate' },
  411. { pattern: /\bembark(?:s|ing|ed)?\b/gi, replace: 'start, begin' },
  412. { pattern: /\bcommenc(?:e|es|ing|ed)\b/gi, replace: 'start, begin', clarity: true },
  413. { pattern: /\bascertain(?:s|ing|ed)?\b/gi, replace: 'find out, determine', clarity: true },
  414. { pattern: /\bendeavou?r(?:s|ing|ed)?\b/gi, replace: 'effort, attempt, try', clarity: true },
  415. { pattern: /\bunderscor(?:es|ing|ed)\b/gi, replace: 'highlights, shows' },
  416. // Hyphen required. The unhyphenated "load bearing" is ordinary English —
  417. // "the load bearing down on the bridge" — where `bearing` is a participle,
  418. // not part of a compound modifier. The tell is always hyphenated.
  419. //
  420. // Only match an immediately following abstract noun from this seed list.
  421. // Unknown nouns, mixed physical/abstract nouns, and predicative uses pass:
  422. // precision over recall (#56). Keep the lookahead out of the matched span.
  423. { pattern: /\bload-bearing\b(?=[ \t]+(?:assumptions?|claims?|invariants?|premises?|constraints?|dependenc(?:y|ies)|arguments?|abstractions?)\b)/gi, replace: 'essential, critical, or say what breaks if you remove it' },
  424. ];
  425. // ─── Tier 2: Flag in clusters (2+ per paragraph) ──────────────────
  426. const TIER2 = {
  427. 'harness': 'use, take advantage of',
  428. 'navigate': 'work through, handle',
  429. 'navigating': 'working through, handling',
  430. 'foster': 'encourage, support, build',
  431. 'elevate': 'improve, raise, strengthen',
  432. 'unleash': 'release, enable, unlock',
  433. 'streamline': 'simplify, speed up',
  434. 'empower': 'enable, let, allow',
  435. 'bolster': 'support, strengthen',
  436. 'spearhead': 'lead, drive, run',
  437. 'resonate': 'connect with, appeal to',
  438. 'resonates': 'connects with, appeals to',
  439. 'revolutionize': 'change, transform',
  440. 'facilitate': 'enable, help, allow',
  441. 'facilitates': 'enables, helps, allows',
  442. 'underpin': 'support, form the basis of',
  443. 'nuanced': 'specific, subtle, detailed',
  444. 'crucial': 'important, key, necessary',
  445. 'multifaceted': 'having several parts, with several aspects',
  446. 'ecosystem': 'system, community, network',
  447. 'myriad': 'many, numerous',
  448. 'plethora': 'many, a lot of',
  449. 'encompass': 'include, cover, span',
  450. 'catalyze': 'start, trigger, accelerate',
  451. 'reimagine': 'rethink, redesign, rebuild',
  452. 'galvanize': 'motivate, rally, push',
  453. 'augment': 'add to, expand, supplement',
  454. 'cultivate': 'build, develop, grow',
  455. 'illuminate': 'clarify, explain, show',
  456. 'elucidate': 'explain, clarify',
  457. 'juxtapose': 'compare, contrast',
  458. 'transformative': 'changes how [X] works',
  459. 'transformation': 'major change, overhaul',
  460. 'cornerstone': 'foundation, basis, key part',
  461. 'paramount': 'most important, top priority',
  462. 'poised': 'ready, set, about to',
  463. 'burgeoning': 'growing, emerging',
  464. 'nascent': 'new, early-stage',
  465. 'quintessential': 'typical, classic, defining',
  466. 'overarching': 'main, central, broad',
  467. 'quietly': 'cut, or name the concrete contrast',
  468. 'underpinning': 'basis, foundation',
  469. 'underpinnings': 'basis, foundations',
  470. 'paradigm-shifting': 'changes the basic model, changes how [X] is understood',
  471. };
  472. // Conditional Tier 2 entries: everyday words whose AI tell is a specific
  473. // significance collocation, not the word itself. They join a paragraph's
  474. // Tier 2 cluster only when the collocation matches — bare uses ("deeply
  475. // nested JSON", "cares deeply") never count, because the base rate of
  476. // these words in innocent prose is far higher than the rest of the table
  477. // and an unconditional entry measurably flags clean human writing.
  478. const TIER2_CONDITIONAL = [
  479. {
  480. word: 'deeply',
  481. pattern: /\bdeeply\s+(?:integrated|committed|rooted|personal|human|flawed|resonant|transformative|interconnected|ingrained|embedded|meaningful)\b/i,
  482. suggestion: 'cut, or name what specifically runs deep',
  483. },
  484. ];
  485. // ─── Tier 3: Flag by density ───────────────────────────────────────
  486. const TIER3 = [
  487. 'significant', 'significantly', 'innovative', 'innovation',
  488. 'effective', 'effectively', 'dynamic', 'dynamics',
  489. 'scalable', 'scalability', 'compelling', 'unprecedented',
  490. 'exceptional', 'exceptionally', 'remarkable', 'remarkably',
  491. 'sophisticated', 'instrumental',
  492. 'world-class', 'state-of-the-art', 'best-in-class',
  493. // `verbatim` is usually redundant with the verb it modifies ("copies X
  494. // verbatim" = "copies X"). It has a genuine term-of-art sense in legal,
  495. // research, and QA registers ("verbatim transcript"), so it lives at Tier 3:
  496. // density-gated, it only fires on overuse, not on a single legitimate use.
  497. 'verbatim',
  498. ];
  499. // Multi-word Tier 3 phrases. Density-gated like single Tier 3 words because
  500. // these legitimately show up in human crypto/web3/dev writing. Threshold is
  501. // intentionally lower than single-word Tier 3 (≥2 occurrences of the same
  502. // phrase) — repeating the same multi-word boilerplate is a stronger AI tell
  503. // than re-using "significant."
  504. const TIER3_PHRASES = [
  505. /\bemerging\s+(?:sector|space|category|industry)\b/gi,
  506. /\bthe\s+integration\s+of\b/gi,
  507. /\bthe\s+intersection\s+of\b/gi,
  508. /\bcommunity-?driven\b/gi,
  509. /\blong-?term\s+sustainability\b/gi,
  510. /\buser\s+engagement\b/gi,
  511. /\bdecentralized\s+compute\b/gi,
  512. /\b(?:sustainable\s+)?reward\s+emissions?\b/gi,
  513. /\btokenized\s+incentive\s+structures?\b/gi,
  514. /\bdesigned\s+for\s+long-?term\b/gi,
  515. ];
  516. // O(1) lookup from any token form (hyphenated or dashless) to its canonical
  517. // Tier 3 word. Counting originally did nested-loop word matching which was
  518. // O(tokens × TIER3) — slow on long pastes.
  519. const TIER3_LOOKUP = new Map();
  520. for (const word of TIER3) {
  521. TIER3_LOOKUP.set(word, word);
  522. const dashless = word.replace(/-/g, '');
  523. if (dashless !== word) TIER3_LOOKUP.set(dashless, word);
  524. }
  525. // Per-category score weights. Applied to distinct (deduplicated) issues so
  526. // the score reflects the same signals the user sees in the issue list.
  527. // Non-uniform on purpose: critical rules like cutoff disclaimers (×10) and
  528. // chatbot artifacts (×8) weigh more than vague attributions (×5), even
  529. // though all three are tagged `critical`.
  530. const ISSUE_WEIGHTS = {
  531. tier1: 5,
  532. // Wordiness, not an AI-frequency marker. Weighted like tier2 so a
  533. // clarity fix cannot push a document toward an AI classification.
  534. 'tier1-clarity': 3,
  535. tier2: 3,
  536. tier3: 2,
  537. transition: 2,
  538. chatbot: 8,
  539. sycophantic: 8,
  540. filler: 2,
  541. 'generic-conclusion': 3,
  542. 'lets-construction': 2,
  543. 'reasoning-artifact': 6,
  544. 'significance-inflation': 4,
  545. 'vague-attribution': 5,
  546. 'hollow-intensifier': 2,
  547. // Issue #82 evidence boundary: this style pattern produced no detector hits
  548. // in either corpus class, so it has no measured authorship direction. Keep
  549. // the finding visible, but do not move authorship scores or probabilities
  550. // until a relevant positive evaluation set supports a direction.
  551. 'emotional-flatline': 0,
  552. 'lingering-attention': 3,
  553. 'novelty-inflation': 3,
  554. 'cutoff-disclaimer': 10,
  555. 'template-phrase': 3,
  556. 'false-concession': 2,
  557. 'rhetorical-question': 2,
  558. 'confidence-calibration': 2,
  559. // Writing-quality guidance whose authorship polarity changes across model
  560. // generations. Keep the flag visible without moving authorship outputs.
  561. 'em-dash': 0,
  562. uniformity: 5,
  563. formatting: 3,
  564. 'tier3-phrase': 3,
  565. // Structural / cluster signals are deliberately weighted high. Unlike
  566. // single-vocabulary hits they're near-dispositive on social-length
  567. // posts (a 15-hashtag block, a 6-item bullet-NP list, three distinct
  568. // crypto-shill phrases stacked) and would otherwise be suppressed by
  569. // the log2(words/50) length divisor on short pastes.
  570. 'tier3-phrase-cluster': 12,
  571. 'hashtag-stuff': 12,
  572. 'bullet-np-list': 10,
  573. 'hedge-stack': 6,
  574. 'future-narrative': 12,
  575. 'real-actual-inflation': 5,
  576. // Social endorsement / CTA closer. Weighted like formulaic-opener: a
  577. // strong single-hit social tell that the length divisor would
  578. // otherwise wash out on a short LinkedIn-length post.
  579. 'social-cta-closer': 8,
  580. // Performed-insight tics: single hits are common in human essays, so
  581. // weighted like tier2 vocabulary — density does the classifying.
  582. 'performed-insight': 3,
  583. // Negation chains are a strong single-hit structural tell.
  584. 'negation-chain': 5,
  585. // Negative parallelism ("It's not just X, it's Y"). The frame is also
  586. // how people state a real correction, so it is weighted below the
  587. // negation chain; the per-piece gate in analyzeText does the rest.
  588. 'negative-parallelism': 4,
  589. 'dev-blog-boilerplate': 3,
  590. 'formulaic-opener': 8,
  591. // Speculative scenario opener ("Imagine a world where…"). Weighted like
  592. // formulaic-opener: a single strong opener tell the length divisor would
  593. // otherwise wash out on a short post.
  594. 'speculative-opener': 8,
  595. // Launch-copy introduction ("Enter X.", "Meet X, your new...").
  596. // Weighted like the other single-hit opener tells: strong on the
  597. // short launch posts where it actually appears.
  598. 'launch-intro': 8,
  599. // Dramatized crowd contrast. Gated hard on the dismissive verb, so
  600. // a hit is meaningful, but the surface shares words with ordinary
  601. // narrative — weighted below the opener tells on purpose.
  602. 'crowd-contrast': 6,
  603. // Fake-casual props (stage directions, wink asides). Near-costume
  604. // when present; same class as the opener tells on short posts.
  605. 'fake-casual-prop': 8,
  606. 'title-case-header': 4,
  607. 'parenthetical-hedge': 3,
  608. // House-style typography is a style review prompt, not authorship
  609. // evidence. Keep it neutral in both the score and classifier (#351).
  610. 'smart-punct-signature': 0,
  611. 'punct-distribution': 6,
  612. 'fnword-trigram-entropy': 5,
  613. 'cross-para-burstiness': 5,
  614. 'normalization-flag': 9,
  615. // Vocabulary-diversity signal (type-token ratio). Weighted modestly
  616. // because the threshold (>=200 tokens AND TTR<0.4) is conservative;
  617. // it stacks with structural signals to push borderline scores up.
  618. 'low-ttr': 3,
  619. // AI-tool fingerprints. Weighted higher than statistical patterns
  620. // because each is a near-definitive single-hit signal — the AI tool
  621. // literally left its mark in the text. citation-markup ranks highest
  622. // (smoking gun: the literal internal markup of ChatGPT/Grok/etc.),
  623. // UTM tracking second (auto-appended by the tool to URLs it writes),
  624. // placeholders third (strong but humans use bracketed slots in
  625. // templates legitimately and forget them — still a publishing bug
  626. // but slightly less definitive AI evidence).
  627. 'ai-placeholder': 10,
  628. 'ai-citation-markup': 15,
  629. 'ai-utm-source': 12,
  630. // Curated P2 copyedits, not authorship evidence. Keep them visible in the
  631. // issue list without moving the AI score, label, probabilities, or
  632. // trinary classification on short documents.
  633. 'unnecessary-hyphenation': 0,
  634. };
  635. // ─── Transition phrases ────────────────────────────────────────────
  636. const TRANSITIONS = [
  637. /\bmoreover\b/gi,
  638. /\bfurthermore\b/gi,
  639. /\badditionally\b/gi,
  640. /\bin\s+today'?s\b/gi,
  641. /\bin\s+an\s+era\s+where\b/gi,
  642. /\bit'?s\s+worth\s+noting\s+that\b/gi,
  643. /\bnotably\b/gi,
  644. /\bin\s+conclusion\b/gi,
  645. /\bin\s+summary\b/gi,
  646. /\bto\s+summarize\b/gi,
  647. /\bwhen\s+it\s+comes\s+to\b/gi,
  648. /\bat\s+the\s+end\s+of\s+the\s+day\b/gi,
  649. /\bthat\s+(?:being\s+)?said\b/gi,
  650. ];
  651. // ─── Chatbot artifacts ─────────────────────────────────────────────
  652. const CHATBOT_ARTIFACTS = [
  653. /\bi\s+hope\s+this\s+helps\b/gi,
  654. /\bcertainly!\b/gi,
  655. /\babsolutely!\b/gi,
  656. /\bgreat\s+question!\b/gi,
  657. /\bexcellent\s+point!\b/gi,
  658. /\bfeel\s+free\s+to\s+reach\s+out\b/gi,
  659. /\blet\s+me\s+know\s+if\s+you\s+need\s+anything\b/gi,
  660. /\bin\s+this\s+article,?\s+we\s+will\s+explore\b/gi,
  661. /\blet'?s\s+dive\s+in!?\b/gi,
  662. ];
  663. // ─── Sycophantic tone ──────────────────────────────────────────────
  664. const SYCOPHANTIC = [
  665. /\byou'?re\s+absolutely\s+right\b/gi,
  666. /\bthat'?s\s+a\s+really\s+insightful\b/gi,
  667. /\bthat'?s\s+a\s+great\s+question\b/gi,
  668. /\bexcellent\s+question\b/gi,
  669. ];
  670. // ─── Filler phrases ────────────────────────────────────────────────
  671. const FILLERS = [
  672. /\bit\s+is\s+important\s+to\s+note\s+that\b/gi,
  673. /\bin\s+terms\s+of\b/gi,
  674. /\bthe\s+reality\s+is\s+that\b/gi,
  675. /\bit'?s\s+important\s+to\s+note\s+that\b/gi,
  676. ];
  677. // ─── Generic conclusions ───────────────────────────────────────────
  678. const GENERIC_CONCLUSIONS = [
  679. /\bthe\s+future\s+looks\s+bright\b/gi,
  680. /\bonly\s+time\s+will\s+tell\b/gi,
  681. /\bone\s+thing\s+is\s+certain\b/gi,
  682. /\bas\s+we\s+move\s+forward\b/gi,
  683. ];
  684. // ─── "Let's" constructions ─────────────────────────────────────────
  685. const LETS_PATTERNS = [
  686. /\blet'?s\s+explore\b/gi,
  687. /\blet'?s\s+take\s+a\s+look\b/gi,
  688. /\blet'?s\s+break\s+this\s+down\b/gi,
  689. /\blet'?s\s+examine\b/gi,
  690. /\blet'?s\s+(?:consider|discuss|delve|unpack|walk\s+through)\b/gi,
  691. ];
  692. // ─── Reasoning chain artifacts ─────────────────────────────────────
  693. const REASONING_ARTIFACTS = [
  694. /\blet\s+me\s+think\s+step\s+by\s+step\b/gi,
  695. /\bbreaking\s+this\s+down\b/gi,
  696. /\bto\s+approach\s+this\s+systematically\b/gi,
  697. /\bhere'?s\s+my\s+thought\s+process\b/gi,
  698. /\bfirst,?\s+let'?s\s+consider\b/gi,
  699. /\bworking\s+through\s+this\s+logically\b/gi,
  700. ];
  701. // NOTE: Acknowledgment loops are judgment-only (#239). The three phrases the
  702. // detector matched ("you're asking about", "the question of whether", "to answer
  703. // your question") are also how people open an ordinary reply and standard
  704. // analytical English. The tell is a restatement that adds nothing, which a
  705. // regex cannot see. See detector/CATEGORIES.md §C.
  706. // ─── Significance inflation ────────────────────────────────────────
  707. const SIGNIFICANCE_INFLATION = [
  708. /\bmarking\s+a\s+(?:pivotal|significant|important)\s+moment\b/gi,
  709. /\ba\s+watershed\s+moment\s+for\b/gi,
  710. // Require an inflating word before "in the evolution of" (the bare phrase
  711. // is allowed for literal uses like "a key stage in the evolution of the
  712. // vertebrate eye"). See issue #212.
  713. /\b(?:moment|milestone|chapter|era|turning\s+point|watershed|inflection\s+point|leap|role|new\s+phase)\s+in\s+the\s+evolution\s+of\b/gi,
  714. /\ba\s+(?:pivotal|defining)\s+moment\s+in\b/gi,
  715. ];
  716. // ─── Vague attributions ────────────────────────────────────────────
  717. const VAGUE_ATTRIBUTIONS = [
  718. /\bexperts\s+(?:believe|say|suggest|agree)\b/gi,
  719. /\bstudies\s+(?:show|suggest|indicate)\b/gi,
  720. /\bresearch\s+(?:shows|suggests|indicates)\b/gi,
  721. /\bindustry\s+leaders\s+(?:agree|believe|say)\b/gi,
  722. ];
  723. // ─── Hollow intensifiers ──────────────────────────────────────────
  724. const HOLLOW_INTENSIFIERS = [
  725. /\bgenuine(?:ly)?\b/gi,
  726. /\btruly\b/gi,
  727. /\bquite\s+frankly\b/gi,
  728. /\bto\s+be\s+honest\b/gi,
  729. /\blet'?s\s+be\s+clear\b/gi,
  730. ];
  731. // ─── Emotional flatline ────────────────────────────────────────────
  732. // The "interesting (part|thing|aspect|piece)" family is matched in two
  733. // shapes: (1) "the most interesting X" inline (the canonical AI list
  734. // intro), and (2) bare "Interesting X:" used as a section-header opener,
  735. // which is the section-break variant that slipped past v3.3.x.
  736. const EMOTIONAL_FLATLINE = [
  737. /\bwhat\s+surprised\s+me\s+most\b/gi,
  738. /\bi\s+was\s+fascinated\s+to\b/gi,
  739. /\bwhat\s+struck\s+me\s+was\b/gi,
  740. /\bi\s+was\s+excited\s+to\s+learn\b/gi,
  741. /\bthe\s+most\s+interesting\s+(?:part|thing|aspect|piece)\b/gi,
  742. // Multiline flag (/m) so `^` matches at every line start, including
  743. // position 0 of a pasted text that has no leading newline. The earlier
  744. // `(?:^|\n)` form silently missed bare openers at the very start of
  745. // input — caught by silent-failure audit 2026-05-16. Leading whitespace
  746. // is `[ \t]*`, not `\s*`: with /m every line start is a match attempt,
  747. // and a `\s*` that can cross newlines rescans the whole blank run from
  748. // each of them, which made a long masked block quadratic (#235).
  749. /^[ \t]*interesting\s+(?:part|thing|aspect|piece)(?:\s+of\s+(?:the\s+)?\w+)?\s*:/gim,
  750. ];
  751. // ─── Lingering-attention claims ────────────────────────────────────
  752. // The share-post opener that claims duration of attention instead of
  753. // saying anything about the thing ("the line I keep coming back to").
  754. //
  755. // Precision note: the bare verb phrase "I keep coming back to X" is NOT
  756. // matched on its own, because it is legitimate whenever a reason follows
  757. // ("I keep coming back to Hirschman because it predicts who quits"), and
  758. // the reason clause is not reliably detectable by regex. Only the
  759. // noun-anchored frame ("the line/quote/bit ... I keep coming back to")
  760. // fires, which is the shape that introduces a subject rather than
  761. // asserting something about it. The bare form stays a skill-prose
  762. // judgment call. See SKILL.md §Lingering-attention claims carve-out.
  763. const LINGERING_ATTENTION = [
  764. // "the line I keep coming back to", "the one quote that I keep coming back to"
  765. // Noun-anchored frame only. "can't stop thinking about" is deliberately
  766. // left to its own pattern below rather than folded into this alternation,
  767. // so "the line I can't stop thinking about" scores once, not twice.
  768. /\b(?:the|that|this)\s+(?:one\s+)?(?:line|quote|bit|part|idea|point|framing|comment|thing)\s+(?:that\s+)?i\s+keep\s+(?:coming\s+back\s+to|thinking\s+about)\b/gi,
  769. /\bi\s+can'?t\s+stop\s+thinking\s+about\b/gi,
  770. /\bstill\s+thinking\s+about\s+(?:this|that)\s+one\b/gi,
  771. /\b(?:been|be)\s+rattling\s+around\s+(?:in\s+)?my\s+(?:head|brain)\b/gi,
  772. /\bi'?ve\s+been\s+chewing\s+on\s+(?:this|that)\b/gi,
  773. ];
  774. // ─── Novelty inflation ─────────────────────────────────────────────
  775. const NOVELTY_INFLATION = [
  776. /\bthe\s+failure\s+mode\s+nobody'?s?\s+naming\b/gi,
  777. /\ba\s+problem\s+nobody\s+talks\s+about\b/gi,
  778. /\bthe\s+insight\s+everyone'?s?\s+missing\b/gi,
  779. /\bwhat\s+nobody\s+tells\s+you\b/gi,
  780. ];
  781. // ─── Cutoff disclaimers ────────────────────────────────────────────
  782. // Includes the canonical LLM self-identification phrases — these are
  783. // near-dispositive on their own (real humans don't write "as an AI
  784. // language model" in first person). Patterns cover the major model
  785. // families' default disclaimer language.
  786. const CUTOFF_DISCLAIMERS = [
  787. /\bas\s+of\s+my\s+last\s+update\b/gi,
  788. /\bas\s+of\s+my\s+(?:knowledge\s+)?(?:cut-?off|last\s+training)\b/gi,
  789. /\bi\s+don'?t\s+have\s+access\s+to\s+real-?time\s+(?:data|information)\b/gi,
  790. /\bbased\s+on\s+available\s+information\b/gi,
  791. /\bas\s+an?\s+(?:ai|artificial\s+intelligence|large\s+language|ai\s+language)\s+(?:language\s+)?model\b/gi,
  792. /\bi\s+(?:am|'m)\s+an?\s+(?:ai|artificial\s+intelligence|large\s+language)\s+(?:assistant|model)?\b/gi,
  793. /\bi\s+cannot\s+(?:provide|give|offer)\s+(?:legal|medical|financial|professional)\s+advice\b/gi,
  794. /\bmy\s+training\s+data\s+(?:only\s+)?(?:goes\s+up\s+to|extends\s+to|ends\s+(?:in|at))\b/gi,
  795. ];
  796. // ─── AI-tool fingerprints ──────────────────────────────────────────
  797. // Three near-definitive AI-origin signals adapted from
  798. // Aboudjem/humanizer-skill P33-P35 (see docs/competitive/audits/
  799. // 2026-05-17-aboudjem-humanizer-skill.md). Unlike the statistical
  800. // patterns above, single hit on any of these is strong evidence —
  801. // the AI tool literally left its fingerprint in the text.
  802. // Unfilled slot-fill placeholders. Catches the canonical "[Your Name]"
  803. // family plus dated stubs and HTML/MD comments with placeholder verbs.
  804. const AI_PLACEHOLDERS = [
  805. // Directive stubs ("[Your Name]", "[INSERT SOURCE URL]",
  806. // "[Describe the specific section]") — verb-led, the user was
  807. // told to fill in.
  808. /\[(?:Your|Insert|Add|Enter|Describe|Specify|Choose|Pick)[^\]\n]{1,80}\]/gi,
  809. // Noun-only template variables common in AI-generated email or
  810. // letter boilerplate. Match the bare noun OR noun + qualifier.
  811. // Conservative list: only nouns that almost never appear as
  812. // bracketed real content (citation refs, code identifiers, etc.
  813. // are excluded because they typically contain dots, slashes,
  814. // hyphens, or version numbers).
  815. /\[(?:Recipient|Sender|Topic|Subject|Salutation|Closing|Position|Department|Project Name|Company Name|Date)(?:\s+[^\]\n]{0,60})?\]/gi,
  816. // All-caps directive forms ("[INSERT X]", "[FILL IN]") — the
  817. // uppercase tells you it's a slot, not real content.
  818. /\[(?:INSERT|FILL\s+IN|ADD|TODO|TBD|PLACEHOLDER)[^\]\n]{0,80}\]/g,
  819. // Date stubs.
  820. /\b(?:19|20)\d{2}-XX-XX\b/g,
  821. /\bXX\/XX\/(?:19|20)\d{2}\b/g,
  822. // HTML/Markdown comment placeholders with placeholder verbs.
  823. /<!--\s*(?:add|fill\s+in|insert|todo|placeholder)[^>]{0,120}-->/gi,
  824. ];
  825. // Chatbot citation/markup tokens that leak through copy-paste.
  826. // `citeturn0search0` / `citeturn0news5` from ChatGPT, contentReference
  827. // tokens, oai_citation, attached_file references, grok_card markers.
  828. // Each is a near-definitive signature of a specific tool.
  829. const AI_CITATION_MARKUP = [
  830. /\bcite(?:turn|news|search|navigation)\d+(?:search|turn|news|navigation)\d+/gi,
  831. /contentReference\s*\[oaicite:[^\]]+\]\s*\{[^}]*\}/gi,
  832. /\boai_citation\b/gi,
  833. /\[attached_file:\d+\]/gi,
  834. /\bgrok_card\b/gi,
  835. ];
  836. // UTM/tracking parameters auto-appended by AI tools to URLs they
  837. // generate. Survives copy-paste even when nothing else does.
  838. const AI_UTM_SOURCE = [
  839. /[?&]utm_source=(?:chatgpt|openai|copilot|claude|grok|gemini|perplexity)(?:\.com|\.ai)?\b/gi,
  840. /[?&]referrer=(?:chatgpt|copilot|grok|claude|gemini|perplexity)\.(?:com|ai)\b/gi,
  841. ];
  842. // ─── Template phrases ──────────────────────────────────────────────
  843. const TEMPLATE_PHRASES = [
  844. // template-phrase: only vague-praise adjectives. "a first step towards the full API",
  845. // "a small step towards cutting our storage bill" stay clean.
  846. /\ban?\s+(?:meaningful|significant|major|crucial|important|big|huge|bold|giant|monumental|pivotal|critical|key|vital|massive|tremendous|substantial|decisive|landmark|historic|exciting|transformative)\s+step\s+(?:towards?|forward)\b/gi,
  847. /\bwhether\s+you'?re\s+\w+\s+or\s+\w+/gi,
  848. /\bi\s+recently\s+had\s+the\s+pleasure\s+of\b/gi,
  849. ];
  850. // ─── False concession ──────────────────────────────────────────────
  851. // "While X is impressive, Y remains a challenge" and "Although X has made
  852. // strides, Y is still an open question" only read as the AI tell when both
  853. // halves are vague: an opener that concedes nothing specific, paired with a
  854. // close that names no actual gap. The subject (X) is widened past a single
  855. // word — "while the underlying model architecture is impressive" is as
  856. // hollow as "while it is impressive" — but stays inside one clause (no
  857. // comma or sentence punctuation) so the opener cannot reach across clauses.
  858. // The close is required in the same sentence: a bare opener followed by a
  859. // concrete, specific continuation ("...at this scale, our write pattern is
  860. // append-only, so we moved the hot table to a log-structured store
  861. // instead") is ordinary technical writing, not the empty frame. See #211.
  862. // "Despite X challenges" is dropped entirely: alone it is too common a
  863. // shape in ordinary prose to carry the tell.
  864. // The close must also follow a clause separator (comma, semicolon or
  865. // colon): without one, "While the model is impressive and remains a
  866. // challenge to maintain, we plan to replace it next month" matched both
  867. // phrases inside the opening clause and never looked at the concrete main
  868. // clause that followed. See #359.
  869. // Stop at the next clause separator too: a concrete continuation followed
  870. // by a later vague phrase is not the empty two-half frame.
  871. const FALSE_CONCESSION_SUBJECT = "[^,;:.!?\\n]{1,60}?";
  872. // Known limit: a period inside an abbreviation ("U.S.", "e.g.") ends the
  873. // gap the same as a real sentence boundary would, so a close on the far
  874. // side of one is deliberately missed to keep that boundary guarantee;
  875. // only a comma, semicolon or colon counts as the clause separator #359
  876. // requires.
  877. const FALSE_CONCESSION_GAP = "[^,;:.!?\\n]{0,80}?[,;:]\\s*[^,;:.!?\\n]{0,80}?";
  878. const FALSE_CONCESSION_VAGUE_CLOSE =
  879. "(?:remains?\\s+a\\s+challenge" +
  880. "|(?:is|are)\\s+still\\s+an?\\s+open\\s+questions?" +
  881. "|there\\s+(?:is|are)\\s+still\\s+work\\s+to\\s+do" +
  882. "|remains?\\s+unanswered)\\b";
  883. const FALSE_CONCESSION = [
  884. new RegExp("\\bwhile\\s+" + FALSE_CONCESSION_SUBJECT + "\\s+is\\s+impressive\\b" +
  885. FALSE_CONCESSION_GAP + FALSE_CONCESSION_VAGUE_CLOSE, 'gi'),
  886. new RegExp("\\balthough\\s+" + FALSE_CONCESSION_SUBJECT + "\\s+has\\s+made\\s+strides\\b" +
  887. FALSE_CONCESSION_GAP + FALSE_CONCESSION_VAGUE_CLOSE, 'gi'),
  888. ];
  889. // ─── Rhetorical question openers ───────────────────────────────────
  890. const RHETORICAL_QUESTIONS = [
  891. /\bbut\s+what\s+does\s+this\s+mean\s+for\b/gi,
  892. /\bso\s+why\s+should\s+you\s+care\b/gi,
  893. /\bwhat'?s\s+next\?\s*/gi,
  894. ];
  895. // ─── Hedge-stacked predictions ─────────────────────────────────────
  896. // Stacks a modal with a hedge adverb: "could potentially create new
  897. // opportunities", "may eventually unlock value." Either word alone is
  898. // fine; the stack is the tell.
  899. const HEDGE_STACK = [
  900. // At most one intervening word, and never a negator. The old {0,2} gap
  901. // matched ordinary English: "could not possibly" (plain emphatic
  902. // negation) and inverted questions like "could a savage possibly" both
  903. // fired. Measured on the human-control corpus, 3 of 4 hedge-stack flags
  904. // were this over-match. See issue #69.
  905. /\b(?:could|may|might)\s+(?:(?!not\b|never\b|hardly\b|scarcely\b|barely\b)\w+\s+)?(?:potentially|eventually|ultimately|possibly|conceivably)\b/gi,
  906. /\b(?:potentially|eventually|ultimately)\s+(?:could|may|might)\b/gi,
  907. ];
  908. // ─── Generic future-narrative closers ──────────────────────────────
  909. // The "may become one of the most important narratives" template — vague
  910. // future significance with no falsifiable claim. Covers narratives /
  911. // stories / trends / themes / chapters / movements.
  912. const FUTURE_NARRATIVE = [
  913. /\b(?:may|could|will|is\s+(?:poised|set)\s+to)\s+become\s+(?:one\s+of\s+)?(?:the\s+)?(?:most\s+)?\w+\s+(?:narratives?|stories|developments?|trends?|movements?|chapters?|themes?|forces?)\b/gi,
  914. /\bone\s+of\s+the\s+most\s+important\s+(?:narratives?|stories|trends?|themes?)\s+of\s+the\s+(?:next|coming)\s+\w+\b/gi,
  915. ];
  916. // ─── "Real/actual" adjective inflation ─────────────────────────────
  917. // "Real on-chain tokenomics", "actual reward sustainability" — using
  918. // real/actual/genuine/true as an empty intensifier on an abstract noun
  919. // to imply the rest of the field is fake/superficial.
  920. const REAL_ACTUAL_INFLATION = [
  921. /\b(?:real|actual|genuine|true)\s+(?:on-?chain\s+)?(?:tokenomics|economics|utility|adoption|sustainability|impact|revenue|fundamentals|demand|value|innovation|traction)\b/gi,
  922. ];
  923. // ─── Formulaic openers ─────────────────────────────────────────────
  924. // The "In the rapidly evolving world of X, Y has emerged as..." family
  925. // — LLM-default essay openers.
  926. const FORMULAIC_OPENERS = [
  927. /\bin\s+the\s+(?:rapidly\s+|ever-?\s*)?(?:evolving|changing|expanding|growing|shifting)\s+(?:world|landscape|realm|space|field|domain|era)\s+of\b/gi,
  928. /\bin\s+(?:an?|the)\s+(?:digital\s+)?age\s+(?:where|of)\b/gi,
  929. /\bas\s+(?:we|the\s+world|society|industries?)\s+(?:continue|move|navigate|enter)\s+(?:to\s+)?(?:evolve|forward|into|through)\b/gi,
  930. // "has emerged as a leader/force/category" — gated to the inflated
  931. // nouns that signal pseudo-significance, since bare "has emerged as
  932. // a" matches normal English ("Rust has emerged as a serious systems
  933. // language"). Same gating for "has become increasingly".
  934. /\bhas\s+emerged\s+as\s+(?:a|the|one\s+of)\s+(?:leading|key|major|critical|essential|fundamental|pivotal|prominent|dominant|important)\s+\w+/gi,
  935. /\bhas\s+become\s+increasingly\s+(?:important|critical|popular|relevant|prominent|essential)\b/gi,
  936. ];
  937. // ─── Speculative scenario openers ──────────────────────────────────
  938. // "Imagine a world where...", "Picture a future in which...", "Envision
  939. // a world where..." — the LLM habit of opening an argument with a
  940. // hypothetical that lists desirable outcomes instead of making a claim.
  941. // Gated to the world/future/reality object plus where/in-which so it
  942. // stays off instructional "imagine you have an array" (a teaching device
  943. // pointing at a concrete example, not a speculative world) and bare
  944. // "imagine that" asides. "consider a scenario where…" is deliberately
  945. // excluded: that is analytical framing common in technical reasoning,
  946. // not the marketing-opener tell. An optional short comma interrupter
  947. // catches the equally common "Imagine, for a moment, a world where…"
  948. // cadence. Known accepted false positive: fiction openings and staged
  949. // thought experiments match too — the engine has no fiction context
  950. // mode, so that call is left to the skill's carve-out (highlight-only;
  951. // a lone hit cannot flip a document's classification).
  952. const SPECULATIVE_OPENERS = [
  953. /\b(?:imagine|picture|envision)(?:\s*,[^,\n]{1,30},)?\s+a\s+(?:world|future|reality)\s+(?:where|in\s+which)\b/gi,
  954. ];
  955. // ─── Launch-copy dramatic introductions ────────────────────────────
  956. // "Meet Flowdesk, your new favorite treasury dashboard" / "Think
  957. // Notion meets Figma" — the LLM-default product-introduction move
  958. // in launch and announcement copy. Both surfaces are gated to the
  959. // sentence-initial imperative followed by ONE capitalized token of 2
  960. // to 30 characters, which is a recall limit: a two-token product name
  961. // ("Meet North Star", "Think Google Docs meets Microsoft Word") is a
  962. // deliberate miss. The Meet surface additionally requires one of four
  963. // launch-copy heads — "your new favorite", "your new go-to", and
  964. // "the new home/way/standard", the last three only when followed by
  965. // "of" / "to" / "in|for" or by end-of-clause punctuation. Without
  966. // that tail the head noun swallows a compound noun and ordinary prose
  967. // fires: "Meet Rosa, the new home secretary" and "Meet Emma, the new
  968. // way station manager" both matched before the tail was required.
  969. // Bare "Meet Sarah, your new account manager" is how humans introduce
  970. // colleagues, pets, and babies, so that form stays with the skill's
  971. // judgment side. Two surfaces from
  972. // the same family are deliberately NOT detected. "Say hello to X",
  973. // because "Say hello to Grandma." is ordinary human prose. And bare
  974. // "Enter X.", because the sentence-initial capitalized-noun form is
  975. // how UI and doc instructions are written: "Enter Password.", "Enter
  976. // Amount.", "Enter Username — your work email." Dropping the dash
  977. // terminator does not reach the period-terminated class, and neither
  978. // does a field-name denylist, so that surface stays with the skill's
  979. // judgment side and the UI forms are pinned as must-not-fire
  980. // fixtures. The anchors are lookbehinds so adjacent intros each
  981. // count and the reported span starts at the tell itself.
  982. const LAUNCH_INTROS = [
  983. /(?<=^|[.!?]\s|\n)Meet\s+[A-Z][\w'-]{1,29}\s*,\s*(?:your\s+new\s+(?:favorite|go-to)\b|the\s+new\s+(?:home\s+of\b|way\s+to\b|standard\s+(?:in|for)\b|(?:standard|way|home)(?=\s*(?:[.!?,;:\u2013\u2014]|$))))/g,
  984. /(?<=^|[.!?]\s|\n)[Tt]hink\s+[A-Z][\w'-]{1,29}\s+meets\s+[A-Z][\w'-]{1,29}\b/g,
  985. ];
  986. // ─── Dramatized contrast against the crowd ─────────────────────────
  987. // "shipped it in 2022, while everyone else was still debating
  988. // timelines" — a claim propped on an implied lagging crowd. The gate
  989. // is a dismissive verb PLUS the "was still" dramatization marker,
  990. // because bare "while everyone else" is ordinary simultaneity ("she
  991. // read while everyone else watched the movie") and even the
  992. // dismissive verbs are ordinary English in literal use ("others
  993. // debated the amendment" in wire copy). That gate is on this FIRST
  994. // branch only. Its stems are restricted to the -ing form, so the
  995. // adjective ("was still deliberate about the tradeoff"), the passive
  996. // ("was still debated by pundits") and the bare present ("was still
  997. // debates timelines") all stay clean — allowing e/es/ed let all
  998. // three through. The other two branches carry no "was still"
  999. // requirement: they match their own stereotyped wording ("writing
  1000. // think-pieces", "playing catch-up"), and the skill entry scopes the
  1001. // claim the same way. Verb stems carry explicit inflection tails so
  1002. // agent nouns ("the market speculators") and adverbs ("deliberately
  1003. // ignored") never match. Measured residue, all accepted: branch one
  1004. // fires on ANY literal progressive use of its verbs ("while the
  1005. // market was still speculating about the price"), not only on "was
  1006. // still debating"; branches two and three fire on literal contrasts
  1007. // of their own ("while everyone else wrote think-pieces from
  1008. // Washington", "while everyone else played catch-up in the spring").
  1009. // Recall is deliberately sacrificed: "was busy debating" without
  1010. // "still" stays a miss, per precision-over-recall.
  1011. const CROWD_CONTRAST = [
  1012. /\bwhile\s+(?:everyone\s+else|the\s+(?:industry|market|competition)|others)\s+(?:was|were|is|are)\s+still\s+(?:busy\s+)?(?:(?:debat|deliberat|hesitat|theoriz|philosophiz|pontificat|speculat|argu)ing|(?:dither|bicker)ing)\b/gi,
  1013. /\bwhile\s+(?:everyone\s+else|the\s+(?:industry|market|competition)|others)\s+(?:was\s+|were\s+)?(?:busy\s+)?(?:writing|wrote)\s+think-?\s?pieces\b/gi,
  1014. /\bwhile\s+(?:everyone\s+else|the\s+(?:industry|market|competition)|others)\s+(?:was\s+|were\s+|is\s+|are\s+)?(?:still\s+)?play(?:ed|ing|s)?\s+catch[-\s]?up\b/gi,
  1015. ];
  1016. // ─── Fake-casual props (stage directions and wink asides) ──────────
  1017. // The regexable props from the fake-casual register: theatrical
  1018. // asterisk stage directions ("*checks notes*", "*chef's kiss*",
  1019. // "*mic drop*") and wink asides. Both lists are closed and short, and
  1020. // that is a recall limit: exactly six stage directions ("checks
  1021. // notes", "chef's kiss", "mic drop", "takes a deep breath", "sips
  1022. // coffee|tea", "nervous laughter") and exactly four parentheticals,
  1023. // the full (yes|no) x (really|seriously) grid. Neighbours in the same
  1024. // register are deliberate misses: "*checks calendar*" and "(yes,
  1025. // honestly)" do not fire. The kiss pattern requires the apostrophe —
  1026. // making it optional matched the ordinary sentence "At midnight,
  1027. // *chefs kiss* their spouses goodbye".
  1028. // The rest of the register (one-word verdict closers, label-prefix
  1029. // openers, the self-QA volley) needs register judgment and stays
  1030. // skill-only — "wild." is a word, not a regex target. "because of
  1031. // course …" joins them: a tense gate does not separate the wink from
  1032. // the ordinary human grumble, because "The build failed because of
  1033. // course it did." is that grumble in the same present-tense-plus-did
  1034. // form the wink uses. Under precision-over-recall the surface is
  1035. // judgment-only, and the grumble is pinned as a fixture. The kiss
  1036. // pattern accepts the curly apostrophe (U+2019) — the form smart
  1037. // punctuation and LLMs actually emit. Known accepted false positive:
  1038. // a human writer using a wink aside on purpose; the props are
  1039. // weighted as a strong single hit, not a classification by
  1040. // themselves.
  1041. const FAKE_CASUAL_PROPS = [
  1042. /\*\s?(?:checks\s+notes|chef['\u2019]s\s+kiss|mic\s+drop|takes\s+a\s+deep\s+breath|sips\s+(?:coffee|tea)|nervous\s+laughter)\s?\*/gi,
  1043. /\(\s?(?:yes|no)\s?,\s?(?:really|seriously)\s?\)/gi,
  1044. ];
  1045. // ─── Performed-insight phrases ─────────────────────────────────────
  1046. // Essayist tics that announce profundity instead of delivering it.
  1047. // Curated noun/complement lists keep precision high: "the whole family"
  1048. // and "sit with him" are ordinary English and must not fire. Adapted
  1049. // from Simon Willison's LLM cliché highlighter
  1050. // (tools.simonwillison.net/llm-cliche-highlighter).
  1051. const PERFORMED_INSIGHT = [
  1052. /\bsit(?:s|ting)?\s+with\s+(?:that|this)(?=\s*(?:[.!?,;:)\u2013\u2014\u2019"']|for\s+a\s+(?:moment|minute|second|beat)\b|$))(?:\s+for\s+a\s+(?:moment|minute|second|beat))?/gi,
  1053. /\bsit(?:s|ting)?\s+with\s+(?:the|your)\s+(?:discomfort|tension|uncertainty|ambiguity|grief|unease)\b/gi,
  1054. /\b(?:that|this|it|which)(?:['\u2019]s|\s+(?:is|was))\s+not\s+nothing\b/gi,
  1055. /\byou\s+already\s+know\s+the\s+answer\b/gi,
  1056. /\b(?:do\s+not|don['\u2019]t)\s+(?:have\s+to\s+)?take\s+my\s+word\s+for\s+it\b/gi,
  1057. /(?<=^|[.!?]\s|\n)Turns\s+out\b/g,
  1058. /(?:['\u2019]s|\b(?:is|was|are|were))\s+the\s+(?:whole|entire)\s+(?:point|game|ballgame|trick|pitch|idea|play|business\s+model|value\s+proposition)\b/gi,
  1059. /\b(?:that|this)(?:['\u2019]s|\s+(?:is|was))\s+the\s+part\s+(?:that|I|you|we|nobody|no\s+one|most\s+people)\b/gi,
  1060. /\bthe\s+only\s+[\w'\u2019-]+\s+that\s+(?:matters|counts)\b/gi,
  1061. // "X is dead; long live Y" also names an essay or a literal succession
  1062. // announcement. Whether the contrast earns its place needs reading (#351).
  1063. /\b(?:that|this)(?:['\u2019]s|\s+(?:is|was))\s+why\s+[^.!?\n]{0,60}\s+mattered\b/gi,
  1064. ];
  1065. const STAGED_DISCOVERY = [
  1066. // Staged discovery: a judgment framed as a twist the writer found ("the recording
  1067. // turned out to be the least interesting part"). Superlative + insight noun keeps
  1068. // ordinary "turned out to be the most expensive option" clean.
  1069. /\b(?:turned|turns|turning)\s+out\s+to\s+be\s+the\s+(?:least|most)\s+(?:interesting|important|surprising|revealing|valuable|useful)\s+(?:part|thing|piece|bit)\b/gi,
  1070. // Require a reveal-style continuation; "the real story was covered by..."
  1071. // describes a literal story and is not a staged discovery.
  1072. /\bthe\s+real\s+story\s+(?:here\s+)?(?:is|was)\b(?=\s+(?:the|that|how|why|what)\b)/gi,
  1073. ];
  1074. // ─── Negation chains ───────────────────────────────────────────────
  1075. // "No fluff, no filler, no jargon" / "It didn't ask, didn't wait" /
  1076. // "Don't call it X. Call it Y." Precision guards, in order: the
  1077. // "no …" chain must open its sentence (mid-sentence inventories like
  1078. // "takes no arguments, no headers, and no body" are factual, not
  1079. // rhetorical); the "did not" chain must be comma-joined with the
  1080. // subject elided ("I did not sleep. I did not eat" is ordinary
  1081. // narration and stays clean); the stop-list keeps idiomatic pairs
  1082. // ("no more, no less", "no matter what") from firing. Adapted from
  1083. // Simon Willison's LLM cliché highlighter.
  1084. const NO_ITEM_STOP = "(?!matter\\b|one\\b|doubt\\b|longer\\b|way\\b|less\\b|more\\b|such\\b|other\\b|means\\b)";
  1085. const NO_ITEM_SECOND_STOP = "(?!(?:in|on|at|of|to|for|with|from|by|is|are|was|were|be|been|being|will|would|can|could|should|shall|may|might|must|have|has|had|do|does|did)\\b)";
  1086. const NEGATION_CHAIN = [
  1087. new RegExp(
  1088. "(?<=^|[.!?]\\s|\\n|[:\\u2013\\u2014]\\s)No\\s+" + NO_ITEM_STOP + "[a-z'\u2019-]+(?:\\s+" + NO_ITEM_SECOND_STOP + "[a-z'\u2019-]+)?" +
  1089. "(?:\\s*,\\s*(?:and\\s+|or\\s+|just\\s+)?no\\s+" + NO_ITEM_STOP + "[a-z'\u2019-]+(?:\\s+" + NO_ITEM_SECOND_STOP + "[a-z'\u2019-]+)?){2,}",
  1090. 'gm'
  1091. ),
  1092. /\b(?:did\s+not|didn['\u2019]t)\s+[a-z]+[^,.;!?\n]{0,20},\s*(?:did\s+not|didn['\u2019]t)\s+[a-z]+/gi,
  1093. /\b(?:do\s+not|don['\u2019]t)\s+(?:just\s+)?(\w+)\s+it\b[^.!?\n]{0,60}[.!?;:,][\s'"\u201d\u2019]*(?:just\s+)?\1\s+it\b/gi,
  1094. ];
  1095. // ─── Negative parallelism ──────────────────────────────────────────
  1096. // "It's not just a search index, it's a foundation for trust." A frame is
  1097. // a negated copula, a body X, then a restatement: "it / this / that /
  1098. // they" + be. The engine splits frames in two (#351):
  1099. //
  1100. // reveal a minimizer (just / merely / simply), then a comma,
  1101. // semicolon, colon, or dash and the restatement ("isn't just
  1102. // raining, it's pouring"). The minimizer-then-upgrade move is
  1103. // the tell itself, so a reveal flags on its own.
  1104. // contrast the same joined frame without the minimizer ("isn't X, it's
  1105. // Y", "isn't about X, it's about Y", "are not only X, they're
  1106. // Y") and the split-sentence reveal ("isn't just X. It's Y.").
  1107. // A single plain correction ("It isn't raining, it's
  1108. // snowing.") is ordinary English, and references/patterns.md
  1109. // allows one frame per piece, so a contrast flags only when
  1110. // another frame of either kind starts within
  1111. // NP_WINDOW_SENTENCES sentences of it. The stacked cadence is
  1112. // the tell; two unrelated corrections paragraphs apart are
  1113. // ordinary prose. "only" is not a reveal minimizer: the human
  1114. // control corpus holds "fossil fuels are not only bad for our
  1115. // environment, they're a losing bet".
  1116. //
  1117. // The restated pronoun is the gate. "not only X but (also) Y" and "not X
  1118. // but Y" are ordinary correlatives and are not matched: on the human
  1119. // control corpus "not only ... but" appeared 16 times in 143k human words
  1120. // against 5 in 115k machine words. X is capped at 80 characters with no
  1121. // comma or sentence punctuation, so a frame cannot reach across clauses.
  1122. const NP_NEG = "(?:\\b(?:is|are|was|were)(?:n['\\u2019]t|\\s+not)|\\b(?:it|this|that|they|he|she|we|you)['\\u2019](?:s|re)\\s+not)";
  1123. const NP_MINIMIZER = "(?:just|merely|simply)";
  1124. const NP_BODY = "[^,;:.!?\\n\\u2014\\u2013]{1,80}?";
  1125. const NP_JOIN = "(?:\\s*[,;:]|\\s*[\\u2014\\u2013]|\\s+--)\\s*";
  1126. const NP_RESTATE = "(?:it|this|that|they)(?:['\\u2019](?:s|re)|\\s+(?:is|are|was|were))\\b";
  1127. const NEGATIVE_PARALLELISM_REVEAL = [
  1128. new RegExp(NP_NEG + "\\s+" + NP_MINIMIZER + "\\s+" + NP_BODY + NP_JOIN + NP_RESTATE, 'gi'),
  1129. ];
  1130. const NEGATIVE_PARALLELISM_CONTRAST = [
  1131. new RegExp(NP_NEG + "\\s+(?!" + NP_MINIMIZER + "\\b)" + NP_BODY + NP_JOIN + NP_RESTATE, 'gi'),
  1132. // Split-sentence reveal: the restatement opens the next sentence.
  1133. new RegExp(NP_NEG + "\\s+" + NP_MINIMIZER + "\\s+" + NP_BODY + "[.!]\\s+" + NP_RESTATE, 'gi'),
  1134. ];
  1135. // Two frames pair when their starting sentences are at most this many
  1136. // sentences apart: the same sentence, the next, or the one after.
  1137. const NP_WINDOW_SENTENCES = 2;
  1138. // A blank line in LF, CRLF, or CR-only text.
  1139. const NP_PARAGRAPH_BREAK = /(?:\r\n|\r(?!\n)|\n)[ \t]*(?:\r\n|\r|\n)/;
  1140. // Reveals always flag. A contrast flags only when some other frame, reveal
  1141. // or contrast, starts nearby. Sentence indexes come from the same coarse
  1142. // splitter the highlight regions use.
  1143. function negativeParallelismIssues(text, reveals, contrasts) {
  1144. // Apply the proximity gate to the same distinct frames the caller reports.
  1145. const distinctReveals = deduplicateIssues(reveals);
  1146. const distinctContrasts = deduplicateIssues(contrasts);
  1147. if (distinctContrasts.length === 0 || distinctReveals.length + distinctContrasts.length < 2) return distinctReveals;
  1148. const frames = [...distinctReveals, ...distinctContrasts];
  1149. const starts = splitSentenceSpans(text).map(([start]) => start);
  1150. const sentenceOf = (index) => {
  1151. let lo = 0;
  1152. let hi = starts.length - 1;
  1153. while (lo < hi) {
  1154. const mid = (lo + hi + 1) >> 1;
  1155. if (starts[mid] <= index) lo = mid;
  1156. else hi = mid - 1;
  1157. }
  1158. return lo;
  1159. };
  1160. const sentences = frames.map((frame) => sentenceOf(frame.index));
  1161. // Paired frames must also share a paragraph: no blank line between them.
  1162. const sameParagraph = (a, b) => !NP_PARAGRAPH_BREAK.test(text.slice(Math.min(a, b), Math.max(a, b)));
  1163. const paired = distinctContrasts.filter((contrast, c) => {
  1164. const i = distinctReveals.length + c;
  1165. return sentences.some((other, j) =>
  1166. j !== i
  1167. && frames[j].index !== contrast.index
  1168. && Math.abs(other - sentences[i]) <= NP_WINDOW_SENTENCES
  1169. && sameParagraph(frames[j].index, contrast.index));
  1170. });
  1171. return [...distinctReveals, ...paired].sort((a, b) => a.index - b.index);
  1172. }
  1173. // ─── Dev-blog boilerplate ──────────────────────────────────────────
  1174. // Stock simplicity slogans from developer marketing. Adapted from
  1175. // Simon Willison's LLM cliché highlighter.
  1176. const DEV_BLOG_BOILERPLATE = [
  1177. // Separators inside "out of the box": ASCII hyphen or whitespace, plus the
  1178. // Unicode hyphens U+2010 and U+2011 and the en dash U+2013 that editors
  1179. // and autocorrect substitute for a hyphen (#140). The em dash and U+2015
  1180. // mark a break ("…works out—of the box office take…"), not a compound, and
  1181. // U+2012 is a figure dash for digit groups, so those stay excluded.
  1182. /\bit\s+just\s+works\b(?!\s+out\b(?![-\s‐‑–]+of[-\s‐‑–]+the[-\s‐‑–]+box\b))/gi,
  1183. /\bzero[-\s]config(?:uration)?\b/gi,
  1184. /\bsane\s+defaults\b/gi,
  1185. /\b(?:hold|fit|fits|holds)\s+in\s+your\s+head\b/gi,
  1186. ];
  1187. // Function words whose presence MID-title marks the AI section-header shape.
  1188. // Word-anchored: without \b the "A" alternative matches inside any word and
  1189. // the guard silently degrades to "four tokens".
  1190. const FUNCTION_WORD = /\b(?:And|Or|Of|The|In|For|To|A|An)\b/;
  1191. // Must accept exactly what TITLE_CASE_HEADER accepts, or the prefix survives
  1192. // into the token count and reintroduces the ##-as-token bug.
  1193. const MD_HEADING_PREFIX = /^#{1,6}[ \t]+/;
  1194. /** Byte ranges covered by fenced code blocks, computed once per scan.
  1195. *
  1196. * A document that documents Markdown is the normal case for this rule -- a
  1197. * fenced `## Heading` example is illustration, not the author's own section
  1198. * header, and flagging it makes every docs page flag itself.
  1199. *
  1200. * This tracks the opening delimiter instead of counting them, because a
  1201. * parity count is wrong on the very case the rule exists for. CommonMark
  1202. * closes a fence only on the same character at the same length or longer, so
  1203. * a four-backtick fence wrapping a three-backtick example -- exactly how you
  1204. * document fences -- nests in practice, and counting delimiters inverts on
  1205. * it. Up to three spaces of indent are legal. An unclosed fence runs to end
  1206. * of document, matching how renderers treat it.
  1207. *
  1208. * Computed once per scan rather than rescanned per hit: the previous version
  1209. * sliced the whole document for every candidate, which is quadratic on a
  1210. * heading-dense file. */
  1211. function fenceRanges(text) {
  1212. const re = /^[ \t]{0,3}(`{3,}|~{3,})([^\n]*)$/gm;
  1213. const ranges = [];
  1214. let open = null;
  1215. let m;
  1216. while ((m = re.exec(text)) !== null) {
  1217. const marker = m[1];
  1218. if (!open) {
  1219. open = { char: marker[0], len: marker.length, start: m.index };
  1220. } else if (
  1221. marker[0] === open.char &&
  1222. marker.length >= open.len &&
  1223. /^[ \t]*\r?$/.test(m[2])
  1224. ) {
  1225. ranges.push([open.start, m.index + m[0].length]);
  1226. open = null;
  1227. }
  1228. }
  1229. if (open) ranges.push([open.start, text.length]);
  1230. return ranges;
  1231. }
  1232. function inFenceRange(ranges, index) {
  1233. return typeof index === 'number' && ranges.some(([a, b]) => index >= a && index < b);
  1234. }
  1235. function blankRange(chars, start, end) {
  1236. for (let i = start; i < end && i < chars.length; i += 1) {
  1237. if (chars[i] !== '\n') chars[i] = ' ';
  1238. }
  1239. }
  1240. function inlineCodeRanges(text) {
  1241. const runs = [];
  1242. for (let i = 0; i < text.length;) {
  1243. if (text[i] === '\n') {
  1244. runs.push(null);
  1245. i += 1;
  1246. continue;
  1247. }
  1248. if (text[i] !== '`') {
  1249. i += 1;
  1250. continue;
  1251. }
  1252. const start = i;
  1253. while (i < text.length && text[i] === '`') i += 1;
  1254. runs.push({ start, end: i, length: i - start });
  1255. }
  1256. const ranges = [];
  1257. let lineStart = 0;
  1258. while (lineStart < runs.length) {
  1259. let lineEnd = runs.indexOf(null, lineStart);
  1260. if (lineEnd === -1) lineEnd = runs.length;
  1261. const nextByLength = new Map();
  1262. const nextSame = new Array(lineEnd - lineStart);
  1263. for (let i = lineEnd - 1; i >= lineStart; i -= 1) {
  1264. nextSame[i - lineStart] = nextByLength.get(runs[i].length);
  1265. nextByLength.set(runs[i].length, i);
  1266. }
  1267. for (let i = lineStart; i < lineEnd;) {
  1268. const close = nextSame[i - lineStart];
  1269. if (close === undefined) {
  1270. i += 1;
  1271. continue;
  1272. }
  1273. ranges.push([runs[i].start, runs[close].end]);
  1274. i = close + 1;
  1275. }
  1276. lineStart = lineEnd + 1;
  1277. }
  1278. return ranges;
  1279. }
  1280. // Copy of the text with fenced blocks and inline code spans blanked out.
  1281. // Index-preserving: each masked character becomes a space and newlines are
  1282. // kept, so offsets into the result still address the same position in the
  1283. // original. For rules where a character inside code is something the author
  1284. // is quoting rather than using — a `#fff` in a CSS sample is not a tag.
  1285. // Fences are blanked first so their backticks cannot pair with a later
  1286. // inline span and swallow the prose between them.
  1287. function maskCode(text) {
  1288. const chars = text.split('');
  1289. for (const [a, b] of fenceRanges(text)) blankRange(chars, a, b);
  1290. // Indented code blocks are deliberately NOT masked. Four spaces is a code
  1291. // block only at top level; under a list marker it is a paragraph
  1292. // continuation, so blanking it silences real tag blocks. #90 reports
  1293. // fences and inline spans, and those are what this masks.
  1294. const withoutFences = chars.join('');
  1295. for (const [start, end] of inlineCodeRanges(withoutFences)) blankRange(chars, start, end);
  1296. return chars.join('');
  1297. }
  1298. function initialFrontmatterRange(text) {
  1299. const lines = [];
  1300. const lineRe = /[^\r\n]*(?:\r\n|\n|\r|$)/g;
  1301. let match;
  1302. while ((match = lineRe.exec(text)) !== null && match[0]) {
  1303. const body = match[0].replace(/(?:\r\n|\n|\r)$/, '');
  1304. lines.push({ body, start: match.index, end: match.index + body.length });
  1305. }
  1306. if (lines.length < 3 || !/^---[ \t]*$/.test(lines[0].body.replace(/^\uFEFF/, ''))) return null;
  1307. let closingLine = -1;
  1308. for (let i = 1; i < lines.length; i += 1) {
  1309. if (/^---[ \t]*$/.test(lines[i].body)) {
  1310. closingLine = i;
  1311. break;
  1312. }
  1313. }
  1314. if (closingLine === -1) return null;
  1315. // A pair of thematic breaks can also surround ordinary Markdown prose.
  1316. // Require the first substantive line to begin like a YAML mapping entry
  1317. // before treating the delimited block as frontmatter. Leading blank lines
  1318. // and YAML comments are valid, and the line parser accepts LF, CRLF, or CR.
  1319. const yamlKey = /^[ \t]*(?:[A-Za-z0-9_.-]+|"[^"\r\n]+"|'[^'\r\n]+')[ \t]*:/;
  1320. const firstContent = lines
  1321. .slice(1, closingLine)
  1322. .find((line) => line.body.trim() && !/^[ \t]*#/.test(line.body));
  1323. if (!firstContent || !yamlKey.test(firstContent.body)) return null;
  1324. return { start: 0, end: lines[closingLine].end };
  1325. }
  1326. // Mask HTML comments in source order while tracking the Markdown constructs
  1327. // that protect a literal `<!--`. A comment wins over code delimiters that
  1328. // occur inside it; a fence, code span, or top-level indented block that
  1329. // starts first wins over comment-looking text inside that code. Each source
  1330. // character participates in a bounded number of forward scans.
  1331. function maskHtmlCommentsOutsideCode(chars) {
  1332. const source = chars.join('');
  1333. const lines = source.split('\n');
  1334. let offset = 0;
  1335. let openFence = null;
  1336. let inIndentedBlock = false;
  1337. let previousBlank = true;
  1338. let listContext = false;
  1339. let maskedHtmlComments = 0;
  1340. const commentClosings = [];
  1341. let closingCursor = 0;
  1342. for (let i = 0; i <= source.length - 3; i += 1) {
  1343. if (source[i] === '-' && source[i + 1] === '-' && source[i + 2] === '>') {
  1344. commentClosings.push(i);
  1345. }
  1346. }
  1347. const backtickRuns = (line) => {
  1348. const runs = [];
  1349. for (let i = 0; i < line.length;) {
  1350. if (line[i] !== '`') {
  1351. i += 1;
  1352. continue;
  1353. }
  1354. const start = i;
  1355. while (i < line.length && line[i] === '`') i += 1;
  1356. runs.push({ start, end: i, length: i - start, next: -1 });
  1357. }
  1358. const nextByLength = new Map();
  1359. for (let i = runs.length - 1; i >= 0; i -= 1) {
  1360. runs[i].next = nextByLength.get(runs[i].length) ?? -1;
  1361. nextByLength.set(runs[i].length, i);
  1362. }
  1363. return runs;
  1364. };
  1365. for (const originalLine of lines) {
  1366. const lineEnd = offset + originalLine.length;
  1367. let visibleLine = chars.slice(offset, lineEnd).join('');
  1368. const fenceMatch = /^[ \t]{0,3}(`{3,}|~{3,})([^\n]*)$/.exec(visibleLine);
  1369. let fencedLine = false;
  1370. if (openFence) {
  1371. fencedLine = true;
  1372. if (
  1373. fenceMatch
  1374. && fenceMatch[1][0] === openFence.char
  1375. && fenceMatch[1].length >= openFence.length
  1376. && /^[ \t]*\r?$/.test(fenceMatch[2])
  1377. ) openFence = null;
  1378. } else if (fenceMatch) {
  1379. fencedLine = true;
  1380. openFence = { char: fenceMatch[1][0], length: fenceMatch[1].length };
  1381. }
  1382. const indented = /^(?: {4}|\t)\S/.test(visibleLine);
  1383. const indentedCode = !fencedLine
  1384. && indented
  1385. && (inIndentedBlock || (previousBlank && !listContext));
  1386. if (!fencedLine && !indentedCode) {
  1387. const runs = backtickRuns(visibleLine);
  1388. let runIndex = 0;
  1389. let cursor = 0;
  1390. while (cursor < visibleLine.length) {
  1391. while (runIndex < runs.length && runs[runIndex].start < cursor) runIndex += 1;
  1392. const commentIndex = visibleLine.indexOf('<!--', cursor);
  1393. const run = runs[runIndex];
  1394. if (run && (commentIndex === -1 || run.start < commentIndex)) {
  1395. if (run.next !== -1) {
  1396. cursor = runs[run.next].end;
  1397. runIndex = run.next + 1;
  1398. } else {
  1399. cursor = run.end;
  1400. runIndex += 1;
  1401. }
  1402. continue;
  1403. }
  1404. if (commentIndex === -1) break;
  1405. const openingIndex = offset + commentIndex;
  1406. while (
  1407. closingCursor < commentClosings.length
  1408. && commentClosings[closingCursor] < openingIndex + 2
  1409. ) closingCursor += 1;
  1410. const closingIndex = commentClosings[closingCursor] ?? -1;
  1411. const end = closingIndex === -1 ? source.length : closingIndex + 3;
  1412. if (closingIndex !== -1) closingCursor += 1;
  1413. blankRange(chars, openingIndex, end);
  1414. maskedHtmlComments += 1;
  1415. cursor = Math.min(visibleLine.length, end - offset);
  1416. }
  1417. }
  1418. visibleLine = chars.slice(offset, lineEnd).join('');
  1419. const layoutChars = visibleLine.split('');
  1420. if (fencedLine) {
  1421. blankRange(layoutChars, 0, layoutChars.length);
  1422. } else {
  1423. const inlineRe = /(`+)(?:(?!\1)[^\n])+\1/g;
  1424. let inlineMatch;
  1425. while ((inlineMatch = inlineRe.exec(visibleLine)) !== null) {
  1426. blankRange(layoutChars, inlineMatch.index, inlineMatch.index + inlineMatch[0].length);
  1427. }
  1428. }
  1429. const layoutLine = layoutChars.join('');
  1430. const blank = layoutLine.trim() === '';
  1431. if (indentedCode) inIndentedBlock = true;
  1432. else if (!blank) inIndentedBlock = false;
  1433. if (!blank && !indentedCode && !fencedLine) {
  1434. if (/^ {0,3}(?:[-*+]|\d{1,9}[.)])(?:\s|$)/.test(layoutLine)) listContext = true;
  1435. else if (/^\S/.test(layoutLine)) listContext = false;
  1436. }
  1437. previousBlank = blank;
  1438. offset = lineEnd + 1;
  1439. }
  1440. return maskedHtmlComments;
  1441. }
  1442. // Mask source-only Markdown spans while preserving source offsets. The
  1443. // detector can then score what a reader sees without making later issue
  1444. // indexes or sentence highlights point at the wrong source location.
  1445. function maskRenderedMarkdown(text) {
  1446. const chars = text.split('');
  1447. let maskedFrontmatter = 0;
  1448. const frontmatter = initialFrontmatterRange(text);
  1449. if (frontmatter) {
  1450. blankRange(chars, frontmatter.start, frontmatter.end);
  1451. maskedFrontmatter = 1;
  1452. }
  1453. const maskedHtmlComments = maskHtmlCommentsOutsideCode(chars);
  1454. return { text: chars.join(''), maskedFrontmatter, maskedHtmlComments };
  1455. }
  1456. // Ignore regions (#351): an author can exclude a passage from scoring, such
  1457. // as a specimen of AI prose quoted on purpose, by wrapping it in
  1458. // <!-- avoid-ai-writing:ignore-start --> … <!-- avoid-ai-writing:ignore-end -->
  1459. // A marker counts only as a whole line: the full comment, at most three
  1460. // spaces of indent, nothing else on the line. Starts nest: each start needs
  1461. // its own end, and the region runs from the outermost start to its matching
  1462. // end. An unclosed start runs to the end of the text; an end with no open
  1463. // start is ignored. The region is blanked in place, so offsets still
  1464. // address the source.
  1465. //
  1466. // Markers are found by ONE left-to-right scan rather than a stack of masks,
  1467. // because separate masks disagree about who owns overlapping text: a fence
  1468. // inside a comment would swallow the comment's `-->`, and a `<pre>` inside
  1469. // a comment would open a container. After skipping initial YAML
  1470. // frontmatter, the scan recognizes, in order of appearance, an HTML
  1471. // comment, a Markdown fence, a top-level indented code line, an inline code
  1472. // span, and an HTML <pre>, <code>, <script>, or <style> element. Whichever
  1473. // construct opens first owns the text until its own close, so a marker, a
  1474. // fence, a tag, or a comment inside another construct is inert. Anything
  1475. // unclosed runs to the end of the text: when in doubt, the marker does
  1476. // nothing and the prose stays scored. Every character is visited a bounded
  1477. // number of times.
  1478. const IGNORE_MARKER_LINE_RE = /^ {0,3}<!--[ \t]*avoid-ai-writing:ignore-(start|end)[ \t]*-->[ \t]*$/i;
  1479. const MARKER_FENCE_RE = /^ {0,3}(`{3,}|~{3,})(.*)$/;
  1480. const HTML_CODE_OPEN_RE = /<(pre|code|script|style)\b[^>]*>/iy;
  1481. const HTML_CODE_CLOSE_RE = {
  1482. pre: /<\/pre/gi,
  1483. code: /<\/code/gi,
  1484. script: /<\/script/gi,
  1485. style: /<\/style/gi,
  1486. };
  1487. function findIgnoreMarkers(text) {
  1488. const n = text.length;
  1489. const markers = [];
  1490. const lineEndAt = (from) => {
  1491. let i = from;
  1492. while (i < n && text[i] !== '\n' && text[i] !== '\r') i += 1;
  1493. return i;
  1494. };
  1495. const nextLineAt = (end) => (text[end] === '\r' && text[end + 1] === '\n' ? end + 2 : end + 1);
  1496. const isLineStart = (i) => i === 0 || text[i - 1] === '\n' || (text[i - 1] === '\r' && text[i] !== '\n');
  1497. // Backtick runs in [from, to), keyed by start, each with the end of the
  1498. // next run of the same length on the line (-1 when unmatched). Built at
  1499. // most once per line segment and never rebuilt after a comment or HTML
  1500. // element closes mid-line: rebuilding made a long line that alternates
  1501. // code spans and <code> elements quadratic. Runs inside a construct the
  1502. // scan jumps over are simply never visited.
  1503. const backtickRuns = (from, to) => {
  1504. const list = [];
  1505. for (let i = from; i < to;) {
  1506. if (text[i] !== '`') { i += 1; continue; }
  1507. const start = i;
  1508. while (i < to && text[i] === '`') i += 1;
  1509. list.push({ start, end: i, length: i - start, closeEnd: -1 });
  1510. }
  1511. const nextByLength = new Map();
  1512. for (let i = list.length - 1; i >= 0; i -= 1) {
  1513. const next = nextByLength.get(list[i].length);
  1514. if (next) list[i].closeEnd = next.end;
  1515. nextByLength.set(list[i].length, list[i]);
  1516. }
  1517. return new Map(list.map((run) => [run.start, run]));
  1518. };
  1519. let pos = 0;
  1520. const frontmatter = initialFrontmatterRange(text);
  1521. if (frontmatter) pos = frontmatter.end;
  1522. let lineEnd = -1;
  1523. let runs = null;
  1524. let prevBlank = true;
  1525. let inIndented = false;
  1526. let listContext = false;
  1527. while (pos < n) {
  1528. if (pos > lineEnd) {
  1529. // Entering a new line, or resuming mid-line after a construct that
  1530. // spanned lines. Only a true line start gets the line-level checks.
  1531. lineEnd = lineEndAt(pos);
  1532. runs = null;
  1533. if (isLineStart(pos)) {
  1534. const line = text.slice(pos, lineEnd);
  1535. const blank = line.trim() === '';
  1536. const fence = MARKER_FENCE_RE.exec(line);
  1537. if (fence && !(fence[1][0] === '`' && fence[2].includes('`'))) {
  1538. let close = n;
  1539. for (let scan = nextLineAt(lineEnd); scan < n;) {
  1540. const end = lineEndAt(scan);
  1541. const closing = MARKER_FENCE_RE.exec(text.slice(scan, end));
  1542. if (
  1543. closing
  1544. && closing[1][0] === fence[1][0]
  1545. && closing[1].length >= fence[1].length
  1546. && closing[2].trim() === ''
  1547. ) {
  1548. close = end;
  1549. break;
  1550. }
  1551. scan = nextLineAt(end);
  1552. }
  1553. pos = close;
  1554. prevBlank = false;
  1555. inIndented = false;
  1556. continue;
  1557. }
  1558. const indented = !blank && /^(?: {4}|\t)/.test(line);
  1559. if (indented && (inIndented || (prevBlank && !listContext))) {
  1560. inIndented = true;
  1561. prevBlank = false;
  1562. pos = lineEnd;
  1563. continue;
  1564. }
  1565. if (!blank) {
  1566. inIndented = false;
  1567. if (/^ {0,3}(?:[-*+]|\d{1,9}[.)])(?:\s|$)/.test(line)) listContext = true;
  1568. else if (/^\S/.test(line)) listContext = false;
  1569. }
  1570. prevBlank = blank;
  1571. const marker = IGNORE_MARKER_LINE_RE.exec(line);
  1572. if (marker) {
  1573. markers.push({ kind: marker[1].toLowerCase(), start: pos, end: lineEnd });
  1574. pos = lineEnd;
  1575. continue;
  1576. }
  1577. }
  1578. }
  1579. const ch = text[pos];
  1580. if (ch === '<') {
  1581. if (text.startsWith('<!--', pos)) {
  1582. const close = text.indexOf('-->', pos + 4);
  1583. pos = close === -1 ? n : close + 3;
  1584. continue;
  1585. }
  1586. HTML_CODE_OPEN_RE.lastIndex = pos;
  1587. const open = HTML_CODE_OPEN_RE.exec(text);
  1588. if (open) {
  1589. const closeRe = HTML_CODE_CLOSE_RE[open[1].toLowerCase()];
  1590. closeRe.lastIndex = pos + open[0].length;
  1591. const close = closeRe.exec(text);
  1592. pos = close === null ? n : close.index;
  1593. continue;
  1594. }
  1595. } else if (ch === '`') {
  1596. if (!runs) runs = backtickRuns(pos, lineEnd);
  1597. const run = runs.get(pos);
  1598. if (run && run.closeEnd !== -1) {
  1599. pos = run.closeEnd;
  1600. continue;
  1601. }
  1602. pos += run ? run.length : 1;
  1603. continue;
  1604. }
  1605. pos += 1;
  1606. }
  1607. return markers;
  1608. }
  1609. function maskIgnoreRegions(text) {
  1610. if (!/avoid-ai-writing:ignore-/i.test(text)) return { text, ignoredRegions: 0 };
  1611. const chars = text.split('');
  1612. let ignoredRegions = 0;
  1613. let depth = 0;
  1614. let openAt = -1;
  1615. for (const marker of findIgnoreMarkers(text)) {
  1616. if (marker.kind === 'start') {
  1617. if (depth === 0) openAt = marker.start;
  1618. depth += 1;
  1619. } else if (depth > 0) {
  1620. depth -= 1;
  1621. if (depth === 0) {
  1622. blankRange(chars, openAt, marker.end);
  1623. ignoredRegions += 1;
  1624. }
  1625. }
  1626. }
  1627. if (depth > 0) {
  1628. blankRange(chars, openAt, chars.length);
  1629. ignoredRegions += 1;
  1630. }
  1631. return { text: chars.join(''), ignoredRegions };
  1632. }
  1633. // Blank the content of double-quoted spans and keep the quote marks, so
  1634. // offsets and the curly-quote signal survive. Based on QUOTED_SPAN in
  1635. // scripts/self-scan.js minus its single-quote branch: apostrophes in
  1636. // contractions and possessives would pair up across ordinary prose. A
  1637. // straight quote touching a letter or digit on its outer side is an inch
  1638. // mark, not a quotation. Empty `""` pairs are consumed so they cannot
  1639. // shift the pairing, and a span never crosses a backtick into code. A
  1640. // nested quotation escaped as a pair (`\"…\"`) stays inside the span. A
  1641. // lone `\"` still closes it, so a path such as `"C:\Temp\"` cannot run on
  1642. // into the next quotation.
  1643. const QUOTED_SPAN_RE = /(?<![\p{L}\p{N}])"(?:\\"[^"`\n]{0,300}\\"|[^"`\n]){0,300}"(?![\p{L}\p{N}])|“[^“”`\n]{0,300}”/gu;
  1644. function maskQuotedSpans(text) {
  1645. let maskedQuotes = 0;
  1646. const masked = text.replace(QUOTED_SPAN_RE, (span) => {
  1647. // Escaped pairs can chain past the 300-character cap; score those.
  1648. // Count code points, as the `u` regex does, so emoji do not trip it.
  1649. if (span.length === 2 || [...span].length > 302) return span;
  1650. maskedQuotes += 1;
  1651. // Keep sentence punctuation so getSentences splits where it did before.
  1652. return span[0] + span.slice(1, -1).replace(/[^.!?]/g, ' ') + span[span.length - 1];
  1653. });
  1654. return { text: masked, maskedQuotes };
  1655. }
  1656. // A `>` opens a blockquote line when a space, the line end, a letter, a
  1657. // nested `>`, an opening double quote, emphasis, a link, or a list marker
  1658. // (`- `, `+ `, `1. `) follows it. That covers compact Markdown (`>text`)
  1659. // without swallowing comparisons such as `>=5` or `>-1`. A single quote is
  1660. // left out, as in the inline quote pass.
  1661. const BLOCKQUOTE_LINE_RE = /^\s*>(?:$|[\s\p{L}>"“*_[]|[-+]\s|\d{1,9}[.)]\s)/u;
  1662. function maskBlockquotes(text) {
  1663. const chars = text.split('');
  1664. const lines = [];
  1665. const lineRe = /[^\r\n]*(?:\r\n|\n|\r|$)/g;
  1666. let match;
  1667. while ((match = lineRe.exec(text)) !== null && match[0]) {
  1668. const body = match[0].replace(/(?:\r\n|\n|\r)$/, '');
  1669. lines.push({ text: body, start: match.index, end: match.index + body.length });
  1670. }
  1671. let quotedLines = 0;
  1672. for (const line of lines) {
  1673. if (BLOCKQUOTE_LINE_RE.test(line.text)) {
  1674. blankRange(chars, line.start, line.end);
  1675. quotedLines += 1;
  1676. }
  1677. }
  1678. return { text: chars.join(''), quotedLines };
  1679. }
  1680. // Keep the historical deletion behavior for default plain mode. Paragraph-
  1681. // scoped rules depend on the surrounding lines being rejoined exactly this
  1682. // way, so changing this prepass would change scores for existing callers.
  1683. function stripBlockquotes(text, sourceMap) {
  1684. const rawLines = text.split(/\r?\n/);
  1685. const stripIndexes = new Set();
  1686. let blankedLines = 0;
  1687. for (let i = 0; i < rawLines.length; i += 1) {
  1688. // A bare CR is not split here, so a CR-only document is one element.
  1689. // Deleting it would take the unquoted lines after the quote with it,
  1690. // so blank each quoted CR-separated line in place instead. Blanking
  1691. // keeps the element length, so the offsets below still line up.
  1692. if (rawLines[i].includes('\r')) {
  1693. rawLines[i] = rawLines[i].split('\r').map((line) => {
  1694. if (!BLOCKQUOTE_LINE_RE.test(line)) return line;
  1695. blankedLines += 1;
  1696. return ' '.repeat(line.length);
  1697. }).join('\r');
  1698. } else if (BLOCKQUOTE_LINE_RE.test(rawLines[i])) {
  1699. stripIndexes.add(i);
  1700. }
  1701. }
  1702. const kept = rawLines
  1703. .map((_, index) => index)
  1704. .filter((index) => !stripIndexes.has(index));
  1705. const result = {
  1706. text: kept.map((index) => rawLines[index]).join('\n'),
  1707. quotedLines: stripIndexes.size + blankedLines,
  1708. };
  1709. if (!Array.isArray(sourceMap)) return result;
  1710. const lineStarts = [];
  1711. let offset = 0;
  1712. for (let i = 0; i < rawLines.length; i += 1) {
  1713. lineStarts.push(offset);
  1714. offset += rawLines[i].length;
  1715. if (i < rawLines.length - 1) {
  1716. if (text[offset] === '\r') offset += 1;
  1717. if (text[offset] === '\n') offset += 1;
  1718. }
  1719. }
  1720. const mapped = [];
  1721. for (let i = 0; i < kept.length; i += 1) {
  1722. const lineIndex = kept[i];
  1723. const start = lineStarts[lineIndex];
  1724. appendMapRange(mapped, sourceMap, start, start + rawLines[lineIndex].length);
  1725. if (i < kept.length - 1) {
  1726. const separatorStart = start + rawLines[lineIndex].length;
  1727. const newlineIndex = text[separatorStart] === '\r' ? separatorStart + 1 : separatorStart;
  1728. mapped.push(sourceMap[newlineIndex]);
  1729. }
  1730. }
  1731. return { ...result, sourceMap: mapped };
  1732. }
  1733. function maskTopLevelIndentedCode(chars, { listAware = false } = {}) {
  1734. const lines = chars.join('').split('\n');
  1735. let offset = 0;
  1736. let inBlock = false;
  1737. let previousBlank = true;
  1738. let listContext = false;
  1739. for (let i = 0; i < lines.length; i += 1) {
  1740. const line = lines[i];
  1741. const indented = /^(?: {4}|\t)\S/.test(line);
  1742. const blank = line.trim() === '';
  1743. const isCode = indented && (inBlock || (previousBlank && (!listAware || !listContext)));
  1744. if (isCode) {
  1745. blankRange(chars, offset, offset + line.length);
  1746. inBlock = true;
  1747. } else if (!blank) {
  1748. inBlock = false;
  1749. }
  1750. if (listAware && !blank && !isCode) {
  1751. if (/^ {0,3}(?:[-*+]|\d{1,9}[.)])(?:\s|$)/.test(line)) listContext = true;
  1752. else if (/^\S/.test(line)) listContext = false;
  1753. }
  1754. previousBlank = blank;
  1755. offset += line.length + 1;
  1756. }
  1757. }
  1758. function maskYamlFrontmatter(chars) {
  1759. const lines = chars.join('').split('\n');
  1760. const bare = (line) => line.replace(/\r$/, '');
  1761. const first = bare(lines[0]).replace(/^\uFEFF/, '');
  1762. if (first !== '---' || lines.length < 2 || /^\s*$/.test(bare(lines[1]))) return;
  1763. let closingLine = -1;
  1764. for (let i = 1; i < lines.length; i += 1) {
  1765. if (bare(lines[i]) === '---') {
  1766. closingLine = i;
  1767. break;
  1768. }
  1769. }
  1770. if (closingLine === -1) return;
  1771. let end = 0;
  1772. for (let i = 0; i <= closingLine; i += 1) {
  1773. end += lines[i].length;
  1774. if (i < lines.length - 1) end += 1;
  1775. }
  1776. blankRange(chars, 0, end);
  1777. }
  1778. function maskYamlMetadata(chars) {
  1779. const lines = chars.join('').split('\n');
  1780. let offset = 0;
  1781. let nestedAfterIndent = null;
  1782. for (const line of lines) {
  1783. const bare = line.replace(/\r$/, '');
  1784. // Lowercase keys are the common unfenced-YAML shape. Keeping this
  1785. // case-sensitive prevents prose labels such as "Note: ..." from being
  1786. // mistaken for metadata and silencing a real copyedit on the line.
  1787. const key = bare.match(/^([ \t]*)(?:-[ \t]+)?[a-z_][a-z0-9_.-]*[ \t]*:(.*)$/);
  1788. const indentation = (bare.match(/^[ \t]*/) || [''])[0].length;
  1789. let shouldMask = false;
  1790. if (key) {
  1791. shouldMask = true;
  1792. nestedAfterIndent = key[2].trim() === '' ? key[1].length : null;
  1793. } else if (nestedAfterIndent !== null && bare.trim() !== '' && indentation > nestedAfterIndent) {
  1794. shouldMask = true;
  1795. } else if (bare.trim() === '') {
  1796. nestedAfterIndent = null;
  1797. } else {
  1798. nestedAfterIndent = null;
  1799. }
  1800. if (shouldMask) blankRange(chars, offset, offset + line.length);
  1801. offset += line.length + 1;
  1802. }
  1803. }
  1804. function maskMarkdownTables(chars) {
  1805. const lines = chars.join('').split('\n');
  1806. // Tested against the trimmed line: the pattern already allows surrounding
  1807. // whitespace, and its adjacent `\s*` groups backtrack quadratically on a
  1808. // long whitespace run, so a line of masked comments or blank padding
  1809. // cost seconds before it was rejected (#235).
  1810. const delimiter = /^\|?\s*:?-{3,}:?\s*(?:\|\s*:?-{3,}:?\s*)+\|?$/;
  1811. const rows = new Set();
  1812. for (let i = 0; i < lines.length; i += 1) {
  1813. const candidate = lines[i].trim();
  1814. if (!candidate.includes('---') || !delimiter.test(candidate)) continue;
  1815. if (i > 0 && lines[i - 1].includes('|')) rows.add(i - 1);
  1816. rows.add(i);
  1817. for (let j = i + 1; j < lines.length && lines[j].includes('|'); j += 1) rows.add(j);
  1818. }
  1819. let offset = 0;
  1820. for (let i = 0; i < lines.length; i += 1) {
  1821. if (rows.has(i)) blankRange(chars, offset, offset + lines[i].length);
  1822. offset += lines[i].length + 1;
  1823. }
  1824. }
  1825. function maskDelimitedQuotes(chars, open, close, apostropheAware = false) {
  1826. const isWord = (char) => char !== undefined && /[a-z0-9]/i.test(char);
  1827. const isEscaped = (index) => {
  1828. let slashes = 0;
  1829. for (let i = index - 1; i >= 0 && chars[i] === '\\'; i -= 1) slashes += 1;
  1830. return slashes % 2 === 1;
  1831. };
  1832. let start = -1;
  1833. for (let i = 0; i < chars.length; i += 1) {
  1834. if (start === -1) {
  1835. if (chars[i] === open && !isEscaped(i) && (!apostropheAware || !isWord(chars[i - 1]))) start = i;
  1836. } else if (chars[i] === close && !isEscaped(i) && (!apostropheAware || !isWord(chars[i + 1]))) {
  1837. blankRange(chars, start, i + 1);
  1838. start = -1;
  1839. }
  1840. }
  1841. }
  1842. // Additional protected spans for the hyphenation copyedit. Unlike general
  1843. // AI-tell matching, this rule must not "correct" a literal spelling in a
  1844. // quote, URL, path, filename, command flag, or Markdown blockquote.
  1845. function maskHyphenationProtected(text) {
  1846. const chars = maskCode(text).split('');
  1847. maskTopLevelIndentedCode(chars);
  1848. maskYamlFrontmatter(chars);
  1849. maskYamlMetadata(chars);
  1850. maskMarkdownTables(chars);
  1851. maskDelimitedQuotes(chars, '"', '"');
  1852. maskDelimitedQuotes(chars, '“', '”');
  1853. maskDelimitedQuotes(chars, "'", "'", true);
  1854. maskDelimitedQuotes(chars, '‘', '’', true);
  1855. const maskMatches = (regex) => {
  1856. const source = chars.join('');
  1857. let match;
  1858. while ((match = regex.exec(source)) !== null) {
  1859. blankRange(chars, match.index, match.index + match[0].length);
  1860. }
  1861. };
  1862. maskMatches(/^[ \t]*>[^\n]*$/gm);
  1863. maskMatches(/\b(?:https?:\/\/|www\.)[^\s<>]+/gi);
  1864. maskMatches(/<[!?/]?[a-z][^<>\n]*>/gi);
  1865. maskMatches(/(?<![a-z0-9_-])--?[a-z0-9][a-z0-9-]{0,127}/gi);
  1866. // Paths and filenames use bounded components. Besides preventing
  1867. // superlinear backtracking on long kebab blobs, the explicit prefix and
  1868. // trailing-slash forms cover single-component paths such as C:\\code-base,
  1869. // ~/code-base, /code-base, and code-base/.
  1870. maskMatches(/(?:[a-z]:[\\/]|\.{1,2}[\\/]|~[\\/]|[\\/])[a-z0-9_.-]{1,255}/gi);
  1871. maskMatches(/(?:[a-z]:[\\/]|(?:\.\.?[\\/])?)(?:[a-z0-9_.-]{1,64}[\\/]){1,128}[a-z0-9_.-]{1,64}/gi);
  1872. maskMatches(/\b[a-z0-9_.]{0,63}-[a-z0-9_.-]{1,64}[\\/]/gi);
  1873. maskMatches(/\b[a-z0-9_.-]{1,64}-[a-z0-9_.-]{1,64}\.[a-z0-9]{1,16}\b/gi);
  1874. // Technical identifiers that are distinguishable from ordinary prose:
  1875. // selectors, scoped packages, versioned tokens, assignments, and kebab
  1876. // names next to an explicit identifier cue. Plain lowercase compounds
  1877. // remain visible to the curated copyedit patterns below.
  1878. maskMatches(/[.#][a-z_][a-z0-9_.-]{0,63}-[a-z0-9_.-]{1,64}\b/gi);
  1879. maskMatches(/@[a-z0-9_.-]{1,64}\/[a-z0-9_.-]{1,64}-[a-z0-9_.-]{1,64}(?:@[^\s,;)\]}]{1,32})?/gi);
  1880. maskMatches(/\b[a-z0-9_.-]{1,64}-[a-z0-9_.-]{1,64}@[~^]?v?\d[a-z0-9*_.+-]{0,31}\b/gi);
  1881. maskMatches(/\b(?:[a-z0-9_.-]{0,64}\d[a-z0-9_.-]{0,64}-[a-z0-9_.-]{1,64}|[a-z0-9_.-]{1,64}-[a-z0-9_.-]{0,64}\d[a-z0-9_.-]{0,64})\b/gi);
  1882. maskMatches(/\b[a-z_][a-z0-9_.]{0,63}(?:-[a-z0-9_.]{1,64}){1,8}(?=[ \t]*[=:])/gi);
  1883. maskMatches(/\b[a-z_][a-z0-9_.]{0,63}(?:-[a-z0-9_.]{1,64}){1,8}(?=[ \t]+(?:npm[ \t]+)?(?:package|module|class|selector|config(?:uration)?[ \t]+key|key|identifier|property|setting|token|slug|command|option)\b)/gi);
  1884. maskMatches(/\b(?:(?:file(?:name)?|directory|folder|package|module|class|selector|config(?:uration)?[ \t]+key|identifier|property|setting|token|slug|command|option)(?:[ \t]+(?:named|called|is|was))?|key[ \t]+(?:named|called|is|was))[ \t]+(?:@[a-z0-9_.-]{1,64}\/)?[a-z_][a-z0-9_.]{0,63}(?:-[a-z0-9_.]{1,64}){1,8}\b/gi);
  1885. maskMatches(/\b(?:npm|pnpm|yarn)[ \t]+(?:add|install)[ \t]+(?:@[a-z0-9_.-]{1,64}\/)?[a-z0-9_.]{1,64}(?:-[a-z0-9_.]{1,64}){1,8}/gi);
  1886. return chars.join('');
  1887. }
  1888. function findUnnecessaryHyphenation(text) {
  1889. const scanText = maskHyphenationProtected(text);
  1890. const issues = [];
  1891. for (const entry of UNNECESSARY_HYPHENATION) {
  1892. const regex = new RegExp(entry.pattern.source, entry.pattern.flags);
  1893. let match;
  1894. while ((match = regex.exec(scanText)) !== null) {
  1895. issues.push({
  1896. type: 'unnecessary-hyphenation',
  1897. text: match[0],
  1898. severity: 'medium',
  1899. suggestion: typeof entry.suggestion === 'function'
  1900. ? entry.suggestion(match[0])
  1901. : entry.suggestion,
  1902. });
  1903. }
  1904. }
  1905. return issues;
  1906. }
  1907. // ─── Forms that open with `#` but are not social tags ──────────────
  1908. // `#` is overloaded in technical prose and the hashtag rule counts every
  1909. // `#word` it sees, so these are subtracted before the threshold applies:
  1910. // #88, #1234 issue and PR references
  1911. // #1a2b3c CSS hex colours, 6 or 8 chars AND containing a digit
  1912. // #include, #ifndef C preprocessor directives
  1913. // Deliberately NOT carving out 3- and 4-digit hex: #dad, #cafe, #b2b, #e2e,
  1914. // #ace, #face and #bad are real tags, and subtracting them cost true
  1915. // positives on exactly the stuffed-block shape this rule exists to catch.
  1916. // Real palettes are dominated by 6-digit values, so a CSS paragraph still
  1917. // lands under the threshold without the short forms.
  1918. // `owner/repo#88` and URL fragments need no carve-out: the char before `#`
  1919. // is a word char, so the rule's own anchor already rejects them.
  1920. // Ambiguous word tags stay counted on purpose. `#general` as a channel and
  1921. // `#general` as a tag are the same token, and separating them needs a guess
  1922. // that costs more precision on real tag blocks than the carve-out buys.
  1923. // Requires at least one digit: #decade, #facade, #deadbeef are a-f words and
  1924. // real tags. Every actual palette value in the wild carries a digit.
  1925. const HEX_COLOUR = /^(?=[0-9a-f]*\d)(?:[0-9a-f]{6}|[0-9a-f]{8})$/i;
  1926. const CPP_DIRECTIVE = /^(?:include|define|undef|if|ifdef|ifndef|elif|else|endif|pragma|error|warning|line)$/;
  1927. function isSocialTag(tag) {
  1928. return !/^\d+$/.test(tag) && !HEX_COLOUR.test(tag) && !CPP_DIRECTIVE.test(tag);
  1929. }
  1930. // ─── Title Case Section Headers in non-technical prose ─────────────
  1931. // "Strategic Negotiations And Key Partnerships" — every content word
  1932. // capitalized. Acceptable in API docs, ML papers, news headlines. Tell
  1933. // in marketing/personal/blog prose. Skipped when contextMode is
  1934. // 'technical'; runs for general, marketing, and personal.
  1935. //
  1936. // The optional `#{1,6}` prefix is load-bearing (#62): without it the `^[A-Z]`
  1937. // anchor required the line to START with a capital, so `## Benefits And
  1938. // Strategic Considerations` never matched — the first character is `#`. The
  1939. // rule missed the single most common way a heading is actually written, while
  1940. // catching the bare-line form it is usually converted from. Reported by a
  1941. // downstream vendoring the detector.
  1942. //
  1943. // Setext headings (`Title`/`=====`) need no prefix: their text line is bare
  1944. // and already matched by this same pattern.
  1945. // Interior tokens accept Title Case words, acronyms (`AI`, `API`, `CLI`) and
  1946. // the capitalised single-letter words `A` and `I`. The first and last tokens
  1947. // stay ordinary `[A-Z][a-z]+` words, which also excludes all-caps banner
  1948. // lines (`## HTTP API REFERENCE`) whose leading token is not Title Case.
  1949. //
  1950. // Separators and trailing whitespace are horizontal only (`[ \t]`), so a
  1951. // match can never run past one physical line: `\s` also eats newlines, which
  1952. // let two unrelated lines or a blank-line-separated fragment combine into a
  1953. // single heading match (GH-291).
  1954. const TITLE_CASE_HEADER = /^(?:#{1,6}[ \t]+)?([A-Z][a-z]+(?:[ \t]+(?:[A-Z][a-z]+|A|I|[A-Z]{2,}|and|or|of|the|in|for|to|a|an))+[ \t]+[A-Z][a-z]+)[ \t]*$/gm;
  1955. // ─── Parenthetical hedging asides ──────────────────────────────────
  1956. // "(and increasingly, X)", "(or more precisely, Y)", "(though to be
  1957. // fair, Z)" — pseudo-aside that adds no information but performs
  1958. // thoughtfulness. Different from genuine human parentheticals which
  1959. // tend to be tangents or clarifications, not hedges.
  1960. const PARENTHETICAL_HEDGE = [
  1961. /\(\s*(?:and\s+)?(?:increasingly|notably|importantly|crucially|interestingly|perhaps)[,]?\s+[^)]{3,60}\)/gi,
  1962. /\(\s*or\s+more\s+(?:precisely|accurately|specifically)[,]?\s+[^)]{3,60}\)/gi,
  1963. /\(\s*though\s+to\s+be\s+fair[,]?\s+[^)]{3,60}\)/gi,
  1964. /\(\s*at\s+least\s+(?:in\s+)?(?:theory|principle|part)[,]?\s+[^)]{0,60}\)/gi,
  1965. ];
  1966. // ─── Confidence calibration ────────────────────────────────────────
  1967. const CONFIDENCE_CALIBRATION = [
  1968. /\binterestingly\b/gi,
  1969. /\bsurprisingly\b/gi,
  1970. /\bimportantly\b/gi,
  1971. /\bsignificantly\b/gi,
  1972. /\bcertainly\b/gi,
  1973. /\bundoubtedly\b/gi,
  1974. /\bwithout\s+a\s+doubt\b/gi,
  1975. ];
  1976. // ─── Social endorsement / CTA closers ──────────────────────────────
  1977. // The curatorial sign-off LLMs append to LinkedIn / X posts that share
  1978. // or recommend something — usually a colon teeing up a link. Distinct
  1979. // from the bare "worth reading" word-table entry (a single weak word)
  1980. // and from infomercial hooks (mid-flow teasers): this is the
  1981. // demonstrative-anchored endorsement — "THIS one is worth your time:",
  1982. // "do yourself a favor and read this", "thank me later" — that performs
  1983. // a recommendation without giving the reader a reason to click.
  1984. //
  1985. // Precision-first: each pattern carries an anchor so it stays off the
  1986. // literal-verb prose a human writes. The demonstrative object ("read
  1987. // THIS", not "read the runbook"), the trailing-terminal lookahead on
  1988. // "miss this" / "bookmark this" (the closing-line shape, not "miss this
  1989. // meeting" / "bookmark this page"), and the sentence-initial lookbehind
  1990. // on "thank me later" / "save this for later" (the imperative CTA, not
  1991. // "she will thank me later") all exist to suppress false positives on
  1992. // ordinary instructional/conversational text. Apostrophe classes admit
  1993. // the curly ' (U+2019) because LinkedIn / Word / macOS auto-curl it —
  1994. // the straight-only form would miss the canonical "you won't" closer.
  1995. const SOCIAL_CTA_CLOSER = [
  1996. /\bthis\s+one['’]?s?\s+(?:is\s+)?(?:well\s+|totally\s+|absolutely\s+|definitely\s+|really\s+|truly\s+|easily\s+|more\s+than\s+)?worth\s+(?:your\s+time|the\s+read|a\s+read|every\s+(?:minute|second)|reading|watching|a\s+listen|a\s+watch|a\s+look|it)\b/gi,
  1997. /\bthis\s+one['’]?s?\s+(?:is\s+)?a\s+must[-\s]?(?:read|watch|listen|see)\b/gi,
  1998. /\b(?:highly|strongly|can['’]?t|cannot)\s+recommend\w*\s+(?:giving\s+)?(?:this|it)\s+(?:one\s+)?a\s+(?:read|listen|watch|look|go)\b/gi,
  1999. /\bdo\s+yourself\s+a\s+favou?r\s+and\s+(?:read|watch|check\s+out)\s+(?:this|it)\b/gi,
  2000. /\byou\s+(?:really\s+)?(?:won['’]?t|do\s*n['’]?t|will\s+not|do\s+not)\s+want\s+to\s+miss\s+this(?:\s+one)?(?=\s*(?:[:.!\n]|$))/gi,
  2001. /(?<=^|[,.!?:\n]\s{0,4})(?:you\s+can\s+)?thank\s+me\s+later\b/gim,
  2002. /(?<=^|[.!?:\n]\s{0,4})save\s+this\s+(?:one\s+)?for\s+later\b/gim,
  2003. /\bbookmark\s+this(?:\s+(?:one|post|thread))?(?=\s*(?:[:.!\n]|$))/gi,
  2004. /\bdo\s*n['’]?t\s+sleep\s+on\s+this\b/gi,
  2005. /\btrust\s+me,?\s+(?:on\s+this|you['’]?ll)\b/gi,
  2006. ];
  2007. // ─── Unnecessary hyphenation (#107) ───────────────────────────────
  2008. // Precision-first subclasses only. The general question of whether a
  2009. // compound modifier is established English needs editorial judgment and
  2010. // remains in SKILL.md; the engine covers only curated open/closed forms and
  2011. // compounds whose surrounding syntax makes the unhyphenated form clear.
  2012. const UNNECESSARY_HYPHENATION = [
  2013. // Welded open noun phrases reported in #107. Match the complete phrase so
  2014. // a project-specific spelling of the pair in another role is not swept in.
  2015. { pattern: /\bresearch-impact\s+aggregat(?:or|ion)s?\b/g, suggestion: (match) => match.replace('research-impact', 'research impact') },
  2016. { pattern: /\bdata-source\s+strateg(?:y|ies)\b/g, suggestion: (match) => match.replace('data-source', 'data source') },
  2017. { pattern: /\bPython-package\s+usage\b/g, suggestion: (match) => match.replace('Python-package', 'Python package') },
  2018. { pattern: /\bRust-crate\s+usage\b/g, suggestion: (match) => match.replace('Rust-crate', 'Rust crate') },
  2019. { pattern: /\bsingle-Project\s+Manifest\b/g, suggestion: (match) => match.replace('single-Project', 'single Project') },
  2020. { pattern: /\btotal-downloads\s+figures?\b/g, suggestion: (match) => match.replace('total-downloads', 'total downloads') },
  2021. { pattern: /\blife-sciences-native\s+citation\s+count\b/g, suggestion: 'citation count from a life sciences source' },
  2022. // Compounds whose standard spelling is closed. Kept as a small curated
  2023. // list rather than guessing that every noun-noun pair should close up.
  2024. { pattern: /\bcode-base\b/g, suggestion: (match) => match.replace('-', '') },
  2025. { pattern: /\bdata-set\b/g, suggestion: (match) => match.replace('-', '') },
  2026. { pattern: /\btime-frame\b/g, suggestion: (match) => match.replace('-', '') },
  2027. { pattern: /\broad-map\b/g, suggestion: (match) => match.replace('-', '') },
  2028. // Attributive-only forms used adverbially or as nouns. The boundary after
  2029. // real-time / long-term is intentionally narrow: "real-time analytics"
  2030. // and "long-term plan" must not fire.
  2031. {
  2032. pattern: /\bin\s+real-time(?=\s*(?:[,.!?;:]|$)|\s+(?:(?:across|as|automatically|because|but|continuously|during|dynamically|every|for|from|immediately|instantly|on|simultaneously|through|throughout|until|via|when|while|with|without)\b))/gi,
  2033. suggestion: 'in real time',
  2034. },
  2035. {
  2036. pattern: /\b(?:for|over)\s+the\s+long-term(?=\s*(?:[,.!?;:]|$)|\s+(?:across|because|but|by|during|for|from|on|through|throughout|until|via|when|while|with|without)\b)/gi,
  2037. suggestion: (match) => match.replace(/long-term/i, 'long term'),
  2038. },
  2039. {
  2040. // Hyphen: ASCII, U+2010, U+2011, or the en dash U+2013 (#140).
  2041. pattern: /\b(?:functions?|functioned|functioning|operates?|operated|operating|runs?|ran|running|works?|worked|working)\s+out[-‐‑–]of[-‐‑–]the[-‐‑–]box\b/gi,
  2042. suggestion: (match) => match.replace(/out[-‐‑–]of[-‐‑–]the[-‐‑–]box/i, 'out of the box'),
  2043. },
  2044. ];
  2045. // ═══ Helpers ═══════════════════════════════════════════════════════
  2046. function tokenize(text) {
  2047. return text.toLowerCase().match(/[\w'-]+/g) || [];
  2048. }
  2049. // Moving-average type-token ratio (MATTR; Covington & McFall 2010,
  2050. // doi:10.1080/09296171003643098): the share of distinct tokens in each
  2051. // run of `size` consecutive tokens, averaged over every such run. Plain
  2052. // TTR falls as a text grows, because common words keep recurring while
  2053. // new ones arrive more slowly, so a fixed threshold on it measures length.
  2054. // The windowed mean does not drift with length, and on a text of exactly
  2055. // `size` tokens it equals plain TTR.
  2056. function movingAverageTTR(tokens, size) {
  2057. const counts = new Map();
  2058. let distinct = 0;
  2059. const add = (token) => {
  2060. const n = (counts.get(token) || 0) + 1;
  2061. counts.set(token, n);
  2062. if (n === 1) distinct += 1;
  2063. };
  2064. const drop = (token) => {
  2065. const n = counts.get(token) - 1;
  2066. if (n === 0) {
  2067. counts.delete(token);
  2068. distinct -= 1;
  2069. } else {
  2070. counts.set(token, n);
  2071. }
  2072. };
  2073. for (let i = 0; i < size; i += 1) add(tokens[i]);
  2074. let sum = distinct;
  2075. for (let i = size; i < tokens.length; i += 1) {
  2076. drop(tokens[i - size]);
  2077. add(tokens[i]);
  2078. sum += distinct;
  2079. }
  2080. return sum / ((tokens.length - size + 1) * size);
  2081. }
  2082. function countWords(text) {
  2083. return (text.match(/\S+/g) || []).length;
  2084. }
  2085. function getParagraphs(text) {
  2086. return text.split(/\n\s*\n/).filter(p => p.trim().length > 0);
  2087. }
  2088. // A Markdown definition entry: optional list marker, then a bold lead term
  2089. // closed by a colon ("**Term:**" or "**Term**:") or followed by an em
  2090. // dash, plus any wrapped continuation lines up to a blank line, list item,
  2091. // bold lead, heading, or blockquote. Group 1 is the term. Used only to
  2092. // keep glossary entries out of the sentence-length uniformity sample.
  2093. const DEFINITION_ENTRY_RE = /^[ \t]*(?:(?:[-*+]|\d+[.)])[ \t]+)?\*\*([^*\n]+?):?\*\*(?:(?<=:\*\*)|[ \t]*[:—])[^\n]*(?:\n(?![ \t]*(?:$|[-*+][ \t]|\d+[.)][ \t]|\*\*|#|>))[^\n]*)*/gm;
  2094. // A glossary is at least this many entries with distinct terms. A label
  2095. // repeated as a lead ("**Note:**" on every line) is not a glossary term,
  2096. // and its lines stay in the sample.
  2097. const GLOSSARY_MIN_ENTRIES = 3;
  2098. // Text with glossary entries replaced by sentence breaks, or null when the
  2099. // text has no glossary.
  2100. function withoutGlossaryEntries(text) {
  2101. if (!text.includes('**')) return null;
  2102. const entries = [...text.matchAll(DEFINITION_ENTRY_RE)];
  2103. const counts = new Map();
  2104. for (const m of entries) {
  2105. const term = m[1].trim().toLowerCase();
  2106. counts.set(term, (counts.get(term) || 0) + 1);
  2107. }
  2108. const unique = entries.filter((m) => counts.get(m[1].trim().toLowerCase()) === 1);
  2109. if (unique.length < GLOSSARY_MIN_ENTRIES) return null;
  2110. let out = '';
  2111. let last = 0;
  2112. for (const m of unique) {
  2113. out += text.slice(last, m.index) + '.';
  2114. last = m.index + m[0].length;
  2115. }
  2116. return out + text.slice(last);
  2117. }
  2118. function getSentences(text) {
  2119. return text.split(/[.!?]+/).filter(s => s.trim().length > 5);
  2120. }
  2121. function matchPatterns(text, patterns, category, severity) {
  2122. const issues = [];
  2123. for (const pat of patterns) {
  2124. const regex = new RegExp(pat.source, pat.flags);
  2125. let match;
  2126. while ((match = regex.exec(text)) !== null) {
  2127. issues.push({
  2128. type: category,
  2129. text: match[0],
  2130. index: match.index,
  2131. severity,
  2132. suggestion: null,
  2133. });
  2134. }
  2135. }
  2136. return issues;
  2137. }
  2138. // ═══ Main analysis ═════════════════════════════════════════════════
  2139. // Upper bound for one scan. Above this we bail rather than running all
  2140. // regex passes over a huge buffer — protects page perf on pasted novels.
  2141. const MAX_WORDS = 10000;
  2142. // V2 contract defaults so early-exit paths (Empty/tooShort/tooLong)
  2143. // still return the same field shape a v2 consumer expects. Without
  2144. // this, `result.document_classification === 'AI_ONLY'` is `undefined`
  2145. // on edge inputs and fails open.
  2146. // UNSCORED is returned on empty / too-short / too-long inputs where
  2147. // we declined to score. Distinct from HUMAN_ONLY (which is a positive
  2148. // classification) so a caller can't mistake a refused scan for a
  2149. // confident human verdict — a 50k-word LLM-generated document is
  2150. // not "human", it's just outside our scoring window.
  2151. function buildV2Defaults(classification, confidence) {
  2152. const probs = classification === 'HUMAN_ONLY'
  2153. ? { human: 1, mixed: 0, ai: 0 }
  2154. : classification === 'AI_ONLY'
  2155. ? { human: 0, mixed: 0, ai: 1 }
  2156. : { human: 0.333, mixed: 0.334, ai: 0.333 };
  2157. return {
  2158. document_classification: classification,
  2159. class_probabilities: probs,
  2160. confidence_category: confidence,
  2161. highlight_sentence_for_ai: [],
  2162. };
  2163. }
  2164. // The analysis contexts and source modes analyzeText() accepts. The CLIs
  2165. // import these lists instead of keeping their own copies.
  2166. const CONTEXT_MODES = Object.freeze(['general', 'technical', 'marketing', 'personal']);
  2167. const SOURCE_MODES = Object.freeze(['plain', 'rendered-markdown']);
  2168. function analyzeText(text, options = {}) {
  2169. if (typeof text !== 'string') {
  2170. throw new TypeError('analyzeText(text): argument must be a string');
  2171. }
  2172. const VALID_CONTEXT_MODES = new Set(CONTEXT_MODES);
  2173.     const requestedMode = options.contextMode === undefined ? 'general' : options.contextMode;
  2174.     const contextMode = VALID_CONTEXT_MODES.has(requestedMode) ? requestedMode : 'general';
  2175.     const contextModeFallback = requestedMode !== contextMode ? requestedMode : null;
  2176.     // Source mode controls which parts of a Markdown file count as prose.
  2177.     // Plain remains the compatibility default. Rendered Markdown masks only
  2178.     // initial YAML frontmatter and HTML comments; source-hygiene checks for
  2179.     // hidden TODO/placeholder comments remain available through plain mode.
  2180. const VALID_SOURCE_MODES = new Set(SOURCE_MODES);
  2181.     const requestedSourceMode = options.sourceMode === undefined ? 'plain' : options.sourceMode;
  2182.     const sourceMode = VALID_SOURCE_MODES.has(requestedSourceMode) ? requestedSourceMode : 'plain';
  2183.     const sourceModeFallback = requestedSourceMode !== sourceMode ? requestedSourceMode : undefined;
  2184. if (!text || text.trim().length === 0) {
  2185. return {
  2186. ...buildV2Defaults('UNSCORED', 'low'),
  2187. score: 0,
  2188. label: 'Empty',
  2189. issues: [],
  2190. stats: {
  2191. wordCount: 0,
  2192. contextMode,
  2193. contextModeFallback,
  2194. sourceMode,
  2195. sourceModeFallback,
  2196. maskedFrontmatter: 0,
  2197. maskedHtmlComments: 0,
  2198. ignoredRegions: 0,
  2199. quotedLines: 0,
  2200. maskedQuotes: 0,
  2201. },
  2202. tooShort: true,
  2203. };
  2204. }
  2205. // Map each working-string code unit back to the caller's source. Every
  2206. // length-changing preprocessing stage composes this map as it removes
  2207. // characters, and results are translated before they leave the function.
  2208. let sourceMap = identitySourceMap(text.length);
  2209. // Context mode selects context-appropriate flagging. Accepted values:
  2210. // 'general' (default) — full ruleset
  2211. // 'technical' — skip title-case headers; individual prose-only rules
  2212. // apply their own technical-context gates (only mode
  2213. // that currently changes scoring)
  2214. // 'marketing' — accepted; recorded in stats; scores same as general
  2215. // 'personal' — accepted; recorded in stats; scores same as general
  2216. // Invalid values fall back to 'general' with stats.contextModeFallback set.
  2217. // Mode validation: an unknown string (e.g. typo "tecnical") would
  2218. // otherwise silently downgrade to general-mode behavior. Coerce to
  2219. // 'general' and surface the original value in stats for traceability.
  2220. let maskedFrontmatter = 0;
  2221. let maskedHtmlComments = 0;
  2222. // Author-marked ignore regions go first, in every source mode, so no
  2223. // later pass sees the excluded passage. Masking keeps the length, so the
  2224. // source map needs no update.
  2225. const ignored = maskIgnoreRegions(text);
  2226. text = ignored.text;
  2227. const { ignoredRegions } = ignored;
  2228. if (sourceMode === 'rendered-markdown') {
  2229. const rendered = maskRenderedMarkdown(text);
  2230. text = rendered.text;
  2231. maskedFrontmatter = rendered.maskedFrontmatter;
  2232. maskedHtmlComments = rendered.maskedHtmlComments;
  2233. }
  2234. // Pre-pass: mask Markdown blockquotes before scoring. A human
  2235. // reacting to AI text by quoting it shouldn't have the quoted block
  2236. // counted against their own writing. A single `> ` line counts too
  2237. // (#238). Masking instead of deleting keeps later issue and highlight
  2238. // offsets aligned with the source file.
  2239. const blockquotes = sourceMode === 'rendered-markdown'
  2240. ? maskBlockquotes(text)
  2241. : stripBlockquotes(text, sourceMap);
  2242. text = blockquotes.text;
  2243. if (blockquotes.sourceMap) sourceMap = blockquotes.sourceMap;
  2244. const { quotedLines } = blockquotes;
  2245. // Pre-pass: the inline half of the blockquote escape hatch. Words inside
  2246. // a double-quoted span belong to whoever is quoted (#238). Masking runs
  2247. // before normalization, as blockquotes do, so bypass characters inside a
  2248. // quotation raise no normalization flag. Masking keeps the length, so
  2249. // the source map needs no update. The smart-punctuation check below
  2250. // reads the unmasked text, because the blanked span would otherwise
  2251. // count as a double space.
  2252. const unquotedText = normalizeText(text).text;
  2253. const quotes = maskQuotedSpans(text);
  2254. text = quotes.text;
  2255. const { maskedQuotes } = quotes;
  2256. // Pre-pass: strip bypass-trick chars before pattern matching so
  2257. // "delve" with a Cyrillic 'е' still hits Tier 1. Compose the map while
  2258. // deleting characters so later offsets still address the source.
  2259. const norm = normalizeText(text, sourceMap);
  2260. text = norm.text;
  2261. sourceMap = norm.sourceMap;
  2262. const wordCount = countWords(text);
  2263. // Unsegmented-script check (GH-241): Chinese and Japanese carry no
  2264. // inter-word spaces, so word segmentation cannot measure them — a long
  2265. // document counts as one \S+ run and would misreport as "Too short",
  2266. // while newline-wrapped lines each count as a word and would score
  2267. // without segmentation. The check therefore runs before the word gate
  2268. // and declines only when CJK characters dominate the non-whitespace
  2269. // text, so short English documents with an incidental place name or
  2270. // single Han character stay scorable. Unicode script properties cover
  2271. // the complete Han, Hiragana, and Katakana repertoires (including
  2272. // supplementary-plane and halfwidth forms); Hangul is space-separated
  2273. // and segments fine, so it is excluded. Both counts use Unicode mode so
  2274. // supplementary characters count as one code point rather than two UTF-16
  2275. // code units.
  2276. const cjkChars = (text.match(/[\p{Script_Extensions=Han}\p{Script_Extensions=Hiragana}\p{Script_Extensions=Katakana}]/gu) || []).length;
  2277. const nonSpaceChars = (text.match(/\S/gu) || []).length;
  2278. if (cjkChars > 0 && cjkChars * 2 >= nonSpaceChars) {
  2279. return {
  2280. ...buildV2Defaults('UNSCORED', 'low'),
  2281. score: 0,
  2282. label: 'Unsupported script',
  2283. issues: [],
  2284. stats: { wordCount, cjkChars, reason: 'unsegmented-script document: no inter-word spaces to count', contextMode, contextModeFallback, sourceMode, sourceModeFallback, maskedFrontmatter, maskedHtmlComments, ignoredRegions, quotedLines, maskedQuotes },
  2285. unsupportedScript: true,
  2286. };
  2287. }
  2288. if (wordCount < 10) {
  2289. return {
  2290. ...buildV2Defaults('UNSCORED', 'low'),
  2291. score: 0,
  2292. label: 'Too short',
  2293. issues: [],
  2294. stats: { wordCount, contextMode, contextModeFallback, sourceMode, sourceModeFallback, maskedFrontmatter, maskedHtmlComments, ignoredRegions, quotedLines, maskedQuotes },
  2295. tooShort: true,
  2296. };
  2297. }
  2298. if (wordCount > MAX_WORDS) {
  2299. return {
  2300. ...buildV2Defaults('UNSCORED', 'low'),
  2301. score: 0,
  2302. label: 'Text too long',
  2303. issues: [],
  2304. stats: { wordCount, contextMode, contextModeFallback, sourceMode, sourceModeFallback, maskedFrontmatter, maskedHtmlComments, ignoredRegions, quotedLines, maskedQuotes },
  2305. tooLong: true,
  2306. };
  2307. }
  2308. const tokens = tokenize(text);
  2309. const paragraphs = getParagraphs(text);
  2310. const sentences = getSentences(text);
  2311. const issues = [];
  2312. let rawScore = 0;
  2313. // ── 1. Tier 1 words ──────────────────────────────────────────
  2314. const tier1Found = new Set();
  2315. for (const token of tokens) {
  2316. if (contextMode === 'technical' && TECHNICAL_EXEMPT.has(token)) continue;
  2317. if (Object.hasOwn(TIER1, token) && !tier1Found.has(token)) {
  2318. tier1Found.add(token);
  2319. issues.push({
  2320. type: 'tier1',
  2321. text: token,
  2322. severity: 'high',
  2323. suggestion: TIER1[token],
  2324. });
  2325. }
  2326. }
  2327. // Tier 1 multi-word phrases. Adds each distinct phrase (lowercased) to
  2328. // `tier1Found` so the same phrase hit multiple times only produces one
  2329. // issue — matches the downstream dedup behavior.
  2330. for (const phrase of TIER1_PHRASES) {
  2331. const regex = new RegExp(phrase.pattern.source, phrase.pattern.flags);
  2332. let match;
  2333. while ((match = regex.exec(text)) !== null) {
  2334. const lower = match[0].toLowerCase();
  2335. if (contextMode === 'technical' && TECHNICAL_EXEMPT.has(lower)) continue;
  2336. if (phrase.skip && phrase.skip(text, match.index)) continue;
  2337. if (tier1Found.has(lower)) continue;
  2338. tier1Found.add(lower);
  2339. issues.push({
  2340. // Clarity-band entries are wordiness edits, not frequency evidence.
  2341. // Same fix, weaker claim — see the Tier 1A/1B split in SKILL.md.
  2342. type: phrase.clarity ? 'tier1-clarity' : 'tier1',
  2343. text: match[0],
  2344. severity: phrase.clarity ? 'medium' : 'high',
  2345. suggestion: phrase.replace,
  2346. });
  2347. }
  2348. }
  2349. // ── 2. Tier 2 clusters ───────────────────────────────────────
  2350. let tier2Clusters = 0;
  2351. for (const para of paragraphs) {
  2352. const paraTokens = tokenize(para);
  2353. const found = [];
  2354. const suggestions = {};
  2355. for (const token of paraTokens) {
  2356. if (contextMode === 'technical' && TECHNICAL_EXEMPT.has(token)) continue;
  2357. if (Object.hasOwn(TIER2, token) && !found.includes(token)) {
  2358. found.push(token);
  2359. suggestions[token] = TIER2[token];
  2360. }
  2361. }
  2362. for (const cond of TIER2_CONDITIONAL) {
  2363. if (contextMode === 'technical' && TECHNICAL_EXEMPT.has(cond.word)) continue;
  2364. if (!found.includes(cond.word) && cond.pattern.test(para)) {
  2365. found.push(cond.word);
  2366. suggestions[cond.word] = cond.suggestion;
  2367. }
  2368. }
  2369. if (found.length >= 2) {
  2370. tier2Clusters++;
  2371. for (const word of found) {
  2372. issues.push({
  2373. type: 'tier2',
  2374. text: word,
  2375. severity: 'medium',
  2376. suggestion: suggestions[word],
  2377. });
  2378. }
  2379. }
  2380. }
  2381. // ── 3. Tier 3 density ────────────────────────────────────────
  2382. const tier3Counts = {};
  2383. for (const token of tokens) {
  2384. const canonical = TIER3_LOOKUP.get(token);
  2385. if (canonical) tier3Counts[canonical] = (tier3Counts[canonical] || 0) + 1;
  2386. }
  2387. // Flag at 3% of word count, but never below 3 occurrences. Previous
  2388. // floor of 1 meant a 50-word text with one "significant" got flagged
  2389. // as Tier 3 overuse, which was noise.
  2390. const densityThreshold = Math.max(3, Math.floor(wordCount * 0.03));
  2391. let tier3Flags = 0;
  2392. for (const [word, count] of Object.entries(tier3Counts)) {
  2393. if (count >= densityThreshold) {
  2394. tier3Flags++;
  2395. issues.push({
  2396. type: 'tier3',
  2397. text: `"${word}" x${count}`,
  2398. severity: 'low',
  2399. suggestion: `Overused (${count} times in ${wordCount} words)`,
  2400. });
  2401. }
  2402. }
  2403. // ── 4–21. Pattern categories ─────────────────────────────────
  2404. issues.push(...matchPatterns(text, TRANSITIONS, 'transition', 'medium'));
  2405. issues.push(...matchPatterns(text, CHATBOT_ARTIFACTS, 'chatbot', 'critical'));
  2406. issues.push(...matchPatterns(text, SYCOPHANTIC, 'sycophantic', 'critical'));
  2407. issues.push(...matchPatterns(text, FILLERS, 'filler', 'medium'));
  2408. issues.push(...matchPatterns(text, GENERIC_CONCLUSIONS, 'generic-conclusion', 'medium'));
  2409. issues.push(...matchPatterns(text, LETS_PATTERNS, 'lets-construction', 'medium'));
  2410. issues.push(...matchPatterns(text, REASONING_ARTIFACTS, 'reasoning-artifact', 'critical'));
  2411. issues.push(...matchPatterns(text, SIGNIFICANCE_INFLATION, 'significance-inflation', 'high'));
  2412. issues.push(...matchPatterns(text, VAGUE_ATTRIBUTIONS, 'vague-attribution', 'critical'));
  2413. issues.push(...matchPatterns(text, HOLLOW_INTENSIFIERS, 'hollow-intensifier', 'medium'));
  2414. issues.push(...matchPatterns(text, EMOTIONAL_FLATLINE, 'emotional-flatline', 'low'));
  2415. issues.push(...matchPatterns(text, LINGERING_ATTENTION, 'lingering-attention', 'medium'));
  2416. issues.push(...matchPatterns(text, NOVELTY_INFLATION, 'novelty-inflation', 'medium'));
  2417. issues.push(...matchPatterns(text, CUTOFF_DISCLAIMERS, 'cutoff-disclaimer', 'critical'));
  2418. issues.push(...matchPatterns(text, AI_PLACEHOLDERS, 'ai-placeholder', 'critical'));
  2419. issues.push(...matchPatterns(text, AI_CITATION_MARKUP, 'ai-citation-markup', 'critical'));
  2420. issues.push(...matchPatterns(text, AI_UTM_SOURCE, 'ai-utm-source', 'critical'));
  2421. issues.push(...matchPatterns(text, TEMPLATE_PHRASES, 'template-phrase', 'high'));
  2422. issues.push(...matchPatterns(text, FALSE_CONCESSION, 'false-concession', 'medium'));
  2423. issues.push(...matchPatterns(text, RHETORICAL_QUESTIONS, 'rhetorical-question', 'medium'));
  2424. issues.push(...matchPatterns(text, HEDGE_STACK, 'hedge-stack', 'high'));
  2425. issues.push(...matchPatterns(text, FUTURE_NARRATIVE, 'future-narrative', 'high'));
  2426. issues.push(...matchPatterns(text, REAL_ACTUAL_INFLATION, 'real-actual-inflation', 'medium'));
  2427. issues.push(...matchPatterns(text, SOCIAL_CTA_CLOSER, 'social-cta-closer', 'high'));
  2428. issues.push(...matchPatterns(text, PERFORMED_INSIGHT, 'performed-insight', 'medium'));
  2429. const stagedDiscoveryIssues = matchPatterns(text, STAGED_DISCOVERY, 'performed-insight', 'medium');
  2430. issues.push(...stagedDiscoveryIssues);
  2431. issues.push(...matchPatterns(text, NEGATION_CHAIN, 'negation-chain', 'high'));
  2432. // Reveals flag alone; contrasts need a nearby frame. See NEGATIVE_PARALLELISM_*.
  2433. issues.push(...negativeParallelismIssues(
  2434. text,
  2435. matchPatterns(text, NEGATIVE_PARALLELISM_REVEAL, 'negative-parallelism', 'high'),
  2436. matchPatterns(text, NEGATIVE_PARALLELISM_CONTRAST, 'negative-parallelism', 'high'),
  2437. ));
  2438. issues.push(...matchPatterns(text, DEV_BLOG_BOILERPLATE, 'dev-blog-boilerplate', 'medium'));
  2439. issues.push(...findUnnecessaryHyphenation(text));
  2440. // ── Tier 1 v2: formulaic openers + parenthetical hedges ──────────
  2441. issues.push(...matchPatterns(text, FORMULAIC_OPENERS, 'formulaic-opener', 'high'));
  2442. issues.push(...matchPatterns(text, SPECULATIVE_OPENERS, 'speculative-opener', 'high'));
  2443. issues.push(...matchPatterns(text, LAUNCH_INTROS, 'launch-intro', 'high'));
  2444. issues.push(...matchPatterns(text, CROWD_CONTRAST, 'crowd-contrast', 'medium'));
  2445. issues.push(...matchPatterns(text, FAKE_CASUAL_PROPS, 'fake-casual-prop', 'high'));
  2446. issues.push(...matchPatterns(text, PARENTHETICAL_HEDGE, 'parenthetical-hedge', 'medium'));
  2447. // Title-case headers — gated to marketing/personal/general modes
  2448. // (technical mode legitimately uses Title Case section headers).
  2449. if (contextMode !== 'technical') {
  2450. const titleHits = matchPatterns(text, [TITLE_CASE_HEADER], 'title-case-header', 'medium');
  2451. // Drop matches that look like proper-noun titles (single line, all
  2452. // tokens capitalized incl. function words) — that's headline style,
  2453. // not the AI-section-header tell which has mid-sentence "And".
  2454. //
  2455. // The prefix strip is load-bearing. matchPatterns reports match[0], so a
  2456. // Markdown hit arrives as "## Terms Of Service" and `##` counts as a
  2457. // token — silently lowering this guard from four content words to three
  2458. // for headings only, which is exactly the class it exists to protect.
  2459. // "## Terms Of Service", "## Bank Of America" and "## Table Of Contents"
  2460. // all flagged as a result: ordinary human headings, on a detector whose
  2461. // stated first priority is not firing on human writing.
  2462. const filtered = titleHits.filter((h) => {
  2463. const title = h.text.replace(MD_HEADING_PREFIX, '');
  2464. const tokens = title.trim().split(/\s+/);
  2465. if (tokens.length < 4) return false;
  2466. // The function word must be MID-title, which is what the comment above
  2467. // has always said and what the test never enforced. A leading "The"
  2468. // satisfied a bare /\bThe\b/, so ordinary human headings flagged:
  2469. // "## The New Security Landscape", "## The Microsoft Approach to
  2470. // Identity", "### The Four Keys to a Successful and Secure Modern
  2471. // Workplace". Measured across 81 files that provably predate LLMs
  2472. // (2018-19 eBooks, 2020 posts): 13 false positives, every one opening
  2473. // with "The", against zero on main.
  2474. //
  2475. // "## Benefits And Strategic Considerations" -- the actual tell, and
  2476. // this rule's own fixture -- is untouched: its "And" is interior.
  2477. return FUNCTION_WORD.test(tokens.slice(1).join(' '));
  2478. });
  2479. const fences = filtered.length ? fenceRanges(text) : [];
  2480. issues.push(...filtered.filter((h) => !inFenceRange(fences, h.index)));
  2481. }
  2482. // ── Normalization-trigger flag ───────────────────────────────────
  2483. // ZWSPs or homoglyphs in pasted prose are near-dispositive: humans
  2484. // don't insert these into their own writing. Single roleplay marker
  2485. // can be a false positive on Markdown emphasis (filtered to multi-
  2486. // word inner already), so requires ≥2.
  2487. if (norm.flags.zeroWidth > 0 || norm.flags.homoglyph >= 2) {
  2488. issues.push({
  2489. type: 'normalization-flag',
  2490. text: `${norm.flags.zeroWidth} zero-width + ${norm.flags.homoglyph} homoglyph swap${norm.flags.homoglyph === 1 ? '' : 's'}`,
  2491. severity: 'critical',
  2492. suggestion: 'Text contains invisible/lookalike chars typical of AI-humanizer bypass tools. Re-type from your own keyboard.',
  2493. });
  2494. }
  2495. if (norm.flags.roleplay >= 2) {
  2496. issues.push({
  2497. type: 'normalization-flag',
  2498. text: `${norm.flags.roleplay} *roleplay-action* markers stripped`,
  2499. severity: 'high',
  2500. suggestion: 'Paired *action* markers are a chat-model artifact.',
  2501. });
  2502. }
  2503. // Em dashes in list-item separator position — a bulleted or numbered
  2504. // list item opening with a bolded lead term or markdown link, then the
  2505. // dash ("- **Term** — desc", "- [label](url) — desc") — are
  2506. // definition-list typography, not prose punctuation. Shared by the
  2507. // smart-punct signature below and the em-dash frequency check (§22).
  2508. // An optional parenthetical or inline-code span may sit between the bold
  2509. // lead term and the dash — "- **Lingering-attention claims**
  2510. // (`lingering-attention`) — the share-post frame…" is the same definition
  2511. // typography as the bare form. Found by the self-scan (see PROOF.md, #67).
  2512. // The leading indent is \s minus line terminators: a bare ^\s* under the
  2513. // m flag rescanned every following blank line from each line start, so a
  2514. // long run of U+2028/U+2029 (line terminators that \s also matches) took
  2515. // quadratic time. Every match still ends at the same dash, so the count is
  2516. // unchanged.
  2517. const SEPARATOR_DASH_RE = /^[^\S\n\r\u2028\u2029]*(?:[-*+]|\d+[.)])\s+(?:\*\*[^*\n]+\*\*|\[[^\]\n]+\]\([^)\n]*\))(?:[ \t]*(?:\([^)\n]*\)|`[^`\n]+`))?[ \t]*—/gm;
  2518. // The same typography in HTML source: an <li> that opens with a
  2519. // <strong>/<b> lead term or an <a> link of at most five words, then the
  2520. // dash (#351). A longer bold span is a clause, not a term. Only the
  2521. // first child of the item counts, so a prose dash later in a list item,
  2522. // or any dash outside <li>, still counts. `&mdash;` entities are never
  2523. // counted as em dashes, so they need no carve-out.
  2524. const SEPARATOR_DASH_HTML_RE = /^[ \t]*<li\b[^>\n]*>[ \t]*(?:<(strong|b)\b[^>\n]*>[ \t]*[^<\s]+(?:[ \t]+[^<\s]+){0,4}[ \t]*<\/\1>|<a\b[^>\n]*>[ \t]*[^<\s]+(?:[ \t]+[^<\s]+){0,4}[ \t]*<\/a>)(?:[ \t]*(?:\([^)\n]*\)|<code\b[^>\n]*>[^<\n]+<\/code>))?[ \t]*—/gim;
  2525. function countSeparatorDashes(value) {
  2526. return (value.match(SEPARATOR_DASH_RE) || []).length
  2527. + (value.match(SEPARATOR_DASH_HTML_RE) || []).length
  2528. + countVersionHeadingDashes(value);
  2529. }
  2530. // Keep-a-Changelog version headings (`## [3.21.0] — 2026-07-30`) join a
  2531. // label to a value exactly as a list separator does. Deliberately narrow:
  2532. // a bracketed or bare semver token, then a dash, then an ISO date, and
  2533. // nothing else on the line. Ordinary prose dashes in headings still count,
  2534. // because SKILL.md applies the em-dash rule to headings too.
  2535. function countVersionHeadingDashes(value) {
  2536. let count = 0;
  2537. for (const rawLine of value.split(/\r\n|\n|\r/)) {
  2538. let end = rawLine.length;
  2539. while (end > 0 && (rawLine[end - 1] === ' ' || rawLine[end - 1] === '\t')) end -= 1;
  2540. if (!/^\d{4}-\d{2}-\d{2}$/.test(rawLine.slice(Math.max(0, end - 10), end))) continue;
  2541. let cursor = end - 10;
  2542. while (cursor > 0 && (rawLine[cursor - 1] === ' ' || rawLine[cursor - 1] === '\t')) cursor -= 1;
  2543. if (rawLine[cursor - 1] !== '\u2014') continue;
  2544. cursor -= 1;
  2545. while (cursor > 0 && (rawLine[cursor - 1] === ' ' || rawLine[cursor - 1] === '\t')) cursor -= 1;
  2546. const prefix = rawLine.slice(0, cursor);
  2547. const heading = /^#{1,6}[ \t]+/.exec(prefix);
  2548. if (!heading) continue;
  2549. let version = prefix.slice(heading[0].length);
  2550. if (version.startsWith('[')) {
  2551. if (!version.endsWith(']')) continue;
  2552. version = version.slice(1, -1);
  2553. } else if (version.endsWith(']')) {
  2554. version = version.slice(0, -1);
  2555. }
  2556. if (version.includes(']')) continue;
  2557. if (/^v?\d+\.\d+\.\d+/.test(version)) count += 1;
  2558. }
  2559. return count;
  2560. }
  2561. // ── Smart-punctuation co-occurrence signature ────────────────────
  2562. // Curly punctuation + prose em dash + Oxford comma, with no sampled
  2563. // spacing or common-contraction anomalies,
  2564. // can also come from house style or editor settings. Keep the existing
  2565. // narrow shape as a low-priority style check in plain source only.
  2566. // Rendered Markdown deliberately preserves publication typography.
  2567. // This finding has no weight or classifier corroborator. Separator-only
  2568. // dashes do not complete the shape, and the spacing/contraction checks
  2569. // below are limited heuristics, not proof that a document has no typos.
  2570. if (sourceMode === 'plain') {
  2571. const hasCurly = /[“”‘’]/.test(text);
  2572. const totalEmDashes = (text.match(/—/g) || []).length;
  2573. const separatorEmDashes = countSeparatorDashes(text);
  2574. const hasEmDash = totalEmDashes > separatorEmDashes;
  2575. const oxfordHit = text.match(/\b\w+,\s+\w+,\s+and\s+\w+/g);
  2576. const hasOxford = (oxfordHit?.length || 0) >= 1;
  2577. const doubleSpaces = (unquotedText.match(/[^.!?] +/g) || []).length;
  2578. const missingApos = /\b(?:dont|wont|cant|isnt|wasnt|shouldnt|wouldnt|couldnt|youre|theyre|its\s+a\s+\w+ing)\b/i.test(text);
  2579. const clean = doubleSpaces === 0 && !missingApos;
  2580. const signals = [hasCurly, hasEmDash, hasOxford, clean].filter(Boolean).length;
  2581. if (signals >= 4 && wordCount >= 80) {
  2582. issues.push({
  2583. type: 'smart-punct-signature',
  2584. text: 'curly punctuation + prose em dash + Oxford comma',
  2585. severity: 'low',
  2586. suggestion: 'Check destination style before changing punctuation. Preserve house-style typography; this combination is not evidence of AI authorship.',
  2587. });
  2588. }
  2589. }
  2590. // ── Punctuation distribution mode ────────────────────────────────
  2591. // Humans cluster trimodal across paragraphs (some paras heavy, some
  2592. // light, some none). AI converges on a normal distribution. We can't
  2593. // run a real modality test client-side, but we can flag the AI
  2594. // signature: low variance of per-paragraph punctuation density.
  2595. // Requires ≥4 paragraphs to be meaningful.
  2596. if (paragraphs.length >= 4) {
  2597. const densities = paragraphs.map((p) => {
  2598. const words = (p.match(/\S+/g) || []).length;
  2599. if (words < 5) return null;
  2600. const puncts = (p.match(/[,;:—()]/g) || []).length;
  2601. return puncts / words;
  2602. }).filter((d) => d !== null);
  2603. if (densities.length >= 4) {
  2604. const mean = densities.reduce((a, b) => a + b, 0) / densities.length;
  2605. const variance = densities.reduce((s, d) => s + (d - mean) ** 2, 0) / densities.length;
  2606. const cv = mean > 0 ? Math.sqrt(variance) / mean : 0;
  2607. // CV < 0.25 across paragraphs means each paragraph has the same
  2608. // punctuation density — the AI signature. Humans usually swing
  2609. // wider. Threshold derived from stylometry papers (arxiv 2507.00838).
  2610. if (cv < 0.25 && mean >= 0.04) {
  2611. issues.push({
  2612. type: 'punct-distribution',
  2613. text: `Punctuation density uniform across paragraphs (CV=${cv.toFixed(2)})`,
  2614. severity: 'medium',
  2615. suggestion: 'AI text holds punctuation density steady; human writers swing between dense and sparse paragraphs.',
  2616. });
  2617. }
  2618. }
  2619. }
  2620. // ── Function-word trigram entropy ────────────────────────────────
  2621. // POS-trigram entropy is the academic signal; function-word trigram
  2622. // entropy approximates it without a tagger (function words ARE the
  2623. // closed-class POS classes). AI text has lower entropy because LLM
  2624. // sampling collapses onto a narrower set of grammatical templates.
  2625. //
  2626. // Method: extract function-word indicators per sentence, build
  2627. // trigrams over the sequence, compute Shannon entropy. Bins below
  2628. // threshold flag.
  2629. if (wordCount >= 150) {
  2630. const FUNC_WORDS = new Set([
  2631. 'the','a','an','and','or','but','of','to','in','on','at','by','for','with',
  2632. 'from','as','is','was','are','were','be','been','being','have','has','had',
  2633. 'do','does','did','will','would','should','could','may','might','must','can',
  2634. 'this','that','these','those','it','its','they','them','their','there','here',
  2635. 'we','our','us','i','you','your','he','she','his','her','him','not','no','so',
  2636. 'if','then','than','when','where','which','who','what','how','why','because',
  2637. ]);
  2638. const seq = tokens.map((t) => FUNC_WORDS.has(t) ? t : '_').filter((_, i, arr) => arr[i] !== '_' || (i > 0 && arr[i - 1] !== '_'));
  2639. if (seq.length >= 50) {
  2640. const trigrams = {};
  2641. for (let i = 0; i < seq.length - 2; i++) {
  2642. const tg = `${seq[i]}|${seq[i + 1]}|${seq[i + 2]}`;
  2643. trigrams[tg] = (trigrams[tg] || 0) + 1;
  2644. }
  2645. const total = seq.length - 2;
  2646. let entropy = 0;
  2647. for (const c of Object.values(trigrams)) {
  2648. const p = c / total;
  2649. entropy -= p * Math.log2(p);
  2650. }
  2651. // Normalize by log2(distinct trigrams) so entropy ranges roughly
  2652. // 0..1 and threshold is interpretable. Empirical threshold: human
  2653. // prose ~0.85-0.95 normalized, AI prose ~0.70-0.82.
  2654. const distinctCount = Object.keys(trigrams).length;
  2655. const normalized = distinctCount > 1 ? entropy / Math.log2(distinctCount) : 1;
  2656. if (normalized < 0.82 && total >= 50) {
  2657. issues.push({
  2658. type: 'fnword-trigram-entropy',
  2659. text: `Function-word trigram entropy ${normalized.toFixed(2)} (low)`,
  2660. severity: 'medium',
  2661. suggestion: 'Grammatical structure is unusually repetitive. AI sampling collapses onto narrower templates than human writing.',
  2662. });
  2663. }
  2664. // Degenerate case: single distinct trigram repeated across the
  2665. // whole document is the strongest possible AI signal but the
  2666. // normalized fallback returns 1.0 (= "fully human"), inverting
  2667. // the signal. Catch it explicitly.
  2668. if (distinctCount === 1 && total >= 50) {
  2669. issues.push({
  2670. type: 'fnword-trigram-entropy',
  2671. text: 'Single function-word trigram repeated across document',
  2672. severity: 'high',
  2673. suggestion: 'Grammatical structure is fully degenerate — every clause uses the same function-word skeleton.',
  2674. });
  2675. }
  2676. }
  2677. }
  2678. // ── Cross-paragraph burstiness ───────────────────────────────────
  2679. // We already check within-paragraph sentence-length uniformity. AI
  2680. // is also flat ACROSS paragraphs — every paragraph has roughly the
  2681. // same sentence-length variance. Humans vary: terse paras next to
  2682. // discursive paras. Measure variance of CV across paragraphs.
  2683. if (paragraphs.length >= 4) {
  2684. const cvs = paragraphs.map((p) => {
  2685. const sents = getSentences(p);
  2686. if (sents.length < 3) return null;
  2687. const lens = sents.map(countWords);
  2688. const m = lens.reduce((a, b) => a + b, 0) / lens.length;
  2689. if (m === 0) return null;
  2690. const v = lens.reduce((s, l) => s + (l - m) ** 2, 0) / lens.length;
  2691. return Math.sqrt(v) / m;
  2692. }).filter((c) => c !== null);
  2693. if (cvs.length >= 4) {
  2694. const cvMean = cvs.reduce((a, b) => a + b, 0) / cvs.length;
  2695. const cvVar = cvs.reduce((s, c) => s + (c - cvMean) ** 2, 0) / cvs.length;
  2696. const cvStd = Math.sqrt(cvVar);
  2697. // Std-of-CV below 0.08 means every paragraph has roughly the same
  2698. // internal rhythm — AI signature. Human prose typically swings
  2699. // 0.15-0.40 across paragraphs of mixed purpose.
  2700. if (cvStd < 0.08 && cvMean < 0.45) {
  2701. issues.push({
  2702. type: 'cross-para-burstiness',
  2703. text: `Sentence-rhythm uniform across paragraphs (σCV=${cvStd.toFixed(2)})`,
  2704. severity: 'medium',
  2705. suggestion: 'Every paragraph has the same internal rhythm. Humans vary cadence between terse and discursive paragraphs.',
  2706. });
  2707. }
  2708. }
  2709. }
  2710. // ── Tier 3 multi-word phrase density ─────────────────────────
  2711. // Two complementary rules:
  2712. // (a) Per-phrase density — same gating as single-word Tier 3: each
  2713. // phrase fine alone, repetition is the tell. Threshold = 2.
  2714. // (b) Cross-phrase clustering — ≥3 *distinct* boilerplate phrases
  2715. // in one piece. LLMs varying their own boilerplate often use
  2716. // each phrase only once but stack 5-10 across the text. The
  2717. // per-phrase rule misses this; the cluster rule catches it.
  2718. // Track non-overlapping match spans so a longer phrase swallowing a
  2719. // shorter one (e.g., "designed for long-term sustainability" matches
  2720. // both "designed for long-term" AND "long-term sustainability") only
  2721. // contributes one distinct hit. Without dedup the cluster threshold
  2722. // can be reached by a single sentence stacking overlapping regexes.
  2723. const claimedSpans = [];
  2724. function spanOverlaps(start, end) {
  2725. for (const [s, e] of claimedSpans) {
  2726. if (start < e && end > s) return true;
  2727. }
  2728. return false;
  2729. }
  2730. let distinctPhrasesHit = 0;
  2731. for (const phrase of TIER3_PHRASES) {
  2732. const regex = new RegExp(phrase.source, phrase.flags);
  2733. const phraseSpans = [];
  2734. let phraseMatch;
  2735. while ((phraseMatch = regex.exec(text)) !== null) {
  2736. const start = phraseMatch.index;
  2737. const end = start + phraseMatch[0].length;
  2738. if (!spanOverlaps(start, end)) {
  2739. phraseSpans.push([start, end, phraseMatch[0]]);
  2740. }
  2741. }
  2742. if (phraseSpans.length === 0) continue;
  2743. for (const [s, e] of phraseSpans) claimedSpans.push([s, e]);
  2744. distinctPhrasesHit++;
  2745. if (phraseSpans.length >= 2) {
  2746. issues.push({
  2747. type: 'tier3-phrase',
  2748. text: `"${phraseSpans[0][2].toLowerCase()}" x${phraseSpans.length}`,
  2749. severity: 'medium',
  2750. suggestion: `Boilerplate phrase repeated ${phraseSpans.length}× — replace at least one with specifics`,
  2751. });
  2752. }
  2753. }
  2754. if (distinctPhrasesHit >= 3) {
  2755. issues.push({
  2756. type: 'tier3-phrase-cluster',
  2757. text: `${distinctPhrasesHit} distinct boilerplate phrases`,
  2758. severity: 'high',
  2759. suggestion: 'Several stock crypto/web3 phrases stacked in one piece. Rewrite around one specific claim or observation.',
  2760. });
  2761. }
  2762. // ── Hashtag stuffing ─────────────────────────────────────────
  2763. // 6+ hashtags in a single post is rare for thoughtful humans and
  2764. // near-universal for LLM-generated social posts. Counted globally,
  2765. // not per-paragraph, since the trailing hashtag block is the shape
  2766. // we care about.
  2767. // Match #tag at start of text or after any non-word char (whitespace,
  2768. // punctuation, line breaks). URL fragments are already excluded
  2769. // because the char immediately before `#` in a URL path is always
  2770. // a word char (e.g. `example.com/page#section` — `e` before `#`).
  2771. // Earlier char class `[\s\\]` had a literal backslash and silently
  2772. // missed hashtags after sentence punctuation; an interim `[\s]` fix
  2773. // on origin only caught whitespace-preceded tags.
  2774. // Code is masked and non-tag `#` forms are subtracted first — see maskCode
  2775. // and isSocialTag. Without them a changelog paragraph citing six issue
  2776. // numbers, or a palette listing six hex colours, scored as a tag block.
  2777. const hashtagMatches = [...maskCode(text).matchAll(/(?:^|\W)#(\w[\w-]*)/g)]
  2778. .filter((m) => isSocialTag(m[1]));
  2779. if (hashtagMatches.length >= 6) {
  2780. issues.push({
  2781. type: 'hashtag-stuff',
  2782. text: `${hashtagMatches.length} hashtags`,
  2783. severity: 'medium',
  2784. suggestion: 'Cut to 2-3 specific tags or none. Long hashtag blocks read as bot output.',
  2785. });
  2786. }
  2787. // ── Bullet list of bare noun phrases ─────────────────────────
  2788. // ≥5 consecutive bullet items that are short (≤6 words) and contain
  2789. // no finite-verb / modal token. Catches the "Stable mining efficiency
  2790. // / Reliable pool connectivity / Optimized RandomX performance ..."
  2791. // shape LLMs default to. Markdown bullets, escaped Markdown bullets,
  2792. // unicode bullets, and dashes are all matched. Numbered lists are
  2793. // excluded — those have a separate "numbered list inflation" rule.
  2794. //
  2795. // Note: verbRe covers auxiliaries and modals ("was", "will", "can",
  2796. // etc.). Regular past-tense verbs ("fixed", "removed") are not
  2797. // matched here; instead, the ≤6-word length gate excludes most
  2798. // real-world changelog lines, which tend to read "fixed the X that
  2799. // was doing Y" (>6 words). Short two-word action items ("* fixed
  2800. // bug") would pass both gates — an acceptable trade-off to avoid
  2801. // false-negative risk from adjectives ending in -ed ("skilled",
  2802. // "advanced") that share the same surface form.
  2803. const lines = text.split(/\r?\n/);
  2804. const parseBullet = (line) => {
  2805. let cursor = 0;
  2806. while (cursor < line.length && /\s/.test(line[cursor])) cursor += 1;
  2807. if (!['*', '-', '•', '+'].includes(line[cursor])) return null;
  2808. cursor += 1;
  2809. const spacingStart = cursor;
  2810. while (cursor < line.length && /\s/.test(line[cursor])) cursor += 1;
  2811. if (cursor === spacingStart) return null;
  2812. if (cursor >= line.length) return cursor - spacingStart >= 2 ? '' : null;
  2813. return line.slice(cursor).trim();
  2814. };
  2815. const verbRe = /\b(?:is|are|was|were|has|have|had|will|would|should|must|do|does|did|can|could|may|might|am|been|being)\b/i;
  2816. const fenceRe = /^\s*(?:```|~~~)/;
  2817. let run = [];
  2818. let blankStreak = 0;
  2819. let inFence = false;
  2820. function flushRun() {
  2821. if (run.length >= 5) {
  2822. const bareNP = run.filter((it) => {
  2823. const wc = (it.match(/\S+/g) || []).length;
  2824. return wc > 0 && wc <= 6 && !verbRe.test(it);
  2825. });
  2826. if (bareNP.length >= 5 && bareNP.length / run.length >= 0.75) {
  2827. issues.push({
  2828. type: 'bullet-np-list',
  2829. text: `${run.length}-item bullet list of bare noun phrases`,
  2830. severity: 'high',
  2831. suggestion: 'Convert to a prose paragraph or merge items. Long lists of bare adj+noun pairs read as AI scaffolding.',
  2832. });
  2833. }
  2834. }
  2835. run = [];
  2836. blankStreak = 0;
  2837. }
  2838. for (const line of lines) {
  2839. if (fenceRe.test(line)) {
  2840. // Code-fence toggle. Bullets inside fences are CLI flag docs or
  2841. // option dumps, not prose AI scaffolding — flush any prose run
  2842. // we were tracking and skip until the fence closes.
  2843. flushRun();
  2844. inFence = !inFence;
  2845. continue;
  2846. }
  2847. if (inFence) continue;
  2848. const bullet = parseBullet(line);
  2849. if (bullet !== null) {
  2850. run.push(bullet);
  2851. blankStreak = 0;
  2852. } else if (line.trim() === '') {
  2853. // A single blank line inside a list is normal Markdown spacing;
  2854. // two or more blank lines break the run, since visually-disjoint
  2855. // bullet sections shouldn't merge into one logical list.
  2856. blankStreak++;
  2857. if (blankStreak >= 2) flushRun();
  2858. } else {
  2859. flushRun();
  2860. }
  2861. }
  2862. flushRun();
  2863. // NOTE: "Wall-of-text replies" (SKILL.md) is deliberately NOT a
  2864. // detector rule here. A first pass tried "reply-length text, >=4
  2865. // sentences, zero newlines" as a structural gate — it broke the
  2866. // "repeated Tier 1 phrase does not inflate score linearly" fixture
  2867. // and, on reflection, would fire on any ordinary short paragraph
  2868. // (a blog intro, a single-paragraph email) since "one paragraph with
  2869. // no internal line break" is simply what continuous prose looks
  2870. // like, not an AI-specific shape. The tell in SKILL.md depends on
  2871. // knowing the text is conversational-reply register in the first
  2872. // place, which the engine can't reliably infer from the bytes alone
  2873. // (CONTRIBUTING.md: "a signal that fires on most normal prose is not
  2874. // worth adding"). Left as an LLM-judgment rule; see CATEGORIES.md §C.
  2875. // Confidence calibration is only flagged when it stacks (3+ instances).
  2876. // Gating happens pre-dedup on raw match count, since that signals actual
  2877. // stacking, not just vocabulary use.
  2878. const confIssues = matchPatterns(text, CONFIDENCE_CALIBRATION, 'confidence-calibration', 'low');
  2879. if (confIssues.length >= 3) issues.push(...confIssues);
  2880. // ── 22. Em dash frequency ────────────────────────────────────
  2881. // Match real em dashes, plus `--` only when surrounded by whitespace on at
  2882. // least one side (skips CLI flags like --save-dev and YAML `---` blocks).
  2883. // Separator-position em dashes (SEPARATOR_DASH_RE above) are excluded
  2884. // from the rate. The list marker is required on purpose: a line-initial
  2885. // "**Bold lead** — full sentence" outside a list is itself an AI tell
  2886. // and still counts, as does a mid-sentence "**bold** — like this"
  2887. // splice. Em dash only — the `--` substitute is never carved out.
  2888. const rawEmDashCount = (text.match(/—|(?<=\s)--(?=\s|$)|(?<=^|\s)--(?=\s)/gm) || []).length;
  2889. const separatorDashCount = countSeparatorDashes(text);
  2890. const emDashCount = rawEmDashCount - separatorDashCount;
  2891. const emDashRate = emDashCount / (wordCount / 1000);
  2892. if (emDashRate > 1) {
  2893. issues.push({
  2894. type: 'em-dash',
  2895. text: `${emDashCount} em dashes in ${wordCount} words`,
  2896. severity: 'medium',
  2897. suggestion: 'Replace with commas, periods, or rewrite',
  2898. });
  2899. }
  2900. // ── 23. Sentence length uniformity ───────────────────────────
  2901. // Glossary entries ("**Term:** gloss", "- **Term** — gloss"; three or
  2902. // more distinct terms, wrapped lines included) are glossary typography: their lengths are parallel by convention, so
  2903. // they say nothing about prose rhythm. They leave the sample here and
  2904. // nowhere else; the five-sentence gate and the 0.25 threshold stay as
  2905. // they are, so prose around a glossary is still measured (#351).
  2906. const glosslessText = withoutGlossaryEntries(text);
  2907. const rhythmSentences = glosslessText === null ? sentences : getSentences(glosslessText);
  2908. if (rhythmSentences.length >= 5) {
  2909. const lengths = rhythmSentences.map(s => countWords(s));
  2910. const avg = lengths.reduce((a, b) => a + b, 0) / lengths.length;
  2911. const variance = lengths.reduce((sum, l) => sum + Math.pow(l - avg, 2), 0) / lengths.length;
  2912. const stdDev = Math.sqrt(variance);
  2913. const cv = avg > 0 ? stdDev / avg : 0;
  2914. if (cv < 0.25 && avg > 10) {
  2915. issues.push({
  2916. type: 'uniformity',
  2917. text: `Sentence lengths cluster around ${Math.round(avg)} words (low variation)`,
  2918. severity: 'medium',
  2919. suggestion: 'Mix short punchy sentences with longer flowing ones',
  2920. });
  2921. }
  2922. }
  2923. // ── Type-token ratio (stylometric — vocabulary diversity) ────
  2924. // TTR = distinct word types / total tokens, taken over each 200-token
  2925. // window and averaged (movingAverageTTR). Whole-text TTR falls with
  2926. // length whoever wrote the text: on the human documents in corpus/ its
  2927. // median is 0.63 at 200 tokens, 0.37 at 2,000 and 0.28 at 6,000, so a
  2928. // fixed threshold on it flagged every 6,000-word public-domain slice.
  2929. // The windowed median stays between 0.62 and 0.64 at every length.
  2930. // Within a window, human prose typically sits around 0.50–0.65 for
  2931. // English; AI prose tends flatter (0.55–0.75 looks normal, but the
  2932. // lower end of the *too-flat* tail at >=200 words is where the signal
  2933. // lives — too FEW unique words for the length). This is the simplest
  2934. // of the four stylometric signals identified in the May 2026 detection-
  2935. // research review: no POS tagger required, no model, pure JS.
  2936. //
  2937. // Threshold tuning: flag only when the sample is large enough
  2938. // that low TTR is meaningfully suspicious (>=200 tokens) AND TTR
  2939. // is below 0.40 (very vocabulary-poor). Conservative on purpose;
  2940. // false positives on short or topic-narrow human prose are easy
  2941. // to trigger and would drown out other signals. The detector-
  2942. // research lens flagged TTR as one of four stylometric add-ons.
  2943. // One of the other three has since shipped in approximated form:
  2944. // `cross-para-burstiness` covers sentence-length burstiness across
  2945. // paragraphs. `fnword-trigram-entropy` is a related tagger-free
  2946. // signal (it approximates POS-trigram entropy, not one of the
  2947. // three). POS-bigram log-odds and function-word z-scores are
  2948. // still TODO.
  2949. if (tokens.length >= 200) {
  2950. const diversity = movingAverageTTR(tokens, 200);
  2951. if (diversity < 0.4) {
  2952. issues.push({
  2953. type: 'low-ttr',
  2954. text: `Vocabulary diversity ${(diversity * 100).toFixed(1)}% (distinct words per 200-token window, ${tokens.length} tokens)`,
  2955. severity: 'low',
  2956. suggestion: 'Text reuses a narrow word set. Vary nouns and verbs deliberately, or check if the topic genuinely warrants the repetition.',
  2957. });
  2958. }
  2959. }
  2960. // ── 24. Paragraph length uniformity ──────────────────────────
  2961. if (paragraphs.length >= 4) {
  2962. const paraLengths = paragraphs.map(p => getSentences(p).length);
  2963. const avg = paraLengths.reduce((a, b) => a + b, 0) / paraLengths.length;
  2964. const allSimilar = paraLengths.every(l => Math.abs(l - avg) <= 1);
  2965. if (allSimilar && avg >= 3) {
  2966. issues.push({
  2967. type: 'uniformity',
  2968. text: `All paragraphs are ~${Math.round(avg)} sentences`,
  2969. severity: 'low',
  2970. suggestion: 'Vary paragraph length deliberately',
  2971. });
  2972. }
  2973. }
  2974. // ── 25. Bold overuse ─────────────────────────────────────────
  2975. const boldMatches = text.match(/\*\*[^*]+\*\*/g) || [];
  2976. if (boldMatches.length > 3) {
  2977. issues.push({
  2978. type: 'formatting',
  2979. text: `${boldMatches.length} bold phrases`,
  2980. severity: 'medium',
  2981. suggestion: 'Strip bold from most; restructure to lead with key info',
  2982. });
  2983. }
  2984. // ── Score from the deduped issue list ───────────────────────
  2985. // Previously rawScore was accumulated inline per pattern hit, so
  2986. // repeated hits of the same phrase (or overlapping matches) inflated
  2987. // the score while the displayed issue list was deduplicated. That
  2988. // produced the UX regression where a "heavy AI patterns" label sat
  2989. // above a list of two items. Now the dedup runs first, then each
  2990. // distinct issue contributes its category weight — so the number
  2991. // reflects the same signals the user actually sees.
  2992. // The staged-discovery phrase can contain the older "the most interesting
  2993. // part" flatline match or sentence-initial "Turns out". Report the
  2994. // enclosing signal once while leaving unrelated findings intact.
  2995. const stagedDiscoverySpans = stagedDiscoveryIssues
  2996. .map((issue) => ({ start: issue.index, end: issue.index + issue.text.length }));
  2997. const nonOverlappingIssues = issues.filter((issue) =>
  2998. (issue.type !== 'emotional-flatline' && issue.type !== 'performed-insight') ||
  2999. stagedDiscoveryIssues.includes(issue) ||
  3000. !Number.isInteger(issue.index) ||
  3001. !stagedDiscoverySpans.some((span) =>
  3002. issue.index >= span.start && issue.index + issue.text.length <= span.end
  3003. )
  3004. );
  3005. const deduped = deduplicateIssues(nonOverlappingIssues);
  3006. for (const issue of deduped) {
  3007. rawScore += ISSUE_WEIGHTS[issue.type] ?? 2;
  3008. }
  3009. // Scale by text length: longer text gets more chances to trigger.
  3010. const lengthFactor = Math.max(1, Math.log2(wordCount / 50));
  3011. const normalizedScore = Math.min(100, Math.round(rawScore / lengthFactor));
  3012. const label = getLabel(normalizedScore);
  3013. // ── Sentence-region smoothing (HMM-style without an HMM) ─────────
  3014. // Map each text-bearing issue back to its sentence indexes, then
  3015. // merge adjacent flagged sentences into contiguous regions for UI
  3016. // highlighting. Borrowed from GPTZero's sentence-highlighting model
  3017. // — gives users "this paragraph is AI" rather than scattered hits.
  3018. const regions = buildSentenceRegions(text, deduped, sourceMode === 'rendered-markdown');
  3019. // Stats derived from the same deduped list so tier counts + patternCount
  3020. // sum to `deduped.length`. Previously patternCount subtracted
  3021. // `tier2Clusters` (a paragraph count) which produced inconsistent totals.
  3022. const tier1Count = deduped.filter((i) => i.type === 'tier1').length;
  3023. const tier2Count = deduped.filter((i) => i.type === 'tier2').length;
  3024. const tier3Count = deduped.filter((i) => i.type === 'tier3').length;
  3025. // ── Trinary classification (GPTZero-shaped) ──────────────────────
  3026. // Decouples confidence from AI-proportion. Maps the 0-100 score plus
  3027. // structural signals into HUMAN_ONLY / MIXED / AI_ONLY with a
  3028. // confidence band. Thresholds are FN-biased: ambiguity routes to
  3029. // MIXED, never AI_ONLY. Quote from GPTZero's design principle:
  3030. // "biases the detector to prefer making less-harmful false-negative
  3031. // errors over false-positive errors."
  3032. // Dense-AI-vocab trifecta: ≥5 distinct tier1 hits + ≥2 tier2 cluster
  3033. // paragraphs + ≥1 transition phrase, AND ≥150 words. Catches
  3034. // saturated ChatGPT prose without firing on dense-jargon human
  3035. // technical writing where the tier1 vocabulary (robust,
  3036. // comprehensive, leverage, ecosystem) legitimately overlaps with
  3037. // systems-programming idiom. Word-count gate prevents short ESL or
  3038. // contrived adversarial sentences from tripping the corroborator.
  3039. const tier1Distinct = new Set(deduped.filter((i) => i.type === 'tier1').map((i) => (i.text || '').toLowerCase())).size;
  3040. const hasTier2Cluster = tier2Clusters >= 2;
  3041. const hasTransition = deduped.some((i) => i.type === 'transition');
  3042. const denseAIVocab = wordCount >= 150 && tier1Distinct >= 5 && hasTier2Cluster && hasTransition;
  3043. const trinary = classifyTrinary({
  3044. score: normalizedScore,
  3045. issues: deduped,
  3046. regions,
  3047. normFlags: norm.flags,
  3048. wordCount,
  3049. denseAIVocab,
  3050. });
  3051. // Translate both dense issue indexes and half-open highlight boundaries.
  3052. // Mapping the region end from its final retained code unit avoids pulling
  3053. // a later removed roleplay marker into the highlighted source slice.
  3054. remapFindingsToSource(deduped, regions, sourceMap);
  3055. return {
  3056. // Legacy fields preserved for existing callers.
  3057. score: normalizedScore,
  3058. label,
  3059. issues: deduped,
  3060. stats: {
  3061. wordCount,
  3062. tier1Count,
  3063. tier2Count,
  3064. tier2Clusters,
  3065. tier3Count,
  3066. tier3Flags,
  3067. patternCount: deduped.length - tier1Count - tier2Count - tier3Count,
  3068. contextMode,
  3069. contextModeFallback,
  3070. sourceMode,
  3071. sourceModeFallback,
  3072. maskedFrontmatter,
  3073. maskedHtmlComments,
  3074. ignoredRegions,
  3075. normalization: norm.flags,
  3076. quotedLines,
  3077. maskedQuotes,
  3078. unmappedHighlights: regions._unmapped ?? 0,
  3079. denseAIVocab,
  3080. tier1Distinct,
  3081. },
  3082. // Trinary API — shape mirrors GPTZero so integrators can swap.
  3083. document_classification: trinary.classification,
  3084. class_probabilities: trinary.probabilities,
  3085. confidence_category: trinary.confidence,
  3086. highlight_sentence_for_ai: regions,
  3087. };
  3088. }
  3089. // ═══ Sentence regions + trinary classifier ═════════════════════════
  3090. // Coarse sentence spans over the whole text, as [start, end) offsets.
  3091. // Produces the same spans as the former /[^.!?]+[.!?]+|\S[^.!?]*$/g scan:
  3092. // each span runs from the end of the previous one through the next run of
  3093. // terminators, and a trailing fragment with no terminator starts at its
  3094. // first non-space character. The regex version backtracked to the end of
  3095. // the input at every position of a long terminator-free run, so a document
  3096. // that ended in blank lines, or whose masked comments became whitespace,
  3097. // cost O(n^2) (#235). This scan touches each character a bounded number
  3098. // of times.
  3099. const SENTENCE_TERMINATOR_RUN = /[.!?]+/g;
  3100. const FIRST_NON_SPACE = /\S/g;
  3101. function splitSentenceSpans(text) {
  3102. const spans = [];
  3103. const length = text.length;
  3104. let pos = 0;
  3105. while (pos < length) {
  3106. SENTENCE_TERMINATOR_RUN.lastIndex = pos;
  3107. const run = SENTENCE_TERMINATOR_RUN.exec(text);
  3108. if (run === null) {
  3109. FIRST_NON_SPACE.lastIndex = pos;
  3110. const head = FIRST_NON_SPACE.exec(text);
  3111. if (head !== null) spans.push([head.index, length]);
  3112. break;
  3113. }
  3114. if (run.index === pos) {
  3115. // Skip the entire bodyless run, not one character at a time: matching
  3116. // every remaining suffix would make a long punctuation run quadratic.
  3117. // If no later sentence terminator exists, the former regex's trailing
  3118. // alternative starts at the LAST terminator of this run.
  3119. const runEnd = pos + run[0].length;
  3120. SENTENCE_TERMINATOR_RUN.lastIndex = runEnd;
  3121. if (SENTENCE_TERMINATOR_RUN.exec(text) === null) {
  3122. spans.push([runEnd - 1, length]);
  3123. break;
  3124. }
  3125. pos = runEnd;
  3126. continue;
  3127. }
  3128. const end = run.index + run[0].length;
  3129. spans.push([pos, end]);
  3130. pos = end;
  3131. }
  3132. return spans;
  3133. }
  3134. function buildSentenceRegions(text, issues, trimBoundaryWhitespace = false) {
  3135. // Split text into sentences with source offsets preserved so the UI
  3136. // can highlight spans accurately. Sentence boundaries are coarse
  3137. // (.!?) — fine for highlighting, not for linguistic correctness.
  3138. const sentences = [];
  3139. for (const [spanStart, spanEnd] of splitSentenceSpans(text)) {
  3140. const raw = text.slice(spanStart, spanEnd);
  3141. const sentenceText = raw.trim();
  3142. if (sentenceText.length < 4) continue;
  3143. let start = spanStart;
  3144. let end = spanEnd;
  3145. if (trimBoundaryWhitespace) {
  3146. // trimStart/trimEnd, not `\s*$`: that regex retries from every
  3147. // position of a leading whitespace run and was quadratic on a span
  3148. // that began with a masked comment block (#235).
  3149. start += raw.length - raw.trimStart().length;
  3150. end -= raw.length - raw.trimEnd().length;
  3151. }
  3152. sentences.push({ start, end, text: sentenceText });
  3153. }
  3154. if (sentences.length === 0) return [];
  3155. // Map issue.text back to sentence indexes via substring search. Two
  3156. // kinds of issue stay out of the AI-highlight regions. Summary signals
  3157. // like "Punctuation density uniform across paragraphs" have no sentence
  3158. // anchor — they contribute to the document-level signal but not to
  3159. // highlights. Any category with authorship weight 0 is style-only by
  3160. // definition, so it belongs in issues[] but never in a field reserved
  3161. // for AI sentence highlights.
  3162. // Filter by issue TYPE not text-regex: text-based filtering used to
  3163. // drop legitimate phrase issues containing "across" / "density".
  3164. const NON_HIGHLIGHT_TYPES = new Set([
  3165. 'punct-distribution',
  3166. 'cross-para-burstiness',
  3167. 'fnword-trigram-entropy',
  3168. 'smart-punct-signature',
  3169. 'normalization-flag',
  3170. 'uniformity',
  3171. 'em-dash',
  3172. 'formatting',
  3173. 'tier3',
  3174. 'tier3-phrase',
  3175. 'tier3-phrase-cluster',
  3176. 'hashtag-stuff',
  3177. 'bullet-np-list',
  3178. 'unnecessary-hyphenation',
  3179. ]);
  3180. const hits = sentences.map(() => ({ count: 0, weight: 0 }));
  3181. const lowerText = text.toLowerCase();
  3182. let unmappedHighlights = 0;
  3183. for (const issue of issues) {
  3184. if (!issue.text || issue.text.length > 200) continue;
  3185. if (NON_HIGHLIGHT_TYPES.has(issue.type)) continue;
  3186. if ((ISSUE_WEIGHTS[issue.type] ?? 2) === 0) continue;
  3187. const needle = issue.text.toLowerCase();
  3188. let idx = 0;
  3189. let matched = false;
  3190. while ((idx = lowerText.indexOf(needle, idx)) !== -1) {
  3191. // Deduplicated verb issues must not highlight skipped noun matches.
  3192. // Reuse the same predicate and retain all qualifying verb locations.
  3193. if (issue.type === 'tier1-clarity' && needle === 'features' && featuresIsNoun(text, idx)) {
  3194. idx += needle.length;
  3195. continue;
  3196. }
  3197. matched = true;
  3198. for (let i = 0; i < sentences.length; i++) {
  3199. if (idx >= sentences[i].start && idx < sentences[i].end) {
  3200. hits[i].count++;
  3201. hits[i].weight += ISSUE_WEIGHTS[issue.type] ?? 2;
  3202. break;
  3203. }
  3204. }
  3205. idx += needle.length;
  3206. }
  3207. if (!matched) unmappedHighlights++;
  3208. }
  3209. // Window-merge contiguous flagged sentences. Allow 1 unflagged
  3210. // sentence gap between two flagged ones (the "smoothing" — keeps
  3211. // a single boring sentence from breaking what's clearly an AI
  3212. // passage). A sentence is "flagged" if it has ≥1 hit.
  3213. const regions = [];
  3214. let cur = null;
  3215. for (let i = 0; i < sentences.length; i++) {
  3216. if (hits[i].count > 0) {
  3217. if (cur === null) {
  3218. cur = { startSentence: i, endSentence: i, start: sentences[i].start, end: sentences[i].end, hitCount: hits[i].count, weight: hits[i].weight };
  3219. } else {
  3220. cur.endSentence = i;
  3221. cur.end = sentences[i].end;
  3222. cur.hitCount += hits[i].count;
  3223. cur.weight += hits[i].weight;
  3224. }
  3225. } else if (cur !== null) {
  3226. // Allow one-sentence gap.
  3227. const next = hits[i + 1];
  3228. if (next && next.count > 0) {
  3229. cur.endSentence = i;
  3230. cur.end = sentences[i].end;
  3231. continue;
  3232. }
  3233. regions.push(finalizeRegion(cur));
  3234. cur = null;
  3235. }
  3236. }
  3237. if (cur !== null) regions.push(finalizeRegion(cur));
  3238. // Expose the unmapped-highlight count via a non-enumerable property
  3239. // so the array length still reads naturally for consumers; the
  3240. // analyzer pulls it into stats.unmappedHighlights for diagnostics.
  3241. Object.defineProperty(regions, '_unmapped', { value: unmappedHighlights, enumerable: false });
  3242. return regions;
  3243. }
  3244. function finalizeRegion(r) {
  3245. // Map cumulative weight inside the region to a 0-1 score. Cap at 20
  3246. // weight = 1.0 (matches Heavy threshold density).
  3247. const score = Math.min(1, r.weight / 20);
  3248. return {
  3249. startSentence: r.startSentence,
  3250. endSentence: r.endSentence,
  3251. start: r.start,
  3252. end: r.end,
  3253. hitCount: r.hitCount,
  3254. score: Math.round(score * 100) / 100,
  3255. };
  3256. }
  3257. // FN-biased: false positives damage trust more than false negatives,
  3258. // so MIXED is wide and AI_ONLY requires multiple signals. Quote from
  3259. // GPTZero: "biases the detector to prefer making less-harmful
  3260. // false-negative errors over false-positive errors."
  3261. function classifyTrinary({ score, issues, regions, normFlags, wordCount, denseAIVocab }) {
  3262. // Strong corroborators — each is near-dispositive on its own:
  3263. // - cutoff-disclaimer (LLM self-identifies as an AI)
  3264. // - reasoning-artifact + chatbot-artifact co-occurrence
  3265. // - normalization-flag at threshold (≥2 ZWSP or homoglyphs).
  3266. // Threshold parity prevents a single stray ZWSP in copy-paste
  3267. // from Word/Notion from flipping to AI_ONLY at score 0.
  3268. // - denseAIVocab: ≥4 distinct tier1 hits AND ≥1 tier2 cluster AND
  3269. // ≥1 transition phrase — the trifecta that saturated ChatGPT
  3270. // prose triggers without needing whitelisted stylometric hits.
  3271. const hasCutoff = issues.some((i) => i.type === 'cutoff-disclaimer');
  3272. const hasNormFlag = normFlags.zeroWidth >= 2 || normFlags.homoglyph >= 2;
  3273. const hasReasoning = issues.some((i) => i.type === 'reasoning-artifact');
  3274. const hasChatbot = issues.some((i) => i.type === 'chatbot');
  3275. const strongCorrob =
  3276. (hasCutoff ? 1 : 0) +
  3277. (hasNormFlag ? 1 : 0) +
  3278. (hasReasoning && hasChatbot ? 1 : 0) +
  3279. (denseAIVocab ? 1 : 0);
  3280. // Weak stylometric corroborators. The style-only smart-punct finding
  3281. // must not alter classification, probabilities, or confidence even when
  3282. // other findings already put the score near a classification boundary.
  3283. const stylometricHits = ['punct-distribution', 'cross-para-burstiness', 'fnword-trigram-entropy']
  3284. .filter((t) => issues.some((i) => i.type === t)).length;
  3285. const weakCorrob = stylometricHits >= 2 ? 1 : 0;
  3286. // Thresholds:
  3287. // score < 15 with no strong → HUMAN_ONLY
  3288. // strong ≥ 1 OR score ≥ 70 → AI_ONLY (lowered from 80; high
  3289. // density of AI vocab is sufficient evidence)
  3290. // score ≥ 40 with any corroborator → AI_ONLY
  3291. // everything else with score ≥ 15 → MIXED
  3292. const totalCorrob = strongCorrob + weakCorrob;
  3293. let classification;
  3294. if (score < 15 && strongCorrob === 0) classification = 'HUMAN_ONLY';
  3295. else if (strongCorrob >= 1 || score >= 70) classification = 'AI_ONLY';
  3296. else if (score >= 40 && totalCorrob >= 1) classification = 'AI_ONLY';
  3297. else classification = 'MIXED';
  3298. // Humanizer-flag escalation: presence of bypass-trick chars is
  3299. // adversarial signal. If a normalization-flag fired we already
  3300. // counted it in strongCorrob → AI_ONLY. Confidence also gets a
  3301. // floor of 'medium' in that case (an adversary actively evading
  3302. // detection should never read as low-confidence noise).
  3303. // Soft probability distribution. Hand-tuned, not calibrated
  3304. // against the labeled corpora the repo now samples
  3305. // (`scripts/dataset-hc3.js`, `scripts/dataset-raid.js`). Largest
  3306. // class is computed as `1 - others` after rounding to guarantee
  3307. // sum=1 exactly. Sub-1% drift would otherwise hide in toFixed.
  3308. const aiSoft = Math.min(0.97, score / 100 + totalCorrob * 0.06 + strongCorrob * 0.08);
  3309. let p;
  3310. if (classification === 'HUMAN_ONLY') p = { human: Math.max(0.6, 1 - aiSoft), mixed: Math.min(0.35, aiSoft * 0.8), ai: Math.min(0.1, aiSoft * 0.3) };
  3311. else if (classification === 'AI_ONLY') p = { human: Math.max(0.02, 1 - aiSoft - 0.05), mixed: 0.1, ai: aiSoft };
  3312. else p = { human: Math.max(0.15, 0.6 - aiSoft * 0.5), mixed: 0.5, ai: aiSoft * 0.7 };
  3313. const rawSum = p.human + p.mixed + p.ai;
  3314. p.human = +(p.human / rawSum).toFixed(3);
  3315. p.mixed = +(p.mixed / rawSum).toFixed(3);
  3316. // Assign ai as the remainder so the three values sum to exactly 1.
  3317. // Clamp to >= 0 in case rounding pushes human+mixed above 1 (the
  3318. // remainder would otherwise show as -0 or -0.001 — surfaces as a
  3319. // negative percentage in any UI doing Math.round(p.ai * 100)).
  3320. p.ai = Math.max(0, +(1 - p.human - p.mixed).toFixed(3));
  3321. const probabilities = p;
  3322. // Confidence band:
  3323. // high — strongCorrob ≥ 2, OR cutoff-disclaimer, OR score < 8 (clean long doc)
  3324. // medium — strongCorrob ≥ 1, OR score ≥ 45 with weak corroborator, OR score < 20
  3325. // low — everything else
  3326. let confidence;
  3327. if (strongCorrob >= 2 || hasCutoff || (score < 8 && wordCount >= 100)) confidence = 'high';
  3328. else if (strongCorrob >= 1 || (score >= 45 && weakCorrob >= 1) || score < 20) confidence = 'medium';
  3329. else confidence = 'low';
  3330. return { classification, probabilities, confidence };
  3331. }
  3332. function getLabel(score) {
  3333. if (score === 0) return 'Clean';
  3334. if (score <= 15) return 'Minimal AI signals';
  3335. if (score <= 35) return 'Some AI patterns';
  3336. if (score <= 60) return 'Moderate AI signals';
  3337. if (score <= 80) return 'Strong AI signals';
  3338. return 'Heavy AI patterns';
  3339. }
  3340. function getColor(score) {
  3341. if (score <= 15) return '#44bb66';
  3342. if (score <= 35) return '#88bb44';
  3343. if (score <= 60) return '#ddaa00';
  3344. if (score <= 80) return '#ff8833';
  3345. return '#ff4444';
  3346. }
  3347. function deduplicateIssues(issues) {
  3348. const seen = new Set();
  3349. return issues.filter(issue => {
  3350. const key = `${issue.type}:${issue.text.toLowerCase()}`;
  3351. if (seen.has(key)) return false;
  3352. seen.add(key);
  3353. return true;
  3354. });
  3355. }
  3356. // ─── Severity labels ──────────────────────────────────────────
  3357. const SEVERITY_LABELS = {
  3358. critical: 'P0',
  3359. high: 'P1',
  3360. medium: 'P2',
  3361. low: 'P3',
  3362. };
  3363. const TYPE_LABELS = {
  3364. 'tier1': 'AI vocabulary',
  3365. 'tier1-clarity': 'Wordiness',
  3366. 'tier2': 'Word cluster',
  3367. 'tier3': 'Overused word',
  3368. 'transition': 'AI transition',
  3369. 'chatbot': 'Chatbot artifact',
  3370. 'sycophantic': 'Sycophantic tone',
  3371. 'filler': 'Filler phrase',
  3372. 'generic-conclusion': 'Generic conclusion',
  3373. 'lets-construction': '"Let\'s" opener',
  3374. 'reasoning-artifact': 'Reasoning artifact',
  3375. 'significance-inflation': 'Significance inflation',
  3376. 'vague-attribution': 'Vague attribution',
  3377. 'hollow-intensifier': 'Hollow intensifier',
  3378. // Keep the public type stable for API consumers; the user-facing name now
  3379. // describes the stock framing that the regexes actually match.
  3380. 'emotional-flatline': 'Stock reaction framing',
  3381. 'lingering-attention': 'Lingering-attention claim',
  3382. 'novelty-inflation': 'Novelty inflation',
  3383. 'cutoff-disclaimer': 'Cutoff disclaimer',
  3384. 'template-phrase': 'Template phrase',
  3385. 'false-concession': 'False concession',
  3386. 'rhetorical-question': 'Rhetorical question',
  3387. 'confidence-calibration': 'Confidence stacking',
  3388. 'em-dash': 'Em dash overuse',
  3389. 'uniformity': 'Rhythm uniformity',
  3390. 'formatting': 'Formatting',
  3391. 'tier3-phrase': 'Boilerplate phrase',
  3392. 'tier3-phrase-cluster': 'Boilerplate cluster',
  3393. 'hashtag-stuff': 'Hashtag stuffing',
  3394. 'bullet-np-list': 'Bullet-NP list',
  3395. 'hedge-stack': 'Hedge-stacked prediction',
  3396. 'future-narrative': 'Generic future narrative',
  3397. 'real-actual-inflation': '"Real/actual" inflation',
  3398. 'social-cta-closer': 'Engagement-bait closer',
  3399. 'formulaic-opener': 'Formulaic opener',
  3400. 'speculative-opener': 'Speculative scenario opener',
  3401. 'launch-intro': 'Launch-copy introduction',
  3402. 'crowd-contrast': 'Dramatized crowd contrast',
  3403. 'fake-casual-prop': 'Fake-casual prop',
  3404. 'title-case-header': 'Title Case header',
  3405. 'parenthetical-hedge': 'Parenthetical hedge',
  3406. 'smart-punct-signature': 'Smart-punct signature',
  3407. 'punct-distribution': 'Punctuation distribution',
  3408. 'fnword-trigram-entropy': 'Grammar repetition',
  3409. 'cross-para-burstiness': 'Cross-paragraph rhythm',
  3410. 'normalization-flag': 'Bypass-trick chars',
  3411. 'low-ttr': 'Low vocabulary diversity',
  3412. 'ai-placeholder': 'Unfilled placeholder',
  3413. 'ai-citation-markup': 'Chatbot citation markup leak',
  3414. 'ai-utm-source': 'AI-tool URL parameter',
  3415. 'unnecessary-hyphenation': 'Unnecessary hyphenation',
  3416. 'performed-insight': 'Performed-insight phrase',
  3417. 'negation-chain': 'Negation chain',
  3418. 'negative-parallelism': 'Negative parallelism',
  3419. 'dev-blog-boilerplate': 'Dev-blog boilerplate',
  3420. };
  3421. // The pattern tables as plain data, keyed by issue type, for tooling that
  3422. // re-emits them in another format (scripts/build-vale.js generates the Vale
  3423. // style in `vale/` from this). Regexes are `{ source, flags }` strings, not
  3424. // the live objects, so no `lastIndex` state is shared. A `gate` names logic
  3425. // analyzeText applies on top of the table, so the table alone does not
  3426. // reproduce a gated finding. Function fields (the `features` noun check,
  3427. // computed hyphenation fixes) cannot be data: they become a `gate` note or
  3428. // are left out. Not used by analyzeText. Bump `schemaVersion` on a change
  3429. // to this shape, not on a change to the patterns.
  3430. const PATTERN_TABLES = (() => {
  3431. const re = (r) => ({ source: r.source, flags: r.flags });
  3432. const list = (patterns) => patterns.map(re);
  3433. const freeze = (value) => {
  3434. if (value && typeof value === 'object') {
  3435. for (const v of Object.values(value)) freeze(v);
  3436. Object.freeze(value);
  3437. }
  3438. return value;
  3439. };
  3440. const NP_GATE = 'Reported only when another negative-parallelism frame starts within two sentences in the same paragraph.';
  3441. return freeze({
  3442. schemaVersion: '0.1.0',
  3443. types: {
  3444. 'tier1': {
  3445. words: Object.entries(TIER1).map(([word, replace]) => ({ word, replace })),
  3446. patterns: TIER1_PHRASES.filter((p) => !p.clarity).map((p) => ({ ...re(p.pattern), replace: p.replace })),
  3447. gate: "`words` match whole lowercase tokens split on [\\w'-]+, so hyphenated and possessive forms do not count. The `technical` context skips TECHNICAL_EXEMPT terms.",
  3448. },
  3449. 'tier1-clarity': {
  3450. patterns: TIER1_PHRASES.filter((p) => p.clarity).map((p) => ({
  3451. ...re(p.pattern),
  3452. replace: p.replace,
  3453. ...(p.skip ? { gate: 'Skipped where "features" reads as a noun (featuresIsNoun).' } : {}),
  3454. })),
  3455. },
  3456. 'tier2': {
  3457. words: Object.entries(TIER2).map(([word, replace]) => ({ word, replace })),
  3458. patterns: TIER2_CONDITIONAL.map((c) => ({ ...re(c.pattern), word: c.word, replace: c.suggestion })),
  3459. gate: 'Reported only when two or more distinct listed words share a paragraph. The `technical` context skips TECHNICAL_EXEMPT terms.',
  3460. },
  3461. 'tier3': {
  3462. words: [...TIER3],
  3463. gate: 'Reported only at max(3, floor(words * 0.03)) uses of one word; hyphenated words also match their dashless form.',
  3464. },
  3465. 'tier3-phrase': {
  3466. patterns: list(TIER3_PHRASES),
  3467. gate: 'Reported only at the second non-overlapping use of one phrase.',
  3468. },
  3469. 'tier3-phrase-cluster': {
  3470. patterns: list(TIER3_PHRASES),
  3471. gate: 'Reported only when three or more distinct phrases appear.',
  3472. },
  3473. 'transition': { patterns: list(TRANSITIONS) },
  3474. 'chatbot': { patterns: list(CHATBOT_ARTIFACTS) },
  3475. 'sycophantic': { patterns: list(SYCOPHANTIC) },
  3476. 'filler': { patterns: list(FILLERS) },
  3477. 'generic-conclusion': { patterns: list(GENERIC_CONCLUSIONS) },
  3478. 'lets-construction': { patterns: list(LETS_PATTERNS) },
  3479. 'reasoning-artifact': { patterns: list(REASONING_ARTIFACTS) },
  3480. 'significance-inflation': { patterns: list(SIGNIFICANCE_INFLATION) },
  3481. 'vague-attribution': { patterns: list(VAGUE_ATTRIBUTIONS) },
  3482. 'hollow-intensifier': { patterns: list(HOLLOW_INTENSIFIERS) },
  3483. 'emotional-flatline': { patterns: list(EMOTIONAL_FLATLINE) },
  3484. 'lingering-attention': { patterns: list(LINGERING_ATTENTION) },
  3485. 'novelty-inflation': { patterns: list(NOVELTY_INFLATION) },
  3486. 'cutoff-disclaimer': { patterns: list(CUTOFF_DISCLAIMERS) },
  3487. 'ai-placeholder': { patterns: list(AI_PLACEHOLDERS) },
  3488. 'ai-citation-markup': { patterns: list(AI_CITATION_MARKUP) },
  3489. 'ai-utm-source': { patterns: list(AI_UTM_SOURCE) },
  3490. 'template-phrase': { patterns: list(TEMPLATE_PHRASES) },
  3491. 'false-concession': { patterns: list(FALSE_CONCESSION) },
  3492. 'rhetorical-question': { patterns: list(RHETORICAL_QUESTIONS) },
  3493. 'hedge-stack': { patterns: list(HEDGE_STACK) },
  3494. 'future-narrative': { patterns: list(FUTURE_NARRATIVE) },
  3495. 'real-actual-inflation': { patterns: list(REAL_ACTUAL_INFLATION) },
  3496. 'social-cta-closer': { patterns: list(SOCIAL_CTA_CLOSER) },
  3497. 'performed-insight': { patterns: list([...PERFORMED_INSIGHT, ...STAGED_DISCOVERY]) },
  3498. 'negation-chain': { patterns: list(NEGATION_CHAIN) },
  3499. 'negative-parallelism': {
  3500. patterns: [
  3501. ...list(NEGATIVE_PARALLELISM_REVEAL),
  3502. ...NEGATIVE_PARALLELISM_CONTRAST.map((r) => ({ ...re(r), gate: NP_GATE })),
  3503. ],
  3504. },
  3505. 'dev-blog-boilerplate': { patterns: list(DEV_BLOG_BOILERPLATE) },
  3506. 'unnecessary-hyphenation': {
  3507. patterns: UNNECESSARY_HYPHENATION.map((e) => ({
  3508. ...re(e.pattern),
  3509. ...(typeof e.suggestion === 'string' ? { replace: e.suggestion } : {}),
  3510. })),
  3511. gate: 'Runs on text with file paths, package names and code identifiers masked (maskHyphenationProtected).',
  3512. },
  3513. 'formulaic-opener': { patterns: list(FORMULAIC_OPENERS) },
  3514. 'speculative-opener': { patterns: list(SPECULATIVE_OPENERS) },
  3515. 'launch-intro': { patterns: list(LAUNCH_INTROS) },
  3516. 'crowd-contrast': { patterns: list(CROWD_CONTRAST) },
  3517. 'fake-casual-prop': { patterns: list(FAKE_CASUAL_PROPS) },
  3518. 'parenthetical-hedge': { patterns: list(PARENTHETICAL_HEDGE) },
  3519. 'confidence-calibration': {
  3520. patterns: list(CONFIDENCE_CALIBRATION),
  3521. gate: 'Reported only when three or more matches occur in one text.',
  3522. },
  3523. },
  3524. });
  3525. })();
  3526. return {
  3527. analyzeText,
  3528. normalizeText,
  3529. getLabel,
  3530. getColor,
  3531. SEVERITY_LABELS,
  3532. TYPE_LABELS,
  3533. CONTEXT_MODES,
  3534. SOURCE_MODES,
  3535. PATTERN_TABLES,
  3536. };
  3537. })();
  3538. if (typeof module !== 'undefined' && module.exports) {
  3539. module.exports = AIDetector;
  3540. }

Ratings and comments

Sign in to rate this skill.

Sign in

Free to use. If it’s helped, pass it on.