claude-skills
Benjamin Vial's skills: own skills plus vetted copies of external ones.
Safety scan
A static text scan, not a guarantee. Select the badge to see what was scanned and every finding.
Check it deeper on your machine
It has changed since its scan on 19 Nov 2026. Its scripts were only partly scanned.
As a Claude Code plugin, install it and then ask Claude Code:
/plugin marketplace add keithmackay/mackayi/plugin install scanskillsafety@mackayiDo a deep scan of https://github.com/benvial/claude-skills with scanskillsafety
Or from a terminal, clone it and run it:
git clone https://github.com/keithmackay/scanskillsafety-skill ~/.claude/skills/scanskillsafetynode ~/.claude/skills/scanskillsafety/cli.cjs --deep 'https://github.com/benvial/claude-skills'
It checks the exact commit: every file, its packages against OSV (api.osv.dev), how it compares with this page's scan (findsafeskills.com receives the repository's address), and, if you agree, Claude reading every file with your own Claude subscription. It never runs the code.
Badge for your README
Markdown
[](/listings/benvial--claude-skills)HTML
<a href="/listings/benvial--claude-skills"><img src="/api/listings/benvial--claude-skills/badge.svg" alt="findsafeskills safety rating" /></a>Install
Install instructions from the README
As written by the repo's author. findsafeskills has not verified these steps.
Claude Code, on each machine:
claude plugin marketplace add benvial/claude-skills claude plugin install skills@claude-skills
Then enable auto-update: `/plugin` → Marketplaces → claude-skills → Enable auto-update.
claude.ai: Customize → Plugins → Add → Add marketplace → Add from a repository → `benvial/claude-skills`, then install `skills`.
What was scanned
The files the safety scan read, as written by the repository's author and shown as plain text. Lines with a finding are marked.
.claude-plugin/marketplace.json · Manifest · 15 lines
- {
- "$schema": "https://json.schemastore.org/claude-code-marketplace.json",
- "name": "claude-skills",
- "description": "Benjamin Vial's skills: own skills plus vetted copies of external ones.",
- "owner": {
- "name": "Benjamin Vial"
- },
- "plugins": [
- {
- "name": "skills",
- "source": "./plugins/skills",
- "description": "All skills and subagents in one plugin."
- }
- ]
- }
README.md · README · 40 lines
- # claude-skills
- One Claude Code plugin marketplace holding all my skills: my own, plus pinned copies of external ones.
- ## Layout
- - `plugins/skills/skills/<name>/` — every skill. Own skills are edited here directly.
- - `plugins/skills/agents/` — subagents that come with external skills.
- - `vendor.json` — external skills: upstream repo, pinned commit, and which paths to copy (or `plugin`: take every skill an upstream plugin.json lists).
- - `LICENSES/<upstream>/` — each upstream's license and the exact commit copied.
- - `scripts/sync.py` — copies the pinned external skills into the plugin.
- ## Install
- Claude Code, on each machine:
- ```bash
- claude plugin marketplace add benvial/claude-skills
- claude plugin install skills@claude-skills
- ```
- Then enable auto-update: `/plugin` → Marketplaces → claude-skills → Enable auto-update.
- claude.ai: Customize → Plugins → Add → Add marketplace → Add from a repository → `benvial/claude-skills`, then install `skills`.
- ## Updating
- - Own skill: edit it under `plugins/skills/skills/`, commit, push.
- - External skills: a weekly workflow moves every pin to upstream HEAD and opens a PR. Review and merge it. To run it by hand: `python scripts/sync.py --bump`.
- - Add or drop an external skill: edit `vendor.json`, run `python scripts/sync.py`, commit.
- ## External sources
- | Upstream | Skills |
- | --- | --- |
- | [JuliusBrussee/caveman](https://github.com/JuliusBrussee/caveman) (Apache-2.0) | caveman, caveman-commit, caveman-review, caveman-compress |
- | [mattpocock/skills](https://github.com/mattpocock/skills) (MIT) | every skill in the upstream plugin (27), read from its `plugin.json` |
- | [pbakaus/impeccable](https://github.com/pbakaus/impeccable) (Apache-2.0) | impeccable + 5 subagents |
- | [wdm0006/python-skills](https://github.com/wdm0006/python-skills) (MIT) | designing-python-apis |
- | [conorbronsdon/avoid-ai-writing](https://github.com/conorbronsdon/avoid-ai-writing) (MIT) | avoid-ai-writing |
plugins/skills/skills/avoid-ai-writing/detector/patterns.js · Bundled script · 3,762 lines
- /**
- * Avoid AI Writing — detection engine (canonical source of truth)
- * Implements regex, structural, and stylometric pattern detection. This repo's SKILL.md
- * catalogs the human-editable pattern rules; this engine is the executable
- * expression of the regex-detectable subset and extends it with stylometric and
- * AI-tool-fingerprint detectors that don't make sense as skill prose
- * (cross-paragraph burstiness, smart-punct signatures, function-word
- * trigram entropy, low type-token ratio, AI-tool URL parameters,
- * chatbot citation markup leaks, unfilled placeholders).
- *
- * Scoring model:
- * Each category has a weight in the ISSUE_WEIGHTS table. Detection runs
- * produce raw (possibly duplicate) issues which are then deduplicated by
- * (type, text) pair. rawScore is the sum of category weights across the
- * deduped list — so the number reflects the same distinct signals the
- * user sees in the issue list.
- *
- * Weights are deliberately non-flat across severity tags. Cutoff
- * disclaimers (10) and chatbot artifacts (8) weigh more than vague
- * attributions (5), even though all three are tagged `critical`, because
- * the skill treats them as stronger or weaker AI-origin signals.
- *
- * rawScore is then normalized to 0-100 via `log2(wordCount/50)` so longer
- * texts don't accumulate unboundedly on the same density of patterns.
- */
- const AIDetector = (() => {
- // ═══ Tier 1 pre-pass: normalize bypass tricks ══════════════════════
- //
- // Humanizer tools and prompt-injection bypass techniques insert
- // invisible / lookalike chars to defeat exact-string detectors. Strip
- // them BEFORE pattern matching so "delve" with a Cyrillic 'е' still
- // hits the Tier 1 list. Unicode ranges sourced from
- // It-s-AI/llm-detection/detection/attacks/.
- //
- // Tracks what was stripped so the trinary classifier can use
- // "normalization triggered" as a corroborating AI signal — humans don't
- // paste ZWSPs into their own writing.
- const CYRILLIC_LOOKALIKES = {
- 'а': 'a', 'е': 'e', 'о': 'o', 'р': 'p', 'с': 'c', 'х': 'x',
- 'у': 'y', 'к': 'k', 'м': 'm', 'н': 'h', 'в': 'b', 'т': 't',
- 'А': 'A', 'Е': 'E', 'О': 'O', 'Р': 'P', 'С': 'C', 'Х': 'X',
- 'У': 'Y', 'К': 'K', 'М': 'M', 'Н': 'H', 'В': 'B', 'Т': 'T',
- };
- const GREEK_LOOKALIKES = { 'ο': 'o', 'Ο': 'O', 'α': 'a', 'Α': 'A', 'ρ': 'p', 'Ρ': 'P' };
- // ─── Source-coordinate mapping (issue #189) ─────────────────────────
- //
- // Each entry maps one code unit in the working string to the matching
- // code unit in the caller's source. Deletion passes copy the entries for
- // retained characters once, so interleaved or overlapping removals cannot
- // double-count offsets. Masking and homoglyph replacement keep their input
- // length and therefore keep the current map unchanged.
- function identitySourceMap(length) {
- return Array.from({ length }, (_, index) => index);
- }
- function appendMapRange(target, source, start, end) {
- for (let index = start; index < end; index += 1) target.push(source[index]);
- }
- function remapFindingsToSource(issues, regions, sourceMap) {
- for (const issue of issues) {
- if (Number.isInteger(issue.index)) issue.index = sourceMap[issue.index];
- }
- for (const region of regions) {
- region.start = sourceMap[region.start];
- region.end = sourceMap[region.end - 1] + 1;
- }
- }
- const ZERO_WIDTH_RE = /[-]/u;
- const ZERO_WIDTH_GLOBAL_RE = /[-]/gu;
- const HOMOGLYPH_RE = /[Ѐ-ӿͰ-Ͽ]/u;
- const HOMOGLYPH_GLOBAL_RE = /[Ѐ-ӿͰ-Ͽ]/gu;
- const LATIN_LETTER_RE = /\p{Script=Latin}/u;
- // Dates, versions and ASCII filenames keep internal dots; prose may hard-wrap.
- // Other sentence punctuation and Markdown/HTML block starts separate units, even
- // without a space, so unrelated Russian text cannot hide an English word.
- const SENTENCE_UNIT_GLOBAL_RE = /(?:[^.!?\r\n]|(?<=[A-Za-z0-9_])\.(?=[A-Za-z0-9_])|\r?\n(?![ \t]*(?:\r?\n|(?:[-*+]|#{1,6}|\d+[.)])\s|[><|]|`{3}|~{3}|(?:(?:-+|=+)[\t ]*|(?:\*[ \t]*){3,}|(?:_[ \t]*){3,}|(?:-[ \t]*){3,})(?:\r?\n|$)|:?-+:?[\t ]*(?:\|[\t ]*:?-+:?[\t ]*)+(?:\|[\t ]*)?(?:\r?\n|$))))+/g;
- // Words are letter runs; a hyphen splits them, so "API-сервис" stays two
- // words and its Russian half is not swapped.
- const LETTER_RUN_GLOBAL_RE = /[\p{L}\p{M}]+/gu;
- const ROLEPLAY_VERBS_RE = /^(?:nods|sighs|laughs|smiles|frowns|shrugs|grins|winks|chuckles|gasps|pauses|thinks|wonders|whispers|shouts|gestures|raises|leans|turns|looks|glances|smirks|blinks|nodding|sighing|laughing|smiling|thinking|gesturing)\b/i;
- const ROLEPLAY_MARKER_RE = /(?<!\*)\*([^*\n]{1,80}?)\*(?!\*)/gu;
- // A word joiner (U+2060) only counts as a bypass character when it splits a
- // word: letters on both sides, the shape a humanizer uses to break "delve"
- // apart. Show-notes and CMS editors insert word joiners next to URLs and
- // punctuation to control line breaking (#351); those are still stripped so
- // matching sees clean text, but they are typesetting, not AI evidence.
- // Neighbours are read past a short adjacent zero-width run, so a doubled
- // joiner inside a word still counts. The bound keeps a long run linear.
- const LETTER_END_RE = /\p{L}$/u;
- const LETTER_START_RE = /^\p{L}/u;
- const MAX_ZERO_WIDTH_SKIP = 8;
- function countsAsBypass(chars, i) {
- if (chars[i] !== '\u2060') return true;
- let before = i - 1;
- while (before >= 0 && i - before <= MAX_ZERO_WIDTH_SKIP && ZERO_WIDTH_RE.test(chars[before])) before -= 1;
- let after = i + 1;
- while (after < chars.length && after - i <= MAX_ZERO_WIDTH_SKIP && ZERO_WIDTH_RE.test(chars[after])) after += 1;
- // A two-unit slice lets the Unicode regex see a supplementary-plane letter.
- return LETTER_END_RE.test(chars.slice(Math.max(0, before - 1), before + 1))
- && LETTER_START_RE.test(chars.slice(after, after + 2));
- }
- function normalizeText(text, sourceMap) {
- const flags = { zeroWidth: 0, homoglyph: 0, roleplay: 0 };
- let out = text;
- let map = Array.isArray(sourceMap) ? sourceMap : null;
- // 1. Strip zero-width chars (ZWSP U+200B, ZWNJ U+200C, ZWJ U+200D,
- // BOM U+FEFF, word joiner U+2060). Word joiners outside a word are
- // stripped without being counted; see countsAsBypass.
- if (map) {
- const chars = [];
- const nextMap = [];
- for (let i = 0; i < out.length; i += 1) {
- if (ZERO_WIDTH_RE.test(out[i])) {
- if (countsAsBypass(out, i)) flags.zeroWidth += 1;
- continue;
- }
- chars.push(out[i]);
- nextMap.push(map[i]);
- }
- out = chars.join('');
- map = nextMap;
- } else {
- out = out.replace(ZERO_WIDTH_GLOBAL_RE, (_, offset, whole) => {
- if (countsAsBypass(whole, offset)) flags.zeroWidth += 1;
- return '';
- });
- }
- // 2. Swap Cyrillic / Greek Latin-lookalike chars back to Latin so
- // pattern matching catches obfuscated tokens. Swapping every а, е, о
- // reported thousands of "homoglyph swaps" on plain Russian text, and
- // swapping inside ordinary Russian words flagged bilingual technical
- // text, so only two word shapes are swapped:
- // - mixed-script words ("pаypal", "dеlve"), anywhere;
- // - words of two or more letters spelled entirely in lookalike
- // letters ("аст" for "act"), when the sentence around them has no
- // Russian or Greek in it: every Cyrillic and Greek letter in the
- // sentence is a lookalike. Deciding per sentence, not per document,
- // keeps Russian padding from hiding such a word in an English
- // sentence.
- // Sentence punctuation, blank lines and Markdown block starts separate units.
- // Internal ASCII-token dots ("07.10", "v1.2", "report.pdf") and a prose
- // hard wrap do not end it: cutting there left pieces such as "поста в
- // r/StableDiffusion от 07" that looked like Latin text.
- // Ordinary Russian words contain non-lookalike letters (з, п, и, л),
- // so short Russian words in Russian technical prose ("на Enum",
- // "от 07.10") stay put, and one-letter prepositions (с, о, у) are too
- // short.
- // Known limits, because no letter-level rule separates these from
- // real Russian: a fully substituted word inside or next to Russian
- // words ("МЕТА поможет", "пароль: аст"), including across prose hard wraps,
- // is left alone; a one-letter
- // lookalike split off by a hyphen ("а-ct") is left alone; and a short
- // Russian sentence spelled only in lookalike letters ("Он сам.") is
- // swapped.
- const lookalikeFor = (m) => CYRILLIC_LOOKALIKES[m] ?? GREEK_LOOKALIKES[m];
- const swapLookalike = (m) => {
- const swap = lookalikeFor(m);
- if (swap) { flags.homoglyph++; return swap; }
- return m;
- };
- const letterCount = (word) => (word.match(/\p{L}/gu) || []).length;
- const allLookalike = (word) => letterCount(word) >= 2
- && [...word].every((ch) => lookalikeFor(ch) || !/\p{L}/u.test(ch));
- out = out.replace(SENTENCE_UNIT_GLOBAL_RE, (unit) => {
- const scriptLetters = unit.match(HOMOGLYPH_GLOBAL_RE) || [];
- const onlyLookalikes = scriptLetters.length > 0 && scriptLetters.every((ch) => lookalikeFor(ch));
- const words = [...unit.matchAll(LETTER_RUN_GLOBAL_RE)];
- let result = '';
- let last = 0;
- words.forEach((match) => {
- const word = match[0];
- let next = word;
- if (HOMOGLYPH_RE.test(word)) {
- if (LATIN_LETTER_RE.test(word) || (allLookalike(word) && onlyLookalikes)) {
- next = word.replace(HOMOGLYPH_GLOBAL_RE, swapLookalike);
- }
- }
- result += unit.slice(last, match.index) + next;
- last = match.index + word.length;
- });
- return result + unit.slice(last);
- });
- // 3. Strip *roleplay-action* markers — paired *...* containing an
- // action verb (nods, sighs, laughs, smiles, etc.) anchored to
- // the start of the inner phrase. This is the actual chat-model
- // artifact shape. Markdown `**bold**` is rejected by the
- // lookbehind/lookahead; legitimate multi-word `*italic*` is
- // preserved because the verb whitelist is narrow.
- if (map) {
- const chars = [];
- const nextMap = [];
- const matcher = new RegExp(ROLEPLAY_MARKER_RE.source, ROLEPLAY_MARKER_RE.flags);
- let cursor = 0;
- let match;
- while ((match = matcher.exec(out)) !== null) {
- if (!ROLEPLAY_VERBS_RE.test(match[1])) continue;
- chars.push(out.slice(cursor, match.index));
- appendMapRange(nextMap, map, cursor, match.index);
- flags.roleplay += 1;
- cursor = match.index + match[0].length;
- }
- chars.push(out.slice(cursor));
- appendMapRange(nextMap, map, cursor, map.length);
- out = chars.join('');
- map = nextMap;
- } else {
- out = out.replace(ROLEPLAY_MARKER_RE, (m, inner) => {
- if (ROLEPLAY_VERBS_RE.test(inner)) {
- flags.roleplay += 1;
- return '';
- }
- return m;
- });
- }
- return map ? { text: out, flags, sourceMap: map } : { text: out, flags };
- }
- // Terms with legitimate technical meaning that are suppressed when contextMode === 'technical'.
- // See references/patterns.md and issue #237.
- const TECHNICAL_EXEMPT = new Set([
- 'robust',
- 'comprehensive',
- 'seamless',
- 'seamlessly',
- 'ecosystem',
- 'leverage',
- 'leverages',
- 'leveraging',
- 'leveraged',
- 'facilitate',
- 'facilitates',
- 'underpin',
- 'underpinning',
- 'underpinnings',
- 'streamline',
- ]);
- // ─── Tier 1: Always flag ───────────────────────────────────────────
- const TIER1 = {
- 'delve': 'explore, dig into, look at',
- 'tapestry': 'complex structure, rich history',
- 'paradigm': 'model, approach, framework',
- 'beacon': 'rewrite entirely',
- 'robust': 'strong, reliable, solid',
- 'comprehensive': 'thorough, complete, full',
- 'cutting-edge': 'latest, newest, advanced',
- 'pivotal': 'important, key, critical',
- 'meticulous': 'careful, detailed, precise',
- 'meticulously': 'carefully, precisely',
- 'seamless': 'smooth, easy, without friction',
- 'seamlessly': 'smoothly, easily',
- 'game-changer': 'changes how [X] works',
- 'game-changing': 'changes how [X] works',
- 'nestled': 'is located, sits',
- 'vibrant': 'lively, active',
- 'thriving': 'growing, active',
- 'bustling': 'busy, active',
- 'intricate': 'complex, detailed',
- 'intricacies': 'complexities, details',
- 'ever-evolving': 'changing, growing',
- 'enduring': 'lasting, long-running',
- 'daunting': 'hard, difficult',
- 'holistic': 'complete, full, whole',
- 'holistically': 'completely, fully',
- 'actionable': 'practical, useful, concrete',
- 'impactful': 'effective, significant',
- 'learnings': 'lessons, findings, takeaways',
- 'synergy': 'combined effect, benefit of working together',
- 'synergies': 'combined effect, benefit of working together',
- 'interplay': 'relationship, connection',
- 'symphony': 'coordination, balance',
- 'embrace': 'adopt, accept, use',
- };
- // Multi-word tier 1 phrases
- // "features" is copula avoidance only as a verb: "the app features a
- // dashboard". The plural noun ("three new features", a "Features" heading,
- // "the features we shipped") is ordinary product and technical writing, and
- // it is most of what the word does in corpus/ (#351). A verb needs a subject
- // before it and an object after it, so the word counts as a noun when it
- // opens a sentence, line or list item; follows a determiner, number,
- // possessive, quantifier, "of", "or" or a compound modifier; or comes
- // before punctuation, a line break, a preposition, a conjunction, a pronoun
- // or a verb.
- // Singular "this", "that" and "each" cannot determine the plural noun:
- // here they are subjects ("each features a...") or a relative pronoun.
- const FEATURES_NOUN_BEFORE_RE = /(?:^|[.!?:;(]\s*|(?:^|\n)[ \t]*(?:[-*+>#]+[ \t]*)?|\b(?:the|a|an|these|those|its|their|our|your|my|his|her|of|or|new|key|main|core|top|other|more|most|many|several|some|all|any|few|no|both|only|about|over|just|free|similar|common|specific|distinct|additional|extra|premium|advanced|basic|best|unique|missing|upcoming|existing|latest|major|minor|useful|important|various|different|certain|such|\d+|two|three|four|five|six|seven|eight|nine|ten|whose)\s+|\w['’]s\s+|\w[-/]\w+\s+)$/i;
- // "on-device" is an object modifier, not the preposition "on".
- const FEATURES_NOUN_AFTER_RE = /^(?:[ \t]*(?:[.,;:!?()[\]"'’”\n]|$)|\s+(?:of|for|in|on|at|into|and|or|but|nor|are|were|is|was|be|been|being|that|which|who|whose|like|such|to|from|with|without|than|help|helps|helped|let|allow|make|keep|give|provide|enable|offer|save|protect|ensure|improve|reduce|support|remain|vary|need|unlock|bring|include|includes|included|work|works|worked|will|can|could|should|would|may|might|must|do|did|does|have|has|had|we|you|they|i|it)(?![\w-]))/i;
- // Bare product objects: "we ship features", "teams build features".
- // A subject is required so noun subjects such as "the ship"/"the build"
- // still flag. One optional modifier preserves "we ship GPT-5 features".
- const FEATURES_NOUN_OBJECT_RE = /\b(?:i|we|you|they|teams?|developers?|engineers?|users?|companies|vendors|the\s+(?:release|update))\s+(?:(?:can|could|will|would|should|must)\s+)?(?:ship|ships|shipped|shipping|build|builds|built|building|add|adds|added|adding)\s+(?:[\w./-]+\s+)?$/i;
- // Leads that make "which/what features" an indirect question (#384): a question
- // verb that opens its clause ("Decide which...", "we should check which..."),
- // or which/what at the start of a sentence or list item. A lead word inside a
- // noun phrase ("a security check which features...") does not count.
- const FEATURES_QUESTION_LEAD_RE = /(?:^|[.!?:;]\s*|(?:^|\n)[ \t]*(?:[-*+>#]+[ \t]*)?|(?:^|[.!?:;,]\s*|(?:^|\n)[ \t]*(?:[-*+>#]+[ \t]*)?|\b(?:i|we|you|they|he|she|let['’]?s|to|can|could|should|must|will|would|please|just|then|first|now|and|or|but|so)\s+)(?:decide|decides|deciding|choose|choosing|pick|check|checking|see|know|knowing|learn|find\s+out|figure\s+out|work\s+out|identify|determine|ask|asking|asked|tell\s+(?:me|us|them|you)|show\s+(?:me|us|them|you)|understand|explain|wonder|discover|confirm|consider|evaluate|prioritize|about)\s+)(?:which|what)\s+$/i;
- // A lead word after a determiner is a noun ("a track which features...",
- // "the review which features..."), not a question verb.
- const FEATURES_LEAD_AS_NOUN_RE = /\b(?:a|an|the|this|that|these|those|my|our|your|their|its|his|her|each|every)\s+[\w-]+\s+(?:which|what)\s+$/i;
- // Subjects that definitively make "features" a verb, avoiding ambiguity with
- // plural-noun subjects like "The experimental features" or "Security features".
- const FEATURES_VERB_SUBJECTS = /\b(?:library|tool|app|application|platform|system|language|service|product|update|release|version|site|website|game|device|model|framework|package|plugin|extension|which|what|that|who|it|he|she)\s+$/i;
- function featuresIsNoun(text, index) {
- const end = index + 'features'.length;
- const before = text.slice(Math.max(0, index - 40), index);
- const after = text.slice(end, end + 40);
- if (FEATURES_NOUN_OBJECT_RE.test(before)) return true;
- // A recognizable model/version followed by an object article is a
- // subject: "GPT-5 features a...", "Windows 11 features a...".
- // Determiner-led counts ("These 3 features a customer requested") and
- // the explicit object contexts above remain nouns. This is a bounded
- // context heuristic, not a grammatical parse of every use of the word.
- const product = /\b([A-Z][\w]*)(?:[-/]\d[\w.-]*|[ \t]+\d+(?:\.\d+)*)[ \t]+$/.exec(before);
- if (product && !FEATURES_NOUN_BEFORE_RE.test(`${product[1]} `)
- && /^\s+(?:a|an|the)(?![\w-])/i.test(after)) return false;
- // A short comma-delimited aside can precede the verb's article-led
- // object. Strong noun contexts still take priority over punctuation.
- if (!FEATURES_NOUN_BEFORE_RE.test(before)
- && /^[ \t]*,[^,\n]{1,80},\s*(?:a|an|the)(?![\w-])/i.test(text.slice(end, end + 110))) return false;
- // "which/what features" is a noun in an indirect question: right after a
- // question verb ("Decide which features matter") or at the start of a
- // sentence ("Which features matter most..."). Every other context falls
- // through to the checks below unchanged, so relative clauses ("a library
- // which features dashboards") keep their verb finding (#384).
- if (FEATURES_QUESTION_LEAD_RE.test(before) && !FEATURES_LEAD_AS_NOUN_RE.test(before)) {
- // If there is no trailing verb (e.g. end of clause), it's the verb case ("Check what features a dashboard and export tools.").
- // Look past a compound subject and optional appositive for a trailing verb before returning the verb classification.
- const match = /^\s+(?:a|an|the)\s+[\w-]+\s*(?:(?:and\b|or\b)\s+(?:a\s+|an\s+|the\s+)?(?:[\w-]+\s+){0,2}[\w-]+\s*)?(?:([,.!?;])|$)/i.exec(after);
- if (match) {
- const predicateRe = /\b(?:can|could|should|will|would|may|might|must|is|are|was|were|has|have|had|do|does|did|include|includes|disable|disables|reject|accept|allow|deny|provide|require|use|make|work|help|give|take|need|become|seem|look|show|offer|support|supports|fail|pass|lack|prefer|choose|prevent|stop)\b/i;
- // If the matched subject ends with a predicate right before the punctuation,
- // the predicate was consumed as part of the compound subject.
- const subjectText = match[0].slice(0, match[1] ? -match[1].length : undefined).trim();
- const lastWord = subjectText.split(/\s+/).pop();
- if (predicateRe.test(lastWord)) {
- return true;
- }
- if (match[1] === ',') {
- // If the comma leads into a phrase, limit noun classification to a predicate
- // belonging to the article-led subject (after the next comma), excluding predicates inside subordinate clauses.
- const remainder = text.slice(end + match[0].length, end + 120);
- // An adverb can sit between the closing comma and the predicate.
- const remainderPredicateRe = new RegExp(`^[^,]+,\\s*(?:(?:[\\w-]+ly|also|not|never)\\s+){0,2}${predicateRe.source}`, 'i');
- if (remainderPredicateRe.test(remainder)) {
- return true;
- }
- }
- return false;
- }
- return true;
- }
- // "The library features support for..." is a verb.
- // "Security features support for..." is a plural noun subject + verb.
- if (FEATURES_VERB_SUBJECTS.test(before) && /^\s+support\s+for\b/i.test(after)) {
- // A nearby quantifier in an earlier clause cannot make a singular pronoun plural.
- if (/\b(?:it|he|she)\s+$/i.test(before)) return false;
- const hasPluralDeterminer = /\b(?:these|those|all|some|many|few|various|multiple|several|both)\s+(?:[\w-]+\s+){1,2}$/i.test(before);
- // Count words modify "features" only when they follow its determiner.
- // A digit inside a product name ("This Windows 11 app") or a date before
- // the subject ("In 2024 the library") is not a plural count.
- const hasPluralCount = /\b(?:the|a|an|this|that|my|our|your|their|its|his|her|each|every)\s+(?:(?:\d{1,2}|two|three|four|five|six|seven|eight|nine|ten|eleven|twelve|thirteen|fourteen|fifteen|sixteen|seventeen|eighteen|nineteen|twenty|dozen)\s+(?:[\w-]+\s+){1,2}|(?!(?:19|20)\d{2}\s)\d{3,}\s+(?!(?:[\w-]+\s+)?(?:app|release|model|version|update)\s+$)(?:[\w-]+\s+){1,2})$/i.test(before);
- const extendedAfter = text.slice(end, end + 100);
- // A predicate inside a relative clause modifying the object (e.g. "teams that connect and remain online")
- // is not coordinated with the main subject. Limit predicate search to the direct prepositional object.
- const hasPartitivePlural = /\b(?:all|some|many|most|few|several|both|\d+|two|three|four|five|six|seven|eight|nine|ten)\s+of\s+(?:the|my|our|your|their|its|his|her)\s+(?:[\w-]+\s+){1,3}$/i.test(before);
- const hasPluralPredicate = /^\s+support\s+for(?:(?!\b(?:that|which|who|whom|whose|where|when)\b)[^.?!;]){0,80}?\b(?:and|but|or|nor)\s+(?:are|were|have|do|reject|accept|allow|deny|provide|require|use|make|work|help|give|take|need|become|seem|look|show|include|offer|support|remain|fail|pass|lack|prefer|choose|prevent|stop)\b/i.test(extendedAfter);
- const hasForObject = /^\s+support\s+for(?:-|\s+)(?:profit|loops?)\b/i.test(after);
- // Numeric modifiers and plural cues exclude a singular-subject reading:
- // "The two platform features..." has plural noun "features".
- // Do not borrow a determiner across a preposition or conjunction:
- // "a suite of system features" still has plural noun "features".
- // "that" can introduce a clause and "which" can determine plural nouns.
- // A relative pronoun needs its own determiner-led subject evidence.
- const hasUnambiguousSingular = /\b(?:a|an|this|that|every|each|one)\s+(?:(?!(?:of|for|in|with|and|or|which|what|that|who|two|three|four|five|six|seven|eight|nine|ten|multiple|several|many|few|both|various)\b)[\w-]+\s+){1,3}$/i.test(before)
- || /\b(?:it|he|she)\s+$/i.test(before)
- || /\b(?:a|an|this|that|each|every|one)\s+[\w-]+\s+(?:which|that|who)\s+$/i.test(before);
- const hasSingularEvidence = hasUnambiguousSingular
- || /\b(?:the|my|our|your|their|its|his|her)\s+(?:(?!(?:of|for|in|with|and|or|which|what|that|who|two|three|four|five|six|seven|eight|nine|ten|multiple|several|many|few|both|various)\b)[\w-]+\s+){1,3}$/i.test(before)
- || /\b(?:the|my|our|your|their|its|his|her)\s+[\w-]+\s+(?:which|that|who)\s+$/i.test(before);
- // Unambiguous singular subjects ("It", "This app") cannot be plural-noun phrases,
- // so hasForObject ("support for loops") only suppresses ambiguous determiners ("The library").
- if (!hasPluralDeterminer && !hasPluralCount && !hasPartitivePlural && !hasPluralPredicate && (hasUnambiguousSingular || (!hasForObject && hasSingularEvidence))) {
- return false;
- }
- }
- return FEATURES_NOUN_BEFORE_RE.test(before) || FEATURES_NOUN_AFTER_RE.test(after);
- }
- const TIER1_PHRASES = [
- { pattern: /\bdelve\s+into\b/gi, replace: 'explore, dig into' },
- { pattern: /\blandscape\b/gi, replace: 'field, space, industry', filter: true },
- { pattern: /\brealm\b/gi, replace: 'area, field, domain' },
- { pattern: /\btestament\s+to\b/gi, replace: 'shows, proves' },
- { pattern: /\bleverag(?:e|es|ing|ed)\b/gi, replace: 'use' },
- { pattern: /\bwatershed\s+moment\b/gi, replace: 'turning point, shift' },
- { pattern: /\bmarking\s+a\s+pivotal\s+moment\b/gi, replace: 'changing [X], leading to [Y]' },
- { pattern: /\bthe\s+future\s+looks\s+bright\b/gi, replace: 'cut or say something specific' },
- { pattern: /\bonly\s+time\s+will\s+tell\b/gi, replace: 'cut or say something specific' },
- { pattern: /\bdespite\s+challenges[^.]*continues?\s+to\s+thrive\b/gi, replace: 'keeps growing despite [named difficulty], continues to succeed despite [named difficulty]' },
- { pattern: /\bdeep\s+dive\b/gi, replace: 'look at, examine' },
- { pattern: /\bdive\s+into\b/gi, replace: 'look at, examine' },
- { pattern: /\bunpack(?:ing)?\b/gi, replace: 'explain, break down' },
- { pattern: /\bcomplexities\b/gi, replace: 'problems, details' },
- { pattern: /\bthought\s+leader(?:ship)?\b/gi, replace: 'expert, authority' },
- { pattern: /\bbest\s+practices\b/gi, replace: 'what works, proven methods' },
- { pattern: /\bat\s+its\s+core\b/gi, replace: 'cut, just state it' },
- { pattern: /\bin\s+order\s+to\b/gi, replace: 'to', clarity: true },
- { pattern: /\bdue\s+to\s+the\s+fact\s+that\b/gi, replace: 'because', clarity: true },
- { pattern: /\bserves\s+as\b/gi, replace: 'is', clarity: true },
- { pattern: /\bfeatures\b/gi, replace: 'has, includes', filter: true, clarity: true, skip: featuresIsNoun },
- { pattern: /\bboasts\b/gi, replace: 'has', clarity: true },
- { pattern: /\butiliz(?:e|es|ing|ed)\b/gi, replace: 'use', clarity: true },
- { pattern: /\bshowcas(?:e|es|ing|ed)\b/gi, replace: 'show, demonstrate' },
- { pattern: /\bembark(?:s|ing|ed)?\b/gi, replace: 'start, begin' },
- { pattern: /\bcommenc(?:e|es|ing|ed)\b/gi, replace: 'start, begin', clarity: true },
- { pattern: /\bascertain(?:s|ing|ed)?\b/gi, replace: 'find out, determine', clarity: true },
- { pattern: /\bendeavou?r(?:s|ing|ed)?\b/gi, replace: 'effort, attempt, try', clarity: true },
- { pattern: /\bunderscor(?:es|ing|ed)\b/gi, replace: 'highlights, shows' },
- // Hyphen required. The unhyphenated "load bearing" is ordinary English —
- // "the load bearing down on the bridge" — where `bearing` is a participle,
- // not part of a compound modifier. The tell is always hyphenated.
- //
- // Only match an immediately following abstract noun from this seed list.
- // Unknown nouns, mixed physical/abstract nouns, and predicative uses pass:
- // precision over recall (#56). Keep the lookahead out of the matched span.
- { pattern: /\bload-bearing\b(?=[ \t]+(?:assumptions?|claims?|invariants?|premises?|constraints?|dependenc(?:y|ies)|arguments?|abstractions?)\b)/gi, replace: 'essential, critical, or say what breaks if you remove it' },
- ];
- // ─── Tier 2: Flag in clusters (2+ per paragraph) ──────────────────
- const TIER2 = {
- 'harness': 'use, take advantage of',
- 'navigate': 'work through, handle',
- 'navigating': 'working through, handling',
- 'foster': 'encourage, support, build',
- 'elevate': 'improve, raise, strengthen',
- 'unleash': 'release, enable, unlock',
- 'streamline': 'simplify, speed up',
- 'empower': 'enable, let, allow',
- 'bolster': 'support, strengthen',
- 'spearhead': 'lead, drive, run',
- 'resonate': 'connect with, appeal to',
- 'resonates': 'connects with, appeals to',
- 'revolutionize': 'change, transform',
- 'facilitate': 'enable, help, allow',
- 'facilitates': 'enables, helps, allows',
- 'underpin': 'support, form the basis of',
- 'nuanced': 'specific, subtle, detailed',
- 'crucial': 'important, key, necessary',
- 'multifaceted': 'having several parts, with several aspects',
- 'ecosystem': 'system, community, network',
- 'myriad': 'many, numerous',
- 'plethora': 'many, a lot of',
- 'encompass': 'include, cover, span',
- 'catalyze': 'start, trigger, accelerate',
- 'reimagine': 'rethink, redesign, rebuild',
- 'galvanize': 'motivate, rally, push',
- 'augment': 'add to, expand, supplement',
- 'cultivate': 'build, develop, grow',
- 'illuminate': 'clarify, explain, show',
- 'elucidate': 'explain, clarify',
- 'juxtapose': 'compare, contrast',
- 'transformative': 'changes how [X] works',
- 'transformation': 'major change, overhaul',
- 'cornerstone': 'foundation, basis, key part',
- 'paramount': 'most important, top priority',
- 'poised': 'ready, set, about to',
- 'burgeoning': 'growing, emerging',
- 'nascent': 'new, early-stage',
- 'quintessential': 'typical, classic, defining',
- 'overarching': 'main, central, broad',
- 'quietly': 'cut, or name the concrete contrast',
- 'underpinning': 'basis, foundation',
- 'underpinnings': 'basis, foundations',
- 'paradigm-shifting': 'changes the basic model, changes how [X] is understood',
- };
- // Conditional Tier 2 entries: everyday words whose AI tell is a specific
- // significance collocation, not the word itself. They join a paragraph's
- // Tier 2 cluster only when the collocation matches — bare uses ("deeply
- // nested JSON", "cares deeply") never count, because the base rate of
- // these words in innocent prose is far higher than the rest of the table
- // and an unconditional entry measurably flags clean human writing.
- const TIER2_CONDITIONAL = [
- {
- word: 'deeply',
- pattern: /\bdeeply\s+(?:integrated|committed|rooted|personal|human|flawed|resonant|transformative|interconnected|ingrained|embedded|meaningful)\b/i,
- suggestion: 'cut, or name what specifically runs deep',
- },
- ];
- // ─── Tier 3: Flag by density ───────────────────────────────────────
- const TIER3 = [
- 'significant', 'significantly', 'innovative', 'innovation',
- 'effective', 'effectively', 'dynamic', 'dynamics',
- 'scalable', 'scalability', 'compelling', 'unprecedented',
- 'exceptional', 'exceptionally', 'remarkable', 'remarkably',
- 'sophisticated', 'instrumental',
- 'world-class', 'state-of-the-art', 'best-in-class',
- // `verbatim` is usually redundant with the verb it modifies ("copies X
- // verbatim" = "copies X"). It has a genuine term-of-art sense in legal,
- // research, and QA registers ("verbatim transcript"), so it lives at Tier 3:
- // density-gated, it only fires on overuse, not on a single legitimate use.
- 'verbatim',
- ];
- // Multi-word Tier 3 phrases. Density-gated like single Tier 3 words because
- // these legitimately show up in human crypto/web3/dev writing. Threshold is
- // intentionally lower than single-word Tier 3 (≥2 occurrences of the same
- // phrase) — repeating the same multi-word boilerplate is a stronger AI tell
- // than re-using "significant."
- const TIER3_PHRASES = [
- /\bemerging\s+(?:sector|space|category|industry)\b/gi,
- /\bthe\s+integration\s+of\b/gi,
- /\bthe\s+intersection\s+of\b/gi,
- /\bcommunity-?driven\b/gi,
- /\blong-?term\s+sustainability\b/gi,
- /\buser\s+engagement\b/gi,
- /\bdecentralized\s+compute\b/gi,
- /\b(?:sustainable\s+)?reward\s+emissions?\b/gi,
- /\btokenized\s+incentive\s+structures?\b/gi,
- /\bdesigned\s+for\s+long-?term\b/gi,
- ];
- // O(1) lookup from any token form (hyphenated or dashless) to its canonical
- // Tier 3 word. Counting originally did nested-loop word matching which was
- // O(tokens × TIER3) — slow on long pastes.
- const TIER3_LOOKUP = new Map();
- for (const word of TIER3) {
- TIER3_LOOKUP.set(word, word);
- const dashless = word.replace(/-/g, '');
- if (dashless !== word) TIER3_LOOKUP.set(dashless, word);
- }
- // Per-category score weights. Applied to distinct (deduplicated) issues so
- // the score reflects the same signals the user sees in the issue list.
- // Non-uniform on purpose: critical rules like cutoff disclaimers (×10) and
- // chatbot artifacts (×8) weigh more than vague attributions (×5), even
- // though all three are tagged `critical`.
- const ISSUE_WEIGHTS = {
- tier1: 5,
- // Wordiness, not an AI-frequency marker. Weighted like tier2 so a
- // clarity fix cannot push a document toward an AI classification.
- 'tier1-clarity': 3,
- tier2: 3,
- tier3: 2,
- transition: 2,
- chatbot: 8,
- sycophantic: 8,
- filler: 2,
- 'generic-conclusion': 3,
- 'lets-construction': 2,
- 'reasoning-artifact': 6,
- 'significance-inflation': 4,
- 'vague-attribution': 5,
- 'hollow-intensifier': 2,
- // Issue #82 evidence boundary: this style pattern produced no detector hits
- // in either corpus class, so it has no measured authorship direction. Keep
- // the finding visible, but do not move authorship scores or probabilities
- // until a relevant positive evaluation set supports a direction.
- 'emotional-flatline': 0,
- 'lingering-attention': 3,
- 'novelty-inflation': 3,
- 'cutoff-disclaimer': 10,
- 'template-phrase': 3,
- 'false-concession': 2,
- 'rhetorical-question': 2,
- 'confidence-calibration': 2,
- // Writing-quality guidance whose authorship polarity changes across model
- // generations. Keep the flag visible without moving authorship outputs.
- 'em-dash': 0,
- uniformity: 5,
- formatting: 3,
- 'tier3-phrase': 3,
- // Structural / cluster signals are deliberately weighted high. Unlike
- // single-vocabulary hits they're near-dispositive on social-length
- // posts (a 15-hashtag block, a 6-item bullet-NP list, three distinct
- // crypto-shill phrases stacked) and would otherwise be suppressed by
- // the log2(words/50) length divisor on short pastes.
- 'tier3-phrase-cluster': 12,
- 'hashtag-stuff': 12,
- 'bullet-np-list': 10,
- 'hedge-stack': 6,
- 'future-narrative': 12,
- 'real-actual-inflation': 5,
- // Social endorsement / CTA closer. Weighted like formulaic-opener: a
- // strong single-hit social tell that the length divisor would
- // otherwise wash out on a short LinkedIn-length post.
- 'social-cta-closer': 8,
- // Performed-insight tics: single hits are common in human essays, so
- // weighted like tier2 vocabulary — density does the classifying.
- 'performed-insight': 3,
- // Negation chains are a strong single-hit structural tell.
- 'negation-chain': 5,
- // Negative parallelism ("It's not just X, it's Y"). The frame is also
- // how people state a real correction, so it is weighted below the
- // negation chain; the per-piece gate in analyzeText does the rest.
- 'negative-parallelism': 4,
- 'dev-blog-boilerplate': 3,
- 'formulaic-opener': 8,
- // Speculative scenario opener ("Imagine a world where…"). Weighted like
- // formulaic-opener: a single strong opener tell the length divisor would
- // otherwise wash out on a short post.
- 'speculative-opener': 8,
- // Launch-copy introduction ("Enter X.", "Meet X, your new...").
- // Weighted like the other single-hit opener tells: strong on the
- // short launch posts where it actually appears.
- 'launch-intro': 8,
- // Dramatized crowd contrast. Gated hard on the dismissive verb, so
- // a hit is meaningful, but the surface shares words with ordinary
- // narrative — weighted below the opener tells on purpose.
- 'crowd-contrast': 6,
- // Fake-casual props (stage directions, wink asides). Near-costume
- // when present; same class as the opener tells on short posts.
- 'fake-casual-prop': 8,
- 'title-case-header': 4,
- 'parenthetical-hedge': 3,
- // House-style typography is a style review prompt, not authorship
- // evidence. Keep it neutral in both the score and classifier (#351).
- 'smart-punct-signature': 0,
- 'punct-distribution': 6,
- 'fnword-trigram-entropy': 5,
- 'cross-para-burstiness': 5,
- 'normalization-flag': 9,
- // Vocabulary-diversity signal (type-token ratio). Weighted modestly
- // because the threshold (>=200 tokens AND TTR<0.4) is conservative;
- // it stacks with structural signals to push borderline scores up.
- 'low-ttr': 3,
- // AI-tool fingerprints. Weighted higher than statistical patterns
- // because each is a near-definitive single-hit signal — the AI tool
- // literally left its mark in the text. citation-markup ranks highest
- // (smoking gun: the literal internal markup of ChatGPT/Grok/etc.),
- // UTM tracking second (auto-appended by the tool to URLs it writes),
- // placeholders third (strong but humans use bracketed slots in
- // templates legitimately and forget them — still a publishing bug
- // but slightly less definitive AI evidence).
- 'ai-placeholder': 10,
- 'ai-citation-markup': 15,
- 'ai-utm-source': 12,
- // Curated P2 copyedits, not authorship evidence. Keep them visible in the
- // issue list without moving the AI score, label, probabilities, or
- // trinary classification on short documents.
- 'unnecessary-hyphenation': 0,
- };
- // ─── Transition phrases ────────────────────────────────────────────
- const TRANSITIONS = [
- /\bmoreover\b/gi,
- /\bfurthermore\b/gi,
- /\badditionally\b/gi,
- /\bin\s+today'?s\b/gi,
- /\bin\s+an\s+era\s+where\b/gi,
- /\bit'?s\s+worth\s+noting\s+that\b/gi,
- /\bnotably\b/gi,
- /\bin\s+conclusion\b/gi,
- /\bin\s+summary\b/gi,
- /\bto\s+summarize\b/gi,
- /\bwhen\s+it\s+comes\s+to\b/gi,
- /\bat\s+the\s+end\s+of\s+the\s+day\b/gi,
- /\bthat\s+(?:being\s+)?said\b/gi,
- ];
- // ─── Chatbot artifacts ─────────────────────────────────────────────
- const CHATBOT_ARTIFACTS = [
- /\bi\s+hope\s+this\s+helps\b/gi,
- /\bcertainly!\b/gi,
- /\babsolutely!\b/gi,
- /\bgreat\s+question!\b/gi,
- /\bexcellent\s+point!\b/gi,
- /\bfeel\s+free\s+to\s+reach\s+out\b/gi,
- /\blet\s+me\s+know\s+if\s+you\s+need\s+anything\b/gi,
- /\bin\s+this\s+article,?\s+we\s+will\s+explore\b/gi,
- /\blet'?s\s+dive\s+in!?\b/gi,
- ];
- // ─── Sycophantic tone ──────────────────────────────────────────────
- const SYCOPHANTIC = [
- /\byou'?re\s+absolutely\s+right\b/gi,
- /\bthat'?s\s+a\s+really\s+insightful\b/gi,
- /\bthat'?s\s+a\s+great\s+question\b/gi,
- /\bexcellent\s+question\b/gi,
- ];
- // ─── Filler phrases ────────────────────────────────────────────────
- const FILLERS = [
- /\bit\s+is\s+important\s+to\s+note\s+that\b/gi,
- /\bin\s+terms\s+of\b/gi,
- /\bthe\s+reality\s+is\s+that\b/gi,
- /\bit'?s\s+important\s+to\s+note\s+that\b/gi,
- ];
- // ─── Generic conclusions ───────────────────────────────────────────
- const GENERIC_CONCLUSIONS = [
- /\bthe\s+future\s+looks\s+bright\b/gi,
- /\bonly\s+time\s+will\s+tell\b/gi,
- /\bone\s+thing\s+is\s+certain\b/gi,
- /\bas\s+we\s+move\s+forward\b/gi,
- ];
- // ─── "Let's" constructions ─────────────────────────────────────────
- const LETS_PATTERNS = [
- /\blet'?s\s+explore\b/gi,
- /\blet'?s\s+take\s+a\s+look\b/gi,
- /\blet'?s\s+break\s+this\s+down\b/gi,
- /\blet'?s\s+examine\b/gi,
- /\blet'?s\s+(?:consider|discuss|delve|unpack|walk\s+through)\b/gi,
- ];
- // ─── Reasoning chain artifacts ─────────────────────────────────────
- const REASONING_ARTIFACTS = [
- /\blet\s+me\s+think\s+step\s+by\s+step\b/gi,
- /\bbreaking\s+this\s+down\b/gi,
- /\bto\s+approach\s+this\s+systematically\b/gi,
- /\bhere'?s\s+my\s+thought\s+process\b/gi,
- /\bfirst,?\s+let'?s\s+consider\b/gi,
- /\bworking\s+through\s+this\s+logically\b/gi,
- ];
- // NOTE: Acknowledgment loops are judgment-only (#239). The three phrases the
- // detector matched ("you're asking about", "the question of whether", "to answer
- // your question") are also how people open an ordinary reply and standard
- // analytical English. The tell is a restatement that adds nothing, which a
- // regex cannot see. See detector/CATEGORIES.md §C.
- // ─── Significance inflation ────────────────────────────────────────
- const SIGNIFICANCE_INFLATION = [
- /\bmarking\s+a\s+(?:pivotal|significant|important)\s+moment\b/gi,
- /\ba\s+watershed\s+moment\s+for\b/gi,
- // Require an inflating word before "in the evolution of" (the bare phrase
- // is allowed for literal uses like "a key stage in the evolution of the
- // vertebrate eye"). See issue #212.
- /\b(?:moment|milestone|chapter|era|turning\s+point|watershed|inflection\s+point|leap|role|new\s+phase)\s+in\s+the\s+evolution\s+of\b/gi,
- /\ba\s+(?:pivotal|defining)\s+moment\s+in\b/gi,
- ];
- // ─── Vague attributions ────────────────────────────────────────────
- const VAGUE_ATTRIBUTIONS = [
- /\bexperts\s+(?:believe|say|suggest|agree)\b/gi,
- /\bstudies\s+(?:show|suggest|indicate)\b/gi,
- /\bresearch\s+(?:shows|suggests|indicates)\b/gi,
- /\bindustry\s+leaders\s+(?:agree|believe|say)\b/gi,
- ];
- // ─── Hollow intensifiers ──────────────────────────────────────────
- const HOLLOW_INTENSIFIERS = [
- /\bgenuine(?:ly)?\b/gi,
- /\btruly\b/gi,
- /\bquite\s+frankly\b/gi,
- /\bto\s+be\s+honest\b/gi,
- /\blet'?s\s+be\s+clear\b/gi,
- ];
- // ─── Emotional flatline ────────────────────────────────────────────
- // The "interesting (part|thing|aspect|piece)" family is matched in two
- // shapes: (1) "the most interesting X" inline (the canonical AI list
- // intro), and (2) bare "Interesting X:" used as a section-header opener,
- // which is the section-break variant that slipped past v3.3.x.
- const EMOTIONAL_FLATLINE = [
- /\bwhat\s+surprised\s+me\s+most\b/gi,
- /\bi\s+was\s+fascinated\s+to\b/gi,
- /\bwhat\s+struck\s+me\s+was\b/gi,
- /\bi\s+was\s+excited\s+to\s+learn\b/gi,
- /\bthe\s+most\s+interesting\s+(?:part|thing|aspect|piece)\b/gi,
- // Multiline flag (/m) so `^` matches at every line start, including
- // position 0 of a pasted text that has no leading newline. The earlier
- // `(?:^|\n)` form silently missed bare openers at the very start of
- // input — caught by silent-failure audit 2026-05-16. Leading whitespace
- // is `[ \t]*`, not `\s*`: with /m every line start is a match attempt,
- // and a `\s*` that can cross newlines rescans the whole blank run from
- // each of them, which made a long masked block quadratic (#235).
- /^[ \t]*interesting\s+(?:part|thing|aspect|piece)(?:\s+of\s+(?:the\s+)?\w+)?\s*:/gim,
- ];
- // ─── Lingering-attention claims ────────────────────────────────────
- // The share-post opener that claims duration of attention instead of
- // saying anything about the thing ("the line I keep coming back to").
- //
- // Precision note: the bare verb phrase "I keep coming back to X" is NOT
- // matched on its own, because it is legitimate whenever a reason follows
- // ("I keep coming back to Hirschman because it predicts who quits"), and
- // the reason clause is not reliably detectable by regex. Only the
- // noun-anchored frame ("the line/quote/bit ... I keep coming back to")
- // fires, which is the shape that introduces a subject rather than
- // asserting something about it. The bare form stays a skill-prose
- // judgment call. See SKILL.md §Lingering-attention claims carve-out.
- const LINGERING_ATTENTION = [
- // "the line I keep coming back to", "the one quote that I keep coming back to"
- // Noun-anchored frame only. "can't stop thinking about" is deliberately
- // left to its own pattern below rather than folded into this alternation,
- // so "the line I can't stop thinking about" scores once, not twice.
- /\b(?:the|that|this)\s+(?:one\s+)?(?:line|quote|bit|part|idea|point|framing|comment|thing)\s+(?:that\s+)?i\s+keep\s+(?:coming\s+back\s+to|thinking\s+about)\b/gi,
- /\bi\s+can'?t\s+stop\s+thinking\s+about\b/gi,
- /\bstill\s+thinking\s+about\s+(?:this|that)\s+one\b/gi,
- /\b(?:been|be)\s+rattling\s+around\s+(?:in\s+)?my\s+(?:head|brain)\b/gi,
- /\bi'?ve\s+been\s+chewing\s+on\s+(?:this|that)\b/gi,
- ];
- // ─── Novelty inflation ─────────────────────────────────────────────
- const NOVELTY_INFLATION = [
- /\bthe\s+failure\s+mode\s+nobody'?s?\s+naming\b/gi,
- /\ba\s+problem\s+nobody\s+talks\s+about\b/gi,
- /\bthe\s+insight\s+everyone'?s?\s+missing\b/gi,
- /\bwhat\s+nobody\s+tells\s+you\b/gi,
- ];
- // ─── Cutoff disclaimers ────────────────────────────────────────────
- // Includes the canonical LLM self-identification phrases — these are
- // near-dispositive on their own (real humans don't write "as an AI
- // language model" in first person). Patterns cover the major model
- // families' default disclaimer language.
- const CUTOFF_DISCLAIMERS = [
- /\bas\s+of\s+my\s+last\s+update\b/gi,
- /\bas\s+of\s+my\s+(?:knowledge\s+)?(?:cut-?off|last\s+training)\b/gi,
- /\bi\s+don'?t\s+have\s+access\s+to\s+real-?time\s+(?:data|information)\b/gi,
- /\bbased\s+on\s+available\s+information\b/gi,
- /\bas\s+an?\s+(?:ai|artificial\s+intelligence|large\s+language|ai\s+language)\s+(?:language\s+)?model\b/gi,
- /\bi\s+(?:am|'m)\s+an?\s+(?:ai|artificial\s+intelligence|large\s+language)\s+(?:assistant|model)?\b/gi,
- /\bi\s+cannot\s+(?:provide|give|offer)\s+(?:legal|medical|financial|professional)\s+advice\b/gi,
- /\bmy\s+training\s+data\s+(?:only\s+)?(?:goes\s+up\s+to|extends\s+to|ends\s+(?:in|at))\b/gi,
- ];
- // ─── AI-tool fingerprints ──────────────────────────────────────────
- // Three near-definitive AI-origin signals adapted from
- // Aboudjem/humanizer-skill P33-P35 (see docs/competitive/audits/
- // 2026-05-17-aboudjem-humanizer-skill.md). Unlike the statistical
- // patterns above, single hit on any of these is strong evidence —
- // the AI tool literally left its fingerprint in the text.
- // Unfilled slot-fill placeholders. Catches the canonical "[Your Name]"
- // family plus dated stubs and HTML/MD comments with placeholder verbs.
- const AI_PLACEHOLDERS = [
- // Directive stubs ("[Your Name]", "[INSERT SOURCE URL]",
- // "[Describe the specific section]") — verb-led, the user was
- // told to fill in.
- /\[(?:Your|Insert|Add|Enter|Describe|Specify|Choose|Pick)[^\]\n]{1,80}\]/gi,
- // Noun-only template variables common in AI-generated email or
- // letter boilerplate. Match the bare noun OR noun + qualifier.
- // Conservative list: only nouns that almost never appear as
- // bracketed real content (citation refs, code identifiers, etc.
- // are excluded because they typically contain dots, slashes,
- // hyphens, or version numbers).
- /\[(?:Recipient|Sender|Topic|Subject|Salutation|Closing|Position|Department|Project Name|Company Name|Date)(?:\s+[^\]\n]{0,60})?\]/gi,
- // All-caps directive forms ("[INSERT X]", "[FILL IN]") — the
- // uppercase tells you it's a slot, not real content.
- /\[(?:INSERT|FILL\s+IN|ADD|TODO|TBD|PLACEHOLDER)[^\]\n]{0,80}\]/g,
- // Date stubs.
- /\b(?:19|20)\d{2}-XX-XX\b/g,
- /\bXX\/XX\/(?:19|20)\d{2}\b/g,
- // HTML/Markdown comment placeholders with placeholder verbs.
- /<!--\s*(?:add|fill\s+in|insert|todo|placeholder)[^>]{0,120}-->/gi,
- ];
- // Chatbot citation/markup tokens that leak through copy-paste.
- // `citeturn0search0` / `citeturn0news5` from ChatGPT, contentReference
- // tokens, oai_citation, attached_file references, grok_card markers.
- // Each is a near-definitive signature of a specific tool.
- const AI_CITATION_MARKUP = [
- /\bcite(?:turn|news|search|navigation)\d+(?:search|turn|news|navigation)\d+/gi,
- /contentReference\s*\[oaicite:[^\]]+\]\s*\{[^}]*\}/gi,
- /\boai_citation\b/gi,
- /\[attached_file:\d+\]/gi,
- /\bgrok_card\b/gi,
- ];
- // UTM/tracking parameters auto-appended by AI tools to URLs they
- // generate. Survives copy-paste even when nothing else does.
- const AI_UTM_SOURCE = [
- /[?&]utm_source=(?:chatgpt|openai|copilot|claude|grok|gemini|perplexity)(?:\.com|\.ai)?\b/gi,
- /[?&]referrer=(?:chatgpt|copilot|grok|claude|gemini|perplexity)\.(?:com|ai)\b/gi,
- ];
- // ─── Template phrases ──────────────────────────────────────────────
- const TEMPLATE_PHRASES = [
- // template-phrase: only vague-praise adjectives. "a first step towards the full API",
- // "a small step towards cutting our storage bill" stay clean.
- /\ban?\s+(?:meaningful|significant|major|crucial|important|big|huge|bold|giant|monumental|pivotal|critical|key|vital|massive|tremendous|substantial|decisive|landmark|historic|exciting|transformative)\s+step\s+(?:towards?|forward)\b/gi,
- /\bwhether\s+you'?re\s+\w+\s+or\s+\w+/gi,
- /\bi\s+recently\s+had\s+the\s+pleasure\s+of\b/gi,
- ];
- // ─── False concession ──────────────────────────────────────────────
- // "While X is impressive, Y remains a challenge" and "Although X has made
- // strides, Y is still an open question" only read as the AI tell when both
- // halves are vague: an opener that concedes nothing specific, paired with a
- // close that names no actual gap. The subject (X) is widened past a single
- // word — "while the underlying model architecture is impressive" is as
- // hollow as "while it is impressive" — but stays inside one clause (no
- // comma or sentence punctuation) so the opener cannot reach across clauses.
- // The close is required in the same sentence: a bare opener followed by a
- // concrete, specific continuation ("...at this scale, our write pattern is
- // append-only, so we moved the hot table to a log-structured store
- // instead") is ordinary technical writing, not the empty frame. See #211.
- // "Despite X challenges" is dropped entirely: alone it is too common a
- // shape in ordinary prose to carry the tell.
- // The close must also follow a clause separator (comma, semicolon or
- // colon): without one, "While the model is impressive and remains a
- // challenge to maintain, we plan to replace it next month" matched both
- // phrases inside the opening clause and never looked at the concrete main
- // clause that followed. See #359.
- // Stop at the next clause separator too: a concrete continuation followed
- // by a later vague phrase is not the empty two-half frame.
- const FALSE_CONCESSION_SUBJECT = "[^,;:.!?\\n]{1,60}?";
- // Known limit: a period inside an abbreviation ("U.S.", "e.g.") ends the
- // gap the same as a real sentence boundary would, so a close on the far
- // side of one is deliberately missed to keep that boundary guarantee;
- // only a comma, semicolon or colon counts as the clause separator #359
- // requires.
- const FALSE_CONCESSION_GAP = "[^,;:.!?\\n]{0,80}?[,;:]\\s*[^,;:.!?\\n]{0,80}?";
- const FALSE_CONCESSION_VAGUE_CLOSE =
- "(?:remains?\\s+a\\s+challenge" +
- "|(?:is|are)\\s+still\\s+an?\\s+open\\s+questions?" +
- "|there\\s+(?:is|are)\\s+still\\s+work\\s+to\\s+do" +
- "|remains?\\s+unanswered)\\b";
- const FALSE_CONCESSION = [
- new RegExp("\\bwhile\\s+" + FALSE_CONCESSION_SUBJECT + "\\s+is\\s+impressive\\b" +
- FALSE_CONCESSION_GAP + FALSE_CONCESSION_VAGUE_CLOSE, 'gi'),
- new RegExp("\\balthough\\s+" + FALSE_CONCESSION_SUBJECT + "\\s+has\\s+made\\s+strides\\b" +
- FALSE_CONCESSION_GAP + FALSE_CONCESSION_VAGUE_CLOSE, 'gi'),
- ];
- // ─── Rhetorical question openers ───────────────────────────────────
- const RHETORICAL_QUESTIONS = [
- /\bbut\s+what\s+does\s+this\s+mean\s+for\b/gi,
- /\bso\s+why\s+should\s+you\s+care\b/gi,
- /\bwhat'?s\s+next\?\s*/gi,
- ];
- // ─── Hedge-stacked predictions ─────────────────────────────────────
- // Stacks a modal with a hedge adverb: "could potentially create new
- // opportunities", "may eventually unlock value." Either word alone is
- // fine; the stack is the tell.
- const HEDGE_STACK = [
- // At most one intervening word, and never a negator. The old {0,2} gap
- // matched ordinary English: "could not possibly" (plain emphatic
- // negation) and inverted questions like "could a savage possibly" both
- // fired. Measured on the human-control corpus, 3 of 4 hedge-stack flags
- // were this over-match. See issue #69.
- /\b(?:could|may|might)\s+(?:(?!not\b|never\b|hardly\b|scarcely\b|barely\b)\w+\s+)?(?:potentially|eventually|ultimately|possibly|conceivably)\b/gi,
- /\b(?:potentially|eventually|ultimately)\s+(?:could|may|might)\b/gi,
- ];
- // ─── Generic future-narrative closers ──────────────────────────────
- // The "may become one of the most important narratives" template — vague
- // future significance with no falsifiable claim. Covers narratives /
- // stories / trends / themes / chapters / movements.
- const FUTURE_NARRATIVE = [
- /\b(?:may|could|will|is\s+(?:poised|set)\s+to)\s+become\s+(?:one\s+of\s+)?(?:the\s+)?(?:most\s+)?\w+\s+(?:narratives?|stories|developments?|trends?|movements?|chapters?|themes?|forces?)\b/gi,
- /\bone\s+of\s+the\s+most\s+important\s+(?:narratives?|stories|trends?|themes?)\s+of\s+the\s+(?:next|coming)\s+\w+\b/gi,
- ];
- // ─── "Real/actual" adjective inflation ─────────────────────────────
- // "Real on-chain tokenomics", "actual reward sustainability" — using
- // real/actual/genuine/true as an empty intensifier on an abstract noun
- // to imply the rest of the field is fake/superficial.
- const REAL_ACTUAL_INFLATION = [
- /\b(?:real|actual|genuine|true)\s+(?:on-?chain\s+)?(?:tokenomics|economics|utility|adoption|sustainability|impact|revenue|fundamentals|demand|value|innovation|traction)\b/gi,
- ];
- // ─── Formulaic openers ─────────────────────────────────────────────
- // The "In the rapidly evolving world of X, Y has emerged as..." family
- // — LLM-default essay openers.
- const FORMULAIC_OPENERS = [
- /\bin\s+the\s+(?:rapidly\s+|ever-?\s*)?(?:evolving|changing|expanding|growing|shifting)\s+(?:world|landscape|realm|space|field|domain|era)\s+of\b/gi,
- /\bin\s+(?:an?|the)\s+(?:digital\s+)?age\s+(?:where|of)\b/gi,
- /\bas\s+(?:we|the\s+world|society|industries?)\s+(?:continue|move|navigate|enter)\s+(?:to\s+)?(?:evolve|forward|into|through)\b/gi,
- // "has emerged as a leader/force/category" — gated to the inflated
- // nouns that signal pseudo-significance, since bare "has emerged as
- // a" matches normal English ("Rust has emerged as a serious systems
- // language"). Same gating for "has become increasingly".
- /\bhas\s+emerged\s+as\s+(?:a|the|one\s+of)\s+(?:leading|key|major|critical|essential|fundamental|pivotal|prominent|dominant|important)\s+\w+/gi,
- /\bhas\s+become\s+increasingly\s+(?:important|critical|popular|relevant|prominent|essential)\b/gi,
- ];
- // ─── Speculative scenario openers ──────────────────────────────────
- // "Imagine a world where...", "Picture a future in which...", "Envision
- // a world where..." — the LLM habit of opening an argument with a
- // hypothetical that lists desirable outcomes instead of making a claim.
- // Gated to the world/future/reality object plus where/in-which so it
- // stays off instructional "imagine you have an array" (a teaching device
- // pointing at a concrete example, not a speculative world) and bare
- // "imagine that" asides. "consider a scenario where…" is deliberately
- // excluded: that is analytical framing common in technical reasoning,
- // not the marketing-opener tell. An optional short comma interrupter
- // catches the equally common "Imagine, for a moment, a world where…"
- // cadence. Known accepted false positive: fiction openings and staged
- // thought experiments match too — the engine has no fiction context
- // mode, so that call is left to the skill's carve-out (highlight-only;
- // a lone hit cannot flip a document's classification).
- const SPECULATIVE_OPENERS = [
- /\b(?:imagine|picture|envision)(?:\s*,[^,\n]{1,30},)?\s+a\s+(?:world|future|reality)\s+(?:where|in\s+which)\b/gi,
- ];
- // ─── Launch-copy dramatic introductions ────────────────────────────
- // "Meet Flowdesk, your new favorite treasury dashboard" / "Think
- // Notion meets Figma" — the LLM-default product-introduction move
- // in launch and announcement copy. Both surfaces are gated to the
- // sentence-initial imperative followed by ONE capitalized token of 2
- // to 30 characters, which is a recall limit: a two-token product name
- // ("Meet North Star", "Think Google Docs meets Microsoft Word") is a
- // deliberate miss. The Meet surface additionally requires one of four
- // launch-copy heads — "your new favorite", "your new go-to", and
- // "the new home/way/standard", the last three only when followed by
- // "of" / "to" / "in|for" or by end-of-clause punctuation. Without
- // that tail the head noun swallows a compound noun and ordinary prose
- // fires: "Meet Rosa, the new home secretary" and "Meet Emma, the new
- // way station manager" both matched before the tail was required.
- // Bare "Meet Sarah, your new account manager" is how humans introduce
- // colleagues, pets, and babies, so that form stays with the skill's
- // judgment side. Two surfaces from
- // the same family are deliberately NOT detected. "Say hello to X",
- // because "Say hello to Grandma." is ordinary human prose. And bare
- // "Enter X.", because the sentence-initial capitalized-noun form is
- // how UI and doc instructions are written: "Enter Password.", "Enter
- // Amount.", "Enter Username — your work email." Dropping the dash
- // terminator does not reach the period-terminated class, and neither
- // does a field-name denylist, so that surface stays with the skill's
- // judgment side and the UI forms are pinned as must-not-fire
- // fixtures. The anchors are lookbehinds so adjacent intros each
- // count and the reported span starts at the tell itself.
- const LAUNCH_INTROS = [
- /(?<=^|[.!?]\s|\n)Meet\s+[A-Z][\w'-]{1,29}\s*,\s*(?:your\s+new\s+(?:favorite|go-to)\b|the\s+new\s+(?:home\s+of\b|way\s+to\b|standard\s+(?:in|for)\b|(?:standard|way|home)(?=\s*(?:[.!?,;:\u2013\u2014]|$))))/g,
- /(?<=^|[.!?]\s|\n)[Tt]hink\s+[A-Z][\w'-]{1,29}\s+meets\s+[A-Z][\w'-]{1,29}\b/g,
- ];
- // ─── Dramatized contrast against the crowd ─────────────────────────
- // "shipped it in 2022, while everyone else was still debating
- // timelines" — a claim propped on an implied lagging crowd. The gate
- // is a dismissive verb PLUS the "was still" dramatization marker,
- // because bare "while everyone else" is ordinary simultaneity ("she
- // read while everyone else watched the movie") and even the
- // dismissive verbs are ordinary English in literal use ("others
- // debated the amendment" in wire copy). That gate is on this FIRST
- // branch only. Its stems are restricted to the -ing form, so the
- // adjective ("was still deliberate about the tradeoff"), the passive
- // ("was still debated by pundits") and the bare present ("was still
- // debates timelines") all stay clean — allowing e/es/ed let all
- // three through. The other two branches carry no "was still"
- // requirement: they match their own stereotyped wording ("writing
- // think-pieces", "playing catch-up"), and the skill entry scopes the
- // claim the same way. Verb stems carry explicit inflection tails so
- // agent nouns ("the market speculators") and adverbs ("deliberately
- // ignored") never match. Measured residue, all accepted: branch one
- // fires on ANY literal progressive use of its verbs ("while the
- // market was still speculating about the price"), not only on "was
- // still debating"; branches two and three fire on literal contrasts
- // of their own ("while everyone else wrote think-pieces from
- // Washington", "while everyone else played catch-up in the spring").
- // Recall is deliberately sacrificed: "was busy debating" without
- // "still" stays a miss, per precision-over-recall.
- const CROWD_CONTRAST = [
- /\bwhile\s+(?:everyone\s+else|the\s+(?:industry|market|competition)|others)\s+(?:was|were|is|are)\s+still\s+(?:busy\s+)?(?:(?:debat|deliberat|hesitat|theoriz|philosophiz|pontificat|speculat|argu)ing|(?:dither|bicker)ing)\b/gi,
- /\bwhile\s+(?:everyone\s+else|the\s+(?:industry|market|competition)|others)\s+(?:was\s+|were\s+)?(?:busy\s+)?(?:writing|wrote)\s+think-?\s?pieces\b/gi,
- /\bwhile\s+(?:everyone\s+else|the\s+(?:industry|market|competition)|others)\s+(?:was\s+|were\s+|is\s+|are\s+)?(?:still\s+)?play(?:ed|ing|s)?\s+catch[-\s]?up\b/gi,
- ];
- // ─── Fake-casual props (stage directions and wink asides) ──────────
- // The regexable props from the fake-casual register: theatrical
- // asterisk stage directions ("*checks notes*", "*chef's kiss*",
- // "*mic drop*") and wink asides. Both lists are closed and short, and
- // that is a recall limit: exactly six stage directions ("checks
- // notes", "chef's kiss", "mic drop", "takes a deep breath", "sips
- // coffee|tea", "nervous laughter") and exactly four parentheticals,
- // the full (yes|no) x (really|seriously) grid. Neighbours in the same
- // register are deliberate misses: "*checks calendar*" and "(yes,
- // honestly)" do not fire. The kiss pattern requires the apostrophe —
- // making it optional matched the ordinary sentence "At midnight,
- // *chefs kiss* their spouses goodbye".
- // The rest of the register (one-word verdict closers, label-prefix
- // openers, the self-QA volley) needs register judgment and stays
- // skill-only — "wild." is a word, not a regex target. "because of
- // course …" joins them: a tense gate does not separate the wink from
- // the ordinary human grumble, because "The build failed because of
- // course it did." is that grumble in the same present-tense-plus-did
- // form the wink uses. Under precision-over-recall the surface is
- // judgment-only, and the grumble is pinned as a fixture. The kiss
- // pattern accepts the curly apostrophe (U+2019) — the form smart
- // punctuation and LLMs actually emit. Known accepted false positive:
- // a human writer using a wink aside on purpose; the props are
- // weighted as a strong single hit, not a classification by
- // themselves.
- const FAKE_CASUAL_PROPS = [
- /\*\s?(?:checks\s+notes|chef['\u2019]s\s+kiss|mic\s+drop|takes\s+a\s+deep\s+breath|sips\s+(?:coffee|tea)|nervous\s+laughter)\s?\*/gi,
- /\(\s?(?:yes|no)\s?,\s?(?:really|seriously)\s?\)/gi,
- ];
- // ─── Performed-insight phrases ─────────────────────────────────────
- // Essayist tics that announce profundity instead of delivering it.
- // Curated noun/complement lists keep precision high: "the whole family"
- // and "sit with him" are ordinary English and must not fire. Adapted
- // from Simon Willison's LLM cliché highlighter
- // (tools.simonwillison.net/llm-cliche-highlighter).
- const PERFORMED_INSIGHT = [
- /\bsit(?:s|ting)?\s+with\s+(?:that|this)(?=\s*(?:[.!?,;:)\u2013\u2014\u2019"']|for\s+a\s+(?:moment|minute|second|beat)\b|$))(?:\s+for\s+a\s+(?:moment|minute|second|beat))?/gi,
- /\bsit(?:s|ting)?\s+with\s+(?:the|your)\s+(?:discomfort|tension|uncertainty|ambiguity|grief|unease)\b/gi,
- /\b(?:that|this|it|which)(?:['\u2019]s|\s+(?:is|was))\s+not\s+nothing\b/gi,
- /\byou\s+already\s+know\s+the\s+answer\b/gi,
- /\b(?:do\s+not|don['\u2019]t)\s+(?:have\s+to\s+)?take\s+my\s+word\s+for\s+it\b/gi,
- /(?<=^|[.!?]\s|\n)Turns\s+out\b/g,
- /(?:['\u2019]s|\b(?:is|was|are|were))\s+the\s+(?:whole|entire)\s+(?:point|game|ballgame|trick|pitch|idea|play|business\s+model|value\s+proposition)\b/gi,
- /\b(?:that|this)(?:['\u2019]s|\s+(?:is|was))\s+the\s+part\s+(?:that|I|you|we|nobody|no\s+one|most\s+people)\b/gi,
- /\bthe\s+only\s+[\w'\u2019-]+\s+that\s+(?:matters|counts)\b/gi,
- // "X is dead; long live Y" also names an essay or a literal succession
- // announcement. Whether the contrast earns its place needs reading (#351).
- /\b(?:that|this)(?:['\u2019]s|\s+(?:is|was))\s+why\s+[^.!?\n]{0,60}\s+mattered\b/gi,
- ];
- const STAGED_DISCOVERY = [
- // Staged discovery: a judgment framed as a twist the writer found ("the recording
- // turned out to be the least interesting part"). Superlative + insight noun keeps
- // ordinary "turned out to be the most expensive option" clean.
- /\b(?:turned|turns|turning)\s+out\s+to\s+be\s+the\s+(?:least|most)\s+(?:interesting|important|surprising|revealing|valuable|useful)\s+(?:part|thing|piece|bit)\b/gi,
- // Require a reveal-style continuation; "the real story was covered by..."
- // describes a literal story and is not a staged discovery.
- /\bthe\s+real\s+story\s+(?:here\s+)?(?:is|was)\b(?=\s+(?:the|that|how|why|what)\b)/gi,
- ];
- // ─── Negation chains ───────────────────────────────────────────────
- // "No fluff, no filler, no jargon" / "It didn't ask, didn't wait" /
- // "Don't call it X. Call it Y." Precision guards, in order: the
- // "no …" chain must open its sentence (mid-sentence inventories like
- // "takes no arguments, no headers, and no body" are factual, not
- // rhetorical); the "did not" chain must be comma-joined with the
- // subject elided ("I did not sleep. I did not eat" is ordinary
- // narration and stays clean); the stop-list keeps idiomatic pairs
- // ("no more, no less", "no matter what") from firing. Adapted from
- // Simon Willison's LLM cliché highlighter.
- const NO_ITEM_STOP = "(?!matter\\b|one\\b|doubt\\b|longer\\b|way\\b|less\\b|more\\b|such\\b|other\\b|means\\b)";
- const NO_ITEM_SECOND_STOP = "(?!(?:in|on|at|of|to|for|with|from|by|is|are|was|were|be|been|being|will|would|can|could|should|shall|may|might|must|have|has|had|do|does|did)\\b)";
- const NEGATION_CHAIN = [
- new RegExp(
- "(?<=^|[.!?]\\s|\\n|[:\\u2013\\u2014]\\s)No\\s+" + NO_ITEM_STOP + "[a-z'\u2019-]+(?:\\s+" + NO_ITEM_SECOND_STOP + "[a-z'\u2019-]+)?" +
- "(?:\\s*,\\s*(?:and\\s+|or\\s+|just\\s+)?no\\s+" + NO_ITEM_STOP + "[a-z'\u2019-]+(?:\\s+" + NO_ITEM_SECOND_STOP + "[a-z'\u2019-]+)?){2,}",
- 'gm'
- ),
- /\b(?:did\s+not|didn['\u2019]t)\s+[a-z]+[^,.;!?\n]{0,20},\s*(?:did\s+not|didn['\u2019]t)\s+[a-z]+/gi,
- /\b(?:do\s+not|don['\u2019]t)\s+(?:just\s+)?(\w+)\s+it\b[^.!?\n]{0,60}[.!?;:,][\s'"\u201d\u2019]*(?:just\s+)?\1\s+it\b/gi,
- ];
- // ─── Negative parallelism ──────────────────────────────────────────
- // "It's not just a search index, it's a foundation for trust." A frame is
- // a negated copula, a body X, then a restatement: "it / this / that /
- // they" + be. The engine splits frames in two (#351):
- //
- // reveal a minimizer (just / merely / simply), then a comma,
- // semicolon, colon, or dash and the restatement ("isn't just
- // raining, it's pouring"). The minimizer-then-upgrade move is
- // the tell itself, so a reveal flags on its own.
- // contrast the same joined frame without the minimizer ("isn't X, it's
- // Y", "isn't about X, it's about Y", "are not only X, they're
- // Y") and the split-sentence reveal ("isn't just X. It's Y.").
- // A single plain correction ("It isn't raining, it's
- // snowing.") is ordinary English, and references/patterns.md
- // allows one frame per piece, so a contrast flags only when
- // another frame of either kind starts within
- // NP_WINDOW_SENTENCES sentences of it. The stacked cadence is
- // the tell; two unrelated corrections paragraphs apart are
- // ordinary prose. "only" is not a reveal minimizer: the human
- // control corpus holds "fossil fuels are not only bad for our
- // environment, they're a losing bet".
- //
- // The restated pronoun is the gate. "not only X but (also) Y" and "not X
- // but Y" are ordinary correlatives and are not matched: on the human
- // control corpus "not only ... but" appeared 16 times in 143k human words
- // against 5 in 115k machine words. X is capped at 80 characters with no
- // comma or sentence punctuation, so a frame cannot reach across clauses.
- const NP_NEG = "(?:\\b(?:is|are|was|were)(?:n['\\u2019]t|\\s+not)|\\b(?:it|this|that|they|he|she|we|you)['\\u2019](?:s|re)\\s+not)";
- const NP_MINIMIZER = "(?:just|merely|simply)";
- const NP_BODY = "[^,;:.!?\\n\\u2014\\u2013]{1,80}?";
- const NP_JOIN = "(?:\\s*[,;:]|\\s*[\\u2014\\u2013]|\\s+--)\\s*";
- const NP_RESTATE = "(?:it|this|that|they)(?:['\\u2019](?:s|re)|\\s+(?:is|are|was|were))\\b";
- const NEGATIVE_PARALLELISM_REVEAL = [
- new RegExp(NP_NEG + "\\s+" + NP_MINIMIZER + "\\s+" + NP_BODY + NP_JOIN + NP_RESTATE, 'gi'),
- ];
- const NEGATIVE_PARALLELISM_CONTRAST = [
- new RegExp(NP_NEG + "\\s+(?!" + NP_MINIMIZER + "\\b)" + NP_BODY + NP_JOIN + NP_RESTATE, 'gi'),
- // Split-sentence reveal: the restatement opens the next sentence.
- new RegExp(NP_NEG + "\\s+" + NP_MINIMIZER + "\\s+" + NP_BODY + "[.!]\\s+" + NP_RESTATE, 'gi'),
- ];
- // Two frames pair when their starting sentences are at most this many
- // sentences apart: the same sentence, the next, or the one after.
- const NP_WINDOW_SENTENCES = 2;
- // A blank line in LF, CRLF, or CR-only text.
- const NP_PARAGRAPH_BREAK = /(?:\r\n|\r(?!\n)|\n)[ \t]*(?:\r\n|\r|\n)/;
- // Reveals always flag. A contrast flags only when some other frame, reveal
- // or contrast, starts nearby. Sentence indexes come from the same coarse
- // splitter the highlight regions use.
- function negativeParallelismIssues(text, reveals, contrasts) {
- // Apply the proximity gate to the same distinct frames the caller reports.
- const distinctReveals = deduplicateIssues(reveals);
- const distinctContrasts = deduplicateIssues(contrasts);
- if (distinctContrasts.length === 0 || distinctReveals.length + distinctContrasts.length < 2) return distinctReveals;
- const frames = [...distinctReveals, ...distinctContrasts];
- const starts = splitSentenceSpans(text).map(([start]) => start);
- const sentenceOf = (index) => {
- let lo = 0;
- let hi = starts.length - 1;
- while (lo < hi) {
- const mid = (lo + hi + 1) >> 1;
- if (starts[mid] <= index) lo = mid;
- else hi = mid - 1;
- }
- return lo;
- };
- const sentences = frames.map((frame) => sentenceOf(frame.index));
- // Paired frames must also share a paragraph: no blank line between them.
- const sameParagraph = (a, b) => !NP_PARAGRAPH_BREAK.test(text.slice(Math.min(a, b), Math.max(a, b)));
- const paired = distinctContrasts.filter((contrast, c) => {
- const i = distinctReveals.length + c;
- return sentences.some((other, j) =>
- j !== i
- && frames[j].index !== contrast.index
- && Math.abs(other - sentences[i]) <= NP_WINDOW_SENTENCES
- && sameParagraph(frames[j].index, contrast.index));
- });
- return [...distinctReveals, ...paired].sort((a, b) => a.index - b.index);
- }
- // ─── Dev-blog boilerplate ──────────────────────────────────────────
- // Stock simplicity slogans from developer marketing. Adapted from
- // Simon Willison's LLM cliché highlighter.
- const DEV_BLOG_BOILERPLATE = [
- // Separators inside "out of the box": ASCII hyphen or whitespace, plus the
- // Unicode hyphens U+2010 and U+2011 and the en dash U+2013 that editors
- // and autocorrect substitute for a hyphen (#140). The em dash and U+2015
- // mark a break ("…works out—of the box office take…"), not a compound, and
- // U+2012 is a figure dash for digit groups, so those stay excluded.
- /\bit\s+just\s+works\b(?!\s+out\b(?![-\s‐‑–]+of[-\s‐‑–]+the[-\s‐‑–]+box\b))/gi,
- /\bzero[-\s]config(?:uration)?\b/gi,
- /\bsane\s+defaults\b/gi,
- /\b(?:hold|fit|fits|holds)\s+in\s+your\s+head\b/gi,
- ];
- // Function words whose presence MID-title marks the AI section-header shape.
- // Word-anchored: without \b the "A" alternative matches inside any word and
- // the guard silently degrades to "four tokens".
- const FUNCTION_WORD = /\b(?:And|Or|Of|The|In|For|To|A|An)\b/;
- // Must accept exactly what TITLE_CASE_HEADER accepts, or the prefix survives
- // into the token count and reintroduces the ##-as-token bug.
- const MD_HEADING_PREFIX = /^#{1,6}[ \t]+/;
- /** Byte ranges covered by fenced code blocks, computed once per scan.
- *
- * A document that documents Markdown is the normal case for this rule -- a
- * fenced `## Heading` example is illustration, not the author's own section
- * header, and flagging it makes every docs page flag itself.
- *
- * This tracks the opening delimiter instead of counting them, because a
- * parity count is wrong on the very case the rule exists for. CommonMark
- * closes a fence only on the same character at the same length or longer, so
- * a four-backtick fence wrapping a three-backtick example -- exactly how you
- * document fences -- nests in practice, and counting delimiters inverts on
- * it. Up to three spaces of indent are legal. An unclosed fence runs to end
- * of document, matching how renderers treat it.
- *
- * Computed once per scan rather than rescanned per hit: the previous version
- * sliced the whole document for every candidate, which is quadratic on a
- * heading-dense file. */
- function fenceRanges(text) {
- const re = /^[ \t]{0,3}(`{3,}|~{3,})([^\n]*)$/gm;
- const ranges = [];
- let open = null;
- let m;
- while ((m = re.exec(text)) !== null) {
- const marker = m[1];
- if (!open) {
- open = { char: marker[0], len: marker.length, start: m.index };
- } else if (
- marker[0] === open.char &&
- marker.length >= open.len &&
- /^[ \t]*\r?$/.test(m[2])
- ) {
- ranges.push([open.start, m.index + m[0].length]);
- open = null;
- }
- }
- if (open) ranges.push([open.start, text.length]);
- return ranges;
- }
- function inFenceRange(ranges, index) {
- return typeof index === 'number' && ranges.some(([a, b]) => index >= a && index < b);
- }
- function blankRange(chars, start, end) {
- for (let i = start; i < end && i < chars.length; i += 1) {
- if (chars[i] !== '\n') chars[i] = ' ';
- }
- }
- function inlineCodeRanges(text) {
- const runs = [];
- for (let i = 0; i < text.length;) {
- if (text[i] === '\n') {
- runs.push(null);
- i += 1;
- continue;
- }
- if (text[i] !== '`') {
- i += 1;
- continue;
- }
- const start = i;
- while (i < text.length && text[i] === '`') i += 1;
- runs.push({ start, end: i, length: i - start });
- }
- const ranges = [];
- let lineStart = 0;
- while (lineStart < runs.length) {
- let lineEnd = runs.indexOf(null, lineStart);
- if (lineEnd === -1) lineEnd = runs.length;
- const nextByLength = new Map();
- const nextSame = new Array(lineEnd - lineStart);
- for (let i = lineEnd - 1; i >= lineStart; i -= 1) {
- nextSame[i - lineStart] = nextByLength.get(runs[i].length);
- nextByLength.set(runs[i].length, i);
- }
- for (let i = lineStart; i < lineEnd;) {
- const close = nextSame[i - lineStart];
- if (close === undefined) {
- i += 1;
- continue;
- }
- ranges.push([runs[i].start, runs[close].end]);
- i = close + 1;
- }
- lineStart = lineEnd + 1;
- }
- return ranges;
- }
- // Copy of the text with fenced blocks and inline code spans blanked out.
- // Index-preserving: each masked character becomes a space and newlines are
- // kept, so offsets into the result still address the same position in the
- // original. For rules where a character inside code is something the author
- // is quoting rather than using — a `#fff` in a CSS sample is not a tag.
- // Fences are blanked first so their backticks cannot pair with a later
- // inline span and swallow the prose between them.
- function maskCode(text) {
- const chars = text.split('');
- for (const [a, b] of fenceRanges(text)) blankRange(chars, a, b);
- // Indented code blocks are deliberately NOT masked. Four spaces is a code
- // block only at top level; under a list marker it is a paragraph
- // continuation, so blanking it silences real tag blocks. #90 reports
- // fences and inline spans, and those are what this masks.
- const withoutFences = chars.join('');
- for (const [start, end] of inlineCodeRanges(withoutFences)) blankRange(chars, start, end);
- return chars.join('');
- }
- function initialFrontmatterRange(text) {
- const lines = [];
- const lineRe = /[^\r\n]*(?:\r\n|\n|\r|$)/g;
- let match;
- while ((match = lineRe.exec(text)) !== null && match[0]) {
- const body = match[0].replace(/(?:\r\n|\n|\r)$/, '');
- lines.push({ body, start: match.index, end: match.index + body.length });
- }
- if (lines.length < 3 || !/^---[ \t]*$/.test(lines[0].body.replace(/^\uFEFF/, ''))) return null;
- let closingLine = -1;
- for (let i = 1; i < lines.length; i += 1) {
- if (/^---[ \t]*$/.test(lines[i].body)) {
- closingLine = i;
- break;
- }
- }
- if (closingLine === -1) return null;
- // A pair of thematic breaks can also surround ordinary Markdown prose.
- // Require the first substantive line to begin like a YAML mapping entry
- // before treating the delimited block as frontmatter. Leading blank lines
- // and YAML comments are valid, and the line parser accepts LF, CRLF, or CR.
- const yamlKey = /^[ \t]*(?:[A-Za-z0-9_.-]+|"[^"\r\n]+"|'[^'\r\n]+')[ \t]*:/;
- const firstContent = lines
- .slice(1, closingLine)
- .find((line) => line.body.trim() && !/^[ \t]*#/.test(line.body));
- if (!firstContent || !yamlKey.test(firstContent.body)) return null;
- return { start: 0, end: lines[closingLine].end };
- }
- // Mask HTML comments in source order while tracking the Markdown constructs
- // that protect a literal `<!--`. A comment wins over code delimiters that
- // occur inside it; a fence, code span, or top-level indented block that
- // starts first wins over comment-looking text inside that code. Each source
- // character participates in a bounded number of forward scans.
- function maskHtmlCommentsOutsideCode(chars) {
- const source = chars.join('');
- const lines = source.split('\n');
- let offset = 0;
- let openFence = null;
- let inIndentedBlock = false;
- let previousBlank = true;
- let listContext = false;
- let maskedHtmlComments = 0;
- const commentClosings = [];
- let closingCursor = 0;
- for (let i = 0; i <= source.length - 3; i += 1) {
- if (source[i] === '-' && source[i + 1] === '-' && source[i + 2] === '>') {
- commentClosings.push(i);
- }
- }
- const backtickRuns = (line) => {
- const runs = [];
- for (let i = 0; i < line.length;) {
- if (line[i] !== '`') {
- i += 1;
- continue;
- }
- const start = i;
- while (i < line.length && line[i] === '`') i += 1;
- runs.push({ start, end: i, length: i - start, next: -1 });
- }
- const nextByLength = new Map();
- for (let i = runs.length - 1; i >= 0; i -= 1) {
- runs[i].next = nextByLength.get(runs[i].length) ?? -1;
- nextByLength.set(runs[i].length, i);
- }
- return runs;
- };
- for (const originalLine of lines) {
- const lineEnd = offset + originalLine.length;
- let visibleLine = chars.slice(offset, lineEnd).join('');
- const fenceMatch = /^[ \t]{0,3}(`{3,}|~{3,})([^\n]*)$/.exec(visibleLine);
- let fencedLine = false;
- if (openFence) {
- fencedLine = true;
- if (
- fenceMatch
- && fenceMatch[1][0] === openFence.char
- && fenceMatch[1].length >= openFence.length
- && /^[ \t]*\r?$/.test(fenceMatch[2])
- ) openFence = null;
- } else if (fenceMatch) {
- fencedLine = true;
- openFence = { char: fenceMatch[1][0], length: fenceMatch[1].length };
- }
- const indented = /^(?: {4}|\t)\S/.test(visibleLine);
- const indentedCode = !fencedLine
- && indented
- && (inIndentedBlock || (previousBlank && !listContext));
- if (!fencedLine && !indentedCode) {
- const runs = backtickRuns(visibleLine);
- let runIndex = 0;
- let cursor = 0;
- while (cursor < visibleLine.length) {
- while (runIndex < runs.length && runs[runIndex].start < cursor) runIndex += 1;
- const commentIndex = visibleLine.indexOf('<!--', cursor);
- const run = runs[runIndex];
- if (run && (commentIndex === -1 || run.start < commentIndex)) {
- if (run.next !== -1) {
- cursor = runs[run.next].end;
- runIndex = run.next + 1;
- } else {
- cursor = run.end;
- runIndex += 1;
- }
- continue;
- }
- if (commentIndex === -1) break;
- const openingIndex = offset + commentIndex;
- while (
- closingCursor < commentClosings.length
- && commentClosings[closingCursor] < openingIndex + 2
- ) closingCursor += 1;
- const closingIndex = commentClosings[closingCursor] ?? -1;
- const end = closingIndex === -1 ? source.length : closingIndex + 3;
- if (closingIndex !== -1) closingCursor += 1;
- blankRange(chars, openingIndex, end);
- maskedHtmlComments += 1;
- cursor = Math.min(visibleLine.length, end - offset);
- }
- }
- visibleLine = chars.slice(offset, lineEnd).join('');
- const layoutChars = visibleLine.split('');
- if (fencedLine) {
- blankRange(layoutChars, 0, layoutChars.length);
- } else {
- const inlineRe = /(`+)(?:(?!\1)[^\n])+\1/g;
- let inlineMatch;
- while ((inlineMatch = inlineRe.exec(visibleLine)) !== null) {
- blankRange(layoutChars, inlineMatch.index, inlineMatch.index + inlineMatch[0].length);
- }
- }
- const layoutLine = layoutChars.join('');
- const blank = layoutLine.trim() === '';
- if (indentedCode) inIndentedBlock = true;
- else if (!blank) inIndentedBlock = false;
- if (!blank && !indentedCode && !fencedLine) {
- if (/^ {0,3}(?:[-*+]|\d{1,9}[.)])(?:\s|$)/.test(layoutLine)) listContext = true;
- else if (/^\S/.test(layoutLine)) listContext = false;
- }
- previousBlank = blank;
- offset = lineEnd + 1;
- }
- return maskedHtmlComments;
- }
- // Mask source-only Markdown spans while preserving source offsets. The
- // detector can then score what a reader sees without making later issue
- // indexes or sentence highlights point at the wrong source location.
- function maskRenderedMarkdown(text) {
- const chars = text.split('');
- let maskedFrontmatter = 0;
- const frontmatter = initialFrontmatterRange(text);
- if (frontmatter) {
- blankRange(chars, frontmatter.start, frontmatter.end);
- maskedFrontmatter = 1;
- }
- const maskedHtmlComments = maskHtmlCommentsOutsideCode(chars);
- return { text: chars.join(''), maskedFrontmatter, maskedHtmlComments };
- }
- // Ignore regions (#351): an author can exclude a passage from scoring, such
- // as a specimen of AI prose quoted on purpose, by wrapping it in
- // <!-- avoid-ai-writing:ignore-start --> … <!-- avoid-ai-writing:ignore-end -->
- // A marker counts only as a whole line: the full comment, at most three
- // spaces of indent, nothing else on the line. Starts nest: each start needs
- // its own end, and the region runs from the outermost start to its matching
- // end. An unclosed start runs to the end of the text; an end with no open
- // start is ignored. The region is blanked in place, so offsets still
- // address the source.
- //
- // Markers are found by ONE left-to-right scan rather than a stack of masks,
- // because separate masks disagree about who owns overlapping text: a fence
- // inside a comment would swallow the comment's `-->`, and a `<pre>` inside
- // a comment would open a container. After skipping initial YAML
- // frontmatter, the scan recognizes, in order of appearance, an HTML
- // comment, a Markdown fence, a top-level indented code line, an inline code
- // span, and an HTML <pre>, <code>, <script>, or <style> element. Whichever
- // construct opens first owns the text until its own close, so a marker, a
- // fence, a tag, or a comment inside another construct is inert. Anything
- // unclosed runs to the end of the text: when in doubt, the marker does
- // nothing and the prose stays scored. Every character is visited a bounded
- // number of times.
- const IGNORE_MARKER_LINE_RE = /^ {0,3}<!--[ \t]*avoid-ai-writing:ignore-(start|end)[ \t]*-->[ \t]*$/i;
- const MARKER_FENCE_RE = /^ {0,3}(`{3,}|~{3,})(.*)$/;
- const HTML_CODE_OPEN_RE = /<(pre|code|script|style)\b[^>]*>/iy;
- const HTML_CODE_CLOSE_RE = {
- pre: /<\/pre/gi,
- code: /<\/code/gi,
- script: /<\/script/gi,
- style: /<\/style/gi,
- };
- function findIgnoreMarkers(text) {
- const n = text.length;
- const markers = [];
- const lineEndAt = (from) => {
- let i = from;
- while (i < n && text[i] !== '\n' && text[i] !== '\r') i += 1;
- return i;
- };
- const nextLineAt = (end) => (text[end] === '\r' && text[end + 1] === '\n' ? end + 2 : end + 1);
- const isLineStart = (i) => i === 0 || text[i - 1] === '\n' || (text[i - 1] === '\r' && text[i] !== '\n');
- // Backtick runs in [from, to), keyed by start, each with the end of the
- // next run of the same length on the line (-1 when unmatched). Built at
- // most once per line segment and never rebuilt after a comment or HTML
- // element closes mid-line: rebuilding made a long line that alternates
- // code spans and <code> elements quadratic. Runs inside a construct the
- // scan jumps over are simply never visited.
- const backtickRuns = (from, to) => {
- const list = [];
- for (let i = from; i < to;) {
- if (text[i] !== '`') { i += 1; continue; }
- const start = i;
- while (i < to && text[i] === '`') i += 1;
- list.push({ start, end: i, length: i - start, closeEnd: -1 });
- }
- const nextByLength = new Map();
- for (let i = list.length - 1; i >= 0; i -= 1) {
- const next = nextByLength.get(list[i].length);
- if (next) list[i].closeEnd = next.end;
- nextByLength.set(list[i].length, list[i]);
- }
- return new Map(list.map((run) => [run.start, run]));
- };
- let pos = 0;
- const frontmatter = initialFrontmatterRange(text);
- if (frontmatter) pos = frontmatter.end;
- let lineEnd = -1;
- let runs = null;
- let prevBlank = true;
- let inIndented = false;
- let listContext = false;
- while (pos < n) {
- if (pos > lineEnd) {
- // Entering a new line, or resuming mid-line after a construct that
- // spanned lines. Only a true line start gets the line-level checks.
- lineEnd = lineEndAt(pos);
- runs = null;
- if (isLineStart(pos)) {
- const line = text.slice(pos, lineEnd);
- const blank = line.trim() === '';
- const fence = MARKER_FENCE_RE.exec(line);
- if (fence && !(fence[1][0] === '`' && fence[2].includes('`'))) {
- let close = n;
- for (let scan = nextLineAt(lineEnd); scan < n;) {
- const end = lineEndAt(scan);
- const closing = MARKER_FENCE_RE.exec(text.slice(scan, end));
- if (
- closing
- && closing[1][0] === fence[1][0]
- && closing[1].length >= fence[1].length
- && closing[2].trim() === ''
- ) {
- close = end;
- break;
- }
- scan = nextLineAt(end);
- }
- pos = close;
- prevBlank = false;
- inIndented = false;
- continue;
- }
- const indented = !blank && /^(?: {4}|\t)/.test(line);
- if (indented && (inIndented || (prevBlank && !listContext))) {
- inIndented = true;
- prevBlank = false;
- pos = lineEnd;
- continue;
- }
- if (!blank) {
- inIndented = false;
- if (/^ {0,3}(?:[-*+]|\d{1,9}[.)])(?:\s|$)/.test(line)) listContext = true;
- else if (/^\S/.test(line)) listContext = false;
- }
- prevBlank = blank;
- const marker = IGNORE_MARKER_LINE_RE.exec(line);
- if (marker) {
- markers.push({ kind: marker[1].toLowerCase(), start: pos, end: lineEnd });
- pos = lineEnd;
- continue;
- }
- }
- }
- const ch = text[pos];
- if (ch === '<') {
- if (text.startsWith('<!--', pos)) {
- const close = text.indexOf('-->', pos + 4);
- pos = close === -1 ? n : close + 3;
- continue;
- }
- HTML_CODE_OPEN_RE.lastIndex = pos;
- const open = HTML_CODE_OPEN_RE.exec(text);
- if (open) {
- const closeRe = HTML_CODE_CLOSE_RE[open[1].toLowerCase()];
- closeRe.lastIndex = pos + open[0].length;
- const close = closeRe.exec(text);
- pos = close === null ? n : close.index;
- continue;
- }
- } else if (ch === '`') {
- if (!runs) runs = backtickRuns(pos, lineEnd);
- const run = runs.get(pos);
- if (run && run.closeEnd !== -1) {
- pos = run.closeEnd;
- continue;
- }
- pos += run ? run.length : 1;
- continue;
- }
- pos += 1;
- }
- return markers;
- }
- function maskIgnoreRegions(text) {
- if (!/avoid-ai-writing:ignore-/i.test(text)) return { text, ignoredRegions: 0 };
- const chars = text.split('');
- let ignoredRegions = 0;
- let depth = 0;
- let openAt = -1;
- for (const marker of findIgnoreMarkers(text)) {
- if (marker.kind === 'start') {
- if (depth === 0) openAt = marker.start;
- depth += 1;
- } else if (depth > 0) {
- depth -= 1;
- if (depth === 0) {
- blankRange(chars, openAt, marker.end);
- ignoredRegions += 1;
- }
- }
- }
- if (depth > 0) {
- blankRange(chars, openAt, chars.length);
- ignoredRegions += 1;
- }
- return { text: chars.join(''), ignoredRegions };
- }
- // Blank the content of double-quoted spans and keep the quote marks, so
- // offsets and the curly-quote signal survive. Based on QUOTED_SPAN in
- // scripts/self-scan.js minus its single-quote branch: apostrophes in
- // contractions and possessives would pair up across ordinary prose. A
- // straight quote touching a letter or digit on its outer side is an inch
- // mark, not a quotation. Empty `""` pairs are consumed so they cannot
- // shift the pairing, and a span never crosses a backtick into code. A
- // nested quotation escaped as a pair (`\"…\"`) stays inside the span. A
- // lone `\"` still closes it, so a path such as `"C:\Temp\"` cannot run on
- // into the next quotation.
- const QUOTED_SPAN_RE = /(?<![\p{L}\p{N}])"(?:\\"[^"`\n]{0,300}\\"|[^"`\n]){0,300}"(?![\p{L}\p{N}])|“[^“”`\n]{0,300}”/gu;
- function maskQuotedSpans(text) {
- let maskedQuotes = 0;
- const masked = text.replace(QUOTED_SPAN_RE, (span) => {
- // Escaped pairs can chain past the 300-character cap; score those.
- // Count code points, as the `u` regex does, so emoji do not trip it.
- if (span.length === 2 || [...span].length > 302) return span;
- maskedQuotes += 1;
- // Keep sentence punctuation so getSentences splits where it did before.
- return span[0] + span.slice(1, -1).replace(/[^.!?]/g, ' ') + span[span.length - 1];
- });
- return { text: masked, maskedQuotes };
- }
- // A `>` opens a blockquote line when a space, the line end, a letter, a
- // nested `>`, an opening double quote, emphasis, a link, or a list marker
- // (`- `, `+ `, `1. `) follows it. That covers compact Markdown (`>text`)
- // without swallowing comparisons such as `>=5` or `>-1`. A single quote is
- // left out, as in the inline quote pass.
- const BLOCKQUOTE_LINE_RE = /^\s*>(?:$|[\s\p{L}>"“*_[]|[-+]\s|\d{1,9}[.)]\s)/u;
- function maskBlockquotes(text) {
- const chars = text.split('');
- const lines = [];
- const lineRe = /[^\r\n]*(?:\r\n|\n|\r|$)/g;
- let match;
- while ((match = lineRe.exec(text)) !== null && match[0]) {
- const body = match[0].replace(/(?:\r\n|\n|\r)$/, '');
- lines.push({ text: body, start: match.index, end: match.index + body.length });
- }
- let quotedLines = 0;
- for (const line of lines) {
- if (BLOCKQUOTE_LINE_RE.test(line.text)) {
- blankRange(chars, line.start, line.end);
- quotedLines += 1;
- }
- }
- return { text: chars.join(''), quotedLines };
- }
- // Keep the historical deletion behavior for default plain mode. Paragraph-
- // scoped rules depend on the surrounding lines being rejoined exactly this
- // way, so changing this prepass would change scores for existing callers.
- function stripBlockquotes(text, sourceMap) {
- const rawLines = text.split(/\r?\n/);
- const stripIndexes = new Set();
- let blankedLines = 0;
- for (let i = 0; i < rawLines.length; i += 1) {
- // A bare CR is not split here, so a CR-only document is one element.
- // Deleting it would take the unquoted lines after the quote with it,
- // so blank each quoted CR-separated line in place instead. Blanking
- // keeps the element length, so the offsets below still line up.
- if (rawLines[i].includes('\r')) {
- rawLines[i] = rawLines[i].split('\r').map((line) => {
- if (!BLOCKQUOTE_LINE_RE.test(line)) return line;
- blankedLines += 1;
- return ' '.repeat(line.length);
- }).join('\r');
- } else if (BLOCKQUOTE_LINE_RE.test(rawLines[i])) {
- stripIndexes.add(i);
- }
- }
- const kept = rawLines
- .map((_, index) => index)
- .filter((index) => !stripIndexes.has(index));
- const result = {
- text: kept.map((index) => rawLines[index]).join('\n'),
- quotedLines: stripIndexes.size + blankedLines,
- };
- if (!Array.isArray(sourceMap)) return result;
- const lineStarts = [];
- let offset = 0;
- for (let i = 0; i < rawLines.length; i += 1) {
- lineStarts.push(offset);
- offset += rawLines[i].length;
- if (i < rawLines.length - 1) {
- if (text[offset] === '\r') offset += 1;
- if (text[offset] === '\n') offset += 1;
- }
- }
- const mapped = [];
- for (let i = 0; i < kept.length; i += 1) {
- const lineIndex = kept[i];
- const start = lineStarts[lineIndex];
- appendMapRange(mapped, sourceMap, start, start + rawLines[lineIndex].length);
- if (i < kept.length - 1) {
- const separatorStart = start + rawLines[lineIndex].length;
- const newlineIndex = text[separatorStart] === '\r' ? separatorStart + 1 : separatorStart;
- mapped.push(sourceMap[newlineIndex]);
- }
- }
- return { ...result, sourceMap: mapped };
- }
- function maskTopLevelIndentedCode(chars, { listAware = false } = {}) {
- const lines = chars.join('').split('\n');
- let offset = 0;
- let inBlock = false;
- let previousBlank = true;
- let listContext = false;
- for (let i = 0; i < lines.length; i += 1) {
- const line = lines[i];
- const indented = /^(?: {4}|\t)\S/.test(line);
- const blank = line.trim() === '';
- const isCode = indented && (inBlock || (previousBlank && (!listAware || !listContext)));
- if (isCode) {
- blankRange(chars, offset, offset + line.length);
- inBlock = true;
- } else if (!blank) {
- inBlock = false;
- }
- if (listAware && !blank && !isCode) {
- if (/^ {0,3}(?:[-*+]|\d{1,9}[.)])(?:\s|$)/.test(line)) listContext = true;
- else if (/^\S/.test(line)) listContext = false;
- }
- previousBlank = blank;
- offset += line.length + 1;
- }
- }
- function maskYamlFrontmatter(chars) {
- const lines = chars.join('').split('\n');
- const bare = (line) => line.replace(/\r$/, '');
- const first = bare(lines[0]).replace(/^\uFEFF/, '');
- if (first !== '---' || lines.length < 2 || /^\s*$/.test(bare(lines[1]))) return;
- let closingLine = -1;
- for (let i = 1; i < lines.length; i += 1) {
- if (bare(lines[i]) === '---') {
- closingLine = i;
- break;
- }
- }
- if (closingLine === -1) return;
- let end = 0;
- for (let i = 0; i <= closingLine; i += 1) {
- end += lines[i].length;
- if (i < lines.length - 1) end += 1;
- }
- blankRange(chars, 0, end);
- }
- function maskYamlMetadata(chars) {
- const lines = chars.join('').split('\n');
- let offset = 0;
- let nestedAfterIndent = null;
- for (const line of lines) {
- const bare = line.replace(/\r$/, '');
- // Lowercase keys are the common unfenced-YAML shape. Keeping this
- // case-sensitive prevents prose labels such as "Note: ..." from being
- // mistaken for metadata and silencing a real copyedit on the line.
- const key = bare.match(/^([ \t]*)(?:-[ \t]+)?[a-z_][a-z0-9_.-]*[ \t]*:(.*)$/);
- const indentation = (bare.match(/^[ \t]*/) || [''])[0].length;
- let shouldMask = false;
- if (key) {
- shouldMask = true;
- nestedAfterIndent = key[2].trim() === '' ? key[1].length : null;
- } else if (nestedAfterIndent !== null && bare.trim() !== '' && indentation > nestedAfterIndent) {
- shouldMask = true;
- } else if (bare.trim() === '') {
- nestedAfterIndent = null;
- } else {
- nestedAfterIndent = null;
- }
- if (shouldMask) blankRange(chars, offset, offset + line.length);
- offset += line.length + 1;
- }
- }
- function maskMarkdownTables(chars) {
- const lines = chars.join('').split('\n');
- // Tested against the trimmed line: the pattern already allows surrounding
- // whitespace, and its adjacent `\s*` groups backtrack quadratically on a
- // long whitespace run, so a line of masked comments or blank padding
- // cost seconds before it was rejected (#235).
- const delimiter = /^\|?\s*:?-{3,}:?\s*(?:\|\s*:?-{3,}:?\s*)+\|?$/;
- const rows = new Set();
- for (let i = 0; i < lines.length; i += 1) {
- const candidate = lines[i].trim();
- if (!candidate.includes('---') || !delimiter.test(candidate)) continue;
- if (i > 0 && lines[i - 1].includes('|')) rows.add(i - 1);
- rows.add(i);
- for (let j = i + 1; j < lines.length && lines[j].includes('|'); j += 1) rows.add(j);
- }
- let offset = 0;
- for (let i = 0; i < lines.length; i += 1) {
- if (rows.has(i)) blankRange(chars, offset, offset + lines[i].length);
- offset += lines[i].length + 1;
- }
- }
- function maskDelimitedQuotes(chars, open, close, apostropheAware = false) {
- const isWord = (char) => char !== undefined && /[a-z0-9]/i.test(char);
- const isEscaped = (index) => {
- let slashes = 0;
- for (let i = index - 1; i >= 0 && chars[i] === '\\'; i -= 1) slashes += 1;
- return slashes % 2 === 1;
- };
- let start = -1;
- for (let i = 0; i < chars.length; i += 1) {
- if (start === -1) {
- if (chars[i] === open && !isEscaped(i) && (!apostropheAware || !isWord(chars[i - 1]))) start = i;
- } else if (chars[i] === close && !isEscaped(i) && (!apostropheAware || !isWord(chars[i + 1]))) {
- blankRange(chars, start, i + 1);
- start = -1;
- }
- }
- }
- // Additional protected spans for the hyphenation copyedit. Unlike general
- // AI-tell matching, this rule must not "correct" a literal spelling in a
- // quote, URL, path, filename, command flag, or Markdown blockquote.
- function maskHyphenationProtected(text) {
- const chars = maskCode(text).split('');
- maskTopLevelIndentedCode(chars);
- maskYamlFrontmatter(chars);
- maskYamlMetadata(chars);
- maskMarkdownTables(chars);
- maskDelimitedQuotes(chars, '"', '"');
- maskDelimitedQuotes(chars, '“', '”');
- maskDelimitedQuotes(chars, "'", "'", true);
- maskDelimitedQuotes(chars, '‘', '’', true);
- const maskMatches = (regex) => {
- const source = chars.join('');
- let match;
- while ((match = regex.exec(source)) !== null) {
- blankRange(chars, match.index, match.index + match[0].length);
- }
- };
- maskMatches(/^[ \t]*>[^\n]*$/gm);
- maskMatches(/\b(?:https?:\/\/|www\.)[^\s<>]+/gi);
- maskMatches(/<[!?/]?[a-z][^<>\n]*>/gi);
- maskMatches(/(?<![a-z0-9_-])--?[a-z0-9][a-z0-9-]{0,127}/gi);
- // Paths and filenames use bounded components. Besides preventing
- // superlinear backtracking on long kebab blobs, the explicit prefix and
- // trailing-slash forms cover single-component paths such as C:\\code-base,
- // ~/code-base, /code-base, and code-base/.
- maskMatches(/(?:[a-z]:[\\/]|\.{1,2}[\\/]|~[\\/]|[\\/])[a-z0-9_.-]{1,255}/gi);
- maskMatches(/(?:[a-z]:[\\/]|(?:\.\.?[\\/])?)(?:[a-z0-9_.-]{1,64}[\\/]){1,128}[a-z0-9_.-]{1,64}/gi);
- maskMatches(/\b[a-z0-9_.]{0,63}-[a-z0-9_.-]{1,64}[\\/]/gi);
- maskMatches(/\b[a-z0-9_.-]{1,64}-[a-z0-9_.-]{1,64}\.[a-z0-9]{1,16}\b/gi);
- // Technical identifiers that are distinguishable from ordinary prose:
- // selectors, scoped packages, versioned tokens, assignments, and kebab
- // names next to an explicit identifier cue. Plain lowercase compounds
- // remain visible to the curated copyedit patterns below.
- maskMatches(/[.#][a-z_][a-z0-9_.-]{0,63}-[a-z0-9_.-]{1,64}\b/gi);
- maskMatches(/@[a-z0-9_.-]{1,64}\/[a-z0-9_.-]{1,64}-[a-z0-9_.-]{1,64}(?:@[^\s,;)\]}]{1,32})?/gi);
- maskMatches(/\b[a-z0-9_.-]{1,64}-[a-z0-9_.-]{1,64}@[~^]?v?\d[a-z0-9*_.+-]{0,31}\b/gi);
- maskMatches(/\b(?:[a-z0-9_.-]{0,64}\d[a-z0-9_.-]{0,64}-[a-z0-9_.-]{1,64}|[a-z0-9_.-]{1,64}-[a-z0-9_.-]{0,64}\d[a-z0-9_.-]{0,64})\b/gi);
- maskMatches(/\b[a-z_][a-z0-9_.]{0,63}(?:-[a-z0-9_.]{1,64}){1,8}(?=[ \t]*[=:])/gi);
- maskMatches(/\b[a-z_][a-z0-9_.]{0,63}(?:-[a-z0-9_.]{1,64}){1,8}(?=[ \t]+(?:npm[ \t]+)?(?:package|module|class|selector|config(?:uration)?[ \t]+key|key|identifier|property|setting|token|slug|command|option)\b)/gi);
- maskMatches(/\b(?:(?:file(?:name)?|directory|folder|package|module|class|selector|config(?:uration)?[ \t]+key|identifier|property|setting|token|slug|command|option)(?:[ \t]+(?:named|called|is|was))?|key[ \t]+(?:named|called|is|was))[ \t]+(?:@[a-z0-9_.-]{1,64}\/)?[a-z_][a-z0-9_.]{0,63}(?:-[a-z0-9_.]{1,64}){1,8}\b/gi);
- maskMatches(/\b(?:npm|pnpm|yarn)[ \t]+(?:add|install)[ \t]+(?:@[a-z0-9_.-]{1,64}\/)?[a-z0-9_.]{1,64}(?:-[a-z0-9_.]{1,64}){1,8}/gi);
- return chars.join('');
- }
- function findUnnecessaryHyphenation(text) {
- const scanText = maskHyphenationProtected(text);
- const issues = [];
- for (const entry of UNNECESSARY_HYPHENATION) {
- const regex = new RegExp(entry.pattern.source, entry.pattern.flags);
- let match;
- while ((match = regex.exec(scanText)) !== null) {
- issues.push({
- type: 'unnecessary-hyphenation',
- text: match[0],
- severity: 'medium',
- suggestion: typeof entry.suggestion === 'function'
- ? entry.suggestion(match[0])
- : entry.suggestion,
- });
- }
- }
- return issues;
- }
- // ─── Forms that open with `#` but are not social tags ──────────────
- // `#` is overloaded in technical prose and the hashtag rule counts every
- // `#word` it sees, so these are subtracted before the threshold applies:
- // #88, #1234 issue and PR references
- // #1a2b3c CSS hex colours, 6 or 8 chars AND containing a digit
- // #include, #ifndef C preprocessor directives
- // Deliberately NOT carving out 3- and 4-digit hex: #dad, #cafe, #b2b, #e2e,
- // #ace, #face and #bad are real tags, and subtracting them cost true
- // positives on exactly the stuffed-block shape this rule exists to catch.
- // Real palettes are dominated by 6-digit values, so a CSS paragraph still
- // lands under the threshold without the short forms.
- // `owner/repo#88` and URL fragments need no carve-out: the char before `#`
- // is a word char, so the rule's own anchor already rejects them.
- // Ambiguous word tags stay counted on purpose. `#general` as a channel and
- // `#general` as a tag are the same token, and separating them needs a guess
- // that costs more precision on real tag blocks than the carve-out buys.
- // Requires at least one digit: #decade, #facade, #deadbeef are a-f words and
- // real tags. Every actual palette value in the wild carries a digit.
- const HEX_COLOUR = /^(?=[0-9a-f]*\d)(?:[0-9a-f]{6}|[0-9a-f]{8})$/i;
- const CPP_DIRECTIVE = /^(?:include|define|undef|if|ifdef|ifndef|elif|else|endif|pragma|error|warning|line)$/;
- function isSocialTag(tag) {
- return !/^\d+$/.test(tag) && !HEX_COLOUR.test(tag) && !CPP_DIRECTIVE.test(tag);
- }
- // ─── Title Case Section Headers in non-technical prose ─────────────
- // "Strategic Negotiations And Key Partnerships" — every content word
- // capitalized. Acceptable in API docs, ML papers, news headlines. Tell
- // in marketing/personal/blog prose. Skipped when contextMode is
- // 'technical'; runs for general, marketing, and personal.
- //
- // The optional `#{1,6}` prefix is load-bearing (#62): without it the `^[A-Z]`
- // anchor required the line to START with a capital, so `## Benefits And
- // Strategic Considerations` never matched — the first character is `#`. The
- // rule missed the single most common way a heading is actually written, while
- // catching the bare-line form it is usually converted from. Reported by a
- // downstream vendoring the detector.
- //
- // Setext headings (`Title`/`=====`) need no prefix: their text line is bare
- // and already matched by this same pattern.
- // Interior tokens accept Title Case words, acronyms (`AI`, `API`, `CLI`) and
- // the capitalised single-letter words `A` and `I`. The first and last tokens
- // stay ordinary `[A-Z][a-z]+` words, which also excludes all-caps banner
- // lines (`## HTTP API REFERENCE`) whose leading token is not Title Case.
- //
- // Separators and trailing whitespace are horizontal only (`[ \t]`), so a
- // match can never run past one physical line: `\s` also eats newlines, which
- // let two unrelated lines or a blank-line-separated fragment combine into a
- // single heading match (GH-291).
- const TITLE_CASE_HEADER = /^(?:#{1,6}[ \t]+)?([A-Z][a-z]+(?:[ \t]+(?:[A-Z][a-z]+|A|I|[A-Z]{2,}|and|or|of|the|in|for|to|a|an))+[ \t]+[A-Z][a-z]+)[ \t]*$/gm;
- // ─── Parenthetical hedging asides ──────────────────────────────────
- // "(and increasingly, X)", "(or more precisely, Y)", "(though to be
- // fair, Z)" — pseudo-aside that adds no information but performs
- // thoughtfulness. Different from genuine human parentheticals which
- // tend to be tangents or clarifications, not hedges.
- const PARENTHETICAL_HEDGE = [
- /\(\s*(?:and\s+)?(?:increasingly|notably|importantly|crucially|interestingly|perhaps)[,]?\s+[^)]{3,60}\)/gi,
- /\(\s*or\s+more\s+(?:precisely|accurately|specifically)[,]?\s+[^)]{3,60}\)/gi,
- /\(\s*though\s+to\s+be\s+fair[,]?\s+[^)]{3,60}\)/gi,
- /\(\s*at\s+least\s+(?:in\s+)?(?:theory|principle|part)[,]?\s+[^)]{0,60}\)/gi,
- ];
- // ─── Confidence calibration ────────────────────────────────────────
- const CONFIDENCE_CALIBRATION = [
- /\binterestingly\b/gi,
- /\bsurprisingly\b/gi,
- /\bimportantly\b/gi,
- /\bsignificantly\b/gi,
- /\bcertainly\b/gi,
- /\bundoubtedly\b/gi,
- /\bwithout\s+a\s+doubt\b/gi,
- ];
- // ─── Social endorsement / CTA closers ──────────────────────────────
- // The curatorial sign-off LLMs append to LinkedIn / X posts that share
- // or recommend something — usually a colon teeing up a link. Distinct
- // from the bare "worth reading" word-table entry (a single weak word)
- // and from infomercial hooks (mid-flow teasers): this is the
- // demonstrative-anchored endorsement — "THIS one is worth your time:",
- // "do yourself a favor and read this", "thank me later" — that performs
- // a recommendation without giving the reader a reason to click.
- //
- // Precision-first: each pattern carries an anchor so it stays off the
- // literal-verb prose a human writes. The demonstrative object ("read
- // THIS", not "read the runbook"), the trailing-terminal lookahead on
- // "miss this" / "bookmark this" (the closing-line shape, not "miss this
- // meeting" / "bookmark this page"), and the sentence-initial lookbehind
- // on "thank me later" / "save this for later" (the imperative CTA, not
- // "she will thank me later") all exist to suppress false positives on
- // ordinary instructional/conversational text. Apostrophe classes admit
- // the curly ' (U+2019) because LinkedIn / Word / macOS auto-curl it —
- // the straight-only form would miss the canonical "you won't" closer.
- const SOCIAL_CTA_CLOSER = [
- /\bthis\s+one['’]?s?\s+(?:is\s+)?(?:well\s+|totally\s+|absolutely\s+|definitely\s+|really\s+|truly\s+|easily\s+|more\s+than\s+)?worth\s+(?:your\s+time|the\s+read|a\s+read|every\s+(?:minute|second)|reading|watching|a\s+listen|a\s+watch|a\s+look|it)\b/gi,
- /\bthis\s+one['’]?s?\s+(?:is\s+)?a\s+must[-\s]?(?:read|watch|listen|see)\b/gi,
- /\b(?:highly|strongly|can['’]?t|cannot)\s+recommend\w*\s+(?:giving\s+)?(?:this|it)\s+(?:one\s+)?a\s+(?:read|listen|watch|look|go)\b/gi,
- /\bdo\s+yourself\s+a\s+favou?r\s+and\s+(?:read|watch|check\s+out)\s+(?:this|it)\b/gi,
- /\byou\s+(?:really\s+)?(?:won['’]?t|do\s*n['’]?t|will\s+not|do\s+not)\s+want\s+to\s+miss\s+this(?:\s+one)?(?=\s*(?:[:.!\n]|$))/gi,
- /(?<=^|[,.!?:\n]\s{0,4})(?:you\s+can\s+)?thank\s+me\s+later\b/gim,
- /(?<=^|[.!?:\n]\s{0,4})save\s+this\s+(?:one\s+)?for\s+later\b/gim,
- /\bbookmark\s+this(?:\s+(?:one|post|thread))?(?=\s*(?:[:.!\n]|$))/gi,
- /\bdo\s*n['’]?t\s+sleep\s+on\s+this\b/gi,
- /\btrust\s+me,?\s+(?:on\s+this|you['’]?ll)\b/gi,
- ];
- // ─── Unnecessary hyphenation (#107) ───────────────────────────────
- // Precision-first subclasses only. The general question of whether a
- // compound modifier is established English needs editorial judgment and
- // remains in SKILL.md; the engine covers only curated open/closed forms and
- // compounds whose surrounding syntax makes the unhyphenated form clear.
- const UNNECESSARY_HYPHENATION = [
- // Welded open noun phrases reported in #107. Match the complete phrase so
- // a project-specific spelling of the pair in another role is not swept in.
- { pattern: /\bresearch-impact\s+aggregat(?:or|ion)s?\b/g, suggestion: (match) => match.replace('research-impact', 'research impact') },
- { pattern: /\bdata-source\s+strateg(?:y|ies)\b/g, suggestion: (match) => match.replace('data-source', 'data source') },
- { pattern: /\bPython-package\s+usage\b/g, suggestion: (match) => match.replace('Python-package', 'Python package') },
- { pattern: /\bRust-crate\s+usage\b/g, suggestion: (match) => match.replace('Rust-crate', 'Rust crate') },
- { pattern: /\bsingle-Project\s+Manifest\b/g, suggestion: (match) => match.replace('single-Project', 'single Project') },
- { pattern: /\btotal-downloads\s+figures?\b/g, suggestion: (match) => match.replace('total-downloads', 'total downloads') },
- { pattern: /\blife-sciences-native\s+citation\s+count\b/g, suggestion: 'citation count from a life sciences source' },
- // Compounds whose standard spelling is closed. Kept as a small curated
- // list rather than guessing that every noun-noun pair should close up.
- { pattern: /\bcode-base\b/g, suggestion: (match) => match.replace('-', '') },
- { pattern: /\bdata-set\b/g, suggestion: (match) => match.replace('-', '') },
- { pattern: /\btime-frame\b/g, suggestion: (match) => match.replace('-', '') },
- { pattern: /\broad-map\b/g, suggestion: (match) => match.replace('-', '') },
- // Attributive-only forms used adverbially or as nouns. The boundary after
- // real-time / long-term is intentionally narrow: "real-time analytics"
- // and "long-term plan" must not fire.
- {
- pattern: /\bin\s+real-time(?=\s*(?:[,.!?;:]|$)|\s+(?:(?:across|as|automatically|because|but|continuously|during|dynamically|every|for|from|immediately|instantly|on|simultaneously|through|throughout|until|via|when|while|with|without)\b))/gi,
- suggestion: 'in real time',
- },
- {
- pattern: /\b(?:for|over)\s+the\s+long-term(?=\s*(?:[,.!?;:]|$)|\s+(?:across|because|but|by|during|for|from|on|through|throughout|until|via|when|while|with|without)\b)/gi,
- suggestion: (match) => match.replace(/long-term/i, 'long term'),
- },
- {
- // Hyphen: ASCII, U+2010, U+2011, or the en dash U+2013 (#140).
- pattern: /\b(?:functions?|functioned|functioning|operates?|operated|operating|runs?|ran|running|works?|worked|working)\s+out[-‐‑–]of[-‐‑–]the[-‐‑–]box\b/gi,
- suggestion: (match) => match.replace(/out[-‐‑–]of[-‐‑–]the[-‐‑–]box/i, 'out of the box'),
- },
- ];
- // ═══ Helpers ═══════════════════════════════════════════════════════
- function tokenize(text) {
- return text.toLowerCase().match(/[\w'-]+/g) || [];
- }
- // Moving-average type-token ratio (MATTR; Covington & McFall 2010,
- // doi:10.1080/09296171003643098): the share of distinct tokens in each
- // run of `size` consecutive tokens, averaged over every such run. Plain
- // TTR falls as a text grows, because common words keep recurring while
- // new ones arrive more slowly, so a fixed threshold on it measures length.
- // The windowed mean does not drift with length, and on a text of exactly
- // `size` tokens it equals plain TTR.
- function movingAverageTTR(tokens, size) {
- const counts = new Map();
- let distinct = 0;
- const add = (token) => {
- const n = (counts.get(token) || 0) + 1;
- counts.set(token, n);
- if (n === 1) distinct += 1;
- };
- const drop = (token) => {
- const n = counts.get(token) - 1;
- if (n === 0) {
- counts.delete(token);
- distinct -= 1;
- } else {
- counts.set(token, n);
- }
- };
- for (let i = 0; i < size; i += 1) add(tokens[i]);
- let sum = distinct;
- for (let i = size; i < tokens.length; i += 1) {
- drop(tokens[i - size]);
- add(tokens[i]);
- sum += distinct;
- }
- return sum / ((tokens.length - size + 1) * size);
- }
- function countWords(text) {
- return (text.match(/\S+/g) || []).length;
- }
- function getParagraphs(text) {
- return text.split(/\n\s*\n/).filter(p => p.trim().length > 0);
- }
- // A Markdown definition entry: optional list marker, then a bold lead term
- // closed by a colon ("**Term:**" or "**Term**:") or followed by an em
- // dash, plus any wrapped continuation lines up to a blank line, list item,
- // bold lead, heading, or blockquote. Group 1 is the term. Used only to
- // keep glossary entries out of the sentence-length uniformity sample.
- const DEFINITION_ENTRY_RE = /^[ \t]*(?:(?:[-*+]|\d+[.)])[ \t]+)?\*\*([^*\n]+?):?\*\*(?:(?<=:\*\*)|[ \t]*[:—])[^\n]*(?:\n(?![ \t]*(?:$|[-*+][ \t]|\d+[.)][ \t]|\*\*|#|>))[^\n]*)*/gm;
- // A glossary is at least this many entries with distinct terms. A label
- // repeated as a lead ("**Note:**" on every line) is not a glossary term,
- // and its lines stay in the sample.
- const GLOSSARY_MIN_ENTRIES = 3;
- // Text with glossary entries replaced by sentence breaks, or null when the
- // text has no glossary.
- function withoutGlossaryEntries(text) {
- if (!text.includes('**')) return null;
- const entries = [...text.matchAll(DEFINITION_ENTRY_RE)];
- const counts = new Map();
- for (const m of entries) {
- const term = m[1].trim().toLowerCase();
- counts.set(term, (counts.get(term) || 0) + 1);
- }
- const unique = entries.filter((m) => counts.get(m[1].trim().toLowerCase()) === 1);
- if (unique.length < GLOSSARY_MIN_ENTRIES) return null;
- let out = '';
- let last = 0;
- for (const m of unique) {
- out += text.slice(last, m.index) + '.';
- last = m.index + m[0].length;
- }
- return out + text.slice(last);
- }
- function getSentences(text) {
- return text.split(/[.!?]+/).filter(s => s.trim().length > 5);
- }
- function matchPatterns(text, patterns, category, severity) {
- const issues = [];
- for (const pat of patterns) {
- const regex = new RegExp(pat.source, pat.flags);
- let match;
- while ((match = regex.exec(text)) !== null) {
- issues.push({
- type: category,
- text: match[0],
- index: match.index,
- severity,
- suggestion: null,
- });
- }
- }
- return issues;
- }
- // ═══ Main analysis ═════════════════════════════════════════════════
- // Upper bound for one scan. Above this we bail rather than running all
- // regex passes over a huge buffer — protects page perf on pasted novels.
- const MAX_WORDS = 10000;
- // V2 contract defaults so early-exit paths (Empty/tooShort/tooLong)
- // still return the same field shape a v2 consumer expects. Without
- // this, `result.document_classification === 'AI_ONLY'` is `undefined`
- // on edge inputs and fails open.
- // UNSCORED is returned on empty / too-short / too-long inputs where
- // we declined to score. Distinct from HUMAN_ONLY (which is a positive
- // classification) so a caller can't mistake a refused scan for a
- // confident human verdict — a 50k-word LLM-generated document is
- // not "human", it's just outside our scoring window.
- function buildV2Defaults(classification, confidence) {
- const probs = classification === 'HUMAN_ONLY'
- ? { human: 1, mixed: 0, ai: 0 }
- : classification === 'AI_ONLY'
- ? { human: 0, mixed: 0, ai: 1 }
- : { human: 0.333, mixed: 0.334, ai: 0.333 };
- return {
- document_classification: classification,
- class_probabilities: probs,
- confidence_category: confidence,
- highlight_sentence_for_ai: [],
- };
- }
- // The analysis contexts and source modes analyzeText() accepts. The CLIs
- // import these lists instead of keeping their own copies.
- const CONTEXT_MODES = Object.freeze(['general', 'technical', 'marketing', 'personal']);
- const SOURCE_MODES = Object.freeze(['plain', 'rendered-markdown']);
- function analyzeText(text, options = {}) {
- if (typeof text !== 'string') {
- throw new TypeError('analyzeText(text): argument must be a string');
- }
- const VALID_CONTEXT_MODES = new Set(CONTEXT_MODES);
- const requestedMode = options.contextMode === undefined ? 'general' : options.contextMode;
- const contextMode = VALID_CONTEXT_MODES.has(requestedMode) ? requestedMode : 'general';
- const contextModeFallback = requestedMode !== contextMode ? requestedMode : null;
- // Source mode controls which parts of a Markdown file count as prose.
- // Plain remains the compatibility default. Rendered Markdown masks only
- // initial YAML frontmatter and HTML comments; source-hygiene checks for
- // hidden TODO/placeholder comments remain available through plain mode.
- const VALID_SOURCE_MODES = new Set(SOURCE_MODES);
- const requestedSourceMode = options.sourceMode === undefined ? 'plain' : options.sourceMode;
- const sourceMode = VALID_SOURCE_MODES.has(requestedSourceMode) ? requestedSourceMode : 'plain';
- const sourceModeFallback = requestedSourceMode !== sourceMode ? requestedSourceMode : undefined;
- if (!text || text.trim().length === 0) {
- return {
- ...buildV2Defaults('UNSCORED', 'low'),
- score: 0,
- label: 'Empty',
- issues: [],
- stats: {
- wordCount: 0,
- contextMode,
- contextModeFallback,
- sourceMode,
- sourceModeFallback,
- maskedFrontmatter: 0,
- maskedHtmlComments: 0,
- ignoredRegions: 0,
- quotedLines: 0,
- maskedQuotes: 0,
- },
- tooShort: true,
- };
- }
- // Map each working-string code unit back to the caller's source. Every
- // length-changing preprocessing stage composes this map as it removes
- // characters, and results are translated before they leave the function.
- let sourceMap = identitySourceMap(text.length);
- // Context mode selects context-appropriate flagging. Accepted values:
- // 'general' (default) — full ruleset
- // 'technical' — skip title-case headers; individual prose-only rules
- // apply their own technical-context gates (only mode
- // that currently changes scoring)
- // 'marketing' — accepted; recorded in stats; scores same as general
- // 'personal' — accepted; recorded in stats; scores same as general
- // Invalid values fall back to 'general' with stats.contextModeFallback set.
- // Mode validation: an unknown string (e.g. typo "tecnical") would
- // otherwise silently downgrade to general-mode behavior. Coerce to
- // 'general' and surface the original value in stats for traceability.
- let maskedFrontmatter = 0;
- let maskedHtmlComments = 0;
- // Author-marked ignore regions go first, in every source mode, so no
- // later pass sees the excluded passage. Masking keeps the length, so the
- // source map needs no update.
- const ignored = maskIgnoreRegions(text);
- text = ignored.text;
- const { ignoredRegions } = ignored;
- if (sourceMode === 'rendered-markdown') {
- const rendered = maskRenderedMarkdown(text);
- text = rendered.text;
- maskedFrontmatter = rendered.maskedFrontmatter;
- maskedHtmlComments = rendered.maskedHtmlComments;
- }
- // Pre-pass: mask Markdown blockquotes before scoring. A human
- // reacting to AI text by quoting it shouldn't have the quoted block
- // counted against their own writing. A single `> ` line counts too
- // (#238). Masking instead of deleting keeps later issue and highlight
- // offsets aligned with the source file.
- const blockquotes = sourceMode === 'rendered-markdown'
- ? maskBlockquotes(text)
- : stripBlockquotes(text, sourceMap);
- text = blockquotes.text;
- if (blockquotes.sourceMap) sourceMap = blockquotes.sourceMap;
- const { quotedLines } = blockquotes;
- // Pre-pass: the inline half of the blockquote escape hatch. Words inside
- // a double-quoted span belong to whoever is quoted (#238). Masking runs
- // before normalization, as blockquotes do, so bypass characters inside a
- // quotation raise no normalization flag. Masking keeps the length, so
- // the source map needs no update. The smart-punctuation check below
- // reads the unmasked text, because the blanked span would otherwise
- // count as a double space.
- const unquotedText = normalizeText(text).text;
- const quotes = maskQuotedSpans(text);
- text = quotes.text;
- const { maskedQuotes } = quotes;
- // Pre-pass: strip bypass-trick chars before pattern matching so
- // "delve" with a Cyrillic 'е' still hits Tier 1. Compose the map while
- // deleting characters so later offsets still address the source.
- const norm = normalizeText(text, sourceMap);
- text = norm.text;
- sourceMap = norm.sourceMap;
- const wordCount = countWords(text);
- // Unsegmented-script check (GH-241): Chinese and Japanese carry no
- // inter-word spaces, so word segmentation cannot measure them — a long
- // document counts as one \S+ run and would misreport as "Too short",
- // while newline-wrapped lines each count as a word and would score
- // without segmentation. The check therefore runs before the word gate
- // and declines only when CJK characters dominate the non-whitespace
- // text, so short English documents with an incidental place name or
- // single Han character stay scorable. Unicode script properties cover
- // the complete Han, Hiragana, and Katakana repertoires (including
- // supplementary-plane and halfwidth forms); Hangul is space-separated
- // and segments fine, so it is excluded. Both counts use Unicode mode so
- // supplementary characters count as one code point rather than two UTF-16
- // code units.
- const cjkChars = (text.match(/[\p{Script_Extensions=Han}\p{Script_Extensions=Hiragana}\p{Script_Extensions=Katakana}]/gu) || []).length;
- const nonSpaceChars = (text.match(/\S/gu) || []).length;
- if (cjkChars > 0 && cjkChars * 2 >= nonSpaceChars) {
- return {
- ...buildV2Defaults('UNSCORED', 'low'),
- score: 0,
- label: 'Unsupported script',
- issues: [],
- stats: { wordCount, cjkChars, reason: 'unsegmented-script document: no inter-word spaces to count', contextMode, contextModeFallback, sourceMode, sourceModeFallback, maskedFrontmatter, maskedHtmlComments, ignoredRegions, quotedLines, maskedQuotes },
- unsupportedScript: true,
- };
- }
- if (wordCount < 10) {
- return {
- ...buildV2Defaults('UNSCORED', 'low'),
- score: 0,
- label: 'Too short',
- issues: [],
- stats: { wordCount, contextMode, contextModeFallback, sourceMode, sourceModeFallback, maskedFrontmatter, maskedHtmlComments, ignoredRegions, quotedLines, maskedQuotes },
- tooShort: true,
- };
- }
- if (wordCount > MAX_WORDS) {
- return {
- ...buildV2Defaults('UNSCORED', 'low'),
- score: 0,
- label: 'Text too long',
- issues: [],
- stats: { wordCount, contextMode, contextModeFallback, sourceMode, sourceModeFallback, maskedFrontmatter, maskedHtmlComments, ignoredRegions, quotedLines, maskedQuotes },
- tooLong: true,
- };
- }
- const tokens = tokenize(text);
- const paragraphs = getParagraphs(text);
- const sentences = getSentences(text);
- const issues = [];
- let rawScore = 0;
- // ── 1. Tier 1 words ──────────────────────────────────────────
- const tier1Found = new Set();
- for (const token of tokens) {
- if (contextMode === 'technical' && TECHNICAL_EXEMPT.has(token)) continue;
- if (Object.hasOwn(TIER1, token) && !tier1Found.has(token)) {
- tier1Found.add(token);
- issues.push({
- type: 'tier1',
- text: token,
- severity: 'high',
- suggestion: TIER1[token],
- });
- }
- }
- // Tier 1 multi-word phrases. Adds each distinct phrase (lowercased) to
- // `tier1Found` so the same phrase hit multiple times only produces one
- // issue — matches the downstream dedup behavior.
- for (const phrase of TIER1_PHRASES) {
- const regex = new RegExp(phrase.pattern.source, phrase.pattern.flags);
- let match;
- while ((match = regex.exec(text)) !== null) {
- const lower = match[0].toLowerCase();
- if (contextMode === 'technical' && TECHNICAL_EXEMPT.has(lower)) continue;
- if (phrase.skip && phrase.skip(text, match.index)) continue;
- if (tier1Found.has(lower)) continue;
- tier1Found.add(lower);
- issues.push({
- // Clarity-band entries are wordiness edits, not frequency evidence.
- // Same fix, weaker claim — see the Tier 1A/1B split in SKILL.md.
- type: phrase.clarity ? 'tier1-clarity' : 'tier1',
- text: match[0],
- severity: phrase.clarity ? 'medium' : 'high',
- suggestion: phrase.replace,
- });
- }
- }
- // ── 2. Tier 2 clusters ───────────────────────────────────────
- let tier2Clusters = 0;
- for (const para of paragraphs) {
- const paraTokens = tokenize(para);
- const found = [];
- const suggestions = {};
- for (const token of paraTokens) {
- if (contextMode === 'technical' && TECHNICAL_EXEMPT.has(token)) continue;
- if (Object.hasOwn(TIER2, token) && !found.includes(token)) {
- found.push(token);
- suggestions[token] = TIER2[token];
- }
- }
- for (const cond of TIER2_CONDITIONAL) {
- if (contextMode === 'technical' && TECHNICAL_EXEMPT.has(cond.word)) continue;
- if (!found.includes(cond.word) && cond.pattern.test(para)) {
- found.push(cond.word);
- suggestions[cond.word] = cond.suggestion;
- }
- }
- if (found.length >= 2) {
- tier2Clusters++;
- for (const word of found) {
- issues.push({
- type: 'tier2',
- text: word,
- severity: 'medium',
- suggestion: suggestions[word],
- });
- }
- }
- }
- // ── 3. Tier 3 density ────────────────────────────────────────
- const tier3Counts = {};
- for (const token of tokens) {
- const canonical = TIER3_LOOKUP.get(token);
- if (canonical) tier3Counts[canonical] = (tier3Counts[canonical] || 0) + 1;
- }
- // Flag at 3% of word count, but never below 3 occurrences. Previous
- // floor of 1 meant a 50-word text with one "significant" got flagged
- // as Tier 3 overuse, which was noise.
- const densityThreshold = Math.max(3, Math.floor(wordCount * 0.03));
- let tier3Flags = 0;
- for (const [word, count] of Object.entries(tier3Counts)) {
- if (count >= densityThreshold) {
- tier3Flags++;
- issues.push({
- type: 'tier3',
- text: `"${word}" x${count}`,
- severity: 'low',
- suggestion: `Overused (${count} times in ${wordCount} words)`,
- });
- }
- }
- // ── 4–21. Pattern categories ─────────────────────────────────
- issues.push(...matchPatterns(text, TRANSITIONS, 'transition', 'medium'));
- issues.push(...matchPatterns(text, CHATBOT_ARTIFACTS, 'chatbot', 'critical'));
- issues.push(...matchPatterns(text, SYCOPHANTIC, 'sycophantic', 'critical'));
- issues.push(...matchPatterns(text, FILLERS, 'filler', 'medium'));
- issues.push(...matchPatterns(text, GENERIC_CONCLUSIONS, 'generic-conclusion', 'medium'));
- issues.push(...matchPatterns(text, LETS_PATTERNS, 'lets-construction', 'medium'));
- issues.push(...matchPatterns(text, REASONING_ARTIFACTS, 'reasoning-artifact', 'critical'));
- issues.push(...matchPatterns(text, SIGNIFICANCE_INFLATION, 'significance-inflation', 'high'));
- issues.push(...matchPatterns(text, VAGUE_ATTRIBUTIONS, 'vague-attribution', 'critical'));
- issues.push(...matchPatterns(text, HOLLOW_INTENSIFIERS, 'hollow-intensifier', 'medium'));
- issues.push(...matchPatterns(text, EMOTIONAL_FLATLINE, 'emotional-flatline', 'low'));
- issues.push(...matchPatterns(text, LINGERING_ATTENTION, 'lingering-attention', 'medium'));
- issues.push(...matchPatterns(text, NOVELTY_INFLATION, 'novelty-inflation', 'medium'));
- issues.push(...matchPatterns(text, CUTOFF_DISCLAIMERS, 'cutoff-disclaimer', 'critical'));
- issues.push(...matchPatterns(text, AI_PLACEHOLDERS, 'ai-placeholder', 'critical'));
- issues.push(...matchPatterns(text, AI_CITATION_MARKUP, 'ai-citation-markup', 'critical'));
- issues.push(...matchPatterns(text, AI_UTM_SOURCE, 'ai-utm-source', 'critical'));
- issues.push(...matchPatterns(text, TEMPLATE_PHRASES, 'template-phrase', 'high'));
- issues.push(...matchPatterns(text, FALSE_CONCESSION, 'false-concession', 'medium'));
- issues.push(...matchPatterns(text, RHETORICAL_QUESTIONS, 'rhetorical-question', 'medium'));
- issues.push(...matchPatterns(text, HEDGE_STACK, 'hedge-stack', 'high'));
- issues.push(...matchPatterns(text, FUTURE_NARRATIVE, 'future-narrative', 'high'));
- issues.push(...matchPatterns(text, REAL_ACTUAL_INFLATION, 'real-actual-inflation', 'medium'));
- issues.push(...matchPatterns(text, SOCIAL_CTA_CLOSER, 'social-cta-closer', 'high'));
- issues.push(...matchPatterns(text, PERFORMED_INSIGHT, 'performed-insight', 'medium'));
- const stagedDiscoveryIssues = matchPatterns(text, STAGED_DISCOVERY, 'performed-insight', 'medium');
- issues.push(...stagedDiscoveryIssues);
- issues.push(...matchPatterns(text, NEGATION_CHAIN, 'negation-chain', 'high'));
- // Reveals flag alone; contrasts need a nearby frame. See NEGATIVE_PARALLELISM_*.
- issues.push(...negativeParallelismIssues(
- text,
- matchPatterns(text, NEGATIVE_PARALLELISM_REVEAL, 'negative-parallelism', 'high'),
- matchPatterns(text, NEGATIVE_PARALLELISM_CONTRAST, 'negative-parallelism', 'high'),
- ));
- issues.push(...matchPatterns(text, DEV_BLOG_BOILERPLATE, 'dev-blog-boilerplate', 'medium'));
- issues.push(...findUnnecessaryHyphenation(text));
- // ── Tier 1 v2: formulaic openers + parenthetical hedges ──────────
- issues.push(...matchPatterns(text, FORMULAIC_OPENERS, 'formulaic-opener', 'high'));
- issues.push(...matchPatterns(text, SPECULATIVE_OPENERS, 'speculative-opener', 'high'));
- issues.push(...matchPatterns(text, LAUNCH_INTROS, 'launch-intro', 'high'));
- issues.push(...matchPatterns(text, CROWD_CONTRAST, 'crowd-contrast', 'medium'));
- issues.push(...matchPatterns(text, FAKE_CASUAL_PROPS, 'fake-casual-prop', 'high'));
- issues.push(...matchPatterns(text, PARENTHETICAL_HEDGE, 'parenthetical-hedge', 'medium'));
- // Title-case headers — gated to marketing/personal/general modes
- // (technical mode legitimately uses Title Case section headers).
- if (contextMode !== 'technical') {
- const titleHits = matchPatterns(text, [TITLE_CASE_HEADER], 'title-case-header', 'medium');
- // Drop matches that look like proper-noun titles (single line, all
- // tokens capitalized incl. function words) — that's headline style,
- // not the AI-section-header tell which has mid-sentence "And".
- //
- // The prefix strip is load-bearing. matchPatterns reports match[0], so a
- // Markdown hit arrives as "## Terms Of Service" and `##` counts as a
- // token — silently lowering this guard from four content words to three
- // for headings only, which is exactly the class it exists to protect.
- // "## Terms Of Service", "## Bank Of America" and "## Table Of Contents"
- // all flagged as a result: ordinary human headings, on a detector whose
- // stated first priority is not firing on human writing.
- const filtered = titleHits.filter((h) => {
- const title = h.text.replace(MD_HEADING_PREFIX, '');
- const tokens = title.trim().split(/\s+/);
- if (tokens.length < 4) return false;
- // The function word must be MID-title, which is what the comment above
- // has always said and what the test never enforced. A leading "The"
- // satisfied a bare /\bThe\b/, so ordinary human headings flagged:
- // "## The New Security Landscape", "## The Microsoft Approach to
- // Identity", "### The Four Keys to a Successful and Secure Modern
- // Workplace". Measured across 81 files that provably predate LLMs
- // (2018-19 eBooks, 2020 posts): 13 false positives, every one opening
- // with "The", against zero on main.
- //
- // "## Benefits And Strategic Considerations" -- the actual tell, and
- // this rule's own fixture -- is untouched: its "And" is interior.
- return FUNCTION_WORD.test(tokens.slice(1).join(' '));
- });
- const fences = filtered.length ? fenceRanges(text) : [];
- issues.push(...filtered.filter((h) => !inFenceRange(fences, h.index)));
- }
- // ── Normalization-trigger flag ───────────────────────────────────
- // ZWSPs or homoglyphs in pasted prose are near-dispositive: humans
- // don't insert these into their own writing. Single roleplay marker
- // can be a false positive on Markdown emphasis (filtered to multi-
- // word inner already), so requires ≥2.
- if (norm.flags.zeroWidth > 0 || norm.flags.homoglyph >= 2) {
- issues.push({
- type: 'normalization-flag',
- text: `${norm.flags.zeroWidth} zero-width + ${norm.flags.homoglyph} homoglyph swap${norm.flags.homoglyph === 1 ? '' : 's'}`,
- severity: 'critical',
- suggestion: 'Text contains invisible/lookalike chars typical of AI-humanizer bypass tools. Re-type from your own keyboard.',
- });
- }
- if (norm.flags.roleplay >= 2) {
- issues.push({
- type: 'normalization-flag',
- text: `${norm.flags.roleplay} *roleplay-action* markers stripped`,
- severity: 'high',
- suggestion: 'Paired *action* markers are a chat-model artifact.',
- });
- }
- // Em dashes in list-item separator position — a bulleted or numbered
- // list item opening with a bolded lead term or markdown link, then the
- // dash ("- **Term** — desc", "- [label](url) — desc") — are
- // definition-list typography, not prose punctuation. Shared by the
- // smart-punct signature below and the em-dash frequency check (§22).
- // An optional parenthetical or inline-code span may sit between the bold
- // lead term and the dash — "- **Lingering-attention claims**
- // (`lingering-attention`) — the share-post frame…" is the same definition
- // typography as the bare form. Found by the self-scan (see PROOF.md, #67).
- // The leading indent is \s minus line terminators: a bare ^\s* under the
- // m flag rescanned every following blank line from each line start, so a
- // long run of U+2028/U+2029 (line terminators that \s also matches) took
- // quadratic time. Every match still ends at the same dash, so the count is
- // unchanged.
- const SEPARATOR_DASH_RE = /^[^\S\n\r\u2028\u2029]*(?:[-*+]|\d+[.)])\s+(?:\*\*[^*\n]+\*\*|\[[^\]\n]+\]\([^)\n]*\))(?:[ \t]*(?:\([^)\n]*\)|`[^`\n]+`))?[ \t]*—/gm;
- // The same typography in HTML source: an <li> that opens with a
- // <strong>/<b> lead term or an <a> link of at most five words, then the
- // dash (#351). A longer bold span is a clause, not a term. Only the
- // first child of the item counts, so a prose dash later in a list item,
- // or any dash outside <li>, still counts. `—` entities are never
- // counted as em dashes, so they need no carve-out.
- const SEPARATOR_DASH_HTML_RE = /^[ \t]*<li\b[^>\n]*>[ \t]*(?:<(strong|b)\b[^>\n]*>[ \t]*[^<\s]+(?:[ \t]+[^<\s]+){0,4}[ \t]*<\/\1>|<a\b[^>\n]*>[ \t]*[^<\s]+(?:[ \t]+[^<\s]+){0,4}[ \t]*<\/a>)(?:[ \t]*(?:\([^)\n]*\)|<code\b[^>\n]*>[^<\n]+<\/code>))?[ \t]*—/gim;
- function countSeparatorDashes(value) {
- return (value.match(SEPARATOR_DASH_RE) || []).length
- + (value.match(SEPARATOR_DASH_HTML_RE) || []).length
- + countVersionHeadingDashes(value);
- }
- // Keep-a-Changelog version headings (`## [3.21.0] — 2026-07-30`) join a
- // label to a value exactly as a list separator does. Deliberately narrow:
- // a bracketed or bare semver token, then a dash, then an ISO date, and
- // nothing else on the line. Ordinary prose dashes in headings still count,
- // because SKILL.md applies the em-dash rule to headings too.
- function countVersionHeadingDashes(value) {
- let count = 0;
- for (const rawLine of value.split(/\r\n|\n|\r/)) {
- let end = rawLine.length;
- while (end > 0 && (rawLine[end - 1] === ' ' || rawLine[end - 1] === '\t')) end -= 1;
- if (!/^\d{4}-\d{2}-\d{2}$/.test(rawLine.slice(Math.max(0, end - 10), end))) continue;
- let cursor = end - 10;
- while (cursor > 0 && (rawLine[cursor - 1] === ' ' || rawLine[cursor - 1] === '\t')) cursor -= 1;
- if (rawLine[cursor - 1] !== '\u2014') continue;
- cursor -= 1;
- while (cursor > 0 && (rawLine[cursor - 1] === ' ' || rawLine[cursor - 1] === '\t')) cursor -= 1;
- const prefix = rawLine.slice(0, cursor);
- const heading = /^#{1,6}[ \t]+/.exec(prefix);
- if (!heading) continue;
- let version = prefix.slice(heading[0].length);
- if (version.startsWith('[')) {
- if (!version.endsWith(']')) continue;
- version = version.slice(1, -1);
- } else if (version.endsWith(']')) {
- version = version.slice(0, -1);
- }
- if (version.includes(']')) continue;
- if (/^v?\d+\.\d+\.\d+/.test(version)) count += 1;
- }
- return count;
- }
- // ── Smart-punctuation co-occurrence signature ────────────────────
- // Curly punctuation + prose em dash + Oxford comma, with no sampled
- // spacing or common-contraction anomalies,
- // can also come from house style or editor settings. Keep the existing
- // narrow shape as a low-priority style check in plain source only.
- // Rendered Markdown deliberately preserves publication typography.
- // This finding has no weight or classifier corroborator. Separator-only
- // dashes do not complete the shape, and the spacing/contraction checks
- // below are limited heuristics, not proof that a document has no typos.
- if (sourceMode === 'plain') {
- const hasCurly = /[“”‘’]/.test(text);
- const totalEmDashes = (text.match(/—/g) || []).length;
- const separatorEmDashes = countSeparatorDashes(text);
- const hasEmDash = totalEmDashes > separatorEmDashes;
- const oxfordHit = text.match(/\b\w+,\s+\w+,\s+and\s+\w+/g);
- const hasOxford = (oxfordHit?.length || 0) >= 1;
- const doubleSpaces = (unquotedText.match(/[^.!?] +/g) || []).length;
- const missingApos = /\b(?:dont|wont|cant|isnt|wasnt|shouldnt|wouldnt|couldnt|youre|theyre|its\s+a\s+\w+ing)\b/i.test(text);
- const clean = doubleSpaces === 0 && !missingApos;
- const signals = [hasCurly, hasEmDash, hasOxford, clean].filter(Boolean).length;
- if (signals >= 4 && wordCount >= 80) {
- issues.push({
- type: 'smart-punct-signature',
- text: 'curly punctuation + prose em dash + Oxford comma',
- severity: 'low',
- suggestion: 'Check destination style before changing punctuation. Preserve house-style typography; this combination is not evidence of AI authorship.',
- });
- }
- }
- // ── Punctuation distribution mode ────────────────────────────────
- // Humans cluster trimodal across paragraphs (some paras heavy, some
- // light, some none). AI converges on a normal distribution. We can't
- // run a real modality test client-side, but we can flag the AI
- // signature: low variance of per-paragraph punctuation density.
- // Requires ≥4 paragraphs to be meaningful.
- if (paragraphs.length >= 4) {
- const densities = paragraphs.map((p) => {
- const words = (p.match(/\S+/g) || []).length;
- if (words < 5) return null;
- const puncts = (p.match(/[,;:—()]/g) || []).length;
- return puncts / words;
- }).filter((d) => d !== null);
- if (densities.length >= 4) {
- const mean = densities.reduce((a, b) => a + b, 0) / densities.length;
- const variance = densities.reduce((s, d) => s + (d - mean) ** 2, 0) / densities.length;
- const cv = mean > 0 ? Math.sqrt(variance) / mean : 0;
- // CV < 0.25 across paragraphs means each paragraph has the same
- // punctuation density — the AI signature. Humans usually swing
- // wider. Threshold derived from stylometry papers (arxiv 2507.00838).
- if (cv < 0.25 && mean >= 0.04) {
- issues.push({
- type: 'punct-distribution',
- text: `Punctuation density uniform across paragraphs (CV=${cv.toFixed(2)})`,
- severity: 'medium',
- suggestion: 'AI text holds punctuation density steady; human writers swing between dense and sparse paragraphs.',
- });
- }
- }
- }
- // ── Function-word trigram entropy ────────────────────────────────
- // POS-trigram entropy is the academic signal; function-word trigram
- // entropy approximates it without a tagger (function words ARE the
- // closed-class POS classes). AI text has lower entropy because LLM
- // sampling collapses onto a narrower set of grammatical templates.
- //
- // Method: extract function-word indicators per sentence, build
- // trigrams over the sequence, compute Shannon entropy. Bins below
- // threshold flag.
- if (wordCount >= 150) {
- const FUNC_WORDS = new Set([
- 'the','a','an','and','or','but','of','to','in','on','at','by','for','with',
- 'from','as','is','was','are','were','be','been','being','have','has','had',
- 'do','does','did','will','would','should','could','may','might','must','can',
- 'this','that','these','those','it','its','they','them','their','there','here',
- 'we','our','us','i','you','your','he','she','his','her','him','not','no','so',
- 'if','then','than','when','where','which','who','what','how','why','because',
- ]);
- const seq = tokens.map((t) => FUNC_WORDS.has(t) ? t : '_').filter((_, i, arr) => arr[i] !== '_' || (i > 0 && arr[i - 1] !== '_'));
- if (seq.length >= 50) {
- const trigrams = {};
- for (let i = 0; i < seq.length - 2; i++) {
- const tg = `${seq[i]}|${seq[i + 1]}|${seq[i + 2]}`;
- trigrams[tg] = (trigrams[tg] || 0) + 1;
- }
- const total = seq.length - 2;
- let entropy = 0;
- for (const c of Object.values(trigrams)) {
- const p = c / total;
- entropy -= p * Math.log2(p);
- }
- // Normalize by log2(distinct trigrams) so entropy ranges roughly
- // 0..1 and threshold is interpretable. Empirical threshold: human
- // prose ~0.85-0.95 normalized, AI prose ~0.70-0.82.
- const distinctCount = Object.keys(trigrams).length;
- const normalized = distinctCount > 1 ? entropy / Math.log2(distinctCount) : 1;
- if (normalized < 0.82 && total >= 50) {
- issues.push({
- type: 'fnword-trigram-entropy',
- text: `Function-word trigram entropy ${normalized.toFixed(2)} (low)`,
- severity: 'medium',
- suggestion: 'Grammatical structure is unusually repetitive. AI sampling collapses onto narrower templates than human writing.',
- });
- }
- // Degenerate case: single distinct trigram repeated across the
- // whole document is the strongest possible AI signal but the
- // normalized fallback returns 1.0 (= "fully human"), inverting
- // the signal. Catch it explicitly.
- if (distinctCount === 1 && total >= 50) {
- issues.push({
- type: 'fnword-trigram-entropy',
- text: 'Single function-word trigram repeated across document',
- severity: 'high',
- suggestion: 'Grammatical structure is fully degenerate — every clause uses the same function-word skeleton.',
- });
- }
- }
- }
- // ── Cross-paragraph burstiness ───────────────────────────────────
- // We already check within-paragraph sentence-length uniformity. AI
- // is also flat ACROSS paragraphs — every paragraph has roughly the
- // same sentence-length variance. Humans vary: terse paras next to
- // discursive paras. Measure variance of CV across paragraphs.
- if (paragraphs.length >= 4) {
- const cvs = paragraphs.map((p) => {
- const sents = getSentences(p);
- if (sents.length < 3) return null;
- const lens = sents.map(countWords);
- const m = lens.reduce((a, b) => a + b, 0) / lens.length;
- if (m === 0) return null;
- const v = lens.reduce((s, l) => s + (l - m) ** 2, 0) / lens.length;
- return Math.sqrt(v) / m;
- }).filter((c) => c !== null);
- if (cvs.length >= 4) {
- const cvMean = cvs.reduce((a, b) => a + b, 0) / cvs.length;
- const cvVar = cvs.reduce((s, c) => s + (c - cvMean) ** 2, 0) / cvs.length;
- const cvStd = Math.sqrt(cvVar);
- // Std-of-CV below 0.08 means every paragraph has roughly the same
- // internal rhythm — AI signature. Human prose typically swings
- // 0.15-0.40 across paragraphs of mixed purpose.
- if (cvStd < 0.08 && cvMean < 0.45) {
- issues.push({
- type: 'cross-para-burstiness',
- text: `Sentence-rhythm uniform across paragraphs (σCV=${cvStd.toFixed(2)})`,
- severity: 'medium',
- suggestion: 'Every paragraph has the same internal rhythm. Humans vary cadence between terse and discursive paragraphs.',
- });
- }
- }
- }
- // ── Tier 3 multi-word phrase density ─────────────────────────
- // Two complementary rules:
- // (a) Per-phrase density — same gating as single-word Tier 3: each
- // phrase fine alone, repetition is the tell. Threshold = 2.
- // (b) Cross-phrase clustering — ≥3 *distinct* boilerplate phrases
- // in one piece. LLMs varying their own boilerplate often use
- // each phrase only once but stack 5-10 across the text. The
- // per-phrase rule misses this; the cluster rule catches it.
- // Track non-overlapping match spans so a longer phrase swallowing a
- // shorter one (e.g., "designed for long-term sustainability" matches
- // both "designed for long-term" AND "long-term sustainability") only
- // contributes one distinct hit. Without dedup the cluster threshold
- // can be reached by a single sentence stacking overlapping regexes.
- const claimedSpans = [];
- function spanOverlaps(start, end) {
- for (const [s, e] of claimedSpans) {
- if (start < e && end > s) return true;
- }
- return false;
- }
- let distinctPhrasesHit = 0;
- for (const phrase of TIER3_PHRASES) {
- const regex = new RegExp(phrase.source, phrase.flags);
- const phraseSpans = [];
- let phraseMatch;
- while ((phraseMatch = regex.exec(text)) !== null) {
- const start = phraseMatch.index;
- const end = start + phraseMatch[0].length;
- if (!spanOverlaps(start, end)) {
- phraseSpans.push([start, end, phraseMatch[0]]);
- }
- }
- if (phraseSpans.length === 0) continue;
- for (const [s, e] of phraseSpans) claimedSpans.push([s, e]);
- distinctPhrasesHit++;
- if (phraseSpans.length >= 2) {
- issues.push({
- type: 'tier3-phrase',
- text: `"${phraseSpans[0][2].toLowerCase()}" x${phraseSpans.length}`,
- severity: 'medium',
- suggestion: `Boilerplate phrase repeated ${phraseSpans.length}× — replace at least one with specifics`,
- });
- }
- }
- if (distinctPhrasesHit >= 3) {
- issues.push({
- type: 'tier3-phrase-cluster',
- text: `${distinctPhrasesHit} distinct boilerplate phrases`,
- severity: 'high',
- suggestion: 'Several stock crypto/web3 phrases stacked in one piece. Rewrite around one specific claim or observation.',
- });
- }
- // ── Hashtag stuffing ─────────────────────────────────────────
- // 6+ hashtags in a single post is rare for thoughtful humans and
- // near-universal for LLM-generated social posts. Counted globally,
- // not per-paragraph, since the trailing hashtag block is the shape
- // we care about.
- // Match #tag at start of text or after any non-word char (whitespace,
- // punctuation, line breaks). URL fragments are already excluded
- // because the char immediately before `#` in a URL path is always
- // a word char (e.g. `example.com/page#section` — `e` before `#`).
- // Earlier char class `[\s\\]` had a literal backslash and silently
- // missed hashtags after sentence punctuation; an interim `[\s]` fix
- // on origin only caught whitespace-preceded tags.
- // Code is masked and non-tag `#` forms are subtracted first — see maskCode
- // and isSocialTag. Without them a changelog paragraph citing six issue
- // numbers, or a palette listing six hex colours, scored as a tag block.
- const hashtagMatches = [...maskCode(text).matchAll(/(?:^|\W)#(\w[\w-]*)/g)]
- .filter((m) => isSocialTag(m[1]));
- if (hashtagMatches.length >= 6) {
- issues.push({
- type: 'hashtag-stuff',
- text: `${hashtagMatches.length} hashtags`,
- severity: 'medium',
- suggestion: 'Cut to 2-3 specific tags or none. Long hashtag blocks read as bot output.',
- });
- }
- // ── Bullet list of bare noun phrases ─────────────────────────
- // ≥5 consecutive bullet items that are short (≤6 words) and contain
- // no finite-verb / modal token. Catches the "Stable mining efficiency
- // / Reliable pool connectivity / Optimized RandomX performance ..."
- // shape LLMs default to. Markdown bullets, escaped Markdown bullets,
- // unicode bullets, and dashes are all matched. Numbered lists are
- // excluded — those have a separate "numbered list inflation" rule.
- //
- // Note: verbRe covers auxiliaries and modals ("was", "will", "can",
- // etc.). Regular past-tense verbs ("fixed", "removed") are not
- // matched here; instead, the ≤6-word length gate excludes most
- // real-world changelog lines, which tend to read "fixed the X that
- // was doing Y" (>6 words). Short two-word action items ("* fixed
- // bug") would pass both gates — an acceptable trade-off to avoid
- // false-negative risk from adjectives ending in -ed ("skilled",
- // "advanced") that share the same surface form.
- const lines = text.split(/\r?\n/);
- const parseBullet = (line) => {
- let cursor = 0;
- while (cursor < line.length && /\s/.test(line[cursor])) cursor += 1;
- if (!['*', '-', '•', '+'].includes(line[cursor])) return null;
- cursor += 1;
- const spacingStart = cursor;
- while (cursor < line.length && /\s/.test(line[cursor])) cursor += 1;
- if (cursor === spacingStart) return null;
- if (cursor >= line.length) return cursor - spacingStart >= 2 ? '' : null;
- return line.slice(cursor).trim();
- };
- const verbRe = /\b(?:is|are|was|were|has|have|had|will|would|should|must|do|does|did|can|could|may|might|am|been|being)\b/i;
- const fenceRe = /^\s*(?:```|~~~)/;
- let run = [];
- let blankStreak = 0;
- let inFence = false;
- function flushRun() {
- if (run.length >= 5) {
- const bareNP = run.filter((it) => {
- const wc = (it.match(/\S+/g) || []).length;
- return wc > 0 && wc <= 6 && !verbRe.test(it);
- });
- if (bareNP.length >= 5 && bareNP.length / run.length >= 0.75) {
- issues.push({
- type: 'bullet-np-list',
- text: `${run.length}-item bullet list of bare noun phrases`,
- severity: 'high',
- suggestion: 'Convert to a prose paragraph or merge items. Long lists of bare adj+noun pairs read as AI scaffolding.',
- });
- }
- }
- run = [];
- blankStreak = 0;
- }
- for (const line of lines) {
- if (fenceRe.test(line)) {
- // Code-fence toggle. Bullets inside fences are CLI flag docs or
- // option dumps, not prose AI scaffolding — flush any prose run
- // we were tracking and skip until the fence closes.
- flushRun();
- inFence = !inFence;
- continue;
- }
- if (inFence) continue;
- const bullet = parseBullet(line);
- if (bullet !== null) {
- run.push(bullet);
- blankStreak = 0;
- } else if (line.trim() === '') {
- // A single blank line inside a list is normal Markdown spacing;
- // two or more blank lines break the run, since visually-disjoint
- // bullet sections shouldn't merge into one logical list.
- blankStreak++;
- if (blankStreak >= 2) flushRun();
- } else {
- flushRun();
- }
- }
- flushRun();
- // NOTE: "Wall-of-text replies" (SKILL.md) is deliberately NOT a
- // detector rule here. A first pass tried "reply-length text, >=4
- // sentences, zero newlines" as a structural gate — it broke the
- // "repeated Tier 1 phrase does not inflate score linearly" fixture
- // and, on reflection, would fire on any ordinary short paragraph
- // (a blog intro, a single-paragraph email) since "one paragraph with
- // no internal line break" is simply what continuous prose looks
- // like, not an AI-specific shape. The tell in SKILL.md depends on
- // knowing the text is conversational-reply register in the first
- // place, which the engine can't reliably infer from the bytes alone
- // (CONTRIBUTING.md: "a signal that fires on most normal prose is not
- // worth adding"). Left as an LLM-judgment rule; see CATEGORIES.md §C.
- // Confidence calibration is only flagged when it stacks (3+ instances).
- // Gating happens pre-dedup on raw match count, since that signals actual
- // stacking, not just vocabulary use.
- const confIssues = matchPatterns(text, CONFIDENCE_CALIBRATION, 'confidence-calibration', 'low');
- if (confIssues.length >= 3) issues.push(...confIssues);
- // ── 22. Em dash frequency ────────────────────────────────────
- // Match real em dashes, plus `--` only when surrounded by whitespace on at
- // least one side (skips CLI flags like --save-dev and YAML `---` blocks).
- // Separator-position em dashes (SEPARATOR_DASH_RE above) are excluded
- // from the rate. The list marker is required on purpose: a line-initial
- // "**Bold lead** — full sentence" outside a list is itself an AI tell
- // and still counts, as does a mid-sentence "**bold** — like this"
- // splice. Em dash only — the `--` substitute is never carved out.
- const rawEmDashCount = (text.match(/—|(?<=\s)--(?=\s|$)|(?<=^|\s)--(?=\s)/gm) || []).length;
- const separatorDashCount = countSeparatorDashes(text);
- const emDashCount = rawEmDashCount - separatorDashCount;
- const emDashRate = emDashCount / (wordCount / 1000);
- if (emDashRate > 1) {
- issues.push({
- type: 'em-dash',
- text: `${emDashCount} em dashes in ${wordCount} words`,
- severity: 'medium',
- suggestion: 'Replace with commas, periods, or rewrite',
- });
- }
- // ── 23. Sentence length uniformity ───────────────────────────
- // Glossary entries ("**Term:** gloss", "- **Term** — gloss"; three or
- // more distinct terms, wrapped lines included) are glossary typography: their lengths are parallel by convention, so
- // they say nothing about prose rhythm. They leave the sample here and
- // nowhere else; the five-sentence gate and the 0.25 threshold stay as
- // they are, so prose around a glossary is still measured (#351).
- const glosslessText = withoutGlossaryEntries(text);
- const rhythmSentences = glosslessText === null ? sentences : getSentences(glosslessText);
- if (rhythmSentences.length >= 5) {
- const lengths = rhythmSentences.map(s => countWords(s));
- const avg = lengths.reduce((a, b) => a + b, 0) / lengths.length;
- const variance = lengths.reduce((sum, l) => sum + Math.pow(l - avg, 2), 0) / lengths.length;
- const stdDev = Math.sqrt(variance);
- const cv = avg > 0 ? stdDev / avg : 0;
- if (cv < 0.25 && avg > 10) {
- issues.push({
- type: 'uniformity',
- text: `Sentence lengths cluster around ${Math.round(avg)} words (low variation)`,
- severity: 'medium',
- suggestion: 'Mix short punchy sentences with longer flowing ones',
- });
- }
- }
- // ── Type-token ratio (stylometric — vocabulary diversity) ────
- // TTR = distinct word types / total tokens, taken over each 200-token
- // window and averaged (movingAverageTTR). Whole-text TTR falls with
- // length whoever wrote the text: on the human documents in corpus/ its
- // median is 0.63 at 200 tokens, 0.37 at 2,000 and 0.28 at 6,000, so a
- // fixed threshold on it flagged every 6,000-word public-domain slice.
- // The windowed median stays between 0.62 and 0.64 at every length.
- // Within a window, human prose typically sits around 0.50–0.65 for
- // English; AI prose tends flatter (0.55–0.75 looks normal, but the
- // lower end of the *too-flat* tail at >=200 words is where the signal
- // lives — too FEW unique words for the length). This is the simplest
- // of the four stylometric signals identified in the May 2026 detection-
- // research review: no POS tagger required, no model, pure JS.
- //
- // Threshold tuning: flag only when the sample is large enough
- // that low TTR is meaningfully suspicious (>=200 tokens) AND TTR
- // is below 0.40 (very vocabulary-poor). Conservative on purpose;
- // false positives on short or topic-narrow human prose are easy
- // to trigger and would drown out other signals. The detector-
- // research lens flagged TTR as one of four stylometric add-ons.
- // One of the other three has since shipped in approximated form:
- // `cross-para-burstiness` covers sentence-length burstiness across
- // paragraphs. `fnword-trigram-entropy` is a related tagger-free
- // signal (it approximates POS-trigram entropy, not one of the
- // three). POS-bigram log-odds and function-word z-scores are
- // still TODO.
- if (tokens.length >= 200) {
- const diversity = movingAverageTTR(tokens, 200);
- if (diversity < 0.4) {
- issues.push({
- type: 'low-ttr',
- text: `Vocabulary diversity ${(diversity * 100).toFixed(1)}% (distinct words per 200-token window, ${tokens.length} tokens)`,
- severity: 'low',
- suggestion: 'Text reuses a narrow word set. Vary nouns and verbs deliberately, or check if the topic genuinely warrants the repetition.',
- });
- }
- }
- // ── 24. Paragraph length uniformity ──────────────────────────
- if (paragraphs.length >= 4) {
- const paraLengths = paragraphs.map(p => getSentences(p).length);
- const avg = paraLengths.reduce((a, b) => a + b, 0) / paraLengths.length;
- const allSimilar = paraLengths.every(l => Math.abs(l - avg) <= 1);
- if (allSimilar && avg >= 3) {
- issues.push({
- type: 'uniformity',
- text: `All paragraphs are ~${Math.round(avg)} sentences`,
- severity: 'low',
- suggestion: 'Vary paragraph length deliberately',
- });
- }
- }
- // ── 25. Bold overuse ─────────────────────────────────────────
- const boldMatches = text.match(/\*\*[^*]+\*\*/g) || [];
- if (boldMatches.length > 3) {
- issues.push({
- type: 'formatting',
- text: `${boldMatches.length} bold phrases`,
- severity: 'medium',
- suggestion: 'Strip bold from most; restructure to lead with key info',
- });
- }
- // ── Score from the deduped issue list ───────────────────────
- // Previously rawScore was accumulated inline per pattern hit, so
- // repeated hits of the same phrase (or overlapping matches) inflated
- // the score while the displayed issue list was deduplicated. That
- // produced the UX regression where a "heavy AI patterns" label sat
- // above a list of two items. Now the dedup runs first, then each
- // distinct issue contributes its category weight — so the number
- // reflects the same signals the user actually sees.
- // The staged-discovery phrase can contain the older "the most interesting
- // part" flatline match or sentence-initial "Turns out". Report the
- // enclosing signal once while leaving unrelated findings intact.
- const stagedDiscoverySpans = stagedDiscoveryIssues
- .map((issue) => ({ start: issue.index, end: issue.index + issue.text.length }));
- const nonOverlappingIssues = issues.filter((issue) =>
- (issue.type !== 'emotional-flatline' && issue.type !== 'performed-insight') ||
- stagedDiscoveryIssues.includes(issue) ||
- !Number.isInteger(issue.index) ||
- !stagedDiscoverySpans.some((span) =>
- issue.index >= span.start && issue.index + issue.text.length <= span.end
- )
- );
- const deduped = deduplicateIssues(nonOverlappingIssues);
- for (const issue of deduped) {
- rawScore += ISSUE_WEIGHTS[issue.type] ?? 2;
- }
- // Scale by text length: longer text gets more chances to trigger.
- const lengthFactor = Math.max(1, Math.log2(wordCount / 50));
- const normalizedScore = Math.min(100, Math.round(rawScore / lengthFactor));
- const label = getLabel(normalizedScore);
- // ── Sentence-region smoothing (HMM-style without an HMM) ─────────
- // Map each text-bearing issue back to its sentence indexes, then
- // merge adjacent flagged sentences into contiguous regions for UI
- // highlighting. Borrowed from GPTZero's sentence-highlighting model
- // — gives users "this paragraph is AI" rather than scattered hits.
- const regions = buildSentenceRegions(text, deduped, sourceMode === 'rendered-markdown');
- // Stats derived from the same deduped list so tier counts + patternCount
- // sum to `deduped.length`. Previously patternCount subtracted
- // `tier2Clusters` (a paragraph count) which produced inconsistent totals.
- const tier1Count = deduped.filter((i) => i.type === 'tier1').length;
- const tier2Count = deduped.filter((i) => i.type === 'tier2').length;
- const tier3Count = deduped.filter((i) => i.type === 'tier3').length;
- // ── Trinary classification (GPTZero-shaped) ──────────────────────
- // Decouples confidence from AI-proportion. Maps the 0-100 score plus
- // structural signals into HUMAN_ONLY / MIXED / AI_ONLY with a
- // confidence band. Thresholds are FN-biased: ambiguity routes to
- // MIXED, never AI_ONLY. Quote from GPTZero's design principle:
- // "biases the detector to prefer making less-harmful false-negative
- // errors over false-positive errors."
- // Dense-AI-vocab trifecta: ≥5 distinct tier1 hits + ≥2 tier2 cluster
- // paragraphs + ≥1 transition phrase, AND ≥150 words. Catches
- // saturated ChatGPT prose without firing on dense-jargon human
- // technical writing where the tier1 vocabulary (robust,
- // comprehensive, leverage, ecosystem) legitimately overlaps with
- // systems-programming idiom. Word-count gate prevents short ESL or
- // contrived adversarial sentences from tripping the corroborator.
- const tier1Distinct = new Set(deduped.filter((i) => i.type === 'tier1').map((i) => (i.text || '').toLowerCase())).size;
- const hasTier2Cluster = tier2Clusters >= 2;
- const hasTransition = deduped.some((i) => i.type === 'transition');
- const denseAIVocab = wordCount >= 150 && tier1Distinct >= 5 && hasTier2Cluster && hasTransition;
- const trinary = classifyTrinary({
- score: normalizedScore,
- issues: deduped,
- regions,
- normFlags: norm.flags,
- wordCount,
- denseAIVocab,
- });
- // Translate both dense issue indexes and half-open highlight boundaries.
- // Mapping the region end from its final retained code unit avoids pulling
- // a later removed roleplay marker into the highlighted source slice.
- remapFindingsToSource(deduped, regions, sourceMap);
- return {
- // Legacy fields preserved for existing callers.
- score: normalizedScore,
- label,
- issues: deduped,
- stats: {
- wordCount,
- tier1Count,
- tier2Count,
- tier2Clusters,
- tier3Count,
- tier3Flags,
- patternCount: deduped.length - tier1Count - tier2Count - tier3Count,
- contextMode,
- contextModeFallback,
- sourceMode,
- sourceModeFallback,
- maskedFrontmatter,
- maskedHtmlComments,
- ignoredRegions,
- normalization: norm.flags,
- quotedLines,
- maskedQuotes,
- unmappedHighlights: regions._unmapped ?? 0,
- denseAIVocab,
- tier1Distinct,
- },
- // Trinary API — shape mirrors GPTZero so integrators can swap.
- document_classification: trinary.classification,
- class_probabilities: trinary.probabilities,
- confidence_category: trinary.confidence,
- highlight_sentence_for_ai: regions,
- };
- }
- // ═══ Sentence regions + trinary classifier ═════════════════════════
- // Coarse sentence spans over the whole text, as [start, end) offsets.
- // Produces the same spans as the former /[^.!?]+[.!?]+|\S[^.!?]*$/g scan:
- // each span runs from the end of the previous one through the next run of
- // terminators, and a trailing fragment with no terminator starts at its
- // first non-space character. The regex version backtracked to the end of
- // the input at every position of a long terminator-free run, so a document
- // that ended in blank lines, or whose masked comments became whitespace,
- // cost O(n^2) (#235). This scan touches each character a bounded number
- // of times.
- const SENTENCE_TERMINATOR_RUN = /[.!?]+/g;
- const FIRST_NON_SPACE = /\S/g;
- function splitSentenceSpans(text) {
- const spans = [];
- const length = text.length;
- let pos = 0;
- while (pos < length) {
- SENTENCE_TERMINATOR_RUN.lastIndex = pos;
- const run = SENTENCE_TERMINATOR_RUN.exec(text);
- if (run === null) {
- FIRST_NON_SPACE.lastIndex = pos;
- const head = FIRST_NON_SPACE.exec(text);
- if (head !== null) spans.push([head.index, length]);
- break;
- }
- if (run.index === pos) {
- // Skip the entire bodyless run, not one character at a time: matching
- // every remaining suffix would make a long punctuation run quadratic.
- // If no later sentence terminator exists, the former regex's trailing
- // alternative starts at the LAST terminator of this run.
- const runEnd = pos + run[0].length;
- SENTENCE_TERMINATOR_RUN.lastIndex = runEnd;
- if (SENTENCE_TERMINATOR_RUN.exec(text) === null) {
- spans.push([runEnd - 1, length]);
- break;
- }
- pos = runEnd;
- continue;
- }
- const end = run.index + run[0].length;
- spans.push([pos, end]);
- pos = end;
- }
- return spans;
- }
- function buildSentenceRegions(text, issues, trimBoundaryWhitespace = false) {
- // Split text into sentences with source offsets preserved so the UI
- // can highlight spans accurately. Sentence boundaries are coarse
- // (.!?) — fine for highlighting, not for linguistic correctness.
- const sentences = [];
- for (const [spanStart, spanEnd] of splitSentenceSpans(text)) {
- const raw = text.slice(spanStart, spanEnd);
- const sentenceText = raw.trim();
- if (sentenceText.length < 4) continue;
- let start = spanStart;
- let end = spanEnd;
- if (trimBoundaryWhitespace) {
- // trimStart/trimEnd, not `\s*$`: that regex retries from every
- // position of a leading whitespace run and was quadratic on a span
- // that began with a masked comment block (#235).
- start += raw.length - raw.trimStart().length;
- end -= raw.length - raw.trimEnd().length;
- }
- sentences.push({ start, end, text: sentenceText });
- }
- if (sentences.length === 0) return [];
- // Map issue.text back to sentence indexes via substring search. Two
- // kinds of issue stay out of the AI-highlight regions. Summary signals
- // like "Punctuation density uniform across paragraphs" have no sentence
- // anchor — they contribute to the document-level signal but not to
- // highlights. Any category with authorship weight 0 is style-only by
- // definition, so it belongs in issues[] but never in a field reserved
- // for AI sentence highlights.
- // Filter by issue TYPE not text-regex: text-based filtering used to
- // drop legitimate phrase issues containing "across" / "density".
- const NON_HIGHLIGHT_TYPES = new Set([
- 'punct-distribution',
- 'cross-para-burstiness',
- 'fnword-trigram-entropy',
- 'smart-punct-signature',
- 'normalization-flag',
- 'uniformity',
- 'em-dash',
- 'formatting',
- 'tier3',
- 'tier3-phrase',
- 'tier3-phrase-cluster',
- 'hashtag-stuff',
- 'bullet-np-list',
- 'unnecessary-hyphenation',
- ]);
- const hits = sentences.map(() => ({ count: 0, weight: 0 }));
- const lowerText = text.toLowerCase();
- let unmappedHighlights = 0;
- for (const issue of issues) {
- if (!issue.text || issue.text.length > 200) continue;
- if (NON_HIGHLIGHT_TYPES.has(issue.type)) continue;
- if ((ISSUE_WEIGHTS[issue.type] ?? 2) === 0) continue;
- const needle = issue.text.toLowerCase();
- let idx = 0;
- let matched = false;
- while ((idx = lowerText.indexOf(needle, idx)) !== -1) {
- // Deduplicated verb issues must not highlight skipped noun matches.
- // Reuse the same predicate and retain all qualifying verb locations.
- if (issue.type === 'tier1-clarity' && needle === 'features' && featuresIsNoun(text, idx)) {
- idx += needle.length;
- continue;
- }
- matched = true;
- for (let i = 0; i < sentences.length; i++) {
- if (idx >= sentences[i].start && idx < sentences[i].end) {
- hits[i].count++;
- hits[i].weight += ISSUE_WEIGHTS[issue.type] ?? 2;
- break;
- }
- }
- idx += needle.length;
- }
- if (!matched) unmappedHighlights++;
- }
- // Window-merge contiguous flagged sentences. Allow 1 unflagged
- // sentence gap between two flagged ones (the "smoothing" — keeps
- // a single boring sentence from breaking what's clearly an AI
- // passage). A sentence is "flagged" if it has ≥1 hit.
- const regions = [];
- let cur = null;
- for (let i = 0; i < sentences.length; i++) {
- if (hits[i].count > 0) {
- if (cur === null) {
- cur = { startSentence: i, endSentence: i, start: sentences[i].start, end: sentences[i].end, hitCount: hits[i].count, weight: hits[i].weight };
- } else {
- cur.endSentence = i;
- cur.end = sentences[i].end;
- cur.hitCount += hits[i].count;
- cur.weight += hits[i].weight;
- }
- } else if (cur !== null) {
- // Allow one-sentence gap.
- const next = hits[i + 1];
- if (next && next.count > 0) {
- cur.endSentence = i;
- cur.end = sentences[i].end;
- continue;
- }
- regions.push(finalizeRegion(cur));
- cur = null;
- }
- }
- if (cur !== null) regions.push(finalizeRegion(cur));
- // Expose the unmapped-highlight count via a non-enumerable property
- // so the array length still reads naturally for consumers; the
- // analyzer pulls it into stats.unmappedHighlights for diagnostics.
- Object.defineProperty(regions, '_unmapped', { value: unmappedHighlights, enumerable: false });
- return regions;
- }
- function finalizeRegion(r) {
- // Map cumulative weight inside the region to a 0-1 score. Cap at 20
- // weight = 1.0 (matches Heavy threshold density).
- const score = Math.min(1, r.weight / 20);
- return {
- startSentence: r.startSentence,
- endSentence: r.endSentence,
- start: r.start,
- end: r.end,
- hitCount: r.hitCount,
- score: Math.round(score * 100) / 100,
- };
- }
- // FN-biased: false positives damage trust more than false negatives,
- // so MIXED is wide and AI_ONLY requires multiple signals. Quote from
- // GPTZero: "biases the detector to prefer making less-harmful
- // false-negative errors over false-positive errors."
- function classifyTrinary({ score, issues, regions, normFlags, wordCount, denseAIVocab }) {
- // Strong corroborators — each is near-dispositive on its own:
- // - cutoff-disclaimer (LLM self-identifies as an AI)
- // - reasoning-artifact + chatbot-artifact co-occurrence
- // - normalization-flag at threshold (≥2 ZWSP or homoglyphs).
- // Threshold parity prevents a single stray ZWSP in copy-paste
- // from Word/Notion from flipping to AI_ONLY at score 0.
- // - denseAIVocab: ≥4 distinct tier1 hits AND ≥1 tier2 cluster AND
- // ≥1 transition phrase — the trifecta that saturated ChatGPT
- // prose triggers without needing whitelisted stylometric hits.
- const hasCutoff = issues.some((i) => i.type === 'cutoff-disclaimer');
- const hasNormFlag = normFlags.zeroWidth >= 2 || normFlags.homoglyph >= 2;
- const hasReasoning = issues.some((i) => i.type === 'reasoning-artifact');
- const hasChatbot = issues.some((i) => i.type === 'chatbot');
- const strongCorrob =
- (hasCutoff ? 1 : 0) +
- (hasNormFlag ? 1 : 0) +
- (hasReasoning && hasChatbot ? 1 : 0) +
- (denseAIVocab ? 1 : 0);
- // Weak stylometric corroborators. The style-only smart-punct finding
- // must not alter classification, probabilities, or confidence even when
- // other findings already put the score near a classification boundary.
- const stylometricHits = ['punct-distribution', 'cross-para-burstiness', 'fnword-trigram-entropy']
- .filter((t) => issues.some((i) => i.type === t)).length;
- const weakCorrob = stylometricHits >= 2 ? 1 : 0;
- // Thresholds:
- // score < 15 with no strong → HUMAN_ONLY
- // strong ≥ 1 OR score ≥ 70 → AI_ONLY (lowered from 80; high
- // density of AI vocab is sufficient evidence)
- // score ≥ 40 with any corroborator → AI_ONLY
- // everything else with score ≥ 15 → MIXED
- const totalCorrob = strongCorrob + weakCorrob;
- let classification;
- if (score < 15 && strongCorrob === 0) classification = 'HUMAN_ONLY';
- else if (strongCorrob >= 1 || score >= 70) classification = 'AI_ONLY';
- else if (score >= 40 && totalCorrob >= 1) classification = 'AI_ONLY';
- else classification = 'MIXED';
- // Humanizer-flag escalation: presence of bypass-trick chars is
- // adversarial signal. If a normalization-flag fired we already
- // counted it in strongCorrob → AI_ONLY. Confidence also gets a
- // floor of 'medium' in that case (an adversary actively evading
- // detection should never read as low-confidence noise).
- // Soft probability distribution. Hand-tuned, not calibrated
- // against the labeled corpora the repo now samples
- // (`scripts/dataset-hc3.js`, `scripts/dataset-raid.js`). Largest
- // class is computed as `1 - others` after rounding to guarantee
- // sum=1 exactly. Sub-1% drift would otherwise hide in toFixed.
- const aiSoft = Math.min(0.97, score / 100 + totalCorrob * 0.06 + strongCorrob * 0.08);
- let p;
- if (classification === 'HUMAN_ONLY') p = { human: Math.max(0.6, 1 - aiSoft), mixed: Math.min(0.35, aiSoft * 0.8), ai: Math.min(0.1, aiSoft * 0.3) };
- else if (classification === 'AI_ONLY') p = { human: Math.max(0.02, 1 - aiSoft - 0.05), mixed: 0.1, ai: aiSoft };
- else p = { human: Math.max(0.15, 0.6 - aiSoft * 0.5), mixed: 0.5, ai: aiSoft * 0.7 };
- const rawSum = p.human + p.mixed + p.ai;
- p.human = +(p.human / rawSum).toFixed(3);
- p.mixed = +(p.mixed / rawSum).toFixed(3);
- // Assign ai as the remainder so the three values sum to exactly 1.
- // Clamp to >= 0 in case rounding pushes human+mixed above 1 (the
- // remainder would otherwise show as -0 or -0.001 — surfaces as a
- // negative percentage in any UI doing Math.round(p.ai * 100)).
- p.ai = Math.max(0, +(1 - p.human - p.mixed).toFixed(3));
- const probabilities = p;
- // Confidence band:
- // high — strongCorrob ≥ 2, OR cutoff-disclaimer, OR score < 8 (clean long doc)
- // medium — strongCorrob ≥ 1, OR score ≥ 45 with weak corroborator, OR score < 20
- // low — everything else
- let confidence;
- if (strongCorrob >= 2 || hasCutoff || (score < 8 && wordCount >= 100)) confidence = 'high';
- else if (strongCorrob >= 1 || (score >= 45 && weakCorrob >= 1) || score < 20) confidence = 'medium';
- else confidence = 'low';
- return { classification, probabilities, confidence };
- }
- function getLabel(score) {
- if (score === 0) return 'Clean';
- if (score <= 15) return 'Minimal AI signals';
- if (score <= 35) return 'Some AI patterns';
- if (score <= 60) return 'Moderate AI signals';
- if (score <= 80) return 'Strong AI signals';
- return 'Heavy AI patterns';
- }
- function getColor(score) {
- if (score <= 15) return '#44bb66';
- if (score <= 35) return '#88bb44';
- if (score <= 60) return '#ddaa00';
- if (score <= 80) return '#ff8833';
- return '#ff4444';
- }
- function deduplicateIssues(issues) {
- const seen = new Set();
- return issues.filter(issue => {
- const key = `${issue.type}:${issue.text.toLowerCase()}`;
- if (seen.has(key)) return false;
- seen.add(key);
- return true;
- });
- }
- // ─── Severity labels ──────────────────────────────────────────
- const SEVERITY_LABELS = {
- critical: 'P0',
- high: 'P1',
- medium: 'P2',
- low: 'P3',
- };
- const TYPE_LABELS = {
- 'tier1': 'AI vocabulary',
- 'tier1-clarity': 'Wordiness',
- 'tier2': 'Word cluster',
- 'tier3': 'Overused word',
- 'transition': 'AI transition',
- 'chatbot': 'Chatbot artifact',
- 'sycophantic': 'Sycophantic tone',
- 'filler': 'Filler phrase',
- 'generic-conclusion': 'Generic conclusion',
- 'lets-construction': '"Let\'s" opener',
- 'reasoning-artifact': 'Reasoning artifact',
- 'significance-inflation': 'Significance inflation',
- 'vague-attribution': 'Vague attribution',
- 'hollow-intensifier': 'Hollow intensifier',
- // Keep the public type stable for API consumers; the user-facing name now
- // describes the stock framing that the regexes actually match.
- 'emotional-flatline': 'Stock reaction framing',
- 'lingering-attention': 'Lingering-attention claim',
- 'novelty-inflation': 'Novelty inflation',
- 'cutoff-disclaimer': 'Cutoff disclaimer',
- 'template-phrase': 'Template phrase',
- 'false-concession': 'False concession',
- 'rhetorical-question': 'Rhetorical question',
- 'confidence-calibration': 'Confidence stacking',
- 'em-dash': 'Em dash overuse',
- 'uniformity': 'Rhythm uniformity',
- 'formatting': 'Formatting',
- 'tier3-phrase': 'Boilerplate phrase',
- 'tier3-phrase-cluster': 'Boilerplate cluster',
- 'hashtag-stuff': 'Hashtag stuffing',
- 'bullet-np-list': 'Bullet-NP list',
- 'hedge-stack': 'Hedge-stacked prediction',
- 'future-narrative': 'Generic future narrative',
- 'real-actual-inflation': '"Real/actual" inflation',
- 'social-cta-closer': 'Engagement-bait closer',
- 'formulaic-opener': 'Formulaic opener',
- 'speculative-opener': 'Speculative scenario opener',
- 'launch-intro': 'Launch-copy introduction',
- 'crowd-contrast': 'Dramatized crowd contrast',
- 'fake-casual-prop': 'Fake-casual prop',
- 'title-case-header': 'Title Case header',
- 'parenthetical-hedge': 'Parenthetical hedge',
- 'smart-punct-signature': 'Smart-punct signature',
- 'punct-distribution': 'Punctuation distribution',
- 'fnword-trigram-entropy': 'Grammar repetition',
- 'cross-para-burstiness': 'Cross-paragraph rhythm',
- 'normalization-flag': 'Bypass-trick chars',
- 'low-ttr': 'Low vocabulary diversity',
- 'ai-placeholder': 'Unfilled placeholder',
- 'ai-citation-markup': 'Chatbot citation markup leak',
- 'ai-utm-source': 'AI-tool URL parameter',
- 'unnecessary-hyphenation': 'Unnecessary hyphenation',
- 'performed-insight': 'Performed-insight phrase',
- 'negation-chain': 'Negation chain',
- 'negative-parallelism': 'Negative parallelism',
- 'dev-blog-boilerplate': 'Dev-blog boilerplate',
- };
- // The pattern tables as plain data, keyed by issue type, for tooling that
- // re-emits them in another format (scripts/build-vale.js generates the Vale
- // style in `vale/` from this). Regexes are `{ source, flags }` strings, not
- // the live objects, so no `lastIndex` state is shared. A `gate` names logic
- // analyzeText applies on top of the table, so the table alone does not
- // reproduce a gated finding. Function fields (the `features` noun check,
- // computed hyphenation fixes) cannot be data: they become a `gate` note or
- // are left out. Not used by analyzeText. Bump `schemaVersion` on a change
- // to this shape, not on a change to the patterns.
- const PATTERN_TABLES = (() => {
- const re = (r) => ({ source: r.source, flags: r.flags });
- const list = (patterns) => patterns.map(re);
- const freeze = (value) => {
- if (value && typeof value === 'object') {
- for (const v of Object.values(value)) freeze(v);
- Object.freeze(value);
- }
- return value;
- };
- const NP_GATE = 'Reported only when another negative-parallelism frame starts within two sentences in the same paragraph.';
- return freeze({
- schemaVersion: '0.1.0',
- types: {
- 'tier1': {
- words: Object.entries(TIER1).map(([word, replace]) => ({ word, replace })),
- patterns: TIER1_PHRASES.filter((p) => !p.clarity).map((p) => ({ ...re(p.pattern), replace: p.replace })),
- gate: "`words` match whole lowercase tokens split on [\\w'-]+, so hyphenated and possessive forms do not count. The `technical` context skips TECHNICAL_EXEMPT terms.",
- },
- 'tier1-clarity': {
- patterns: TIER1_PHRASES.filter((p) => p.clarity).map((p) => ({
- ...re(p.pattern),
- replace: p.replace,
- ...(p.skip ? { gate: 'Skipped where "features" reads as a noun (featuresIsNoun).' } : {}),
- })),
- },
- 'tier2': {
- words: Object.entries(TIER2).map(([word, replace]) => ({ word, replace })),
- patterns: TIER2_CONDITIONAL.map((c) => ({ ...re(c.pattern), word: c.word, replace: c.suggestion })),
- gate: 'Reported only when two or more distinct listed words share a paragraph. The `technical` context skips TECHNICAL_EXEMPT terms.',
- },
- 'tier3': {
- words: [...TIER3],
- gate: 'Reported only at max(3, floor(words * 0.03)) uses of one word; hyphenated words also match their dashless form.',
- },
- 'tier3-phrase': {
- patterns: list(TIER3_PHRASES),
- gate: 'Reported only at the second non-overlapping use of one phrase.',
- },
- 'tier3-phrase-cluster': {
- patterns: list(TIER3_PHRASES),
- gate: 'Reported only when three or more distinct phrases appear.',
- },
- 'transition': { patterns: list(TRANSITIONS) },
- 'chatbot': { patterns: list(CHATBOT_ARTIFACTS) },
- 'sycophantic': { patterns: list(SYCOPHANTIC) },
- 'filler': { patterns: list(FILLERS) },
- 'generic-conclusion': { patterns: list(GENERIC_CONCLUSIONS) },
- 'lets-construction': { patterns: list(LETS_PATTERNS) },
- 'reasoning-artifact': { patterns: list(REASONING_ARTIFACTS) },
- 'significance-inflation': { patterns: list(SIGNIFICANCE_INFLATION) },
- 'vague-attribution': { patterns: list(VAGUE_ATTRIBUTIONS) },
- 'hollow-intensifier': { patterns: list(HOLLOW_INTENSIFIERS) },
- 'emotional-flatline': { patterns: list(EMOTIONAL_FLATLINE) },
- 'lingering-attention': { patterns: list(LINGERING_ATTENTION) },
- 'novelty-inflation': { patterns: list(NOVELTY_INFLATION) },
- 'cutoff-disclaimer': { patterns: list(CUTOFF_DISCLAIMERS) },
- 'ai-placeholder': { patterns: list(AI_PLACEHOLDERS) },
- 'ai-citation-markup': { patterns: list(AI_CITATION_MARKUP) },
- 'ai-utm-source': { patterns: list(AI_UTM_SOURCE) },
- 'template-phrase': { patterns: list(TEMPLATE_PHRASES) },
- 'false-concession': { patterns: list(FALSE_CONCESSION) },
- 'rhetorical-question': { patterns: list(RHETORICAL_QUESTIONS) },
- 'hedge-stack': { patterns: list(HEDGE_STACK) },
- 'future-narrative': { patterns: list(FUTURE_NARRATIVE) },
- 'real-actual-inflation': { patterns: list(REAL_ACTUAL_INFLATION) },
- 'social-cta-closer': { patterns: list(SOCIAL_CTA_CLOSER) },
- 'performed-insight': { patterns: list([...PERFORMED_INSIGHT, ...STAGED_DISCOVERY]) },
- 'negation-chain': { patterns: list(NEGATION_CHAIN) },
- 'negative-parallelism': {
- patterns: [
- ...list(NEGATIVE_PARALLELISM_REVEAL),
- ...NEGATIVE_PARALLELISM_CONTRAST.map((r) => ({ ...re(r), gate: NP_GATE })),
- ],
- },
- 'dev-blog-boilerplate': { patterns: list(DEV_BLOG_BOILERPLATE) },
- 'unnecessary-hyphenation': {
- patterns: UNNECESSARY_HYPHENATION.map((e) => ({
- ...re(e.pattern),
- ...(typeof e.suggestion === 'string' ? { replace: e.suggestion } : {}),
- })),
- gate: 'Runs on text with file paths, package names and code identifiers masked (maskHyphenationProtected).',
- },
- 'formulaic-opener': { patterns: list(FORMULAIC_OPENERS) },
- 'speculative-opener': { patterns: list(SPECULATIVE_OPENERS) },
- 'launch-intro': { patterns: list(LAUNCH_INTROS) },
- 'crowd-contrast': { patterns: list(CROWD_CONTRAST) },
- 'fake-casual-prop': { patterns: list(FAKE_CASUAL_PROPS) },
- 'parenthetical-hedge': { patterns: list(PARENTHETICAL_HEDGE) },
- 'confidence-calibration': {
- patterns: list(CONFIDENCE_CALIBRATION),
- gate: 'Reported only when three or more matches occur in one text.',
- },
- },
- });
- })();
- return {
- analyzeText,
- normalizeText,
- getLabel,
- getColor,
- SEVERITY_LABELS,
- TYPE_LABELS,
- CONTEXT_MODES,
- SOURCE_MODES,
- PATTERN_TABLES,
- };
- })();
- if (typeof module !== 'undefined' && module.exports) {
- module.exports = AIDetector;
- }
Ratings and comments
Sign in to rate this skill.
Sign inFree to use. If it’s helped, pass it on.