Skip to content
Back to search

ontoship

MarketplaceWorks with Claude CodeGitHub

OntoShip — ontology-driven docs + spec-driven dev-flow for Claude Code.

Safety scan

A static text scan, not a guarantee. Select the badge to see what was scanned and every finding.

Checked today
Badge for your README
findsafeskills safety rating

Markdown

[![findsafeskills safety rating](/api/listings/vakovalskii--ontoship/badge.svg)](/listings/vakovalskii--ontoship)

HTML

<a href="/listings/vakovalskii--ontoship"><img src="/api/listings/vakovalskii--ontoship/badge.svg" alt="findsafeskills safety rating" /></a>

Install

Install instructions from the README

As written by the repo's author. findsafeskills has not verified these steps.

python3 skills/kb-search/gitmark.py index            # build .gitmark/index.db
python3 skills/kb-search/gitmark.py search "auth flow"
python3 skills/kb-search/gitmark.py map -o docs-map.html && open docs-map.html

Pure `python3` (≥3.7) with SQLite **FTS5**; the **trigram** tokenizer (SQLite ≥ 3.34) adds fuzzy/substring/non-Latin matching — detected automatically, degrades gracefully. Add `.gitmark/` to `.gitignore` — it's a build artifact.

What was scanned

The files the safety scan read, as written by the repository's author and shown as plain text. Lines with a finding are marked.

.claude-plugin/marketplace.json · Manifest · 14 lines

View this file on GitHub (read at the repository's current version: the scan did not record which commit it read)

This file has changed since it was scanned, so this is not exactly the text that was scanned, and no line is marked. The findings are listed with the safety scan above.

  1. {
  2. "name": "ontoship",
  3. "owner": {
  4. "name": "Valera Kovalskii"
  5. },
  6. "description": "OntoShip — ontology-driven docs + spec-driven dev-flow for Claude Code.",
  7. "plugins": [
  8. {
  9. "name": "gitmark",
  10. "source": ".",
  11. "description": "md+README+git knowledge base: FTS5 search (bm25 + trigram/fuzzy), HTML+graph generation, code-ontology linter."
  12. }
  13. ]
  14. }
README.md · README · 197 lines

View this file on GitHub (read at the repository's current version: the scan did not record which commit it read)

This file has changed since it was scanned, so this is not exactly the text that was scanned, and no line is marked. The findings are listed with the safety scan above.

  1. # OntoShip
  2. > **A project knowledge base is just `markdown` + a `README` index + `git`.** Search, the
  3. > link graph and the checks are all built from those files.
  4. ![Zooming into a real knowledge base: repo, folder, document, three entities](docs/img/microscope.gif)
  5. ▶ [Watch the full 60-second video](docs/media/ontoship-microscope.mp4): from the whole
  6. knowledge base down to a single line of text and back.
  7. ![OntoShip graph of a real project](docs/img/graph.png)
  8. *Above: the documentation graph of a live production project
  9. ([hub.neuraldeep.ru](https://hub.neuraldeep.ru/)), generated by `gitmark map` with gravity
  10. mode on. Colors are services. The team reads and edits this KB every day.*
  11. On that project today: **421** markdown docs, **568** links between them, **88** `README`
  12. indexes, **617** commits to the KB since March 2026. `gitmark index` rebuilds the whole
  13. thing in **1.5 s** with ~50 MB of RAM.
  14. ---
  15. ## The idea
  16. Markdown is the source of truth. You and your AI assistant read it directly, in any
  17. editor. Everything derived from it (the search index, this HTML overview, the link graph)
  18. is regenerated from the md, so git stays clean. No service, no database, no embeddings,
  19. no vendor. The entry point is `AGENTS.md`, which every agent reads; `CLAUDE.md` just
  20. imports it with `@AGENTS.md`. Every folder's `README.md` is its index.
  21. ```
  22. AGENTS.md ← entry point: header, rules, short tree of /docs
  23. CLAUDE.md ← one line: @AGENTS.md
  24. /docs
  25. README.md ← master index
  26. /services
  27. /<service>
  28. README.md ← always the folder index
  29. ```
  30. We have built this way with AI assistants for months. Agents and people want the same
  31. thing: plain md they can open anywhere, plus fast search so they open the right file
  32. instead of grepping blind.
  33. ## Under the microscope
  34. Zoom in on the knowledge base and every level is still something you can open in a text
  35. editor:
  36. | zoom | level | what it is |
  37. |---|---|---|
  38. | ×1 | the whole KB | a graph of md files linked to each other; agents and humans read the same thing |
  39. | ×10 | a folder | a `README.md` index plus the docs it points to |
  40. | ×100 | a document | plain markdown: a small frontmatter header, headings, links |
  41. | ×1 000 | the header | three entities: object, properties, links (see the ontology below) |
  42. | ×10 000 | the atom | one line of text, e.g. `node_type: runbook`. git diffs it, `gitmark`/`grep` find it, an agent reads it |
  43. ![Four zoom levels: folder, document, three entities, a line of text](docs/img/microscope-levels.png)
  44. No layer underneath needs a database, a schema or embeddings.
  45. ## What's in here
  46. **`gitmark`** is one self-contained Python-stdlib CLI with zero dependencies:
  47. | command | what |
  48. |---|---|
  49. | `index` | build a SQLite **FTS5** index of sections + frontmatter (title, description, tags, type, status, trust) and typed links |
  50. | `search` | ranked per file: section bm25 + title/description bm25 + trigram substring + fuzzy (typos, Russian morphology), fused with RRF; `--type/--status/--service` filters; `--json` |
  51. | `query` | structured questions over frontmatter, straight from md: `query type=decision status!=deprecated`, `query type=plan --count-by service`, `query trust=unverified 'generated.by~claude'` |
  52. | `map` | self-contained **HTML**: collapsible tree + rendered markdown + a force/radial **graph** of links |
  53. | `serve` | tiny local HTTP server to view the map |
  54. | `stat` | index/KB stats |
  55. | `lint` | *(optional)* ontology checks I1–I9: frontmatter, vocabularies, orphans, broken links, typed links to missing code, past `stale_after`, entry point (`AGENTS.md` + `@AGENTS.md`) |
  56. | `lint --baseline <ref>` | ratchet for a KB with debt: fails only if there are **more** findings than in `<ref>` |
  57. | `toc [--write]` | indexes that are **built, not written**: the `<!-- gitmark:toc -->` block of a folder README is rebuilt from the docs' `title`/`description` |
  58. | `secrets [files…]` | tokens, keys, passwords in the KB, value masked; exit 1 when found |
  59. | `attrs [--write]` | `.gitattributes` with `merge=union` for README indexes, so parallel branches never conflict on them |
  60. | `health` | KB scorecard: typed %, description %, typed links, orphans, trust tiers, stale, duplicate docs |
  61. | `version` | — |
  62. OntoShip ships the **`gitmark`** plugin (KB + dev-flow) as a Claude Code marketplace:
  63. three skills and five commands.
  64. | skill | what |
  65. |---|---|
  66. | `kb-search` | search the KB (FTS5) instead of grepping blind |
  67. | `kb-curate` | light ontology rules when adding/editing docs (types, frontmatter, typed links) |
  68. | `dev-flow` | the spec-driven loop to ship a feature: research → tasks → goal → **spec (md via `kb-curate`)** → isolated **git worktree** → implement → tests → independent review → dev-tests → prod-tests → ship (MR → `dev` → `main`). Feature to prod in ~40 min – 2 h. |
  69. | command | what |
  70. |---|---|
  71. | `/kb` | search the KB (FTS5) and answer from the top hits |
  72. | `/kb-map` | build the self-contained HTML graph of the KB and open it |
  73. | `/doc` | compose/update **one** KB doc following the ontology (wraps `kb-curate`) |
  74. | `/onto-doc` | build the **whole** KB: fans out kb-curate curator agents per area, then lint + index + map |
  75. | `/ship` | run the dev-flow on a feature/fix: research → … → ship (MR → dev → main) |
  76. ### What to write after a command
  77. The argument after the command is the prompt. The more specific it is, the better the result.
  78. **`/kb <query>`**: a topic, symbol, concept or question to find in the KB.
  79. - Good: `/kb how is the billing webhook signature verified`, `/kb tier rate limits`, `/kb sandbox isolation`.
  80. - Use concrete nouns, not full sentences. Cyrillic, typos and substrings are fine (trigram/fuzzy).
  81. - Bare `/kb` prints index stats and the syntax.
  82. **`/kb-map [output-path]`**: usually nothing (defaults to `docs-map.html`).
  83. - Optional path: `/kb-map build/graph.html`.
  84. **`/doc <topic / what to document>`**: name the thing to write up. The agent picks the
  85. node_type and folder, writes frontmatter + typed links, and indexes it.
  86. - Good: `/doc how the billing webhook verifies the YooKassa signature`, `/doc decision: drop Firecracker-per-session`.
  87. - It searches first and edits an existing doc instead of creating a duplicate.
  88. **`/onto-doc [scope]`**: bootstrap or rebuild the entire KB. Empty means the whole repo, or
  89. give it a scope (`/onto-doc services/api services/billing`, `/onto-doc only reference docs`).
  90. - Surveys the codebase, splits it into doc areas and dispatches one curator agent per area
  91. in parallel, then lints, reindexes and regenerates the graph.
  92. - Use it on a fresh repo to stand up docs/ from nothing, or to backfill coverage.
  93. **`/ship <what + why + done>`**: describe the change as fully as you can, the dev-flow is
  94. only as good as this description. Say what to do, why (the goal) and the done-criteria,
  95. and name the target service or file if you know it.
  96. - Good: `/ship add a 60 rpm rate-limit to /api/search; goal: stop abuse; done = HTTP 429 over the limit + a unit test + a note in docs/reference/limits.md`.
  97. - Weak: `/ship fix search`. The agent has to guess the goal and the done-criteria.
  98. - Add "deploy it" or "don't deploy" if you want to control the last step.
  99. > **Looking for `destructive-guard`?** The `PreToolUse` safety hook that intercepts
  100. > destructive commands (`rm -rf`, `git reset --hard`, `terraform destroy`, `DROP TABLE`…)
  101. > and turns them into a y/n confirmation now lives in its own repo:
  102. > **[github.com/vakovalskii/destructive-guard](https://github.com/vakovalskii/destructive-guard)**
  103. > · [landing + rules reference](https://vakovalskii.github.io/destructive-guard/).
  104. ## Quickstart
  105. ```bash
  106. python3 skills/kb-search/gitmark.py index # build .gitmark/index.db
  107. python3 skills/kb-search/gitmark.py search "auth flow"
  108. python3 skills/kb-search/gitmark.py map -o docs-map.html && open docs-map.html
  109. ```
  110. Pure `python3` (≥3.7) with SQLite **FTS5**. The **trigram** tokenizer (SQLite ≥ 3.34) adds
  111. fuzzy, substring and non-Latin matching; it is detected automatically, and search still
  112. works without it. Add `.gitmark/` to `.gitignore`, it's a build artifact.
  113. ## Install as a Claude Code plugin
  114. ```text
  115. /plugin marketplace add vakovalskii/ontoship
  116. /plugin install gitmark@ontoship
  117. ```
  118. Or copy `skills/` and `commands/` into your repo's `.claude/`. That keeps it project-local,
  119. no marketplace needed.
  120. ## Optional: the ontology
  121. The core needs no frontmatter at all: `index`/`search`/`map` work on raw markdown. If a
  122. team wants the KB to stay in shape as it grows, there's a light code ontology (inspired by
  123. [Palantir's Ontology](https://www.palantir.com/docs/foundry/ontology/overview)). Each doc
  124. gets a `node_type`, a few frontmatter properties and typed links, and `gitmark lint`
  125. checks them (no broken links, no orphans, every folder has a README index).
  126. See [`docs/ontology.md`](docs/ontology.md). The tool works the same if you skip it.
  127. The frontmatter is compatible with Google's [Open Knowledge Format](https://github.com/GoogleCloudPlatform/knowledge-catalog/blob/main/okf/SPEC.md):
  128. `type` works as `node_type`, and `description`, `generated`/`verified` (trust tier) and
  129. `stale_after` are understood by search, `query` and `lint`. On a 198-question benchmark
  130. over seven real project KBs, gitmark 0.2 puts the right doc first 69% of the time
  131. (grep: 30%, 0.1: 47%). A one-line `description` on each doc lifts it further,
  132. see [docs/decisions/okf-alignment.md](docs/decisions/okf-alignment.md) and [`bench/`](bench/README.md).
  133. Palantir models an organization as objects, their properties and the links between them.
  134. OntoShip takes the same three entities and applies them to knowledge about code:
  135. | Palantir Ontology | in OntoShip | example |
  136. |---|---|---|
  137. | **Object** (object type) | `node_type`: every `.md` is one object of one type | `runbook`, `service`, `gotcha`, `plan`, `decision`, `reference` |
  138. | **Properties** | YAML frontmatter, the object's columns | `title`, `service`, `status`, `updated`, `tags` |
  139. | **Links** (link types) | typed links under `links:`, including links into code | `documents`, `depends_on`, `supersedes`, `part_of` |
  140. The `kb-curate` skill holds the rules for writing them; `gitmark lint` checks the invariants
  141. (every doc has a type, values come from the vocabulary, no orphans, no broken links, every
  142. folder has a README, a superseded doc is marked deprecated).
  143. ![Three entities in one real frontmatter header](docs/img/ontology-3-entities.png)
  144. ## Why not a wiki, Notion or vector-DB memory
  145. You already read md and already use git. The CLI is one stdlib Python file you can read in
  146. an afternoon. It works offline and sends nothing to a SaaS. There are plenty of memory
  147. tools that are a service to run; this one is a few files in your repo, and the graph above
  148. shows it on a real product.
  149. ## License
  150. MIT, see [LICENSE](LICENSE).
skills/kb-search/gitmark.py · Bundled script · 1,640 lines
  1. #!/usr/bin/env python3
  2. """GitMark Memory Bank — CLI для базы знаний на чистом md + README + git.
  3. Source of truth = markdown. Всё производное (поисковый индекс, HTML-обзор, граф)
  4. регенерится из md → git остаётся чистым (.gitmark/ в .gitignore).
  5. Поиск: SQLite FTS5 — несколько каналов (bm25 по чанкам, bm25 по title/description/
  6. tags, trigram-подстрока, fuzzy 4-граммы), слитые RRF на уровне
  7. файлов. Frontmatter (тип, статус, trust, stale) — в индексе и в фильтрах.
  8. Чистый python stdlib (PyYAML — опционально, для полного YAML). Оффлайн.
  9. Команды:
  10. gitmark [--root .] index построить индекс
  11. gitmark search "<q>" [-k 8] [--json] [--type T] [--status S] [--service S]
  12. [--scope all|live|history]
  13. gitmark query [key=val key!=val key~sub key>val key? !key …] [--fields …]
  14. [--count-by key] [--json] структурный запрос по frontmatter
  15. gitmark map [-o docs/docs-map.html] self-contained HTML: дерево+рендер+граф
  16. gitmark serve [-p 8799] локальный http для просмотра HTML
  17. gitmark stat статистика индекса/БЗ
  18. gitmark lint [paths…] [--strict] [--json] [--baseline REF] проверить онтологию (I1–I9);
  19. с --baseline падает, только если замечаний стало больше, чем в REF (храповик)
  20. gitmark health [--json] скоркард БЗ: типизация, связи, trust, stale
  21. gitmark toc [--write] собрать блоки <!-- gitmark:toc --> в README-индексах
  22. gitmark secrets [files…] найти в БЗ токены/ключи/пароли (exit 1 — нашёл)
  23. gitmark attrs [--write] .gitattributes: README-индексы docs/ — merge=union
  24. gitmark version
  25. Совместимо с Open Knowledge Format (OKF v0.2): `type` = синоним `node_type`,
  26. `description` (синоним `summary`), `generated`/`verified` (trust tier), `stale_after`,
  27. `status: stable`, ссылки от корня (`/path.md`). `status: frozen` — снимок на дату, не правится.
  28. """
  29. from __future__ import annotations
  30. import argparse
  31. import datetime as _dt
  32. import hashlib
  33. import json
  34. import math
  35. import os
  36. import re
  37. import sqlite3
  38. import subprocess
  39. import sys
  40. import unicodedata
  41. from collections import Counter, defaultdict
  42. from pathlib import Path
  43. _yaml = False # PyYAML грузится лениво (поиску он не нужен)
  44. def _load_yaml():
  45. """PyYAML, если установлен (полный YAML); иначе None → мини-парсер."""
  46. global _yaml
  47. if _yaml is False:
  48. try:
  49. import yaml as _y
  50. _yaml = _y
  51. except Exception:
  52. _yaml = None
  53. return _yaml
  54. VERSION = "0.3.1"
  55. EXCLUDE_DIRS = {
  56. ".git", "node_modules", ".next", "dist", "build", "__pycache__",
  57. ".pytest_cache", "_vendor", ".venv", "venv", "vendor",
  58. ".gitmark", ".worktrees",
  59. }
  60. IGNORE_FILE = ".gitmarkignore"
  61. DB_REL = ".gitmark/index.db"
  62. HEAD_RE = re.compile(r"^(#{1,6})\s+(.+?)\s*#*$")
  63. H1_RE = re.compile(r"^#\s+(.+)$", re.MULTILINE)
  64. LINK_RE = re.compile(r"(?<!\!)\[[^\]]*\]\(([^)]+)\)")
  65. WORD_RE = re.compile(r"[\w\-]+", re.UNICODE)
  66. FM_RE = re.compile(r"\A?---[ \t]*\r?\n(.*?)\r?\n---[ \t]*(?:\r?\n|\Z)", re.DOTALL)
  67. # ─────────────────────────── discovery ───────────────────────────
  68. def repo_root(start: Path) -> Path:
  69. p = start.resolve()
  70. for cand in [p, *p.parents]:
  71. if (cand / ".git").exists():
  72. return cand
  73. return p
  74. def ignored_prefixes(root: Path) -> tuple:
  75. """.gitmarkignore в корне: путь на строку (папка или файл от корня), # — комментарий."""
  76. f = root / IGNORE_FILE
  77. if not f.exists():
  78. return ()
  79. return tuple(l.strip().strip("/") for l in f.read_text(encoding="utf-8").splitlines()
  80. if l.strip() and not l.lstrip().startswith("#"))
  81. def _git_md(root: Path) -> list | None:
  82. """*.md, которые видит git (tracked + untracked, без .gitignore). None — не git-репо."""
  83. try:
  84. r = subprocess.run(["git", "-C", str(root), "ls-files", "-co", "--exclude-standard",
  85. "-z", "--", "*.md"], capture_output=True, timeout=60)
  86. except (OSError, subprocess.SubprocessError):
  87. return None
  88. if r.returncode != 0:
  89. return None
  90. return [x for x in r.stdout.decode("utf-8", "replace").split("\0") if x]
  91. def iter_md(root: Path):
  92. """Markdown базы знаний. В git-репо — то, что видит git (вендорные/игнорируемые
  93. папки отпадают сами), иначе обход ФС. Плюс EXCLUDE_DIRS и .gitmarkignore."""
  94. skip = ignored_prefixes(root)
  95. rels = _git_md(root)
  96. if rels is None or os.environ.get("GITMARK_NO_GIT"):
  97. rels = [p.relative_to(root).as_posix() for p in root.rglob("*.md")]
  98. for rel in sorted(set(rels)):
  99. parts = rel.split("/")
  100. if any(part in EXCLUDE_DIRS for part in parts[:-1]):
  101. continue
  102. if any(rel == s or rel.startswith(s + "/") for s in skip):
  103. continue
  104. p = root / rel
  105. if p.is_file():
  106. yield p
  107. def area_of(rel: str) -> str:
  108. parts = rel.split("/")
  109. if parts[0] == "docs":
  110. # docs/services/<svc> разносим по сервисам (отдельная группа/цвет), прочее — docs/<sub>
  111. if len(parts) > 3 and parts[1] == "services":
  112. return "docs/services/" + parts[2]
  113. return "docs/" + parts[1] if len(parts) > 2 else "docs"
  114. if parts[0] == "services" and len(parts) > 1:
  115. return "services/" + parts[1]
  116. if len(parts) == 1:
  117. return "(root)"
  118. return parts[0]
  119. def title_of(text: str, rel: str) -> str:
  120. m = H1_RE.search(text)
  121. if m:
  122. return re.sub(r"[`*]", "", m.group(1)).strip()[:90]
  123. return Path(rel).name
  124. def chunk_md(text: str, line_offset: int = 0):
  125. """Чанкуем по заголовкам → (line_start, heading, body). line_offset — строки frontmatter."""
  126. lines = text.split("\n")
  127. chunks, cur = [], {"line": 1 + line_offset, "heading": "", "body": []}
  128. in_fence = False
  129. for i, ln in enumerate(lines, 1 + line_offset):
  130. if ln.lstrip().startswith("```"):
  131. in_fence = not in_fence
  132. m = None if in_fence else HEAD_RE.match(ln)
  133. if m:
  134. if cur["body"] or cur["heading"]:
  135. chunks.append((cur["line"], cur["heading"], "\n".join(cur["body"])))
  136. cur = {"line": i, "heading": m.group(2).strip(), "body": [ln]}
  137. else:
  138. cur["body"].append(ln)
  139. if any(x.strip() for x in cur["body"]) or cur["heading"]:
  140. chunks.append((cur["line"], cur["heading"], "\n".join(cur["body"])))
  141. return chunks
  142. def _nfc(s: str) -> str:
  143. return unicodedata.normalize("NFC", s)
  144. def resolve_link(src_rel: str, href: str, known: set) -> str | None:
  145. """md-ссылка → путь известного дока. `/x.md` — от корня (OKF), иначе от папки src."""
  146. import posixpath
  147. href = _nfc(href.split("#")[0].strip().strip("<>"))
  148. if not href or not href.endswith(".md") or href.startswith(("http", "mailto:")):
  149. return None
  150. src_dir = Path(_nfc(src_rel)).parent
  151. if href.startswith("/"):
  152. cands = [href.lstrip("/")]
  153. else:
  154. cands = [posixpath.normpath((src_dir / href).as_posix()), href.lstrip("./")]
  155. for c in cands:
  156. if c in known:
  157. return c
  158. base = Path(href).name
  159. hits = [r for r in known if r.endswith("/" + base) or r == base]
  160. return hits[0] if len(hits) == 1 else None
  161. # ─────────────────────────── frontmatter ───────────────────────────
  162. def split_frontmatter(text: str):
  163. """→ (yaml-текст | None, тело, число строк, занятых frontmatter)."""
  164. m = FM_RE.match(text)
  165. if not m:
  166. return None, text.lstrip(""), 0
  167. return m.group(1), text[m.end():], m.group(0).count("\n")
  168. def _plain(v):
  169. """PyYAML-значения → JSON-совместимые (даты → ISO-строки)."""
  170. if isinstance(v, dict):
  171. return {str(k): _plain(x) for k, x in v.items()}
  172. if isinstance(v, (list, tuple)):
  173. return [_plain(x) for x in v]
  174. if isinstance(v, (_dt.date, _dt.datetime)):
  175. return v.isoformat().replace("+00:00", "Z")
  176. return v
  177. def _strip_comment(line: str) -> str:
  178. """Убрать YAML-комментарий ` # …` вне кавычек."""
  179. q = None
  180. for i, ch in enumerate(line):
  181. if q:
  182. if ch == q:
  183. q = None
  184. elif ch in "'\"":
  185. q = ch
  186. elif ch == "#" and (i == 0 or line[i - 1] in " \t"):
  187. return line[:i]
  188. return line
  189. def _split_top(s: str) -> list:
  190. """Разбить по запятым верхнего уровня (не внутри [] {} и кавычек)."""
  191. out, depth, q, cur = [], 0, None, ""
  192. for ch in s:
  193. if q:
  194. q = None if ch == q else q
  195. elif ch in "'\"":
  196. q = ch
  197. elif ch in "[{":
  198. depth += 1
  199. elif ch in "]}":
  200. depth -= 1
  201. elif ch == "," and depth == 0:
  202. out.append(cur)
  203. cur = ""
  204. continue
  205. cur += ch
  206. if cur.strip():
  207. out.append(cur)
  208. return [x.strip() for x in out]
  209. _KV_RE = re.compile(r"^([^\s'\"\[{#-][^:]*?|\"[^\"]*\"|'[^']*'):(?:\s+(.*)|\s*)$")
  210. def _scalar(v: str):
  211. v = v.strip()
  212. if not v:
  213. return None
  214. if v[0] == "[" and v[-1] == "]":
  215. return [_scalar(x) for x in _split_top(v[1:-1])]
  216. if v[0] == "{" and v[-1] == "}":
  217. d = {}
  218. for part in _split_top(v[1:-1]):
  219. m = _KV_RE.match(part)
  220. if m:
  221. d[m.group(1).strip("'\"")] = _scalar(m.group(2) or "")
  222. return d
  223. if len(v) > 1 and v[0] == v[-1] and v[0] in "'\"":
  224. return v[1:-1]
  225. low = v.lower()
  226. if low in ("true", "false"):
  227. return low == "true"
  228. if low in ("null", "~"):
  229. return None
  230. if re.fullmatch(r"-?\d+", v):
  231. return int(v)
  232. return v
  233. def _mini_yaml(raw: str) -> dict:
  234. """Мини-парсер YAML (stdlib): скаляры, flow [..]/{..}, вложенные map/list по отступам,
  235. блочные скаляры | и >, комментарии. Покрывает то, что реально пишут во frontmatter."""
  236. root: dict = {}
  237. stack = [(-1, root)] # (мин. отступ элементов, контейнер)
  238. pending = None # (отступ, dict, ключ) — ключ без значения ждёт блок
  239. block = None # (отступ ключа, dict, ключ, style, [строки]) — | или >
  240. for line in raw.split("\n"):
  241. if block:
  242. ind = len(line) - len(line.lstrip(" \t"))
  243. if not line.strip() or ind > block[0]:
  244. block[4].append(line.strip())
  245. continue
  246. sep = "\n" if block[3] == "|" else " "
  247. block[1][block[2]] = sep.join(block[4]).strip()
  248. block = None
  249. line = _strip_comment(line).rstrip()
  250. if not line.strip():
  251. continue
  252. ind = len(line) - len(line.lstrip(" \t"))
  253. s = line.strip()
  254. is_item = s == "-" or s.startswith("- ")
  255. if pending:
  256. pind, pd, pkey = pending
  257. pending = None
  258. if ind > pind or (is_item and ind == pind):
  259. cont = [] if is_item else {}
  260. pd[pkey] = cont
  261. stack.append((ind, cont))
  262. while len(stack) > 1:
  263. mind, cont = stack[-1]
  264. if ind < mind or (ind == mind and isinstance(cont, list) != is_item):
  265. stack.pop()
  266. else:
  267. break
  268. cont = stack[-1][1]
  269. if is_item and isinstance(cont, list):
  270. val = s[1:].strip()
  271. m = _KV_RE.match(val) if val and val[0] not in "[{'\"" else None
  272. if m: # "- key: v" — map внутри списка
  273. d = {}
  274. cont.append(d)
  275. stack.append((ind + 2, d))
  276. k, v = m.group(1).strip("'\""), m.group(2) or ""
  277. d[k] = _scalar(v) if v else None
  278. if not v:
  279. pending = (ind + 2, d, k)
  280. else:
  281. cont.append(_scalar(val))
  282. continue
  283. if not isinstance(cont, dict):
  284. continue
  285. m = _KV_RE.match(s)
  286. if not m:
  287. continue
  288. k, v = m.group(1).strip("'\""), (m.group(2) or "").strip()
  289. if v in ("|", ">", "|-", ">-", "|+", ">+"):
  290. block = (ind, cont, k, v[0], [])
  291. cont[k] = ""
  292. elif not v:
  293. cont[k] = None
  294. pending = (ind, cont, k)
  295. else:
  296. cont[k] = _scalar(v)
  297. if block:
  298. block[1][block[2]] = ("\n" if block[3] == "|" else " ").join(block[4]).strip()
  299. return root
  300. def parse_frontmatter(text: str) -> dict | None:
  301. """YAML-frontmatter → dict (None — нет frontmatter). PyYAML если есть, иначе мини-парсер;
  302. мини-парсер же — фолбэк на «почти-YAML», который PyYAML не принимает (`title: a: b`)."""
  303. raw, _, _ = split_frontmatter(text)
  304. if raw is None:
  305. return None
  306. y = _load_yaml()
  307. if y is not None:
  308. try:
  309. data = y.safe_load(raw)
  310. if isinstance(data, dict):
  311. return _plain(data)
  312. except Exception:
  313. pass
  314. return _mini_yaml(raw)
  315. def as_list(v) -> list:
  316. if v is None or v == "":
  317. return []
  318. if isinstance(v, list):
  319. return [x for x in v if x is not None and x != ""]
  320. return [v]
  321. def trust_tier(fm: dict) -> str:
  322. """OKF §5.3: нет verified → unverified; только машины → machine; есть human: → human."""
  323. ver = (fm or {}).get("verified")
  324. if not ver:
  325. return "unverified"
  326. evs = ver if isinstance(ver, list) else [ver]
  327. bys = [str(e.get("by", "")) if isinstance(e, dict) else str(e) for e in evs]
  328. return "human" if any(b.startswith("human:") for b in bys) else "machine"
  329. def _parse_when(v) -> _dt.datetime | None:
  330. if not v:
  331. return None
  332. s = str(v).strip().replace("Z", "+00:00")
  333. try:
  334. d = _dt.datetime.fromisoformat(s)
  335. except ValueError:
  336. try:
  337. d = _dt.datetime.fromisoformat(s[:10])
  338. except ValueError:
  339. return None
  340. return d if d.tzinfo else d.replace(tzinfo=_dt.timezone.utc)
  341. def is_stale(fm: dict, now: _dt.datetime | None = None) -> bool:
  342. """OKF §5.5: stale, когда now >= stale_after."""
  343. when = _parse_when((fm or {}).get("stale_after"))
  344. return bool(when and (now or _dt.datetime.now(_dt.timezone.utc)) >= when)
  345. def node_type_of(fm: dict | None) -> str | None:
  346. """`node_type` (OntoShip) или `type` (OKF)."""
  347. fm = fm or {}
  348. nt = fm.get("node_type") or fm.get("type")
  349. return str(nt).strip() if nt else None
  350. def fm_links(fm: dict | None) -> list:
  351. """Типизированные связи из `links:` → [(link_type, target), …]."""
  352. links = (fm or {}).get("links")
  353. if not isinstance(links, dict):
  354. return []
  355. return [(str(lt), str(t).strip()) for lt, ts in links.items()
  356. for t in as_list(ts) if str(t).strip()]
  357. def load_doc(root: Path, p: Path) -> dict | None:
  358. """Прочитать md → {rel, text, body, offset, fm, meta}. None — не читается."""
  359. rel = p.relative_to(root).as_posix()
  360. try:
  361. text = p.read_text(encoding="utf-8", errors="replace").replace("\r\n", "\n")
  362. except OSError:
  363. return None
  364. _, body, offset = split_frontmatter(text)
  365. fm = parse_frontmatter(text) or {}
  366. gen = fm.get("generated") if isinstance(fm.get("generated"), dict) else {}
  367. meta = {
  368. "node_type": node_type_of(fm) or "",
  369. "title": str(fm.get("title") or title_of(body, rel)),
  370. "description": str(fm.get("description") or fm.get("summary") or ""), # summary — синоним
  371. "tags": [str(x) for x in as_list(fm.get("tags"))],
  372. "aliases": [str(x) for x in as_list(fm.get("aliases"))],
  373. "status": str(fm.get("status") or ""),
  374. "service": [x.strip() for v in as_list(fm.get("service")) for x in str(v).split(",") if x.strip()],
  375. "updated": str(fm.get("updated") or gen.get("at") or ""), # OKF: generated.at
  376. "trust": trust_tier(fm),
  377. "stale": is_stale(fm),
  378. }
  379. return {"rel": rel, "text": text, "body": body, "offset": offset,
  380. "fm": fm if text.startswith(("---", "---")) else None, "meta": meta}
  381. def load_docs(root: Path) -> dict:
  382. out = {}
  383. for p in iter_md(root):
  384. d = load_doc(root, p)
  385. if d:
  386. out[_nfc(d["rel"])] = d
  387. return out
  388. def doc_edges(d: dict, known: set) -> list:
  389. """Все связи дока: (dst, link_type, kind). kind: doc|code|url|missing."""
  390. rel, out, seen = d["rel"], [], set()
  391. for lt, t in fm_links(d["fm"]):
  392. dst = resolve_link(rel, t, known)
  393. if dst:
  394. kind = "doc"
  395. elif t.startswith(("http://", "https://")):
  396. dst, kind = t, "url"
  397. else:
  398. dst, kind = t, "code"
  399. if (dst, lt) not in seen:
  400. seen.add((dst, lt))
  401. out.append((dst, lt, kind))
  402. for href in LINK_RE.findall(strip_code(d["body"])):
  403. dst = resolve_link(rel, href.split()[0] if href.split() else href, known)
  404. if dst and dst != rel and not any(o[0] == dst for o in out):
  405. out.append((dst, "relates_to", "doc"))
  406. return out
  407. # ─────────────────────────── index ───────────────────────────
  408. def _has_trigram(con) -> bool:
  409. try:
  410. con.execute("CREATE VIRTUAL TABLE _tri_probe USING fts5(x, tokenize='trigram')")
  411. con.execute("DROP TABLE _tri_probe")
  412. return True
  413. except sqlite3.OperationalError:
  414. return False
  415. def cmd_index(root: Path, force: bool = False) -> dict:
  416. """Индекс — кэш: всегда пересобирается целиком (быстро; схема меняется между версиями)."""
  417. db = root / DB_REL
  418. db.parent.mkdir(parents=True, exist_ok=True)
  419. if db.exists():
  420. db.unlink()
  421. con = sqlite3.connect(db)
  422. tok = "tokenize='unicode61 remove_diacritics 2'"
  423. try:
  424. con.execute(f"CREATE VIRTUAL TABLE fts USING fts5(path UNINDEXED, lineno UNINDEXED, heading, body, {tok})")
  425. con.execute(f"CREATE VIRTUAL TABLE dfts USING fts5(path UNINDEXED, title, meta, {tok})")
  426. except sqlite3.OperationalError as e:
  427. print(f"ОШИБКА: SQLite без FTS5 ({e}). Нужен python с FTS5.", file=sys.stderr)
  428. sys.exit(2)
  429. has_tri = _has_trigram(con)
  430. if has_tri:
  431. con.execute("CREATE VIRTUAL TABLE tri USING fts5(path UNINDEXED, lineno UNINDEXED, body, tokenize='trigram')")
  432. con.execute("CREATE TABLE files(path TEXT PRIMARY KEY, title TEXT, area TEXT, size INT, chunks INT, "
  433. "node_type TEXT, status TEXT, service TEXT, description TEXT, updated TEXT, "
  434. "trust TEXT, stale INT, fm TEXT)")
  435. con.execute("CREATE TABLE links(src TEXT, dst TEXT, type TEXT, kind TEXT)")
  436. con.execute("CREATE TABLE meta(k TEXT PRIMARY KEY, v TEXT)")
  437. docs = load_docs(root)
  438. known = set(docs)
  439. nchunks = 0
  440. for rel, d in docs.items():
  441. m = d["meta"]
  442. chs = chunk_md(d["body"], d["offset"])
  443. for line, heading, body in chs:
  444. con.execute("INSERT INTO fts VALUES(?,?,?,?)", (rel, str(line), heading, body))
  445. if has_tri:
  446. con.execute("INSERT INTO tri VALUES(?,?,?)", (rel, str(line), heading + "\n" + body))
  447. nchunks += len(chs)
  448. path_words = " ".join(re.split(r"[/_.\-]+", rel.rsplit(".", 1)[0]))
  449. con.execute("INSERT INTO dfts VALUES(?,?,?)", (
  450. rel, m["title"],
  451. " ".join([m["description"], " ".join(m["tags"]), " ".join(m["aliases"]), path_words])))
  452. con.execute("INSERT INTO files VALUES(?,?,?,?,?,?,?,?,?,?,?,?,?)", (
  453. rel, m["title"], area_of(rel), len(d["text"].encode("utf-8")), len(chs),
  454. m["node_type"], m["status"], ",".join(m["service"]), m["description"], m["updated"],
  455. m["trust"], int(m["stale"]), json.dumps(d["fm"], ensure_ascii=False, default=str)))
  456. for dst, lt, kind in doc_edges(d, known):
  457. con.execute("INSERT INTO links VALUES(?,?,?,?)", (rel, dst, lt, kind))
  458. con.execute("INSERT OR REPLACE INTO meta VALUES('trigram', ?)", ("1" if has_tri else "0",))
  459. con.execute("INSERT OR REPLACE INTO meta VALUES('version', ?)", (VERSION,))
  460. con.commit()
  461. n_links = con.execute("SELECT count(*) FROM links WHERE kind='doc'").fetchone()[0]
  462. con.close()
  463. return {"files": len(docs), "chunks": nchunks, "links": n_links,
  464. "trigram": has_tri, "db": str(db.relative_to(root))}
  465. # ─────────────────────────── search ───────────────────────────
  466. STOPWORDS = set("""
  467. и в во не что он на я с со как а то все она так его но да ты к у же вы за бы по только ее её мне
  468. было вот от меня еще ещё нет о об из ему теперь когда ли если уже или ни быть был него до вас
  469. опять уж вам ведь там потом себя ничего ей может они тут где есть надо ней для мы тебя их чем
  470. была сам чтоб без чего раз тоже себе под будет ж тогда кто этот того потому этого какой какая
  471. какие каким каких какую здесь этом один мой тем чтобы нее сейчас были куда зачем всех можно при
  472. после над больше тот через эти нас про всего них много три эту эта это моя свою этой перед лучше
  473. том нельзя такой им более всегда между почему нужно надо сколько чтобы ли как-то который которая
  474. которые если именно сделать делать
  475. a an the and or but if of to in on at by for with from as is are was were be been being do does
  476. did how what where which who whom why when can could should would will shall may might must i we
  477. you he she it they our your my me us this that these those there here into about than then so not
  478. no have has had get got its it's any some all each just also
  479. """.split())
  480. _RU_END = sorted("""ями ами иями ией иях ях ах ов ев ей ий ый ой ая яя ое ее ые ие ых их ого его
  481. ому ему ыми ими ом ем ам ям ую юю ть ться ется ются ет ют ит ят ил ила ило или ешь ишь ете ите ся
  482. ость ости остью ения ение ений ением ениям ированный ировать ирование ующий ующая ующие а я о е ы
  483. и у ю ь й""".split(), key=len, reverse=True)
  484. _EN_END = sorted("ations ation ings ing ments ment ness ies ied ers er es ed ly s".split(),
  485. key=len, reverse=True)
  486. _CYR = re.compile(r"[а-яё]")
  487. def stem(w: str) -> str:
  488. """Лёгкий стеммер ru/en: срезать одно окончание, оставив основу ≥4 символов.
  489. Запрос ищется префиксом основы — «деплоймента» найдёт «деплоймент», «деплойменты»."""
  490. if len(w) <= 4:
  491. return w
  492. for e in (_RU_END if _CYR.search(w) else _EN_END):
  493. if w.endswith(e) and len(w) - len(e) >= 4:
  494. return w[:-len(e)]
  495. return w
  496. def query_terms(query: str) -> list:
  497. """Значимые термины запроса (lower, без стоп-слов; если всё стоп-слова — оставить всё)."""
  498. words = [w.lower().strip("-") for w in WORD_RE.findall(query)]
  499. words = [w for w in words if len(w) >= 2]
  500. sig = [w for w in words if w not in STOPWORDS]
  501. out, seen = [], set()
  502. for w in sig or words:
  503. if w not in seen:
  504. seen.add(w)
  505. out.append(w)
  506. return out
  507. def _fts_match_query(query: str) -> str:
  508. return " OR ".join(f'"{stem(t)}"*' for t in query_terms(query))
  509. def _fuzzy_phrases(s: str) -> list:
  510. """Фразы из пар соседних 3-грамм = 4-символьные подстроки запроса.
  511. Матч по 4-символьным окнам (а не одиночным 3-граммам) отсекает мусор: опечатка
  512. сохраняет длинные общие подстроки (`firecraker`↔`firecracker`: `fire`,`ecra`),
  513. а случайная строка делит с доками лишь разрозненные частые 3-граммы (`ent`,`non`).
  514. """
  515. wins, seen = [], set()
  516. for w in query_terms(s):
  517. if len(w) < 4:
  518. continue
  519. for i in range(len(w) - 3):
  520. win = w[i:i + 4] # 4-символьное окно; FTS5-trigram сам разложит
  521. if '"' in win or win in seen:
  522. continue
  523. seen.add(win)
  524. wins.append(win)
  525. return wins
  526. # Веса каналов в RRF и константа сглаживания. Подобраны на bench/ (см. bench/README.md).
  527. CHANNEL_W = {"chunk": 1.0, "doc": 0.5, "phrase": 0.6, "fuzzy": 0.5}
  528. RRF_K = 5
  529. PRIOR = {"typed": 1.3, "index": 0.7, "retired": 0.6, "stale": 0.85}
  530. HUB_NAMES = {"changelog.md", "log.md", "history.md"}
  531. # Исторические типы — датированные снимки (замысел до реализации, разовый аудит), которые
  532. # расходятся с фактом. `--scope live` их прячет, `--scope history` ищет только по ним.
  533. HISTORICAL_TYPES = {"plan", "report"} # хроники упоминают всё подряд
  534. if os.environ.get("GITMARK_TUNE"): # для bench/: подбор весов без правки кода
  535. _t = json.loads(os.environ["GITMARK_TUNE"])
  536. CHANNEL_W.update(_t.get("CHANNEL_W", {}))
  537. PRIOR.update(_t.get("PRIOR", {}))
  538. RRF_K = _t.get("RRF_K", RRF_K)
  539. def _allowed(con, types=None, statuses=None, services=None, scope="all") -> set | None:
  540. if not (types or statuses or services) and scope == "all":
  541. return None
  542. ok = set()
  543. for path, nt, st, svc in con.execute("SELECT path,node_type,status,service FROM files"):
  544. if types and nt not in types:
  545. continue
  546. if scope == "live" and nt in HISTORICAL_TYPES or scope == "history" and nt not in HISTORICAL_TYPES:
  547. continue
  548. if statuses and st not in statuses:
  549. continue
  550. if services and not set(svc.split(",")) & set(services):
  551. continue
  552. ok.add(path)
  553. return ok
  554. def cmd_search(root: Path, query: str, k: int = 8, types=None, statuses=None,
  555. services=None, scope: str = "all") -> list:
  556. """Поиск: каналы → ранги файлов → RRF → приоры (типизирован / index / deprecated / stale).
  557. Один результат на файл — лучший чанк (path:line + сниппет)."""
  558. db = root / DB_REL
  559. if not db.exists():
  560. raise SystemExit("Индекс не найден — запусти `gitmark index`")
  561. con = sqlite3.connect(db)
  562. try:
  563. con.execute("SELECT trust FROM files LIMIT 1")
  564. except sqlite3.OperationalError:
  565. raise SystemExit("Индекс старой версии — запусти `gitmark index`")
  566. has_tri = (con.execute("SELECT v FROM meta WHERE k='trigram'").fetchone() or ("0",))[0] == "1"
  567. allowed = _allowed(con, types, statuses, services, scope)
  568. ranks: dict = defaultdict(dict) # channel → {path: rank}
  569. best: dict = {} # path → лучший чанк {line, heading, snippet}
  570. N = max(60, k * 8)
  571. def take(channel, rows, with_chunk=True):
  572. r = 0
  573. for row in rows:
  574. path = row[0]
  575. if allowed is not None and path not in allowed:
  576. continue
  577. if path in ranks[channel]:
  578. continue
  579. r += 1
  580. ranks[channel][path] = r
  581. if with_chunk and path not in best:
  582. best[path] = {"line": int(row[1]), "heading": row[2], "snippet": " ".join(row[3].split())}
  583. bm_q = _fts_match_query(query)
  584. if bm_q:
  585. try: # (1) bm25 по чанкам; заголовок весит больше тела
  586. take("chunk", con.execute(
  587. "SELECT path,lineno,heading,snippet(fts,3,'»','«','…',14) FROM fts "
  588. "WHERE fts MATCH ? ORDER BY bm25(fts,0,0,2.0,1.0) LIMIT ?", (bm_q, N)))
  589. except sqlite3.OperationalError:
  590. pass
  591. try: # (2) bm25 по документу: title, description/tags/aliases/путь
  592. take("doc", con.execute(
  593. "SELECT path FROM dfts WHERE dfts MATCH ? ORDER BY bm25(dfts,0,3.0,1.5) LIMIT ?",
  594. (bm_q, N)), with_chunk=False)
  595. except sqlite3.OperationalError:
  596. pass
  597. if has_tri and len(query.strip()) >= 3:
  598. try: # (3) точная подстрока
  599. tq = '"' + query.replace('"', " ").strip() + '"'
  600. take("phrase", con.execute(
  601. "SELECT path,lineno,'',snippet(tri,2,'»','«','…',14) FROM tri "
  602. "WHERE tri MATCH ? ORDER BY bm25(tri) LIMIT ?", (tq, N)))
  603. except sqlite3.OperationalError:
  604. pass
  605. # (4) fuzzy: OR по 4-символьным окнам → опечатки/морфология/кириллица.
  606. # Порог покрытия: чанк принимается, только если содержит ≥ ceil(20%) окон запроса.
  607. grams = _fuzzy_phrases(query)
  608. if grams:
  609. need = max(1, (len(grams) + 4) // 5)
  610. fq = " OR ".join(f'"{g}"' for g in grams)
  611. try:
  612. rows = []
  613. for path, lineno, snip, body in con.execute(
  614. "SELECT path,lineno,snippet(tri,2,'»','«','…',14),body FROM tri "
  615. "WHERE tri MATCH ? ORDER BY bm25(tri) LIMIT ?", (fq, N * 2)):
  616. bl = body.lower()
  617. if sum(1 for g in grams if g in bl) >= need:
  618. rows.append((path, lineno, "", snip))
  619. take("fuzzy", rows)
  620. except sqlite3.OperationalError:
  621. pass
  622. fused: dict = defaultdict(float)
  623. via: dict = defaultdict(list)
  624. for ch, rk in ranks.items():
  625. for path, r in rk.items():
  626. fused[path] += CHANNEL_W[ch] / (RRF_K + r)
  627. via[path].append(ch)
  628. if not fused:
  629. con.close()
  630. return []
  631. info = {r[0]: r[1:] for r in con.execute(
  632. "SELECT path,title,node_type,status,trust,stale FROM files WHERE path IN (%s)"
  633. % ",".join("?" * len(fused)), tuple(fused))}
  634. for path in fused:
  635. title, nt, st, trust, stale = info.get(path, ("", "", "", "", 0))
  636. f = PRIOR["typed"] if nt else 1.0
  637. if nt == "index" or Path(path).name.lower() in HUB_NAMES:
  638. f *= PRIOR["index"]
  639. if st in ("deprecated", "archived", "superseded"):
  640. f *= PRIOR["retired"]
  641. if stale:
  642. f *= PRIOR["stale"]
  643. fused[path] *= f
  644. top = sorted(fused, key=lambda p: -fused[p])[:k]
  645. out = []
  646. for path in top:
  647. title, nt, st, trust, stale = info.get(path, ("", "", "", "", 0))
  648. b = best.get(path)
  649. if b is None: # файл найден только по doc-каналу — берём первый чанк
  650. row = con.execute("SELECT lineno,heading,substr(body,1,240) FROM fts WHERE path=? "
  651. "ORDER BY CAST(lineno AS INT) LIMIT 1", (path,)).fetchone() or (1, "", "")
  652. b = {"line": int(row[0]), "heading": row[1], "snippet": " ".join(row[2].split())}
  653. out.append({"path": path, "line": b["line"], "heading": b["heading"],
  654. "snippet": b["snippet"], "score": round(fused[path], 5),
  655. "via": "+".join(via[path]), "title": title, "node_type": nt,
  656. "status": st, "trust": trust, "stale": bool(stale)})
  657. con.close()
  658. return out
  659. # ─────────────────────────── query (структурные вопросы к frontmatter) ───────────────────────────
  660. _COND_RE = re.compile(r"^(!)?([\w.\-]+)(?:(!=|>=|<=|=|~|>|<)(.*)|(\?))?$")
  661. def _get_path(fm: dict, key: str) -> list:
  662. """`a.b` по frontmatter → плоский список значений (сквозь списки)."""
  663. vals = [fm]
  664. for part in key.split("."):
  665. nxt = []
  666. for v in vals:
  667. if isinstance(v, list):
  668. v_items = v
  669. else:
  670. v_items = [v]
  671. for it in v_items:
  672. if isinstance(it, dict) and part in it:
  673. x = it[part]
  674. nxt.extend(x if isinstance(x, list) else [x])
  675. vals = nxt
  676. return [v for v in vals if v is not None and v != ""]
  677. def doc_record(d: dict) -> dict:
  678. """frontmatter + вычисляемые поля: path, area, type, trust, stale, links.<type>."""
  679. rec = dict(d["fm"] or {})
  680. m = d["meta"]
  681. rec.update({"path": d["rel"], "area": area_of(d["rel"]), "type": m["node_type"],
  682. "node_type": m["node_type"], "title": m["title"], "trust": m["trust"],
  683. "stale": m["stale"]})
  684. return rec
  685. def match_cond(rec: dict, cond: str) -> bool:
  686. m = _COND_RE.match(cond.strip())
  687. if not m:
  688. raise SystemExit(f"не понял условие: {cond!r} (ожидаю key=val, key!=val, key~sub, "
  689. f"key>val, key<val, key?, !key)")
  690. neg, key, op, val, exists = m.groups()
  691. vals = [str(v).lower() if not isinstance(v, bool) else str(v).lower()
  692. for v in _get_path(rec, key)]
  693. if neg and not op:
  694. return not vals
  695. if exists or not op:
  696. return bool(vals)
  697. val = (val or "").strip().strip("'\"").lower()
  698. alts = [x.strip() for x in val.split("|")]
  699. if op == "=":
  700. return any(v in alts for v in vals)
  701. if op == "!=":
  702. return not any(v in alts for v in vals)
  703. if op == "~":
  704. return any(a in v for v in vals for a in alts)
  705. cmp = {">": lambda a, b: a > b, "<": lambda a, b: a < b,
  706. ">=": lambda a, b: a >= b, "<=": lambda a, b: a <= b}[op]
  707. return any(cmp(v, val) for v in vals)
  708. def cmd_query(root: Path, conds: list, count_by: str | None = None) -> dict:
  709. """Фильтр по frontmatter прямо из md (источник истины), без индекса."""
  710. rows = [doc_record(d) for d in load_docs(root).values() if d["fm"] is not None]
  711. hits = [r for r in rows if all(match_cond(r, c) for c in conds)]
  712. res = {"total": len(rows), "matched": len(hits), "rows": hits}
  713. if count_by:
  714. cnt = Counter()
  715. for r in hits:
  716. for v in _get_path(r, count_by) or ["(нет)"]:
  717. cnt[str(v)] += 1
  718. res["counts"] = cnt.most_common()
  719. return res
  720. # ─────────────────────────── stat ───────────────────────────
  721. def cmd_stat(root: Path) -> dict:
  722. db = root / DB_REL
  723. if not db.exists():
  724. return {"indexed": False}
  725. con = sqlite3.connect(db)
  726. try:
  727. f = con.execute("SELECT count(*), coalesce(sum(size),0), coalesce(sum(chunks),0) FROM files").fetchone()
  728. links = con.execute("SELECT count(*) FROM links WHERE kind='doc'").fetchone()[0]
  729. typed = con.execute("SELECT count(*) FROM links WHERE type!='relates_to'").fetchone()[0]
  730. areas = con.execute("SELECT count(DISTINCT area) FROM files").fetchone()[0]
  731. types = con.execute("SELECT node_type, count(*) FROM files WHERE node_type!='' "
  732. "GROUP BY 1 ORDER BY 2 DESC").fetchall()
  733. except sqlite3.OperationalError:
  734. con.close()
  735. return {"indexed": False, "stale_schema": True}
  736. has_tri = (con.execute("SELECT v FROM meta WHERE k='trigram'").fetchone() or ("0",))[0] == "1"
  737. con.close()
  738. return {"indexed": True, "files": f[0], "bytes": f[1], "chunks": f[2], "links": links,
  739. "typed_links": typed, "areas": areas, "trigram": has_tri, "types": types}
  740. # ─────────────────────────── lint (онтология) ───────────────────────────
  741. # Словари по умолчанию. Проект расширяет их в frontmatter своего ontology.md:
  742. # vocab: { node_type: [meeting], service: [api, billing], status: [done] }
  743. NODE_TYPES = {"service", "reference", "runbook", "gotcha", "decision",
  744. "plan", "guide", "report", "index", "memory"}
  745. # Реальный словарь сервисов выводится per-repo из имён папок docs/** + vocab.service.
  746. SERVICES = {"_platform"}
  747. STATUSES = {"active", "draft", "deprecated", "archived", "stable", # stable — OKF
  748. "frozen"} # снимок на дату: не правится, факт — в живом доке
  749. LOAD_BEARING = {"service", "reference", "runbook", "plan", "decision"}
  750. BEARING_DIRS = {"reference", "ops", "plans", "decisions", "services"}
  751. CODE_DIRS = ("services", "apps", "packages", "libs", "modules")
  752. LINK_KEYS = {"documents", "depends_on", "supersedes", "relates_to",
  753. "implemented_by", "part_of"}
  754. FENCE_RE = re.compile(r"```.*?```", re.DOTALL)
  755. INLINE_CODE_RE = re.compile(r"`[^`]*`")
  756. INVARIANTS = {
  757. "I1": "несущий документ без frontmatter с node_type/type",
  758. "I2": "значение вне словаря (node_type/service/status/тип связи)",
  759. "I3": "сирота — нет связей",
  760. "I4": "битая md-ссылка",
  761. "I5": "папка docs/ без README.md",
  762. "I6": "supersedes-цель не deprecated/archived",
  763. "I7": "links: documents/implemented_by → путь не существует",
  764. "I8": "просрочен stale_after",
  765. "I9": "точка входа: AGENTS.md с шапкой и ссылкой на docs/README.md, CLAUDE.md подключает его (@AGENTS.md)",
  766. }
  767. def strip_code(text: str) -> str:
  768. """Убрать fenced ``` и inline `code` — чтобы не ловить ссылки-примеры из кода."""
  769. return INLINE_CODE_RE.sub(" ", FENCE_RE.sub(" ", text))
  770. def in_kb(rel: str) -> bool:
  771. """Что линтуется: docs/** и md в корне (README/CLAUDE/AGENTS). Прочие md (код, фикстуры) —
  772. ищутся, но онтологией не считаются."""
  773. return rel.startswith("docs/") or "/" not in rel
  774. def project_vocab(docs: dict) -> dict:
  775. """vocab: из frontmatter любого ontology.md."""
  776. voc = defaultdict(set)
  777. for rel, d in docs.items():
  778. if Path(rel).name != "ontology.md":
  779. continue
  780. v = (d["fm"] or {}).get("vocab")
  781. if isinstance(v, dict):
  782. for key, vals in v.items():
  783. voc[key] |= {str(x) for x in as_list(vals)}
  784. return voc
  785. def _code_path_exists(root: Path, src_rel: str, target: str) -> bool:
  786. t = target.split("#")[0].strip()
  787. if not t or "://" in t:
  788. return True
  789. base = root if t.startswith("/") else (root / src_rel).parent
  790. return (base / t.lstrip("/")).exists() or (root / t.lstrip("/")).exists()
  791. def cmd_lint(root: Path, paths: list | None = None) -> dict:
  792. """Проверка инвариантов онтологии I1–I9. Возвращает {issues, errors, warnings, checked}."""
  793. docs = load_docs(root)
  794. known = set(docs)
  795. voc = project_vocab(docs)
  796. node_types = NODE_TYPES | voc["node_type"]
  797. statuses = STATUSES | voc["status"]
  798. link_keys = LINK_KEYS | voc["link"]
  799. if voc["service"]: # словарь задан явно — он и есть закон
  800. services_vocab = set(SERVICES) | voc["service"]
  801. else: # иначе — из папок docs/** и кода services|apps|packages/*
  802. services_vocab = set(SERVICES)
  803. docs_root = root / "docs"
  804. if docs_root.exists():
  805. services_vocab |= {d.name for d in docs_root.rglob("*") if d.is_dir()}
  806. for top in CODE_DIRS:
  807. if (root / top).is_dir():
  808. services_vocab |= {d.name for d in (root / top).iterdir() if d.is_dir()}
  809. out_links, in_links, issues = {}, defaultdict(set), []
  810. sel = set(paths) if paths else None
  811. edges = {}
  812. for rel, d in docs.items():
  813. edges[rel] = doc_edges(d, known)
  814. outs = set()
  815. for dst, lt, kind in edges[rel]:
  816. if kind == "doc":
  817. outs.add(dst)
  818. in_links[dst].add(rel)
  819. out_links[rel] = outs
  820. if not in_kb(rel):
  821. continue
  822. for href in LINK_RE.findall(strip_code(d["body"])):
  823. h = href.split()[0] if href.split() else href
  824. if (h.split("#")[0].endswith(".md") and not h.startswith(("http", "mailto:"))
  825. and not resolve_link(rel, h, known) and not _code_path_exists(root, rel, h)):
  826. issues.append(("ERR", "I4", rel, f"битая ссылка → {h}"))
  827. # README на каждую docs/-папку (I5)
  828. docs_dirs = {(root / rel).parent for rel in docs if rel.startswith("docs/")}
  829. for dd in sorted(docs_dirs):
  830. if not (dd / "README.md").exists() and not (dd / "index.md").exists():
  831. issues.append(("WARN", "I5", dd.relative_to(root).as_posix() + "/", "нет README.md (индекс папки)"))
  832. for rel, d in docs.items():
  833. if sel and rel not in sel:
  834. continue
  835. if not rel.startswith("docs/"):
  836. continue
  837. fm, nt = d["fm"], d["meta"]["node_type"]
  838. # I1 — несущий документ без frontmatter/типа
  839. parts = set(rel.split("/")[1:-1])
  840. if not fm or not nt:
  841. if parts & BEARING_DIRS and Path(rel).name not in ("README.md", "index.md", "log.md"):
  842. issues.append(("ERR", "I1", rel, "нет frontmatter с node_type"))
  843. continue
  844. # I2 — значения в словарях
  845. if nt not in node_types:
  846. issues.append(("ERR", "I2", rel, f"node_type='{nt}' вне словаря"))
  847. for svc in d["meta"]["service"]:
  848. if svc not in services_vocab and not svc.startswith("_"):
  849. issues.append(("WARN", "I2", rel, f"service='{svc}' вне словаря"))
  850. st = d["meta"]["status"]
  851. if st and st not in statuses:
  852. issues.append(("WARN", "I2", rel, f"status='{st}' вне словаря"))
  853. for lt, t in fm_links(fm):
  854. if lt not in link_keys:
  855. issues.append(("WARN", "I2", rel, f"тип связи '{lt}' вне словаря"))
  856. # I3 — сироты (несущий тип без входящих/исходящих связей, в т.ч. на код)
  857. if nt in LOAD_BEARING and not out_links.get(rel) and not in_links.get(rel) and not fm_links(fm):
  858. issues.append(("WARN", "I3", rel, "сирота — нет связей (ни in, ни out)"))
  859. for dst, lt, kind in edges[rel]:
  860. # I6 — supersedes-цель должна быть deprecated/archived
  861. if lt == "supersedes" and kind == "doc":
  862. if docs[dst]["meta"]["status"] not in ("deprecated", "archived"):
  863. issues.append(("WARN", "I6", rel, f"supersedes {dst}, но он не deprecated/archived"))
  864. # I4/I7 — цели типизированных связей
  865. if kind == "code" and dst.endswith(".md") and not _code_path_exists(root, rel, dst):
  866. issues.append(("ERR", "I4", rel, f"links.{lt} → {dst} — нет такого дока"))
  867. elif kind == "code" and not _code_path_exists(root, rel, dst):
  868. issues.append(("WARN", "I7", rel, f"links.{lt} → {dst} — путь не существует"))
  869. # I8 — просрочен
  870. if d["meta"]["stale"]:
  871. issues.append(("WARN", "I8", rel, f"stale_after={fm.get('stale_after')} прошёл — перепроверь"))
  872. if not sel:
  873. issues += lint_entry(root)
  874. errs = [i for i in issues if i[0] == "ERR"]
  875. warns = [i for i in issues if i[0] == "WARN"]
  876. return {"issues": issues, "errors": errs, "warnings": warns, "checked": len(docs)}
  877. AGENTS_IMPORT_RE = re.compile(r"^@\.?/?AGENTS\.md\s*$", re.MULTILINE)
  878. def lint_entry(root: Path) -> list:
  879. """I9 — точка входа. AGENTS.md читают Codex, Qwen Code, Cursor и др., CLAUDE.md — только Claude Code:
  880. правила живут в AGENTS.md (шапка + короткое дерево docs/), CLAUDE.md подключает его строкой @AGENTS.md,
  881. без копии — иначе два файла разъедутся."""
  882. agents, claude = root / "AGENTS.md", root / "CLAUDE.md"
  883. has_docs = (root / "docs").is_dir()
  884. out = []
  885. if not agents.exists():
  886. if claude.exists():
  887. out.append(("WARN", "I9", "CLAUDE.md",
  888. "вход только в CLAUDE.md — перенеси правила в AGENTS.md, в CLAUDE.md оставь @AGENTS.md"))
  889. elif has_docs:
  890. out.append(("WARN", "I9", "AGENTS.md", "нет точки входа AGENTS.md"))
  891. return out
  892. text = agents.read_text(encoding="utf-8", errors="replace")
  893. # индекс базы знаний: обычно docs/README.md, но БЗ может жить и в другой папке (b24/, kb/)
  894. if has_docs and not re.search(r"\]\(\.?/?[\w.-]+(?:/[\w.-]+)*/(?:README|index)\.md\)", text):
  895. out.append(("WARN", "I9", "AGENTS.md", "нет ссылки на индекс базы знаний (docs/README.md или <папка>/README.md)"))
  896. if claude.exists() and not AGENTS_IMPORT_RE.search(claude.read_text(encoding="utf-8", errors="replace")):
  897. out.append(("WARN", "I9", "CLAUDE.md", "не подключает AGENTS.md строкой @AGENTS.md"))
  898. return out
  899. # ─────────────────────────── храповик lint ───────────────────────────
  900. def _issue_key(i) -> tuple:
  901. return (i[1], i[2], i[3])
  902. def cmd_lint_baseline(root: Path, ref: str) -> dict:
  903. """Сравнить lint с состоянием REF (ветка/коммит): то же дерево из `git archive` во временной папке.
  904. Отвечает на один вопрос — стало ли замечаний больше. Старый долг ветку не валит, новый — валит.
  905. Не смогли сравнить → error (exit 2): «не посмотрел» нельзя выдавать за «не выросло»."""
  906. import tarfile, tempfile, io
  907. head = cmd_lint(root)
  908. try:
  909. r = subprocess.run(["git", "-C", str(root), "archive", "--format=tar", ref],
  910. capture_output=True, timeout=300)
  911. except (OSError, subprocess.SubprocessError) as e:
  912. return {"error": f"git archive {ref}: {e}"}
  913. if r.returncode != 0:
  914. return {"error": f"git archive {ref}: {r.stderr.decode('utf-8', 'replace').strip()}"}
  915. with tempfile.TemporaryDirectory(prefix="gitmark-base-") as tmp:
  916. with tarfile.open(fileobj=io.BytesIO(r.stdout)) as tar:
  917. tar.extractall(tmp, filter="data") if hasattr(tarfile, "data_filter") else tar.extractall(tmp)
  918. prev = os.environ.get("GITMARK_NO_GIT")
  919. os.environ["GITMARK_NO_GIT"] = "1" # во временной папке нет .git — обход ФС
  920. try:
  921. base = cmd_lint(Path(tmp))
  922. finally:
  923. if prev is None:
  924. os.environ.pop("GITMARK_NO_GIT", None)
  925. else:
  926. os.environ["GITMARK_NO_GIT"] = prev
  927. from collections import Counter
  928. new = Counter(map(_issue_key, head["issues"])) - Counter(map(_issue_key, base["issues"]))
  929. fresh = []
  930. for i in head["issues"]: # по одному на каждое превышение счётчика: дубли считаются
  931. if new[_issue_key(i)] > 0:
  932. new[_issue_key(i)] -= 1
  933. fresh.append(i)
  934. return {"head": len(head["issues"]), "base": len(base["issues"]), "new": fresh,
  935. "grew": len(head["issues"]) > len(base["issues"]), "head_issues": head["issues"]}
  936. # ─────────────────────────── toc: собираемые индексы ───────────────────────────
  937. TOC_OPEN, TOC_CLOSE = "<!-- gitmark:toc -->", "<!-- /gitmark:toc -->"
  938. TOC_RE = re.compile(re.escape(TOC_OPEN) + r".*?" + re.escape(TOC_CLOSE), re.DOTALL)
  939. INDEX_NAMES = ("README.md", "index.md")
  940. def _toc_lines(root: Path, folder: Path, docs: dict) -> list:
  941. """Строка на документ папки и на подпапку с индексом: `- [title](file) — description`."""
  942. rows = []
  943. for rel, d in sorted(docs.items()):
  944. p = root / rel
  945. if p.parent == folder and p.name not in INDEX_NAMES:
  946. rows.append((p.name, d))
  947. elif p.parent.parent == folder and p.name in INDEX_NAMES:
  948. rows.append((f"{p.parent.name}/{p.name}", d))
  949. out = []
  950. for href, d in rows:
  951. desc = d["meta"]["description"].strip().replace("\n", " ")
  952. out.append(f"- [{d['meta']['title']}]({href})" + (f" — {desc}" if desc else ""))
  953. return out
  954. def cmd_toc(root: Path, write: bool = False) -> dict:
  955. """Индексы, которые СОБИРАЮТСЯ, а не пишутся: в README/index.md папки блок между
  956. <!-- gitmark:toc --> и <!-- /gitmark:toc --> пересобирается из title/description документов.
  957. Папка без маркеров не трогается. Без --write — проверка: stale = индексы, разошедшиеся с папкой."""
  958. docs = load_docs(root)
  959. stale, checked = [], 0
  960. for rel in sorted(docs):
  961. p = root / rel
  962. if p.name not in INDEX_NAMES:
  963. continue
  964. text = p.read_text(encoding="utf-8")
  965. if TOC_OPEN not in text:
  966. continue
  967. checked += 1
  968. block = "\n".join([TOC_OPEN, *_toc_lines(root, p.parent, docs), TOC_CLOSE])
  969. new = TOC_RE.sub(lambda _m: block, text, count=1)
  970. if new != text:
  971. stale.append(rel)
  972. if write:
  973. p.write_text(new, encoding="utf-8")
  974. return {"checked": checked, "stale": stale, "written": write}
  975. # ─────────────────────────── secrets ───────────────────────────
  976. SECRET_PATTERNS = [
  977. ("private-key", re.compile(r"-----BEGIN [A-Z ]*PRIVATE KEY-----")),
  978. ("openai-key", re.compile(r"\bsk-(?:proj-|ant-)?[A-Za-z0-9_-]{20,}")),
  979. ("github-token", re.compile(r"\bgh[pousr]_[A-Za-z0-9]{30,}")),
  980. ("gitlab-token", re.compile(r"\bglpat-[A-Za-z0-9_-]{20,}")),
  981. ("slack-token", re.compile(r"\bxox[abprs]-[A-Za-z0-9-]{10,}")),
  982. ("aws-key", re.compile(r"\bAKIA[0-9A-Z]{16}\b")),
  983. ("bearer", re.compile(r"(?i)\bbearer\s+[A-Za-z0-9._~+/=-]{24,}")),
  984. ("assigned-secret", re.compile(
  985. r"(?i)\b(?:password|passwd|secret|token|api[_-]?key)\s*[:=]\s*['\"]?([^\s'\"<>`]{8,})")),
  986. ("url-credentials", re.compile(r"\b[a-z][a-z0-9+.-]*://[^/\s:@]+:[^/\s:@]{4,}@")),
  987. ]
  988. # заглушки в примерах — не секрет
  989. PLACEHOLDER_RE = re.compile(r"(?i)^(?:\$\{?[A-Z_]+\}?|<[^>]*>|x{4,}|\*{4,}|\.{3,}|your[_-].*|changeme|example.*|placeholder.*)$")
  990. def _mask(s: str) -> str:
  991. return s[:4] + "…" + str(len(s)) if len(s) > 8 else "…"
  992. def cmd_secrets(root: Path, files: list | None = None) -> dict:
  993. """Токены, ключи и пароли в markdown БЗ (docs/** и md в корне) или в заданных файлах.
  994. Значение маскируется: в выводе только вид и первые символы."""
  995. if files:
  996. targets = [(f, (root / f) if not Path(f).is_absolute() else Path(f)) for f in files]
  997. else:
  998. targets = [(p.relative_to(root).as_posix(), p) for p in iter_md(root)
  999. if in_kb(p.relative_to(root).as_posix())]
  1000. found = []
  1001. for rel, p in targets:
  1002. try:
  1003. lines = p.read_text(encoding="utf-8", errors="replace").splitlines()
  1004. except OSError:
  1005. continue
  1006. for n, line in enumerate(lines, 1):
  1007. for name, rx in SECRET_PATTERNS:
  1008. for m in rx.finditer(line):
  1009. val = m.group(m.lastindex or 0)
  1010. if PLACEHOLDER_RE.match(val.strip()):
  1011. continue
  1012. found.append((rel, n, name, _mask(val)))
  1013. return {"checked": len(targets), "found": found}
  1014. # ─────────────────────────── .gitattributes ───────────────────────────
  1015. UNION_LINES = ("docs/**/README.md merge=union", "docs/**/index.md merge=union",
  1016. "docs/README.md merge=union")
  1017. def cmd_attrs(root: Path, write: bool = False) -> dict:
  1018. """README-индексы — append-списки: параллельные ветки дописывают строки, `merge=union` склеивает их
  1019. без конфликта. Дубли и порядок потом выравнивает `gitmark toc --write`."""
  1020. f = root / ".gitattributes"
  1021. have = f.read_text(encoding="utf-8").splitlines() if f.exists() else []
  1022. missing = [l for l in UNION_LINES if l not in have]
  1023. if write and missing:
  1024. tail = "" if not have or have[-1] == "" else "\n"
  1025. with f.open("a", encoding="utf-8") as fh:
  1026. fh.write(tail + "# gitmark: индексы базы знаний склеиваются при слиянии\n" + "\n".join(missing) + "\n")
  1027. return {"missing": missing, "written": write and bool(missing)}
  1028. # ─────────────────────────── health (скоркард) ───────────────────────────
  1029. def cmd_health(root: Path) -> dict:
  1030. """Насколько БЗ — онтология, а не куча файлов. Считается по docs/** (или по всем,
  1031. если docs/ нет): доля с типом/title/description/updated, типизированные связи,
  1032. сироты, битые ссылки, trust-tier, stale, дубли по содержимому."""
  1033. docs = load_docs(root)
  1034. known = set(docs)
  1035. scope = {r: d for r, d in docs.items() if r.startswith("docs/")} or docs
  1036. content = {r: d for r, d in scope.items()
  1037. if Path(r).name not in ("README.md", "index.md", "log.md")}
  1038. n = len(content) or 1
  1039. typed = [d for d in content.values() if d["meta"]["node_type"]]
  1040. fm_of = lambda d: d["fm"] or {}
  1041. linked_in = defaultdict(int)
  1042. edges = {}
  1043. for r, d in docs.items():
  1044. edges[r] = doc_edges(d, known)
  1045. for dst, _, kind in edges[r]:
  1046. if kind == "doc":
  1047. linked_in[dst] += 1
  1048. orphans = [r for r in content if not edges[r] and not linked_in[r]]
  1049. broken = sum(1 for r, d in scope.items() for h in LINK_RE.findall(strip_code(d["body"]))
  1050. if h.split("#")[0].endswith(".md") and not h.startswith("http")
  1051. and not resolve_link(r, h.split()[0], known)
  1052. and not _code_path_exists(root, r, h.split()[0]))
  1053. hashes = defaultdict(list)
  1054. for r, d in docs.items():
  1055. body = re.sub(r"\s+", " ", d["body"]).strip()
  1056. if len(body) > 200:
  1057. hashes[hashlib.sha1(body.encode()).hexdigest()].append(r)
  1058. dups = [v for v in hashes.values() if len(v) > 1]
  1059. trust = Counter(d["meta"]["trust"] for d in content.values())
  1060. pct = lambda c: round(100 * c / n)
  1061. res = {
  1062. "docs": len(content), "indexes": len(scope) - len(content), "all_md": len(docs),
  1063. "typed_pct": pct(len(typed)),
  1064. "title_pct": pct(sum(1 for d in content.values() if fm_of(d).get("title"))),
  1065. "description_pct": pct(sum(1 for d in content.values() if fm_of(d).get("description") or fm_of(d).get("summary"))),
  1066. "updated_pct": pct(sum(1 for d in content.values()
  1067. if fm_of(d).get("updated") or isinstance(fm_of(d).get("generated"), dict))),
  1068. "typed_links_pct": pct(sum(1 for d in content.values() if fm_links(d["fm"]))),
  1069. "code_links_pct": pct(sum(1 for r in content
  1070. if any(k == "code" for _, _, k in edges[r]))),
  1071. "orphans": len(orphans), "broken_links": broken,
  1072. "stale": sum(1 for d in content.values() if d["meta"]["stale"]),
  1073. "trust": dict(trust), "duplicate_groups": len(dups),
  1074. "duplicates": dups[:10],
  1075. "types": Counter(d["meta"]["node_type"] for d in typed).most_common(),
  1076. }
  1077. # интегральный балл 0–100: структура (тип/title/updated), связность, гигиена
  1078. score = (0.30 * res["typed_pct"] + 0.10 * res["title_pct"] + 0.10 * res["description_pct"]
  1079. + 0.10 * res["updated_pct"] + 0.15 * res["typed_links_pct"]
  1080. + 0.15 * (100 - pct(len(orphans))) + 0.10 * (100 if broken == 0 else max(0, 100 - 5 * broken)))
  1081. res["score"] = round(score)
  1082. return res
  1083. # ─────────────────────────── map (HTML обзор + граф) ───────────────────────────
  1084. def cmd_map(root: Path, out: Path) -> dict:
  1085. """Self-contained HTML: дерево + рендер md + радиальный граф от точки входа."""
  1086. try:
  1087. import markdown as _md
  1088. render = lambda t: _md.markdown(t, extensions=["extra", "sane_lists", "tables", "toc"], output_format="html5")
  1089. pretty = True
  1090. except Exception:
  1091. import html as _html
  1092. render = lambda t: f"<pre>{_html.escape(t)}</pre>"
  1093. pretty = False
  1094. docs = load_docs(root)
  1095. known = set(docs)
  1096. files, edges = {}, []
  1097. for rel, d in docs.items():
  1098. files[rel] = {"rel": rel, "area": area_of(rel), "title": d["meta"]["title"],
  1099. "size": len(d["text"].encode("utf-8")), "html": render(d["body"]),
  1100. "type": d["meta"]["node_type"], "status": d["meta"]["status"]}
  1101. for rel, d in docs.items():
  1102. for dst, lt, kind in doc_edges(d, known):
  1103. if kind == "doc" and dst != rel:
  1104. edges.append({"s": rel, "t": dst, "kind": "ref", "lt": lt})
  1105. areas = sorted({f["area"] for f in files.values()})
  1106. nodes = [{"id": "area::" + a, "label": a, "kind": "area", "area": a, "size": 0} for a in areas]
  1107. nodes += [{"id": r, "label": f["title"], "kind": "doc", "area": f["area"],
  1108. "size": f["size"], "rel": r} for r, f in files.items()]
  1109. edges += [{"s": "area::" + f["area"], "t": r, "kind": "own"} for r, f in files.items()]
  1110. # BFS-уровни/углы от точки входа (AGENTS.md|CLAUDE.md|README.md|первый)
  1111. import math
  1112. from collections import defaultdict, deque
  1113. nid = {n["id"]: n for n in nodes}
  1114. root_id = next((c for c in ("AGENTS.md", "CLAUDE.md", "README.md") if c in nid), nodes[0]["id"])
  1115. adj = defaultdict(set)
  1116. for e in edges:
  1117. adj[e["s"]].add(e["t"]); adj[e["t"]].add(e["s"])
  1118. level, parent, dq = {root_id: 0}, {root_id: None}, deque([root_id])
  1119. while dq:
  1120. u = dq.popleft()
  1121. for v in sorted(adj[u]):
  1122. if v not in level:
  1123. level[v] = level[u] + 1; parent[v] = u; dq.append(v)
  1124. children = defaultdict(list)
  1125. for v, p in parent.items():
  1126. if p is not None:
  1127. children[p].append(v)
  1128. leaf = {}
  1129. def _cnt(n):
  1130. if not children[n]:
  1131. leaf[n] = 1; return 1
  1132. s = sum(_cnt(c) for c in children[n]); leaf[n] = s; return s
  1133. _cnt(root_id)
  1134. ang = {}
  1135. def _asn(n, a0, a1):
  1136. ang[n] = (a0 + a1) / 2
  1137. ch = sorted(children[n], key=lambda c: c)
  1138. if not ch:
  1139. return
  1140. tot = sum(leaf[c] for c in ch) or 1; cur = a0
  1141. for c in ch:
  1142. sp = (a1 - a0) * leaf[c] / tot; _asn(c, cur, cur + sp); cur += sp
  1143. _asn(root_id, 0, 2 * math.pi)
  1144. maxlvl = max(level.values()) if level else 0
  1145. unreached = [n["id"] for n in nodes if n["id"] not in level]
  1146. for i, u in enumerate(unreached):
  1147. level[u] = maxlvl + 1; ang[u] = 2 * math.pi * i / max(1, len(unreached))
  1148. treeset = set()
  1149. for v, p in parent.items():
  1150. if p is not None:
  1151. treeset.add((p, v)); treeset.add((v, p))
  1152. for n in nodes:
  1153. n["lvl"] = level.get(n["id"], maxlvl + 1); n["ang"] = round(ang.get(n["id"], 0), 4)
  1154. n["root"] = n["id"] == root_id
  1155. for e in edges:
  1156. e["tree"] = (e["s"], e["t"]) in treeset
  1157. data = {"files": files, "nodes": nodes, "edges": edges, "root": root_id,
  1158. "stats": {"files": len(files), "areas": len(areas),
  1159. "refs": sum(1 for e in edges if e["kind"] == "ref"),
  1160. "bytes": sum(f["size"] for f in files.values()),
  1161. "maxlvl": maxlvl, "unreached": len(unreached), "pretty": pretty}}
  1162. payload = json.dumps(data, ensure_ascii=False).replace("</", "<\\/")
  1163. html = _MAP_HTML.replace("__DATA__", payload).replace("__ROOTNAME__", root.name)
  1164. out.parent.mkdir(parents=True, exist_ok=True)
  1165. out.write_text(html, encoding="utf-8")
  1166. return {"out": str(out), "files": len(files), "refs": data["stats"]["refs"],
  1167. "unreached": len(unreached), "pretty": pretty}
  1168. # ─────────────────────────── serve ───────────────────────────
  1169. def cmd_serve(root: Path, port: int):
  1170. import http.server, socketserver, functools
  1171. docroot = root / "docs"
  1172. if not (docroot / "docs-map.html").exists():
  1173. docroot = root
  1174. handler = functools.partial(http.server.SimpleHTTPRequestHandler, directory=str(docroot))
  1175. with socketserver.TCPServer(("127.0.0.1", port), handler) as httpd:
  1176. print(f"GitMark serve → http://127.0.0.1:{port}/docs-map.html (Ctrl-C стоп)")
  1177. try:
  1178. httpd.serve_forever()
  1179. except KeyboardInterrupt:
  1180. print("\nстоп")
  1181. # ─────────────────────────── CLI ───────────────────────────
  1182. def _csv(v):
  1183. return [x.strip() for x in v.split(",") if x.strip()] if v else None
  1184. def main(argv=None):
  1185. ap = argparse.ArgumentParser(prog="gitmark", description="GitMark Memory Bank — md+git knowledge base CLI")
  1186. ap.add_argument("--root", default=None, help="корень репо (по умолчанию — авто по .git)")
  1187. sub = ap.add_subparsers(dest="cmd", required=True)
  1188. sub.add_parser("index", help="построить индекс").add_argument("--force", action="store_true",
  1189. help="(совместимость; индекс всегда пересобирается)")
  1190. sp = sub.add_parser("search", help="искать")
  1191. sp.add_argument("query"); sp.add_argument("-k", type=int, default=8); sp.add_argument("--json", action="store_true")
  1192. sp.add_argument("--type", help="только эти node_type/type, через запятую")
  1193. sp.add_argument("--status", help="только эти status, через запятую")
  1194. sp.add_argument("--service", help="только эти service, через запятую")
  1195. sp.add_argument("--scope", choices=["all", "live", "history"], default="all",
  1196. help="live — без исторических plan/report; history — только они")
  1197. qp = sub.add_parser("query", help="структурный запрос по frontmatter",
  1198. description="Условия (AND): key=val key!=val key~sub key>val key<val key? !key. "
  1199. "Альтернативы через |: status=draft|active. Вложенные ключи: verified.by, "
  1200. "links.depends_on. Вычисляемые: path, area, type, trust, stale.")
  1201. qp.add_argument("cond", nargs="*")
  1202. qp.add_argument("--fields", default="path,type,title,status", help="колонки вывода")
  1203. qp.add_argument("--count-by", help="посчитать по значениям ключа (списки разворачиваются)")
  1204. qp.add_argument("--json", action="store_true")
  1205. mp = sub.add_parser("map", help="HTML обзор+граф"); mp.add_argument("-o", "--out", default=None)
  1206. sv = sub.add_parser("serve", help="локальный http"); sv.add_argument("-p", "--port", type=int, default=8799)
  1207. sub.add_parser("stat", help="статистика")
  1208. lp = sub.add_parser("lint", help="проверить онтологию (I1–I9)")
  1209. lp.add_argument("paths", nargs="*", help="ограничить файлами (по умолчанию — все docs/)")
  1210. lp.add_argument("--strict", action="store_true", help="exit 1 при любых ERR")
  1211. lp.add_argument("--json", action="store_true")
  1212. lp.add_argument("--baseline", metavar="REF",
  1213. help="храповик: exit 1, только если замечаний больше, чем в REF (exit 2 — не смог сравнить)")
  1214. hp = sub.add_parser("health", help="скоркард БЗ"); hp.add_argument("--json", action="store_true")
  1215. tp = sub.add_parser("toc", help="собрать блоки gitmark:toc в индексах")
  1216. tp.add_argument("--write", action="store_true", help="записать; без флага — проверка (exit 1, если разошлись)")
  1217. scp = sub.add_parser("secrets", help="токены/ключи/пароли в БЗ")
  1218. scp.add_argument("files", nargs="*", help="проверить эти файлы (по умолчанию — md базы знаний)")
  1219. atp = sub.add_parser("attrs", help=".gitattributes: merge=union для README-индексов")
  1220. atp.add_argument("--write", action="store_true")
  1221. sub.add_parser("version", help="версия")
  1222. a = ap.parse_args(argv)
  1223. root = Path(a.root).resolve() if a.root else repo_root(Path.cwd())
  1224. if a.cmd == "index":
  1225. r = cmd_index(root, a.force)
  1226. print(f"✓ index: {r['files']} файлов · {r['chunks']} чанков · {r['links']} ссылок · "
  1227. f"trigram={'on' if r['trigram'] else 'OFF'} → {r['db']}")
  1228. elif a.cmd == "search":
  1229. res = cmd_search(root, a.query, a.k, _csv(a.type), _csv(a.status), _csv(a.service), a.scope)
  1230. if a.json:
  1231. print(json.dumps(res, ensure_ascii=False, indent=2)); return
  1232. if not res:
  1233. print("ничего не найдено"); return
  1234. for r in res:
  1235. head = f" › {r['heading']}" if r["heading"] else ""
  1236. tags = " ".join(x for x in (r["node_type"], r["status"],
  1237. "stale" if r["stale"] else "",
  1238. "✓human" if r["trust"] == "human" else "") if x)
  1239. tags = f" \033[36m{tags}\033[0m" if tags else ""
  1240. print(f"\033[32m{r['path']}:{r['line']}\033[0m{head}{tags} \033[90m[{r['via']}]\033[0m")
  1241. print(f" {r['snippet'][:200]}")
  1242. elif a.cmd == "query":
  1243. r = cmd_query(root, a.cond, a.count_by)
  1244. if a.json:
  1245. print(json.dumps(r, ensure_ascii=False, indent=1, default=str)); return
  1246. if a.count_by:
  1247. for v, c in r["counts"]:
  1248. print(f"{c:>5} {v}")
  1249. else:
  1250. fields = _csv(a.fields)
  1251. for row in r["rows"]:
  1252. print(" ".join(", ".join(str(x) for x in _get_path(row, f)) or "—" for f in fields))
  1253. print(f"\n{r['matched']} из {r['total']} документов с frontmatter")
  1254. elif a.cmd == "map":
  1255. out = Path(a.out) if a.out else (root / "docs" / "docs-map.html")
  1256. r = cmd_map(root, out)
  1257. warn = "" if r["pretty"] else " (raw md — `pip install markdown` для рендера)"
  1258. print(f"✓ map: {r['out']} · {r['files']} файлов · {r['refs']} ссылок · "
  1259. f"{r['unreached']} вне индекса{warn}")
  1260. elif a.cmd == "serve":
  1261. cmd_serve(root, a.port)
  1262. elif a.cmd == "stat":
  1263. s = cmd_stat(root)
  1264. if not s.get("indexed"):
  1265. print("индекс не построен — `gitmark index`"); return
  1266. types = " ".join(f"{t}:{n}" for t, n in s["types"])
  1267. print(f"GitMark · {s['files']} файлов · {s['areas']} папок · {s['chunks']} чанков · "
  1268. f"{s['links']} ссылок ({s['typed_links']} типизир.) · {s['bytes']//1024} KB · "
  1269. f"trigram={'on' if s['trigram'] else 'off'}\n типы: {types or '—'}")
  1270. elif a.cmd == "lint" and a.baseline:
  1271. r = cmd_lint_baseline(root, a.baseline)
  1272. if "error" in r:
  1273. print(f"\033[31mне смог сравнить\033[0m — {r['error']}", file=sys.stderr); sys.exit(2)
  1274. for lvl, code, path, msg in r["new"]:
  1275. col = "\033[31m" if lvl == "ERR" else "\033[33m"
  1276. print(f"{col}{lvl}\033[0m \033[90m{code}\033[0m {path} — {msg}")
  1277. verdict = "\033[31mвыросло\033[0m" if r["grew"] else "\033[32mне выросло\033[0m"
  1278. print(f"\n{verdict}: {a.baseline} {r['base']} → сейчас {r['head']} · новых {len(r['new'])}")
  1279. if r["grew"]:
  1280. sys.exit(1)
  1281. elif a.cmd == "toc":
  1282. r = cmd_toc(root, a.write)
  1283. for rel in r["stale"]:
  1284. print(f"{'✓ собран' if a.write else 'разошёлся'}: {rel}")
  1285. print(f"индексов с блоком gitmark:toc: {r['checked']} · {'обновлено' if a.write else 'разошлось'}: {len(r['stale'])}")
  1286. if r["stale"] and not a.write:
  1287. sys.exit(1)
  1288. elif a.cmd == "secrets":
  1289. r = cmd_secrets(root, a.files or None)
  1290. for rel, n, name, masked in r["found"]:
  1291. print(f"\033[31m{rel}:{n}\033[0m {name} {masked}")
  1292. print(f"{'✓ чисто' if not r['found'] else f'найдено: {len(r[chr(102)+chr(111)+chr(117)+chr(110)+chr(100)])}'} ({r['checked']} файлов)")
  1293. if r["found"]:
  1294. sys.exit(1)
  1295. elif a.cmd == "attrs":
  1296. r = cmd_attrs(root, a.write)
  1297. if not r["missing"]:
  1298. print("✓ .gitattributes: индексы склеиваются (merge=union)")
  1299. else:
  1300. print(("✓ дописано в .gitattributes:\n " if a.write else "нет в .gitattributes (gitmark attrs --write):\n ")
  1301. + "\n ".join(r["missing"]))
  1302. if not a.write:
  1303. sys.exit(1)
  1304. elif a.cmd == "lint":
  1305. r = cmd_lint(root, a.paths or None)
  1306. if a.json:
  1307. print(json.dumps({"errors": len(r["errors"]), "warnings": len(r["warnings"]),
  1308. "checked": r["checked"],
  1309. "issues": [dict(zip(("level", "code", "path", "msg"), i)) for i in r["issues"]]},
  1310. ensure_ascii=False, indent=1))
  1311. else:
  1312. order = {"ERR": 0, "WARN": 1}
  1313. codes = {}
  1314. for lvl, code, path, msg in sorted(r["issues"], key=lambda x: (order[x[0]], x[1], x[2])):
  1315. codes[code] = codes.get(code, 0) + 1
  1316. col = "\033[31m" if lvl == "ERR" else "\033[33m"
  1317. print(f"{col}{lvl}\033[0m \033[90m{code}\033[0m {path} — {msg}")
  1318. ne, nw = len(r["errors"]), len(r["warnings"])
  1319. summary = " · ".join(f"{c}×{n}" for c, n in sorted(codes.items())) or "—"
  1320. mark = "\033[32m✓ чисто\033[0m" if ne == 0 and nw == 0 else f"\033[31m{ne} ERR\033[0m · \033[33m{nw} WARN\033[0m"
  1321. print(f"\n{mark} ({r['checked']} файлов · {summary})")
  1322. if a.strict and r["errors"]:
  1323. sys.exit(1)
  1324. elif a.cmd == "health":
  1325. h = cmd_health(root)
  1326. if a.json:
  1327. print(json.dumps(h, ensure_ascii=False, indent=1)); return
  1328. print(f"GitMark health · {h['score']}/100 · {h['docs']} доков (+{h['indexes']} индексов)")
  1329. print(f" типизировано {h['typed_pct']}% · title {h['title_pct']}% · description {h['description_pct']}%"
  1330. f" · updated {h['updated_pct']}%")
  1331. print(f" типизир. связи {h['typed_links_pct']}% · связь с кодом {h['code_links_pct']}%"
  1332. f" · сирот {h['orphans']} · битых ссылок {h['broken_links']}")
  1333. print(f" trust {h['trust']} · stale {h['stale']} · дублей {h['duplicate_groups']}")
  1334. for g in h["duplicates"][:5]:
  1335. print(f" дубль: {' = '.join(g)}")
  1336. print(f" типы: {' '.join(f'{t}:{n}' for t, n in h['types']) or '—'}")
  1337. elif a.cmd == "version":
  1338. print(f"gitmark {VERSION}")
  1339. # HTML-шаблон map (дерево + рендер + радиальный граф). __DATA__/__ROOTNAME__ инжектятся.
  1340. _MAP_HTML = r"""<!doctype html><html lang="ru"><head>
  1341. <meta charset="utf-8"><meta name="viewport" content="width=device-width,initial-scale=1">
  1342. <title>GitMark · __ROOTNAME__</title>
  1343. <style>
  1344. :root{--bg:#0a0a0a;--panel:#121212;--panel2:#171717;--border:#262626;--fg:#e8e8e8;--muted:#8a8a8a;--accent:#00ff88;
  1345. --mono:"SF Mono",ui-monospace,"JetBrains Mono",Menlo,Consolas,monospace;--sans:-apple-system,BlinkMacSystemFont,"Segoe UI",Inter,Roboto,Arial,sans-serif;
  1346. --docs:#00ff88;--services:#7c9cff;--root:#9aa0a6;}
  1347. *{box-sizing:border-box}html,body{margin:0;height:100%;background:var(--bg);color:var(--fg);font-family:var(--sans);font-size:15px}
  1348. a{color:var(--accent);text-decoration:none}a:hover{text-decoration:underline}
  1349. #top{display:flex;align-items:center;gap:14px;padding:10px 16px;border-bottom:1px solid var(--border);background:var(--panel)}
  1350. #top h1{font-family:var(--mono);font-size:15px;margin:0;font-weight:600}
  1351. #top .stat{color:var(--muted);font-size:12.5px;font-family:var(--mono)}
  1352. #search{flex:0 0 240px;margin-left:auto;background:var(--panel2);border:1px solid var(--border);color:var(--fg);padding:6px 10px;border-radius:7px;font-family:var(--mono);font-size:13px;outline:none}
  1353. .tabs{display:flex;gap:4px}.tab{cursor:pointer;padding:6px 14px;border-radius:7px;border:1px solid var(--border);background:var(--panel2);color:var(--muted);font-family:var(--mono);font-size:13px}
  1354. .tab.on{color:var(--bg);background:var(--accent);border-color:var(--accent);font-weight:600}
  1355. #main{display:flex;height:calc(100vh - 53px)}
  1356. #side{flex:0 0 320px;overflow:auto;border-right:1px solid var(--border);background:var(--panel);padding:8px 0}
  1357. .grp{padding:7px 14px 3px;font-family:var(--mono);font-size:11.5px;color:var(--muted);text-transform:uppercase;cursor:pointer;display:flex;justify-content:space-between;position:sticky;top:0;background:var(--panel)}
  1358. .grp:hover{color:var(--fg)}.grp .car{display:inline-block;width:10px;color:var(--muted);font-size:10px}
  1359. .it{padding:4px 14px 4px 26px;cursor:pointer;font-size:13.5px;border-left:2px solid transparent;color:#cfcfcf}
  1360. .it:hover{background:var(--panel2)}.it.on{border-left-color:var(--accent);background:var(--panel2);color:#fff}
  1361. .it .p{display:block;font-family:var(--mono);font-size:10.5px;color:var(--muted)}.hidden{display:none}
  1362. #doc{flex:1;overflow:auto;padding:34px 44px 80px}#docwrap{max-width:860px;margin:0 auto}
  1363. #crumb{font-family:var(--mono);font-size:12px;color:var(--muted);margin-bottom:18px}#crumb b{color:var(--accent)}
  1364. .md h1{font-family:var(--mono);font-size:1.9rem;border-bottom:1px solid var(--border);padding-bottom:.35em}
  1365. .md h2{font-family:var(--mono);font-size:1.4rem;border-bottom:1px solid var(--border);padding-bottom:.3em;margin-top:2em}
  1366. .md h3{font-family:var(--mono);color:var(--accent)}.md{line-height:1.7}
  1367. .md code{font-family:var(--mono);background:#1d1d1d;padding:.12em .4em;border-radius:4px;color:#ffd479}
  1368. .md pre{background:#0e0e0e;border:1px solid var(--border);border-radius:8px;padding:14px;overflow:auto}.md pre code{background:none;color:#dcdcdc}
  1369. .md table{border-collapse:collapse;font-size:13.5px;display:block;overflow:auto}.md th,.md td{border:1px solid var(--border);padding:6px 10px}
  1370. .md blockquote{border-left:3px solid var(--accent);padding:.3em 1em;color:var(--muted);background:#0e0e0e}
  1371. #gp{flex:1;position:relative;overflow:hidden}#cv{display:block;width:100%;height:100%;cursor:grab}
  1372. #legend{position:absolute;top:12px;left:12px;background:rgba(18,18,18,.9);border:1px solid var(--border);border-radius:8px;padding:10px;font-family:var(--mono);font-size:12px}
  1373. #legend .row{display:flex;align-items:center;gap:7px;margin:3px 0}#legend .dot{width:10px;height:10px;border-radius:50%}
  1374. #tip{position:absolute;pointer-events:none;background:#000;border:1px solid var(--accent);border-radius:6px;padding:6px 9px;font-family:var(--mono);font-size:12px;max-width:340px;display:none}
  1375. #tip .s{color:var(--muted);font-size:10.5px}
  1376. #gctl{position:absolute;top:12px;right:12px;background:rgba(18,18,18,.92);border:1px solid var(--border);border-radius:8px;padding:9px 11px;font-family:var(--mono);font-size:12px;display:flex;flex-direction:column;gap:6px}
  1377. #gctl label{display:flex;align-items:center;gap:6px;cursor:pointer;color:var(--muted)}#gctl label:hover{color:var(--fg)}
  1378. #gctl input{accent-color:var(--accent);cursor:pointer}#gctl #reheat{color:var(--accent);border-top:1px solid var(--border);padding-top:6px}
  1379. #ghint{position:absolute;bottom:12px;left:50%;transform:translateX(-50%);background:rgba(18,18,18,.85);border:1px solid var(--border);border-radius:7px;padding:5px 12px;font-family:var(--mono);font-size:11.5px;color:var(--muted);white-space:nowrap}
  1380. </style></head><body>
  1381. <div id="top"><h1>🧠 GitMark · __ROOTNAME__</h1><span class="stat" id="stat"></span>
  1382. <div class="tabs"><div class="tab on" data-v="browse">Дерево</div><div class="tab" data-v="graph">Граф</div></div>
  1383. <input id="search" placeholder="фильтр файлов…"></div>
  1384. <div id="main">
  1385. <div id="side"></div>
  1386. <div id="doc"><div id="docwrap"><div id="crumb"></div><div class="md" id="mdbody"></div></div></div>
  1387. <div id="gp" class="hidden"><canvas id="cv"></canvas><div id="legend"></div>
  1388. <div id="gctl">
  1389. <label><input type="checkbox" id="showref" checked> ссылки между доками</label>
  1390. <label><input type="checkbox" id="showown" checked> папка→док</label>
  1391. <label><input type="checkbox" id="physics"> свободный режим (гравитация)</label>
  1392. <label id="reheat">↻ пересобрать рост</label>
  1393. </div>
  1394. <div id="ghint">🕸 центр = точка входа · ЛКМ-таскать · колесо-зум · клик по точке → открыть док</div>
  1395. <div id="tip"></div></div>
  1396. </div>
  1397. <script>
  1398. const DATA=__DATA__,F=DATA.files,$=s=>document.querySelector(s);
  1399. // группа узла = сервис (docs/services/<svc> или services/<svc>), иначе docs/root.
  1400. function svcOf(a){const m=a.match(/^docs\/services\/([^/]+)/)||a.match(/^services\/([^/]+)/);return m?m[1]:null;}
  1401. function grp(a){return svcOf(a)||(a.startsWith('docs')?'docs':'root');}
  1402. const PAL=['#7c9cff','#ff6ec7','#ffb454','#4ec9ff','#c586ff','#f7768e','#9ece6a','#e0af68','#bb9af7','#73daca','#ff9e64','#7dcfff'];
  1403. const _cc={};
  1404. function colorOf(g){if(g==='docs')return '#00ff88';if(g==='root')return '#9aa0a6';
  1405. if(_cc[g])return _cc[g];let h=0;for(let i=0;i<g.length;i++)h=(h*31+g.charCodeAt(i))>>>0;return _cc[g]=PAL[h%PAL.length];}
  1406. const COL=new Proxy({},{get:(_,k)=>colorOf(k)}); // COL[grp(area)] совместимость
  1407. function esc(s){return(s||'').replace(/[&<>]/g,c=>({'&':'&amp;','<':'&lt;','>':'&gt;'}[c]));}
  1408. $('#stat').textContent=DATA.stats.files+' файлов · '+DATA.stats.areas+' папок · '+DATA.stats.refs+' ссылок · '+(DATA.stats.bytes/1024).toFixed(0)+' KB';
  1409. const byArea={};DATA.nodes.filter(n=>n.kind==='doc').forEach(n=>{(byArea[n.area]=byArea[n.area]||[]).push(n);});
  1410. const side=$('#side');
  1411. Object.keys(byArea).sort().forEach(a=>{const items=byArea[a].sort((x,y)=>x.rel.localeCompare(y.rel));
  1412. const g=document.createElement('div');g.className='grp';
  1413. g.innerHTML='<span><span class="car">▸</span> <span style="color:'+COL[grp(a)]+'">'+a+'</span></span><span>'+items.length+'</span>';side.appendChild(g);
  1414. const box=document.createElement('div');box.className='gbox hidden'; // свёрнуто по умолчанию
  1415. items.forEach(n=>{const d=document.createElement('div');d.className='it';d.dataset.rel=n.rel;
  1416. d.innerHTML=esc(n.label)+'<span class="p">'+esc(n.rel)+'</span>';d.onclick=()=>open(n.rel);box.appendChild(d);});side.appendChild(box);
  1417. g.onclick=()=>{const h=box.classList.toggle('hidden');g.querySelector('.car').textContent=h?'▸':'▾';};});
  1418. function open(rel){const f=F[rel];if(!f)return;$('#mdbody').innerHTML=f.html;
  1419. $('#crumb').innerHTML='<b>'+esc(f.area)+'</b> / '+esc(rel.split('/').pop())+' · '+(f.size/1024).toFixed(1)+' KB';
  1420. document.querySelectorAll('.it').forEach(x=>x.classList.toggle('on',x.dataset.rel===rel));
  1421. const act=document.querySelector('.it.on');if(act){const bx=act.parentElement;if(bx.classList.contains('hidden')){bx.classList.remove('hidden');const gg=bx.previousElementSibling,c=gg&&gg.querySelector('.car');if(c)c.textContent='▾';}}
  1422. $('#mdbody').querySelectorAll('a[href]').forEach(an=>{const h=an.getAttribute('href');const b=(h||'').split('#')[0].split('/').pop();
  1423. const hit=Object.keys(F).filter(r=>r.endsWith('/'+b)||r===b);if(b&&b.endsWith('.md')&&hit.length===1){an.onclick=e=>{e.preventDefault();open(hit[0]);$('#doc').scrollTop=0;};}});
  1424. $('#doc').scrollTop=0;}
  1425. $('#search').oninput=e=>{const q=e.target.value.toLowerCase();
  1426. document.querySelectorAll('.it').forEach(it=>{it.style.display=it.textContent.toLowerCase().includes(q)?'':'none';});
  1427. document.querySelectorAll('.gbox').forEach(b=>b.classList.toggle('hidden',!q)); // при поиске разворачиваем, пусто — сворачиваем
  1428. document.querySelectorAll('.grp .car').forEach(c=>c.textContent=q?'▾':'▸');};
  1429. document.querySelectorAll('.tab').forEach(t=>t.onclick=()=>{document.querySelectorAll('.tab').forEach(x=>x.classList.toggle('on',x===t));
  1430. const g=t.dataset.v==='graph';$('#side').classList.toggle('hidden',g);$('#doc').classList.toggle('hidden',g);$('#gp').classList.toggle('hidden',!g);if(g){resize();if(!G.on)startG();}});
  1431. /* радиальный граф */
  1432. const cv=$('#cv'),ctx=cv.getContext('2d');const G={on:false,view:{x:0,y:0,k:1},drag:null,hover:null,grow:0,maxlvl:DATA.stats.maxlvl};
  1433. function resize(){const r=cv.getBoundingClientRect();cv.width=r.width*devicePixelRatio;cv.height=r.height*devicePixelRatio;ctx.setTransform(devicePixelRatio,0,0,devicePixelRatio,0,0);G.W=r.width;G.H=r.height;if(G.on)layout();}
  1434. addEventListener('resize',()=>{if(!$('#gp').classList.contains('hidden'))resize();});
  1435. function layout(){const cx=G.W/2,cy=G.H/2,gap=Math.min(G.W,G.H)*0.46/(G.maxlvl+2);G.cx=cx;G.cy=cy;G.gap=gap;
  1436. G.nodes.forEach(n=>{n.tx=n.root?cx:cx+n.lvl*gap*Math.cos(n.ang);n.ty=n.root?cy:cy+n.lvl*gap*Math.sin(n.ang);});}
  1437. function startG(){G.on=true;const idx={};G.nodes=DATA.nodes.map((n,i)=>{idx[n.id]=i;return{...n,x:G.W/2,y:G.H/2,vx:0,vy:0,
  1438. r:n.root?13:(n.kind==='area'?7:3+Math.min(6,Math.sqrt(n.size/1100))),col:n.root?'#fff':COL[grp(n.area)]};});
  1439. G.edges=DATA.edges.map(e=>({s:idx[e.s],t:idx[e.t],kind:e.kind,tree:e.tree})).filter(e=>e.s!=null&&e.t!=null);
  1440. layout();legend();G.grow=0;loop();}
  1441. function spider(){if(G.grow<1)G.grow=Math.min(1,G.grow+0.02);const g=1-Math.pow(1-G.grow,3);
  1442. G.nodes.forEach(n=>{if(n===G.drag)return;n.x+=(G.cx+(n.tx-G.cx)*g-n.x)*0.14;n.y+=(G.cy+(n.ty-G.cy)*g-n.y)*0.14;});}
  1443. function physics(){const N=G.nodes,rep=2200,k=.045;
  1444. for(let i=0;i<N.length;i++){const a=N[i];for(let j=i+1;j<N.length;j++){const b=N[j];let dx=a.x-b.x,dy=a.y-b.y,d2=dx*dx+dy*dy+.01,d=Math.sqrt(d2),f=rep/d2;a.vx+=dx/d*f;a.vy+=dy/d*f;b.vx-=dx/d*f;b.vy-=dy/d*f;}}
  1445. G.edges.forEach(e=>{const a=N[e.s],b=N[e.t];let dx=b.x-a.x,dy=b.y-a.y,d=Math.sqrt(dx*dx+dy*dy)+.01,L=e.kind==='own'?70:150,f=(d-L)*k*(e.kind==='own'?1.4:.5);a.vx+=dx/d*f;a.vy+=dy/d*f;b.vx-=dx/d*f;b.vy-=dy/d*f;});
  1446. N.forEach(n=>{n.vx+=(G.W/2-n.x)*.001;n.vy+=(G.H/2-n.y)*.001;if(n===G.drag)return;n.x+=n.vx*=.85;n.y+=n.vy*=.85;});}
  1447. function loop(){($('#physics').checked?physics:spider)();draw();requestAnimationFrame(loop);}
  1448. function draw(){ctx.clearRect(0,0,G.W,G.H);const v=G.view;ctx.save();ctx.translate(v.x,v.y);ctx.scale(v.k,v.k);
  1449. const sr=$('#showref').checked,so=$('#showown').checked,phys=$('#physics').checked;
  1450. if(!phys){ctx.strokeStyle='rgba(255,255,255,.04)';for(let l=1;l<=G.maxlvl;l++){ctx.beginPath();ctx.arc(G.cx,G.cy,l*G.gap*(1-Math.pow(1-G.grow,3)),0,7);ctx.stroke();}}
  1451. G.edges.forEach(e=>{if(e.tree||(e.kind==='ref'&&!sr)||(e.kind==='own'&&!so))return;const a=G.nodes[e.s],b=G.nodes[e.t];ctx.strokeStyle=e.kind==='ref'?'rgba(0,255,136,.13)':'rgba(120,120,120,.07)';ctx.lineWidth=.6;ctx.beginPath();ctx.moveTo(a.x,a.y);ctx.lineTo(b.x,b.y);ctx.stroke();});
  1452. G.edges.forEach(e=>{if(!e.tree||(e.kind==='ref'&&!sr)||(e.kind==='own'&&!so))return;const a=G.nodes[e.s],b=G.nodes[e.t],par=a.lvl<=b.lvl?a:b,hot=G.hover&&(a===G.hover||b===G.hover);
  1453. ctx.strokeStyle=hot?'rgba(255,255,255,.7)':(COL[grp(par.area)]+'66');ctx.lineWidth=hot?1.6:1;ctx.beginPath();ctx.moveTo(a.x,a.y);ctx.lineTo(b.x,b.y);ctx.stroke();});
  1454. G.nodes.forEach(n=>{if(n.root){ctx.beginPath();ctx.arc(n.x,n.y,n.r+7,0,7);ctx.fillStyle='rgba(255,255,255,.1)';ctx.fill();}
  1455. ctx.beginPath();ctx.arc(n.x,n.y,n.r,0,7);ctx.fillStyle=n.col;ctx.fill();if(n.kind==='area'||n.root){ctx.strokeStyle=n.root?'#00ff88':'#000';ctx.lineWidth=n.root?2:1.4;ctx.stroke();}
  1456. if(n.root||n.kind==='area'||n===G.hover||v.k>1.7){ctx.fillStyle=n.root?'#00ff88':(n===G.hover?'#fff':'#bdbdbd');ctx.font=((n.root||n.kind==='area')?'600 ':'')+(n.root?13:11)+'px '+getComputedStyle(document.body).getPropertyValue('--mono');
  1457. ctx.fillText(n.root?(DATA.root+' ◀ вход'):(n.kind==='area'?n.label:n.label.slice(0,30)),n.x+n.r+4,n.y+3);}});ctx.restore();}
  1458. function legend(){const groups={};DATA.nodes.forEach(n=>{const g=grp(n.area);groups[g]=(groups[g]||0)+1;});
  1459. const rank=x=>x==='docs'?1:x==='root'?2:0; // сервисы первыми, docs/root в конце
  1460. const order=Object.keys(groups).sort((a,b)=>rank(a)-rank(b)||a.localeCompare(b));
  1461. const lbl={docs:'docs/* (сквозное)',root:'корень/прочее'};
  1462. $('#legend').innerHTML='<div class="row" style="color:#00ff88">● '+DATA.root+' = вход</div>'+
  1463. order.map(g=>'<div class="row"><span class="dot" style="background:'+colorOf(g)+'"></span>'+(lbl[g]||g)+' <span style="color:var(--muted)">'+groups[g]+'</span></div>').join('')+
  1464. '<div class="row" style="color:var(--muted);margin-top:4px">кольцо = шагов от входа</div>';}
  1465. function tw(mx,my){return{x:(mx-G.view.x)/G.view.k,y:(my-G.view.y)/G.view.k};}
  1466. function pick(mx,my){const w=tw(mx,my);let b=null,bd=1e9;G.nodes.forEach(n=>{const d=(n.x-w.x)**2+(n.y-w.y)**2;if(d<bd&&d<(n.r+6)**2){bd=d;b=n;}});return b;}
  1467. cv.addEventListener('mousedown',e=>{const r=cv.getBoundingClientRect();const n=pick(e.clientX-r.left,e.clientY-r.top);G.moved=false;if(n){G.drag=n;G.ds={x:e.clientX,y:e.clientY};}else G.pan={x:e.clientX,y:e.clientY,vx:G.view.x,vy:G.view.y};});
  1468. cv.addEventListener('mousemove',e=>{const r=cv.getBoundingClientRect(),mx=e.clientX-r.left,my=e.clientY-r.top;
  1469. if(G.drag){const w=tw(mx,my);G.drag.x=w.x;G.drag.y=w.y;if(G.ds&&Math.abs(e.clientX-G.ds.x)+Math.abs(e.clientY-G.ds.y)>4)G.moved=true;return;}
  1470. if(G.pan){G.view.x=G.pan.vx+(e.clientX-G.pan.x);G.view.y=G.pan.vy+(e.clientY-G.pan.y);return;}
  1471. const n=pick(mx,my);G.hover=n;const tip=$('#tip');if(n){tip.style.display='block';tip.style.left=(mx+16)+'px';tip.style.top=(my+12)+'px';
  1472. tip.innerHTML='<div>'+esc(n.root?DATA.root:n.label)+'</div><div class="s">'+esc(n.kind==='area'?n.area:(n.rel||'')+' · '+n.lvl+' шаг(ов)')+'</div>';}else tip.style.display='none';});
  1473. addEventListener('mouseup',()=>{if(G.drag){const n=G.drag;G.drag=null;if(!G.moved&&n.kind==='doc'){document.querySelector('.tab[data-v=browse]').click();open(n.rel);}}G.pan=null;});
  1474. cv.addEventListener('wheel',e=>{e.preventDefault();const r=cv.getBoundingClientRect(),mx=e.clientX-r.left,my=e.clientY-r.top,f=e.deltaY<0?1.12:.89,w=tw(mx,my);G.view.k=Math.max(.3,Math.min(5,G.view.k*f));G.view.x=mx-w.x*G.view.k;G.view.y=my-w.y*G.view.k;},{passive:false});
  1475. $('#reheat').onclick=()=>{if(!G.on)return;layout();G.nodes.forEach(n=>{n.x=G.cx;n.y=G.cy;n.vx=0;n.vy=0;});G.grow=0;};
  1476. $('#physics').onchange=()=>{if(!$('#physics').checked&&G.on){layout();G.grow=1;}};
  1477. open(DATA.files[DATA.root]?DATA.root:Object.keys(F)[0]);
  1478. </script></body></html>"""
  1479. if __name__ == "__main__":
  1480. main()

Ratings and comments

Sign in to rate this skill.

Sign in

Free to use. If it’s helped, pass it on.