overmind
玄霖超脑 · 无量网络 v4 重构版— 66 模块6通道 AI 认知神经系统 · 装一次,你所有的 AI 工具从此共享一个永远不失忆的大脑。跨会话记忆 · 多 Agent 互通 · 自动代码审查 · 零配置
Safety scan
A static text scan, not a guarantee. Select the badge to see what was scanned and every finding.
Badge for your README
Markdown
[](/listings/xuanlinai--overmind)HTML
<a href="/listings/xuanlinai--overmind"><img src="/api/listings/xuanlinai--overmind/badge.svg" alt="findsafeskills safety rating" /></a>Install
Install instructions from the README
As written by the repo's author. findsafeskills has not verified these steps.
git clone https://github.com/xuanlinAI/overmind.git && cd overmind && node install.js
一条命令完成全部 6 步:
| 步骤 | 内容 | |:--:|------| | 1 | **环境探测** — OS/CPU/内存/磁盘/Python/Node/AI Agent 全检 | | 2 | **安全快照** — 安装前备份所有关键文件 | | 3 | **完整性校验** — 检测缺失/损坏,自动修复 | | 4 | **依赖安装** — npm + pip 按需安装 | | 5 | **配置生成** — .overmind_env.json + hooks/MCP/CLAUDE.md 自动写入 | | 6 | **冒烟测试** — 6 通道全链验证,失败自动回滚 |
安装后设置 `DEEPSEEK_API_KEY`,重启 Claude Code 即可。
What was scanned
The files the safety scan read, as written by the repository's author and shown as plain text. Lines with a finding are marked.
plugin.json · Manifest · 51 lines
View this file on GitHub (read at the repository's current version: the scan did not record which commit it read)
- {
- "name": "xuanlin-overmind",
- "version": "v4",
- "description": "CC 的外挂认知引擎 — 66 模块 · 5 层记忆 · 知识图谱推理 · 红队审查 · 自进化",
- "author": "玄霖AI (xuanlinAI)",
- "hooks": {
- "SessionStart": [
- {
- "matcher": "",
- "hooks": [
- {
- "type": "command",
- "command": "mshta vbscript:Execute(\"CreateObject(\"\"WScript.Shell\"\").Run \"\"node <INSTALL_DIR>/inject.js\"\",0,False:close\")"
- }
- ]
- }
- ],
- "UserPromptSubmit": [
- {
- "matcher": "",
- "hooks": [
- {
- "type": "command",
- "command": "mshta vbscript:Execute(\"CreateObject(\"\"WScript.Shell\"\").Run \"\"node <INSTALL_DIR>/inject.js\"\",0,False:close\")"
- }
- ]
- }
- ],
- "SessionEnd": [
- {
- "matcher": "",
- "hooks": [
- {
- "type": "command",
- "command": "mshta vbscript:Execute(\"CreateObject(\"\"WScript.Shell\"\").Run \"\"node <INSTALL_DIR>/consolidate.js\"\",0,False:close\")"
- }
- ]
- }
- ]
- },
- "mcpServers": {
- "ctxproxy": {
- "type": "stdio",
- "command": "python",
- "args": [
- "<INSTALL_DIR>/daemon.py"
- ],
- "env": {}
- }
- }
- }
README.md · README · 341 lines
View this file on GitHub (read at the repository's current version: the scan did not record which commit it read)
- # 玄霖超脑 · 无量网络 v4 重构版
- <p align="center">
- <b>66 模块 AI 认知神经系统 · 6 通道架构 · 知识图谱推理 · 红队审查 · 自进化</b>
- </p>
- <p align="center">
- <a href="#中文">🇨🇳 中文</a> | <a href="#english">English</a>
- </p>
- <p align="center">
- <b>不是记忆,是推理。不是工具,是神经系统。不是插件,是认知引擎。</b>
- </p>
- <p align="center">
- 🥇 6 通道架构 — 串并联管道 + z2 中枢 + n2 终端<br>
- 🥈 零配置自适应安装 — 探针 + 自修复 + 诚实降级<br>
- 🥉 零进程触发 — 根治 Windows 窗口闪烁<br>
- 🔮 舰队模式 — 多 CC 自发现 + 跨 CC 技能 boosting + 冲突检测<br>
- 🧠 66 模块实时互连 — 知识图谱 4500+ 节点 · 10 种关系 · 被动推理
- </p>
- ---
- <span id="中文"></span>
- # 🇨🇳 中文
- ## 这是什么
- 玄霖超脑 v4(无量网络)是 Claude Code 的外挂认知引擎。66 个模块通过 6 条通道 + 事件总线实时互连,形成 AI 界首个认知神经系统。
- **它在你不在的时候仍在思考。你写的每行代码先被红队分身攻击才到你面前。你做的每个决策都有"选了另一个会怎样"的阴影追踪。你开的每个 CC 实例都能看到彼此在做什么。**
- ## 重构版 vs 初版 v4
- | 维度 | 初版 v4 | 重构版 v4 |
- |------|---------|-----------|
- | **安装方式** | 手动配置,Windows 特化 | `node install.js` 一键,全平台自适应 |
- | **通道架构** | 2 通道 (串+并) | **6 通道** (CH1-CH6) |
- | **管道阶段** | 21 阶段 | **37 阶段** + 量子缓存 |
- | **广播模块** | 18 模块 | **48 模块** |
- | **AI Agent 适配** | 仅 Claude Code | 自动检测 CC/Cursor/Aider/Codex/Gemini |
- | **z2 中枢** | 无 | daemon.py FleetWatcher,5s 舰队广播 |
- | **n2 终端** | 无 | communicator AI 筛选 + 串并联后处理 |
- | **舰队模式** | 无 | 多 CC 自发现 + 跨 CC 协同 |
- | **窗口闪烁** | Windows 多次闪现 | **彻底根治**(零进程触发架构) |
- | **自修复** | 无 | vault.js 校验 + 快照回滚 |
- | **错误处理** | 裸 except 吞信号 | 全部 `except Exception` + error 监听 |
- | **execSync 安全** | 14 处 cmd 窗口 | 全替换 execHidden (spawnSync) |
- | **自进化** | Hermes Fusion 基础 | 14 轮融合 + 70 次自动晋升 + 4 个 AI 工作流 |
- | **紧急恢复** | 无 | 安装失败自动快照回滚 |
- ## 版本演进
- | 版本 | 代号 | 模块数 | 核心 |
- |------|------|--------|------|
- | v1 | Context Proxy | 9 | 基础记忆 |
- | v2 | Xuanlin Overmind | 9+ | 知识图谱 · 自进化 |
- | v3 | (跳过发布) | 30 | 5 层记忆 · 18 MCP 工具 |
- | **v4 初版** | **无量网络** | **66** | **事件总线 · 红队 · 反事实 · 心理模型** |
- | **v4 重构版** | **无量网络重构** | **66** | **6 通道 · z2 中枢 · n2 终端 · 自适应安装 · 零进程触发** |
- ## 性能
- | 指标 | 数值 |
- |------|------|
- | 核心延迟 p50 | 0.001ms (60/66 模块亚毫秒) |
- | 管道吞吐 | 270 stages/s (缓存命中 ∞) |
- | 广播延迟 p99 | <1ms |
- | 事件 I/O | 2,500 write/s, 2,800 drain/s |
- | 6 通道并发 | 77ms/周期, 0% 崩溃率 |
- | 极限压力测试 | 200 周期 · 50 CC · 10,000 事件, 0 错误 |
- | 内存占用 | <500MB (66 模块) |
- | 日均 API 成本 | $0.05-0.15 |
- | 发布自测 | **42/42 PASSED** |
- [完整自测说明 → docs/SELF_TEST_CHECKLIST.md](docs/SELF_TEST_CHECKLIST.md)
- ## 安装
- ```bash
- git clone https://github.com/xuanlinAI/overmind.git && cd overmind && node install.js
- ```
- 一条命令完成全部 6 步:
- | 步骤 | 内容 |
- |:--:|------|
- | 1 | **环境探测** — OS/CPU/内存/磁盘/Python/Node/AI Agent 全检 |
- | 2 | **安全快照** — 安装前备份所有关键文件 |
- | 3 | **完整性校验** — 检测缺失/损坏,自动修复 |
- | 4 | **依赖安装** — npm + pip 按需安装 |
- | 5 | **配置生成** — .overmind_env.json + hooks/MCP/CLAUDE.md 自动写入 |
- | 6 | **冒烟测试** — 6 通道全链验证,失败自动回滚 |
- 安装后设置 `DEEPSEEK_API_KEY`,重启 Claude Code 即可。
- ## 六通道架构
- ```
- ┌── 核心模块 ──┐ ┌── z2中枢 ──┐ ┌── n2终端 ──┐
- │ │ │ │ │ │
- 串联 ────┤ CH1 │ │ CH3 │ │ CH5 │
- │ 37 stages │ │ !include │ │ 8链补全 │
- │ → injection │ │ → LLM自觉 │ │ → enrich │
- │ │ │ │ │ │
- 并联 ────┤ CH2 │ │ CH4 │ │ CH6 │
- │ 48 fire │ │ event→bus │ │ bus→11通知 │
- │ → sideFx │ │ → 9模块 │ │ → modules │
- └──────────────┘ └────────────┘ └────────────┘
- ```
- ## 核心能力 (66 模块)
- ### 记忆系统 (5层)
- - 语义记忆:SQLite + FTS5 + jieba 中文分词,AI 精选
- - 程序性记忆:同模式 ≥3 次自动晋升为可复用模板
- - 情景记忆:AI 生成会话摘要存档,按项目检索
- - 知识图谱:10 种关系类型,BFS 被动推理,因果链追踪
- - 反馈回路:4,500+ 事件驱动有效性评分自动调节
- ### 知识图谱 (10 种关系)
- `depends_on → part_of → blocked_by → causes → solves → related_to → extends → conflicts_with → alternative_to → triggers`
- 4,500+ 节点,4,600+ 边。被动推理:搜索一个记忆自动带出因果链。
- ### 智能分析 (CH1 37 阶段管道)
- 意图预判 · 安全门禁 · 提交门禁 · 输出盾 · 人格画像(8维) · 红队审查 · 异常检测 · 预测调试 · 成本分析 · 跨会话续接 · 技能编排 · 反事实引擎 · 知识验证 · 预测调试 · 检查点 · TDD约束 · 环境预取 · 噪声学习 · 心理模型 · 梦境研究(Pro深度分析) · 创意合成 · 思维链透明 · 自主研究 · 预算拦截 · 跨项目迁移(6领域) · 技能血统 · 记忆预算(5维) · 反流失 · 预判加载 · 会话简报 · 因果可视化 · 时间旅行 · 僵尸检测(90天) · 自愈检查 · 模块链接
- ### z2 中枢 (daemon.py)
- - FleetWatcher 5 秒扫描所有 CC 会话,提取 Q/A,生成舰队广播
- - CH3 直连:`.fleet_broadcast.md` → `!include` → LLM 自动感知同伴
- - CH4 中枢广播:`.event_queue/` → wiring.js drain → bus → 9 模块
- - 零进程触发:hook 写 `.trigger.tmp` → daemon 原子 rename → spawn inject.js
- - 20 个 MCP 工具,含 `fleet_status` `fleet_peek` 舰队查询
- ### n2 终端 (communicator.js)
- - AI 筛选:flash 模型过滤注入文档,94% 压缩率
- - CH5 串联补全:8 模块后处理链
- - CH6 并联通知:11 模块并行接收过滤后输出
- ### 涌现功能
- 自我镜像回路 · 舰队自省记忆 · 跨 CC 技能 boosting · 舰队冲突检测 · 跨 CC 创意碰撞 · CH4 去重 · 技能自动贴标签 (30/30)
- ### 技能生态
- 双阶段选择(AI精选+激进退底) · 偏好学习 · 技能编排(A→B→C链) · 技能热加载 · 技能血统 · 技能市场协议(manifest) · 30 个自创建技能(全标签)
- ### 安全与质量
- 红队审查(恶魔分身攻击) · 安全拦截(rm -rf检测) · 输出盾 · 提交门禁 · TDD约束 · 预算拦截(3次止损) · 补丁模式
- ### 部署与运维
- 自适应安装器(probe/vault/config/ready) · 舰队模式(多CC共享大脑) · 自愈系统(崩→重启,坏→恢复) · 看门狗(每分钟检查) · 日志旋转(5000行) · 跨平台(Win/Mac/Linux) · 多Agent(CC/Cursor/Aider等8个) · 零窗口闪烁
- ### MCP 工具 (20 个)
- `search_memory` · `save_memory` · `list_skills` · `create_skill` · `memory_stats` · `current_context` · `hermes_fusion` · `search_procedural` · `search_episodes` · `save_episodic` · `search_graph` · `expand_keys` · `create_edge` · `graph_stats` · `search_warnings` · `record_feedback` · `skill_rankings` · `skill_prefs` · **`fleet_status`** · **`fleet_peek`**
- ## 与竞品对比
- | | Xuanlin Overmind v4 | Mem0 | AgentMemory |
- |---|---|---|---|
- | 部署 | 100% 本地免费 | 云服务 | 本地 |
- | 模块数 | **66** | ~8 | ~12 |
- | 通道架构 | **6 通道** | 无 | 无 |
- | 图推理 | **10 种关系**,免费 | 锁 $249/月 | 基础 BM25 |
- | 认知功能 | 预警/预测/进化/梦境/红队/反事实/心理模型/舰队 | 无 | 无 |
- | 舰队协同 | **多 CC 自发现+协同** | 无 | 无 |
- | 自适应安装 | **全平台零配置** | 手动 | 手动 |
- | 日均成本 | **$0.05-0.15** | $8+/天 (Pro) | $0 (本地) |
- | 自测覆盖 | **42 项全绿** | 未公开 | 未公开 |
- 他们做的是 AI 的硬盘。Overmind 做的是 AI 的脑。
- ---
- <span id="english"></span>
- # 🇬🇧 English
- ## What is this
- Xuanlin Overmind v4 (Immeasurable Network) is a cognitive engine for Claude Code. 66 modules interconnected through 6 channels and an event bus form the first cognitive nervous system for AI coding agents.
- **It thinks while you're away. Every line of code you write is attacked by a red-team shadow before you see it. Every decision you make has a counterfactual shadow tracking "what if you chose differently." Every CC instance you open can see what the others are doing.**
- ## Refactored v4 vs Original v4
- | Dimension | Original v4 | Refactored v4 |
- |-----------|-------------|---------------|
- | **Install** | Manual config, Windows-only | `node install.js` one-click, cross-platform |
- | **Channels** | 2 (serial+parallel) | **6 channels** (CH1-CH6) |
- | **Pipeline stages** | 21 | **37** + quantum caching |
- | **Broadcast modules** | 18 | **48** |
- | **Agent support** | Claude Code only | Auto-detect CC/Cursor/Aider/Codex/Gemini |
- | **z2 Hub** | None | daemon.py FleetWatcher, 5s fleet broadcast |
- | **n2 Terminal** | None | AI filter + serial/parallel post-processing |
- | **Fleet mode** | None | Multi-CC auto-discovery + cross-CC collaboration |
- | **Window flash** | Frequent Windows console flash | **Completely eliminated** |
- | **Self-repair** | None | vault.js verification + snapshot rollback |
- | **Error handling** | Naked except swallowing signals | All `except Exception` + error listeners |
- | **execSync safety** | 14 cmd.exe window sites | All replaced with execHidden (spawnSync) |
- | **Self-evolution** | Basic Hermes Fusion | 14 fusions + 70 auto-promotions + 4 workflows |
- | **Emergency recovery** | None | Install failure → auto snapshot rollback |
- ## Version History
- | Version | Codename | Modules | Highlights |
- |---------|----------|---------|------------|
- | v1 | Context Proxy | 9 | Basic memory |
- | v2 | Xuanlin Overmind | 9+ | Knowledge graph · Self-evolution |
- | v3 | (Skipped) | 30 | 5-layer memory · 18 MCP tools |
- | **v4 Original** | **Immeasurable Network** | **66** | **Event bus · Red team · Counterfactual · Theory of mind** |
- | **v4 Refactored** | **Immeasurable Network R2** | **66** | **6-channel · z2 hub · n2 terminal · Adaptive install** |
- ## Performance
- | Metric | Value |
- |--------|-------|
- | Core latency p50 | 0.001ms (60/66 modules sub-ms) |
- | Pipeline throughput | 270 stages/s (∞ with cache hit) |
- | Broadcast latency p99 | <1ms |
- | Event I/O | 2,500 write/s, 2,800 drain/s |
- | 6-channel concurrent | 77ms/cycle, 0% crash rate |
- | Extreme stress test | 200 cycles · 50 CC · 10,000 events, 0 errors |
- | Memory footprint | <500MB (66 modules) |
- | Daily API cost | $0.05-0.15 |
- | Release self-test | **42/42 PASSED** |
- [Full self-test documentation → docs/SELF_TEST_CHECKLIST.md](docs/SELF_TEST_CHECKLIST.md)
- ## Installation
- ```bash
- git clone https://github.com/xuanlinAI/overmind.git && cd overmind && node install.js
- ```
- One command completes all 6 steps:
- | Step | Description |
- |:--:|------|
- | 1 | **Environment Probe** — OS/CPU/RAM/Disk/Python/Node/AI Agent detection |
- | 2 | **Safety Snapshot** — Backup all critical files before changes |
- | 3 | **Integrity Check** — Detect missing/corrupted files, auto-repair |
- | 4 | **Dependencies** — npm + pip install as needed |
- | 5 | **Config Generation** — .overmind_env.json + hooks/MCP/CLAUDE.md auto-written |
- | 6 | **Smoke Test** — Full 6-channel verification, auto-rollback on failure |
- Set `DEEPSEEK_API_KEY` after install, restart Claude Code.
- ## Six-Channel Architecture
- ```
- ┌── Core Modules ──┐ ┌── z2 Hub ──┐ ┌── n2 Terminal ──┐
- │ │ │ │ │ │
- Serial ──┤ CH1 │ │ CH3 │ │ CH5 │
- │ 37 stages │ │ !include │ │ 8-chain enrich │
- │ → injection │ │ → LLM │ │ → enrich doc │
- │ │ │ │ │ │
- Parallel ┤ CH2 │ │ CH4 │ │ CH6 │
- │ 48 fire │ │ event→bus │ │ bus→11 notify │
- │ → side effects │ │ → 9 modules│ │ → modules │
- └──────────────────┘ └────────────┘ └─────────────────┘
- ```
- ## Core Capabilities (66 Modules)
- ### Memory System (5 Layers)
- - **Semantic Memory** — SQLite + FTS5 + jieba Chinese tokenizer, AI-curated selection
- - **Procedural Memory** — Patterns occurring ≥3 times auto-promoted to reusable templates
- - **Episodic Memory** — AI-generated session summaries, searchable by project
- - **Knowledge Graph** — 10 relation types, BFS passive reasoning, causal chain tracking
- - **Feedback Loop** — 4,500+ events driving automated effectiveness scoring
- ### Knowledge Graph (10 Relations)
- `depends_on → part_of → blocked_by → causes → solves → related_to → extends → conflicts_with → alternative_to → triggers`
- 4,500+ nodes, 4,600+ edges. Passive reasoning: searching one memory automatically surfaces its causal chain.
- ### Intelligent Analysis (CH1 37-Stage Pipeline)
- Intent prediction · Security gatekeeper · Commit gate · Output shield · Persona profiling (8-dim) · Red-team adversarial review · Anomaly detection · Forecast debugging · Cost optimization · Cross-session continuity · Skill composition · Counterfactual engine · Knowledge verification · Predictive debugging · Checkpoints · TDD enforcement · Environment prefetch · Noise learning · Theory of mind · Dream research (pro model) · Creative synthesis · Reasoning transparency · Autonomous research · Budget interception · Cross-project transfer (6 domains) · Skill lineage · Memory budgeting (5-dim) · Anti-compaction · Preloading · Session briefing · Causal visualization · Time travel · Zombie detection (90-day) · Self-healing · Module nexus
- ### z2 Hub (daemon.py)
- - FleetWatcher: 5-second scan of all CC sessions, Q/A extraction, fleet broadcast generation
- - CH3 Direct: `.fleet_broadcast.md` → `!include` → LLM auto-awareness of peer CCs
- - CH4 Hub Broadcast: `.event_queue/` → wiring.js drain → bus → 9 modules
- - Zero-process trigger: hook writes `.trigger.tmp` → daemon atomic rename → spawn inject.js
- - 20 MCP tools, including `fleet_status` and `fleet_peek`
- ### n2 Terminal (communicator.js)
- - AI Filter: flash model filtering, 94% compression ratio
- - CH5 Serial Enrichment: 8-module post-processing chain
- - CH6 Parallel Notification: 11 modules receive filtered output simultaneously
- ### Emergent Capabilities
- Self-mirror loop · Fleet self-aware memory · Cross-CC skill boosting · Fleet conflict detection · Cross-CC creative collision · CH4 dedup · Auto skill tagging (30/30)
- ### Skill Ecosystem
- Dual-stage AI selection with aggressive fallback · Preference learning · Skill chain detection (A→B→C) · Hot-reload · Skill lineage · Marketplace protocol · 30 auto-created skills (fully tagged)
- ### Security & Quality
- Red-team adversarial review (shadow twin attack) · Security interception (rm -rf detection) · Output shield · Commit gate · TDD enforcement · Budget interceptor (3-failure stop-loss) · Patch mode
- ### Deployment & Operations
- Adaptive installer (probe/vault/config/ready) · Fleet mode (multi-CC shared brain) · Self-healing (crash→restart, corrupt→recover) · Watchdog (per-minute checks) · Log rotation (5,000 lines) · Cross-platform (Win/Mac/Linux) · Multi-agent (CC/Cursor/Aider + 5 more) · Zero window flash
- ### MCP Tools (20)
- `search_memory` · `save_memory` · `list_skills` · `create_skill` · `memory_stats` · `current_context` · `hermes_fusion` · `search_procedural` · `search_episodes` · `save_episodic` · `search_graph` · `expand_keys` · `create_edge` · `graph_stats` · `search_warnings` · `record_feedback` · `skill_rankings` · `skill_prefs` · **`fleet_status`** · **`fleet_peek`**
- ## vs Competitors
- | | Xuanlin Overmind v4 | Mem0 | AgentMemory |
- |---|---|---|---|
- | Deployment | 100% local, free | Cloud service | Local |
- | Modules | **66** | ~8 | ~12 |
- | Channel architecture | **6 channels** | None | None |
- | Graph reasoning | **10 relations**, free | Locked behind $249/mo | Basic BM25 |
- | Cognitive functions | Warning/Prediction/Evolution/Dream/Red-team/Counterfactual/ToM/Fleet | None | None |
- | Fleet collaboration | **Multi-CC auto-discovery** | None | None |
- | Adaptive install | **Cross-platform, zero-config** | Manual | Manual |
- | Daily cost | **$0.05-0.15** | $8+/day (Pro) | $0 (local) |
- | Self-test coverage | **42 items all-green** | Unknown | Unknown |
- They built AI's hard drive. Overmind is AI's brain.
- ## Platform Compatibility
- | Agent | Status |
- |-------|--------|
- | Claude Code | ✅ Native |
- | Cursor | ✅ Adapted (auto-detect) |
- | Aider | ✅ Adapted (auto-detect) |
- | Hermes Agent | ✅ Adapter |
- | Codex CLI | ✅ Adapter |
- | Gemini CLI | ✅ Adapter |
- | OpenClaw | ✅ Adapter |
- ---
- <p align="center">
- Made by <a href="https://github.com/xuanlinAI">玄霖AI (xuanlinAI)</a>
- </p>
daemon.py · Bundled script · 1,500 lines · 3 findings marked
- """Xuanlin Overmind z2 Hub — MCP server + FleetWatcher broadcast"""
- import json, sys, os, sqlite3, hashlib, re, time, glob, jieba, threading, tempfile, subprocess
- # pythonw on Windows has sys.stdout = None — redirect to devnull to prevent
- # process hang when print() buffer fills (subsystem-level deadlock)
- if sys.stdout is None:
- sys.stdout = open(os.devnull, 'w')
- if sys.stderr is None:
- sys.stderr = open(os.devnull, 'w')
- ROOT = os.path.dirname(os.path.abspath(__file__))
- HOME = os.environ.get('HOME') or os.environ.get('USERPROFILE') or os.path.expanduser('~')
- CC_SESSIONS_DIR = os.path.join(HOME, '.claude', 'sessions')
- CC_PROJECTS_DIR = os.path.join(HOME, '.claude', 'projects')
- FLEET_BROADCAST_FILE = os.path.join(ROOT, '.fleet_broadcast.md')
- EVENT_QUEUE_DIR = os.path.join(ROOT, '.event_queue')
- FLEET_STATE_FILE = os.path.join(ROOT, '.fleet_state.json')
- SCAN_INTERVAL = 5 # seconds
- SKILL_DIRS = [
- os.path.join(ROOT, 'skills'), # custom skills
- os.path.join(HOME, '.claude', 'skills'), # CC skills dir
- os.path.join(HOME, '.claude', 'plugins', 'cache'), # installed plugins
- ]
- DB_PATH = os.path.join(ROOT, 'memory.db')
- GRAPH_DB_PATH = os.path.join(ROOT, 'graph.db')
- RELATION_TYPES = [
- 'depends_on', 'part_of', 'blocked_by', 'causes', 'solves',
- 'related_to', 'extends', 'conflicts_with', 'alternative_to', 'triggers'
- ]
- def init_db():
- db = sqlite3.connect(DB_PATH)
- db.execute('CREATE TABLE IF NOT EXISTS semantic (id INTEGER PRIMARY KEY AUTOINCREMENT, key TEXT UNIQUE, content TEXT, tags TEXT DEFAULT "", created_at TEXT DEFAULT (datetime("now")), updated_at TEXT DEFAULT (datetime("now")), access_count INTEGER DEFAULT 0)')
- for col, ctype in [('confidence', 'REAL DEFAULT 0.5'), ('last_accessed', 'TEXT'), ('promotion_count', 'INTEGER DEFAULT 0'), ('content_hash', 'TEXT')]:
- try: db.execute(f'ALTER TABLE semantic ADD COLUMN {col} {ctype}')
- except Exception: pass
- for col, ctype in [('version', "TEXT DEFAULT '1.0.0'"), ('requires', 'TEXT'), ('provides', 'TEXT'), ('invoke_count', 'INTEGER DEFAULT 0'), ('last_invoked', 'TEXT'), ('quality_score', 'REAL DEFAULT 0.3')]:
- try: db.execute(f'ALTER TABLE skill_index ADD COLUMN {col} {ctype}')
- except Exception: pass
- try: db.execute('CREATE VIRTUAL TABLE IF NOT EXISTS semantic_fts USING fts5(key, content, tags, tokenize="unicode61")')
- except Exception: pass
- try: db.execute('ALTER TABLE semantic ADD COLUMN source_session TEXT')
- except Exception: pass
- try: db.execute('ALTER TABLE semantic ADD COLUMN dedup_key TEXT')
- except Exception: pass
- try: db.execute('ALTER TABLE semantic ADD COLUMN procedural_source TEXT')
- except Exception: pass
- try: db.execute('CREATE INDEX IF NOT EXISTS idx_semantic_dedup ON semantic(dedup_key)')
- except Exception: pass
- db.execute('''CREATE TABLE IF NOT EXISTS episodic (
- id INTEGER PRIMARY KEY AUTOINCREMENT,
- session_id TEXT NOT NULL,
- summary TEXT DEFAULT "",
- task TEXT DEFAULT "",
- message_count INTEGER DEFAULT 0,
- transcript_path TEXT DEFAULT "",
- project_name TEXT DEFAULT "",
- created_at TEXT NOT NULL
- )''')
- db.execute('CREATE TABLE IF NOT EXISTS skill_index (name TEXT PRIMARY KEY, description TEXT, triggers TEXT DEFAULT "", file_path TEXT, installed_at TEXT DEFAULT (datetime("now")))')
- db.execute('CREATE TABLE IF NOT EXISTS evolution_log (id INTEGER PRIMARY KEY AUTOINCREMENT, session_id TEXT, action TEXT, detail TEXT, created_at TEXT DEFAULT (datetime("now")))')
- db.execute('''CREATE TABLE IF NOT EXISTS procedural (
- id INTEGER PRIMARY KEY AUTOINCREMENT,
- name TEXT UNIQUE NOT NULL,
- description TEXT NOT NULL,
- steps TEXT NOT NULL,
- trigger_patterns TEXT DEFAULT '',
- use_count INTEGER DEFAULT 0,
- status TEXT DEFAULT "active",
- created_at TEXT DEFAULT (datetime("now"))
- )''')
- try: db.execute('CREATE VIRTUAL TABLE IF NOT EXISTS procedural_fts USING fts5(name, description, steps, trigger_patterns, tokenize="unicode61")')
- except Exception: pass
- db.execute("INSERT OR IGNORE INTO semantic (key, content, tags) VALUES ('_schema_version', '1', 'system')")
- db.execute('CREATE INDEX IF NOT EXISTS idx_semantic_stale ON semantic(last_accessed, updated_at)')
- db.commit()
- return db
- db = init_db()
- # ---- FEEDBACK LOOP ----
- for col, ctype in [('effectiveness_score', 'REAL DEFAULT 0.5'), ('last_effective_at', 'TEXT'), ('ineffective_count', 'INTEGER DEFAULT 0'), ('injected_count', 'INTEGER DEFAULT 0')]:
- try: db.execute(f'ALTER TABLE semantic ADD COLUMN {col} {ctype}')
- except Exception: pass
- db.execute('''CREATE TABLE IF NOT EXISTS feedback_events (
- id INTEGER PRIMARY KEY AUTOINCREMENT,
- memory_key TEXT NOT NULL,
- event_type TEXT NOT NULL,
- session_id TEXT,
- detail TEXT DEFAULT "",
- created_at TEXT DEFAULT (datetime("now")))''')
- try: db.execute('CREATE INDEX IF NOT EXISTS idx_feedback_key ON feedback_events(memory_key)')
- except Exception: pass
- db.execute('''CREATE TABLE IF NOT EXISTS skill_feedback (
- id INTEGER PRIMARY KEY AUTOINCREMENT,
- skill_name TEXT NOT NULL,
- event_type TEXT NOT NULL,
- task_context TEXT DEFAULT "",
- session_id TEXT,
- effectiveness REAL DEFAULT 0.5,
- created_at TEXT DEFAULT (datetime("now")))''')
- try: db.execute('CREATE INDEX IF NOT EXISTS idx_skillfb_name ON skill_feedback(skill_name)')
- except Exception: pass
- db.execute('''CREATE TABLE IF NOT EXISTS skill_prefs (
- skill_name TEXT NOT NULL,
- task_pattern TEXT NOT NULL,
- use_count INTEGER DEFAULT 1,
- effectiveness REAL DEFAULT 0.5,
- last_used TEXT,
- UNIQUE(skill_name, task_pattern))''')
- def record_feedback(memory_key, event_type, session_id='', detail=''):
- valid = {'injected', 'referenced', 'helped', 'did_not_help', 'caused_confusion'}
- if event_type not in valid: return False
- try:
- db.execute('INSERT INTO feedback_events (memory_key, event_type, session_id, detail) VALUES (?,?,?,?)', (memory_key, event_type, session_id, detail))
- if event_type == 'injected':
- db.execute("UPDATE semantic SET injected_count = COALESCE(injected_count, 0) + 1, last_accessed = datetime('now') WHERE key = ?", (memory_key,))
- elif event_type == 'helped':
- db.execute("UPDATE semantic SET effectiveness_score = MIN(1.0, COALESCE(effectiveness_score, 0.5) + 0.15), last_effective_at = datetime('now'), confidence = MIN(1.0, COALESCE(confidence, 0.5) + 0.1) WHERE key = ?", (memory_key,))
- elif event_type in ('did_not_help', 'caused_confusion'):
- db.execute("UPDATE semantic SET effectiveness_score = MAX(0.05, COALESCE(effectiveness_score, 0.5) - 0.2), ineffective_count = COALESCE(ineffective_count, 0) + 1, confidence = MAX(0.1, COALESCE(confidence, 0.5) - 0.1) WHERE key = ?", (memory_key,))
- elif event_type == 'referenced':
- db.execute("UPDATE semantic SET access_count = COALESCE(access_count, 0) + 1, last_accessed = datetime('now') WHERE key = ?", (memory_key,))
- db.commit()
- return True
- except Exception: return False
- def prune_ineffective_mems():
- bad = db.execute("SELECT key, content, COALESCE(effectiveness_score,0.5) as eff, COALESCE(ineffective_count,0) as nc FROM semantic WHERE key != '_schema_version' AND ineffective_count >= 3 AND COALESCE(effectiveness_score, 0.5) < 0.2").fetchall()
- if not bad: return 0
- archive_dir = os.path.join(ROOT, 'memory', 'archive')
- os.makedirs(archive_dir, exist_ok=True)
- archive_file = os.path.join(archive_dir, f'ineffective_{time.strftime("%Y%m%d_%H%M%S")}.json')
- archived = [{'key': r[0], 'content': r[1], 'effectiveness': r[2], 'ineffective_count': r[3]} for r in bad]
- with open(archive_file, 'w', encoding='utf-8') as f:
- json.dump(archived, f, ensure_ascii=False, indent=2)
- for r in bad:
- db.execute('DELETE FROM semantic WHERE key = ?', (r[0],))
- db.execute("DELETE FROM semantic_fts WHERE rowid = (SELECT rowid FROM semantic_fts WHERE key = ?)", (r[0],))
- db.execute('DELETE FROM feedback_events WHERE memory_key = ?', (r[0],))
- db.execute("INSERT INTO evolution_log (session_id, action, detail) VALUES ('hermes', 'prune_ineffective', ?)", (json.dumps({'count': len(bad)}),))
- db.commit()
- return len(bad)
- # ---- GRAPH DATABASE ----
- def init_graph():
- gdb = sqlite3.connect(GRAPH_DB_PATH)
- gdb.execute('PRAGMA journal_mode = WAL')
- gdb.execute('PRAGMA wal_autocheckpoint = 200')
- gdb.execute('''CREATE TABLE IF NOT EXISTS nodes (
- key TEXT PRIMARY KEY, label TEXT DEFAULT '',
- node_type TEXT DEFAULT "memory", importance REAL DEFAULT 0.5,
- created_at TEXT DEFAULT (datetime("now")), updated_at TEXT DEFAULT (datetime("now")))''')
- gdb.execute('''CREATE TABLE IF NOT EXISTS edges (
- id INTEGER PRIMARY KEY AUTOINCREMENT,
- source TEXT NOT NULL, target TEXT NOT NULL, relation_type TEXT NOT NULL,
- confidence REAL DEFAULT 0.5, evidence TEXT DEFAULT "",
- source_session TEXT DEFAULT "",
- created_at TEXT DEFAULT (datetime("now")),
- access_count INTEGER DEFAULT 0,
- UNIQUE(source, target, relation_type))''')
- gdb.execute('CREATE INDEX IF NOT EXISTS idx_gedges_src ON edges(source)')
- gdb.execute('CREATE INDEX IF NOT EXISTS idx_gedges_tgt ON edges(target)')
- gdb.execute('CREATE INDEX IF NOT EXISTS idx_gedges_type ON edges(relation_type)')
- try: gdb.execute("ALTER TABLE edges ADD COLUMN updated_at TEXT")
- except Exception: pass
- try: gdb.execute("CREATE VIRTUAL TABLE IF NOT EXISTS edges_fts USING fts5(source, target, relation_type, evidence, tokenize='unicode61')")
- except Exception: pass
- gdb.commit()
- return gdb
- gdb = init_graph()
- def upsert_edge(source, target, relation_type, confidence=0.5, evidence='', session_id=''):
- if relation_type not in RELATION_TYPES:
- relation_type = 'related_to'
- if source == target: return None
- row = gdb.execute('SELECT id, confidence FROM edges WHERE source=? AND target=? AND relation_type=?', (source, target, relation_type)).fetchone()
- if row:
- new_conf = min(1.0, max(row[1], confidence) + 0.05)
- gdb.execute('UPDATE edges SET confidence=?, evidence=CASE WHEN ?!="" AND (evidence IS NULL OR evidence NOT LIKE "%"||?||"%") THEN COALESCE(evidence,"") || " | " || ? ELSE evidence END, access_count=access_count+1 WHERE id=?', (new_conf, evidence, evidence, evidence, row[0]))
- return row[0]
- try:
- cur = gdb.execute('INSERT OR IGNORE INTO edges (source, target, relation_type, confidence, evidence, source_session) VALUES (?,?,?,?,?,?)', (source, target, relation_type, confidence, evidence, session_id))
- gdb.commit()
- if cur.lastrowid:
- try: gdb.execute('INSERT INTO edges_fts(rowid, source, target, relation_type, evidence) VALUES (?,?,?,?,?)', (cur.lastrowid, source, target, relation_type, evidence))
- except Exception: pass
- gdb.execute('INSERT OR IGNORE INTO nodes (key) VALUES (?)', (source,))
- gdb.execute('INSERT OR IGNORE INTO nodes (key) VALUES (?)', (target,))
- gdb.commit()
- return cur.lastrowid
- except Exception: pass
- return None
- def get_neighbors(key, depth=2):
- visited = {key}
- nodes_map = {}
- edges = []
- frontier = [key]
- for d in range(depth):
- next_frontier = []
- for nk in frontier:
- rows = gdb.execute('SELECT source, target, relation_type, confidence, evidence FROM edges WHERE source=? OR target=? ORDER BY confidence DESC', (nk, nk)).fetchall()
- for src, tgt, rtype, conf, ev in rows:
- edges.append({'source': src, 'target': tgt, 'relation_type': rtype, 'confidence': conf, 'evidence': ev or ''})
- nb = tgt if src == nk else src
- if nb not in visited:
- visited.add(nb)
- next_frontier.append(nb)
- node = gdb.execute('SELECT key, label, node_type, importance FROM nodes WHERE key=?', (nb,)).fetchone()
- if node: nodes_map[nb] = {'key': node[0], 'label': node[1], 'node_type': node[2], 'importance': node[3]}
- else: nodes_map[nb] = {'key': nb, 'label': '', 'node_type': 'memory', 'importance': 0.5}
- frontier = next_frontier
- return {'nodes': list(nodes_map.values()), 'edges': edges, 'root': key, 'depth': depth}
- def expand_keys(memory_keys, depth=1):
- if not memory_keys: return {'keys': [], 'subgraph': {'nodes': [], 'edges': []}}
- all_edges = []
- all_nodes = {}
- expanded = set(memory_keys)
- for k in memory_keys:
- sub = get_neighbors(k, depth)
- for e in sub['edges']: all_edges.append(e)
- for n in sub['nodes']:
- all_nodes[n['key']] = n
- expanded.add(n['key'])
- seen = set()
- deduped = []
- for e in all_edges:
- sig = (e['source'], e['target'], e['relation_type'])
- if sig not in seen:
- seen.add(sig)
- deduped.append(e)
- return {'keys': list(expanded), 'subgraph': {'nodes': list(all_nodes.values()), 'edges': deduped}}
- def search_graph(query, limit=10):
- if not query:
- rows = gdb.execute('SELECT source, target, relation_type, confidence, evidence FROM edges ORDER BY confidence DESC LIMIT ?', (limit,)).fetchall()
- return [{'source': r[0], 'target': r[1], 'relation_type': r[2], 'confidence': r[3], 'evidence': r[4] or ''} for r in rows]
- try:
- rows = gdb.execute("SELECT e.source, e.target, e.relation_type, e.confidence, e.evidence FROM edges_fts f JOIN edges e ON f.rowid = e.id WHERE edges_fts MATCH ? ORDER BY rank LIMIT ?", (query, limit)).fetchall()
- except Exception:
- like = f'%{query}%'
- rows = gdb.execute('SELECT source, target, relation_type, confidence, evidence FROM edges WHERE source LIKE ? OR target LIKE ? OR relation_type LIKE ? OR evidence LIKE ? ORDER BY confidence DESC LIMIT ?', (like, like, like, like, limit)).fetchall()
- return [{'source': r[0], 'target': r[1], 'relation_type': r[2], 'confidence': r[3], 'evidence': r[4] or ''} for r in rows]
- def get_graph_stats():
- nodes = gdb.execute('SELECT COUNT(*) FROM nodes').fetchone()[0]
- edges = gdb.execute('SELECT COUNT(*) FROM edges').fetchone()[0]
- types = gdb.execute('SELECT relation_type, COUNT(*) as c FROM edges GROUP BY relation_type ORDER BY c DESC').fetchall()
- return {'nodes': nodes, 'edges': edges, 'relation_types': [{'type': r[0], 'count': r[1]} for r in types]}
- # Graph maintenance: prune low-confidence edges, merge duplicates
- def prune_graph():
- deleted = gdb.execute('DELETE FROM edges WHERE confidence < 0.15').rowcount
- if deleted:
- gdb.execute("INSERT INTO evolution_log (session_id, action, detail) VALUES ('hermes', 'graph_prune', ?)", (json.dumps({'deleted': deleted}),))
- db.commit()
- return deleted
- def read_msg():
- line = sys.stdin.readline()
- if not line:
- return None
- return json.loads(line)
- def send_msg(obj):
- sys.stdout.write(json.dumps(obj, ensure_ascii=False) + '\n')
- sys.stdout.flush()
- def search_skills(query='', limit=3):
- if not query:
- rows = db.execute('SELECT * FROM skill_index ORDER BY installed_at DESC LIMIT ?', (limit,)).fetchall()
- return [{'name': r[0], 'description': r[1], 'triggers': r[2], 'file_path': r[3]} for r in rows]
- words = [w.lower() for w in query.split() if len(w) > 1]
- if not words:
- rows = db.execute('SELECT * FROM skill_index LIMIT ?', (limit,)).fetchall()
- return [{'name': r[0], 'description': r[1], 'triggers': r[2], 'file_path': r[3]} for r in rows]
- # Get all skills, score each by word hit count
- rows = db.execute('SELECT * FROM skill_index').fetchall()
- scored = []
- for r in rows:
- name = (r[0] or '').lower()
- desc = (r[1] or '').lower()
- triggers = (r[2] or '').lower()
- combined = name + ' ' + desc + ' ' + triggers
- score = 0
- for w in words:
- if w in name: score += 3
- elif w in triggers: score += 3
- elif w in desc: score += 1
- # Partial match (substring)
- elif any(w in part for part in combined.split()): score += 0.5
- if score > 0:
- scored.append((score, r))
- scored.sort(key=lambda x: x[0], reverse=True)
- return [{'name': r[0], 'description': r[1], 'triggers': r[2], 'file_path': r[3]} for _, r in scored[:limit]]
- def char_bigrams(text):
- """Generate character bigrams for CJK text matching"""
- chars = re.sub(r'[\s\d\W]', '', text)
- bigrams = set()
- for i in range(len(chars) - 1):
- bigrams.add(chars[i:i+2])
- return list(bigrams)
- def jieba_segment(text):
- """Segment Chinese text with jieba, preserve non-Chinese tokens"""
- words = jieba.cut(text)
- return ' '.join(w for w in words if w.strip())
- def search_hybrid(query, limit=10):
- q = jieba_segment(query).split()
- fts_query = ' OR '.join(w for w in q if w and len(w) > 1)
- rows = []
- if fts_query:
- try:
- rows = db.execute(
- "SELECT s.rowid, s.key, s.content, s.tags, rank, COALESCE(m.effectiveness_score, 0.5), COALESCE(m.injected_count, 0), COALESCE(m.ineffective_count, 0) FROM semantic_fts s JOIN semantic m ON s.key = m.key WHERE semantic_fts MATCH ? AND s.key != '_schema_version' ORDER BY rank LIMIT ?",
- (fts_query, limit)).fetchall()
- except Exception:
- rows = db.execute(
- "SELECT id, key, content, tags, COALESCE(effectiveness_score, 0.5), COALESCE(injected_count, 0), COALESCE(ineffective_count, 0) FROM semantic WHERE key != '_schema_version' ORDER BY updated_at DESC LIMIT ?",
- (limit,)).fetchall()
- else:
- rows = db.execute(
- "SELECT id, key, content, tags, COALESCE(effectiveness_score, 0.5), COALESCE(injected_count, 0), COALESCE(ineffective_count, 0) FROM semantic WHERE key != '_schema_version' ORDER BY updated_at DESC LIMIT ?",
- (limit,)).fetchall()
- seen = set()
- result = []
- for r in rows:
- k = r[1]
- if k not in seen:
- seen.add(k)
- update_confidence(k, 0.03)
- # FTS row: (rowid, key, content, tags, rank, eff, inj_cnt, ineff_cnt) = 8 cols
- # Non-FTS row: (id, key, content, tags, eff, inj_cnt, ineff_cnt) = 7 cols
- n = len(r)
- eff = round(r[n-3], 2) if n >= 7 and r[n-3] is not None else 0.5
- inj = int(r[n-2] or 0) if n >= 7 else 0
- ineff = int(r[n-1] or 0) if n >= 7 else 0
- result.append({'key': k, 'content': r[2], 'tags': r[3] if r[3] else '', 'effectiveness': eff, 'injected_count': inj, 'ineffective_count': ineff})
- auto_promote()
- return result[:limit]
- def call_ai_api(messages, max_tokens=16384, timeout=120):
- import urllib.request
- API_KEY = os.environ.get('DEEPSEEK_API_KEY') or os.environ.get('ANTHROPIC_AUTH_TOKEN') or 'YOUR_LLM_API_KEY'
- Warning: If no DeepSeek key is set, it falls back to the user's ANTHROPIC_AUTH_TOKEN and sends it as a Bearer token to api.deepseek.com, together with memory and session text.
- Line 363 on GitHub
- body = json.dumps({
- 'model': 'deepseek-v4-flash',
- 'max_tokens': max_tokens,
- 'messages': messages
- }).encode('utf-8')
- req = urllib.request.Request('https://api.deepseek.com/v1/chat/completions',
- data=body,
- headers={'Content-Type': 'application/json', 'Authorization': f'Bearer {API_KEY}'})
- resp = urllib.request.urlopen(req, timeout=timeout)
- data = json.loads(resp.read())
- return data['choices'][0]['message']['content']
- def select_memories_ai(query, all_mems, limit=5):
- """FTS5 pre-filter → flash model picks best memories. Same pattern as inject.js selectMemoriesAI."""
- if not all_mems:
- return []
- # Pre-filter: sliding bigram keyword scoring
- raw = (query or '').lower()
- keywords = []
- for i in range(len(raw) - 1):
- ch = raw[i:i+2]
- if ch and len(ch) == 2:
- keywords.append(ch)
- eng_words = re.findall(r'[a-z0-9]{3,}', raw)
- keywords.extend(eng_words)
- has_cjk = bool(re.search(r'[一-鿿]', query or ''))
- scored = []
- for m in all_mems:
- key_low = (m.get('key') or '').lower()
- content_low = (m.get('content') or '').lower()
- txt = key_low + ' ' + content_low
- s = 0
- for k in keywords:
- if k in key_low: s += 2
- elif k in content_low: s += 1
- if has_cjk and len(content_low) > 20:
- s += 0.5
- eff = float(m.get('effectiveness', 0.5) or 0.5)
- inj = int(m.get('injected_count', 0) or 0)
- # Effectiveness boost: proven memories get up to +1.5, unproven get baseline
- eff_boost = eff * 1.5 + (0.1 if inj > 0 else 0)
- scored.append((s + eff_boost, m))
- scored.sort(key=lambda x: x[0], reverse=True)
- shortlist = [m for _, m in scored[:40]]
- if len(shortlist) < 3:
- return shortlist[:limit]
- catalog = '\n'.join([f"- {m['key']}: {(m.get('content') or '')[:60]}" for m in shortlist])
- prompt = f"Pick the {limit} BEST memories for this task. Prefer specific technical facts. Skip plugin-internal noise. You MUST pick at least 2. Return ONLY JSON array: [\"key1\",\"key2\",\"key3\"]\n\nTask: {(query or '')[:300]}\n\n{catalog}"
- try:
- result = call_ai_api([{'role': 'user', 'content': prompt}], max_tokens=16384, timeout=120)
- if not result:
- return shortlist[:limit]
- names = []
- try: names = json.loads(result.strip())
- except Exception:
- m = re.search(r'\[.*\]', result, re.DOTALL)
- if m:
- try: names = json.loads(m.group(0))
- except Exception: pass
- if not isinstance(names, list):
- return shortlist[:limit]
- selected = []
- for n in names:
- for m in shortlist:
- if m.get('key') == n:
- selected.append(m)
- break
- return selected[:limit] if selected else shortlist[:limit]
- except Exception:
- return shortlist[:limit]
- def search_hybrid_ai(query, limit=10):
- """FTS5 pre-filter → AI selection. Fall back to pure FTS5 on API failure."""
- pre = search_hybrid(query, 30)
- if not pre:
- return []
- all_mems = [{'key': r['key'], 'content': r['content'], 'tags': r.get('tags', '')} for r in pre]
- result = select_memories_ai(query, all_mems, limit)
- for r in result:
- update_confidence(r['key'], 0.02)
- return result
- def auto_tag_skill(name, desc, content=''):
- """Extract matching keywords from skill name + description + body"""
- tags = set()
- text = (name + ' ' + desc + ' ' + (content or '')[:1000]).lower()
- # 1. Extract English technical terms (only camelCase, snake_case, dot.case — not plain words)
- for m in re.finditer(r'\b([a-z]+(?:[A-Z][a-z]+)+|[a-z]+_[a-z]+|[a-z]+\.[a-z]+)\b', text):
- word = m.group(1)
- if len(word) >= 4:
- tags.add(word.lower())
- # Also extract ALL_CAPS acronyms (FTS5, JSON, API, etc) — 2-6 uppercase letters
- for m in re.finditer(r'\b([A-Z]{2,6})\b', text):
- tags.add(m.group(1))
- # 2. Chinese technical bigrams (2-4 chars, filter common words)
- cjk = re.findall(r'[一-鿿]{2,4}', text)
- stop_cjk = {'使用','可以','需要','一个','这个','进行','通过','所有','什么','怎么','如何','如果','或者','但是','因为','所以','已经','没有','不是','只是','还是','并且','然后','其中','之后','之前','包含','以及'}
- for w in cjk:
- if w not in stop_cjk:
- tags.add(w)
- # 3. Tech domain keywords from name
- name_low = name.lower()
- domain_map = {
- 'reverse': ['逆向', 'reverse', '反汇编', 'decompile'],
- 'encrypt': ['加密', 'encrypt', '签名', 'sign', 'AES', 'RSA', 'SM2', 'SM4', 'HMAC', 'MD5'],
- 'frida': ['frida', 'hook', '注入', 'inject'],
- 'js': ['javascript', 'JS', 'AST', '反混淆', 'deobfuscate'],
- 'binary': ['二进制', 'binary', 'native', 'so文件', 'elf'],
- 'api': ['API', 'http', 'fetch', '接口', '请求'],
- 'wechat': ['微信', 'wechat', '小程序', 'miniprogram'],
- 'proxy': ['代理', 'proxy', 'mitmproxy', '抓包'],
- 'ticket': ['抢票', 'ticket', '预约', 'booking'],
- 'skill': ['技能', 'skill', '自动', 'auto'],
- 'review': ['审查', 'review', '代码质量'],
- 'test': ['测试', 'test', 'benchmark'],
- 'debug': ['调试', 'debug', '日志', 'log'],
- 'clean': ['清理', 'clean', '删除', 'remove'],
- 'inject': ['注入', 'inject', 'injection'],
- 'evolve': ['进化', 'evolve', '自进化', 'self'],
- }
- for key, terms in domain_map.items():
- if key in name_low:
- for t in terms:
- tags.add(t)
- # 4. Fallback: extract keywords from name itself (split on hyphen/underscore)
- if not tags:
- for part in re.split(r'[-_]', name):
- if len(part) >= 3:
- tags.add(part.lower())
- # 5. Merge and return
- result = ','.join(sorted(tags)[:80]) # cap at 80 tags
- return result
- def index_skills():
- skills = []
- for skills_dir in SKILL_DIRS:
- if not os.path.exists(skills_dir):
- continue
- for root, dirs, files in os.walk(skills_dir):
- depth = root.count(os.sep) - skills_dir.count(os.sep)
- if depth > 7:
- continue
- if 'SKILL.md' not in files:
- continue
- try:
- with open(os.path.join(root, 'SKILL.md'), encoding='utf-8') as f:
- content = f.read()
- fm_match = re.match(r'^---\n(.*?)\n---', content, re.DOTALL)
- if fm_match:
- meta = {}
- for line in fm_match.group(1).split('\n'):
- m = re.match(r'(\w+):\s*(.+)', line)
- if m:
- meta[m.group(1)] = m.group(2).strip()
- if 'name' in meta:
- desc = meta.get('description', '')
- triggers = re.search(r'TRIGGERS?:' + '\s*(.+)', desc, re.IGNORECASE)
- trigger_str = triggers.group(1) if triggers else ''
- db.execute('INSERT OR REPLACE INTO skill_index (name, description, triggers, file_path) VALUES (?,?,?,?)', (meta['name'], desc, trigger_str, os.path.join(root, 'SKILL.md')))
- skills.append({'name': meta['name'], 'description': desc, 'triggers': trigger_str})
- except Exception:
- pass
- # Also index flat .md files from skills/all/ (auto-created by Worker)
- all_dir = os.path.join(ROOT, 'skills', 'all')
- if os.path.exists(all_dir):
- for fname in os.listdir(all_dir):
- if not fname.endswith('.md'):
- continue
- fpath = os.path.join(all_dir, fname)
- try:
- with open(fpath, encoding='utf-8') as f:
- content = f.read()
- fm_match = re.match(r'^---\n(.*?)\n---', content, re.DOTALL)
- if fm_match:
- meta = {}
- for line in fm_match.group(1).split('\n'):
- m = re.match(r'(\w+):\s*(.+)', line)
- if m:
- meta[m.group(1)] = m.group(2).strip()
- if 'name' in meta:
- desc = meta.get('description', '')
- triggers = re.search(r'TRIGGERS?:' + '\s*(.+)', desc, re.IGNORECASE)
- trigger_str = triggers.group(1) if triggers else ''
- name = meta['name']
- db.execute('INSERT OR REPLACE INTO skill_index (name, description, triggers, file_path) VALUES (?,?,?,?)', (name, desc, trigger_str, fpath))
- # Auto-tag: generate matching_tags from name+desc+content if empty
- existing_tags = db.execute('SELECT matching_tags FROM skill_index WHERE name=?', (name,)).fetchone()
- if not existing_tags or not (existing_tags[0] or '').strip():
- # Use full content as fallback if desc is empty (YAML block scalars)
- tag_text = desc if desc and len(desc) > 10 else content
- tags = auto_tag_skill(name, tag_text, content)
- if tags:
- db.execute('UPDATE skill_index SET matching_tags=? WHERE name=?', (tags, name))
- skills.append({'name': name, 'description': desc, 'triggers': trigger_str})
- except Exception:
- pass
- db.commit()
- return skills
- def search_procedural(query, limit=5):
- q = re.sub(r'[^\w\s]', ' ', query).split()
- q = ' OR '.join(w for w in q if w)
- if not q:
- rows = db.execute("SELECT id, name, description, steps, trigger_patterns FROM procedural WHERE status='active' ORDER BY use_count DESC LIMIT ?", (limit,)).fetchall()
- else:
- try:
- rows = db.execute("SELECT rowid, name, description, steps, trigger_patterns, rank FROM procedural_fts WHERE procedural_fts MATCH ? ORDER BY rank LIMIT ?", (q, limit)).fetchall()
- except Exception:
- rows = db.execute("SELECT id, name, description, steps, trigger_patterns FROM procedural WHERE (description LIKE ? OR name LIKE ?) AND status='active' LIMIT ?", (f'%{query.split()[0]}%', f'%{query.split()[0]}%', limit)).fetchall()
- return [{'name': r[1], 'description': r[2], 'steps': r[3], 'triggers': r[4]} for r in rows]
- def save_procedural(name, description, steps, triggers=''):
- existing = db.execute('SELECT id FROM procedural WHERE name = ?', (name,)).fetchone()
- if existing:
- db.execute('UPDATE procedural SET description=?, steps=?, trigger_patterns=? WHERE name=?', (description, steps, triggers, name))
- else:
- db.execute('INSERT INTO procedural (name, description, steps, trigger_patterns) VALUES (?,?,?,?)', (name, description, steps, triggers))
- try:
- db.execute('DELETE FROM procedural_fts WHERE name = ?', (name,))
- db.execute('INSERT INTO procedural_fts(name, description, steps, trigger_patterns) VALUES (?,?,?,?)', (name, description, steps, triggers))
- except Exception: pass
- db.commit()
- def score_skills():
- """Calculate quality score for each skill based on invoke_count and recency."""
- db.execute("""
- UPDATE skill_index SET quality_score =
- CASE
- WHEN invoke_count >= 5 THEN 1.0
- WHEN invoke_count >= 2 THEN 0.8
- WHEN invoke_count = 1 THEN 0.5
- WHEN last_invoked IS NULL AND julianday('now') - julianday(installed_at) > 7 THEN 0.1
- ELSE 0.3
- END
- WHERE name IS NOT NULL
- """)
- db.commit()
- def prune_low_quality_skills():
- """Archive skills with quality_score < 0.2 and >7 days without invocation."""
- rows = db.execute("SELECT name, description, file_path, COALESCE(quality_score, 0) as qs FROM skill_index WHERE COALESCE(quality_score, 0) < 0.2 AND julianday('now') - julianday(installed_at) > 7").fetchall()
- if rows:
- archive_dir = os.path.join(ROOT, 'memory', 'archive')
- os.makedirs(archive_dir, exist_ok=True)
- archive_file = os.path.join(archive_dir, f'skills_pruned_{time.strftime("%Y%m%d")}.json')
- pruned = [{'name': r[0], 'description': r[1], 'file_path': r[2], 'quality_score': r[3]} for r in rows]
- with open(archive_file, 'w', encoding='utf-8') as f:
- json.dump(pruned, f, ensure_ascii=False, indent=2)
- for r in rows:
- db.execute('DELETE FROM skill_index WHERE name = ?', (r[0],))
- try: os.remove(r[2]) if r[2] and os.path.exists(r[2]) else None
- except Exception: pass
- db.execute("INSERT INTO evolution_log (session_id, action, detail) VALUES ('hermes', 'prune_skills', ?)", (json.dumps({'count': len(rows)}),))
- db.commit()
- return len(rows)
- return 0
- def auto_compact():
- """Merge similar memories by key prefix AND content similarity"""
- merged = 0
- # 1. Key-prefix merge
- rows = db.execute("SELECT key, COUNT(*) as cnt, GROUP_CONCAT(id) as ids FROM semantic WHERE key != '_schema_version' GROUP BY substr(key, 1, instr(key||'_','_')-1) HAVING cnt > 1").fetchall()
- for r in rows:
- ids = [int(x) for x in r[2].split(',')]
- if len(ids) > 1:
- for rid in ids[1:]:
- db.execute('DELETE FROM semantic WHERE id = ?', (rid,))
- merged += len(ids) - 1
- # 2. Content similarity: hash-bucket dedup (first 50 chars)
- buckets = {}
- all_mems = db.execute("SELECT id, key, content FROM semantic WHERE key != '_schema_version' AND LENGTH(content) > 30 ORDER BY id").fetchall()
- for mem in all_mems:
- h = hashlib.md5((mem[2] or '')[:50].encode()).hexdigest()[:8]
- if h not in buckets: buckets[h] = []
- buckets[h].append(mem)
- to_delete = set()
- for h, mems in buckets.items():
- if len(mems) < 2: continue
- for i in range(1, len(mems)):
- to_delete.add(mems[i][0])
- merged += 1
- for rid in to_delete:
- db.execute('DELETE FROM semantic WHERE id = ?', (rid,))
- if merged:
- db.commit()
- return merged
- def get_stats():
- sem = db.execute('SELECT COUNT(*) FROM semantic').fetchone()[0]
- proc = db.execute('SELECT COUNT(*) FROM procedural').fetchone()[0]
- skill = db.execute('SELECT COUNT(*) FROM skill_index').fetchone()[0]
- evo = db.execute('SELECT COUNT(*) FROM evolution_log').fetchone()[0]
- epi = get_episodic_count()
- return {'semanticCount': sem, 'proceduralCount': proc, 'skillCount': skill, 'evoCount': evo, 'episodeCount': epi}
- def save_semantic(key, content, tags='', source_session=None):
- existing = db.execute('SELECT id FROM semantic WHERE key = ?', (key,)).fetchone()
- seg = jieba_segment(content)
- if existing:
- db.execute('UPDATE semantic SET content=?, tags=?, updated_at=datetime("now") WHERE key=?', (content, tags, key))
- try: db.execute('DELETE FROM semantic_fts WHERE key = ?', (key,))
- except Exception: pass
- try: db.execute('INSERT INTO semantic_fts(key, content, tags) VALUES (?,?,?)', (key, seg, tags))
- except Exception: pass
- else:
- db.execute('INSERT INTO semantic (key, content, tags, source_session) VALUES (?,?,?,?)', (key, content, tags, source_session))
- try: db.execute('INSERT INTO semantic_fts(key, content, tags) VALUES (?,?,?)', (key, seg, tags))
- except Exception: pass
- db.commit()
- def save_episodic(session_id, summary='', task='', message_count=0, transcript_path='', project_name=''):
- db.execute("INSERT INTO episodic (session_id, summary, task, message_count, transcript_path, project_name, created_at) VALUES (?,?,?,?,?,?,datetime('now'))",
- (session_id, summary, task, message_count, transcript_path, project_name))
- db.commit()
- def search_episodes(query, limit=10):
- q = f'%{query}%'
- return [{'session_id': r[1], 'summary': r[2], 'task': r[3], 'message_count': r[4], 'project_name': r[6], 'created_at': r[7]}
- for r in db.execute('SELECT * FROM episodic WHERE summary LIKE ? OR task LIKE ? OR project_name LIKE ? ORDER BY created_at DESC LIMIT ?',
- (q, q, q, limit)).fetchall()]
- def get_episodic_count():
- return db.execute('SELECT COUNT(*) FROM episodic').fetchone()[0]
- # ─── Hermes Memory Engine ───
- def update_confidence(key, delta=0.05):
- """Boost confidence on access, cap at 1.0"""
- db.execute("UPDATE semantic SET confidence = MIN(1.0, COALESCE(confidence, 0.5) + ?), last_accessed = datetime('now'), access_count = access_count + 1 WHERE key = ?", (delta, key))
- db.commit()
- def decay_stale_memories():
- """Confidence decay: 30+ days no access → confidence *= 0.9. Archive if confidence < 0.2."""
- db.execute("UPDATE semantic SET confidence = MAX(0.1, COALESCE(confidence, 0.5) * 0.9) WHERE key != '_schema_version' AND (last_accessed < datetime('now', '-30 days') OR (last_accessed IS NULL AND updated_at < datetime('now', '-30 days')))")
- # Archive memories that decayed below 0.2
- stale = db.execute("SELECT key, content, tags, COALESCE(confidence,0.5) as conf, updated_at FROM semantic WHERE key != '_schema_version' AND COALESCE(confidence, 0.5) < 0.2").fetchall()
- if stale:
- archive_dir = os.path.join(ROOT, 'memory', 'archive')
- os.makedirs(archive_dir, exist_ok=True)
- archive_file = os.path.join(archive_dir, f'decay_{time.strftime("%Y%m%d")}.json')
- archived = []
- for r in stale:
- archived.append({'key': r[0], 'content': r[1], 'tags': r[2], 'confidence': r[3], 'updated_at': r[4]})
- db.execute('DELETE FROM semantic WHERE key = ?', (r[0],))
- db.execute("DELETE FROM semantic_fts WHERE rowid = (SELECT rowid FROM semantic_fts WHERE key = ?)", (r[0],))
- with open(archive_file, 'w', encoding='utf-8') as f:
- json.dump(archived, f, ensure_ascii=False, indent=2)
- db.execute("INSERT INTO evolution_log (session_id, action, detail) VALUES ('hermes', 'archive_decay', ?)", (json.dumps({'count': len(stale)}),))
- db.commit()
- def auto_promote():
- """Episodic→Semantic promotion check, Semantic→Procedural pattern creation"""
- events = []
- # Lower threshold: access_count >= 1, confidence >= 0.3
- rows = db.execute("""SELECT key, content, access_count, COALESCE(confidence, 0.3) as conf, tags
- FROM semantic WHERE key != '_schema_version' AND access_count >= 1 AND COALESCE(confidence, 0.3) >= 0.3
- AND COALESCE(promotion_count, 0) = 0
- ORDER BY access_count DESC, conf DESC LIMIT 5""").fetchall()
- # Meta/noise patterns — skip these for procedural promotion
- META_PATTERNS = [
- 'test_question', 'memory.*contain', 'knowledge.*in.*memory',
- 'losing.*cc.*history', 'cc.*history.*loses',
- 'inject_log', 'worker_log', 'mem_size', 'catalog.*chars',
- 'selectSkillsAI', 'selectMemoriesAI', 'getAllMemory',
- 'restarted.*session', 'user_says_restart', 'user_agrees',
- 'semanticCount', 'skillCount', 'promotion_count'
- ]
- for r in rows:
- key, content, ac, conf, tags = r
- # Skip meta/noise — self-referential or plugin-internal trivia
- txt = (key + ' ' + (content or '')).lower()
- if any(re.search(p, txt) for p in META_PATTERNS):
- db.execute("UPDATE semantic SET promotion_count = -1 WHERE key = ?", (key,))
- continue
- try:
- db.execute("""INSERT OR IGNORE INTO procedural (name, description, steps, trigger_patterns)
- VALUES (?, ?, ?, ?)""",
- (key, content, content, tags or ''))
- try:
- seg = ' '.join(jieba.cut(content))
- db.execute("INSERT OR IGNORE INTO procedural_fts(name, description, trigger_patterns) VALUES (?,?,?)", (key, seg, tags or ''))
- except Exception: pass
- except Exception: pass
- db.execute("UPDATE semantic SET promotion_count = 1, confidence = MIN(1.0, ?), procedural_source = ? WHERE key = ?", (ac * 0.1 + conf, key, key))
- db.execute("INSERT INTO evolution_log (session_id, action, detail) VALUES ('hermes', 'auto_promote', ?)", (json.dumps({'key': key, 'reason': 'procedural_pattern'}),))
- events.append({'key': key, 'action': 'promoted_to_procedural'})
- db.commit()
- return events
- def analyze_skills_with_pro():
- """Offline: pro reads all SKILL.md files, generates matching_tags for each"""
- import urllib.request
- API_KEY = os.environ.get('DEEPSEEK_API_KEY') or os.environ.get('ANTHROPIC_AUTH_TOKEN') or 'YOUR_LLM_API_KEY'
- # Add matching_tags column if needed
- try: db.execute('ALTER TABLE skill_index ADD COLUMN matching_tags TEXT DEFAULT \"\"')
- except Exception: pass
- rows = db.execute('SELECT name, file_path, description FROM skill_index WHERE 1=1').fetchall()
- if not rows: return {'analyzed': 0}
- # Build compact catalog
- catalog = ''
- for r in rows:
- name, fp, desc = r
- try:
- with open(fp, encoding='utf-8') as f:
- body = f.read()[:600]
- except Exception:
- body = desc or ''
- catalog += f'### {name}\n{body}\n\n'
- prompt = f"""Analyze each skill below. For each, output one line: SKILL_NAME: comma-separated Chinese+English trigger keywords that describe WHEN to use this skill.
- Keywords should cover: task types, domains, technologies, verbs. Be GENEROUS — include broad and specific matches.
- Example format:
- binary-analysis: 逆向,二进制,反汇编,加密算法,AES,SM4,IDA Pro,Ghidra,x64dbg,reverse engineering,binary RE
- my-review: 审查,review,代码质量,安全审计,code quality,bug finding
- Skills:
- {catalog}"""
- try:
- body = json.dumps({
- 'model': 'deepseek-v4-pro[1m]',
- 'max_tokens': 16384,
- 'messages': [{'role': 'user', 'content': prompt}]
- }).encode('utf-8')
- req = urllib.request.Request('https://api.deepseek.com/anthropic/v1/messages',
- Warning: Uploads the first 600 characters of every indexed skill file to api.deepseek.com, authenticated with a key taken from the environment. This sends local skill content to a third-party service.
- Line 806 on GitHub
- data=body,
- headers={'Content-Type': 'application/json', 'x-api-key': API_KEY, 'anthropic-version': '2023-06-01'})
- resp = urllib.request.urlopen(req, timeout=300)
- data = json.loads(resp.read())
- text_block = next((c for c in data.get('content', []) if c.get('type') == 'text'), None)
- result = text_block.get('text', '') if text_block else ''
- except Exception as e:
- db.execute("INSERT INTO evolution_log (session_id, action, detail) VALUES ('hermes', 'analyze_skills_error', ?)", (str(e)[:500],))
- db.commit()
- return {'analyzed': 0, 'error': str(e)[:100]}
- # Parse and apply
- analyzed = 0
- for line in result.split('\n'):
- line = line.strip()
- if ':' not in line: continue
- m = re.match(r'^([\w_-]+):\s*(.+)', line)
- if not m: continue
- name, tags = m.group(1), m.group(2)
- try:
- db.execute('UPDATE skill_index SET matching_tags = ? WHERE name = ?', (tags, name))
- analyzed += 1
- except Exception: pass
- db.commit()
- db.execute("INSERT INTO evolution_log (session_id, action, detail) VALUES ('hermes', 'analyze_skills', ?)", (json.dumps({'count': analyzed}),))
- db.commit()
- return {'analyzed': analyzed}
- def evolve_with_ai():
- """AI-driven evolution: merge similar, detect contradictions, create structured workflows"""
- import urllib.request
- API_KEY = os.environ.get('DEEPSEEK_API_KEY') or os.environ.get('ANTHROPIC_AUTH_TOKEN') or 'YOUR_LLM_API_KEY'
- Warning: Sends stored memory contents to api.deepseek.com, using ANTHROPIC_AUTH_TOKEN as the API key if no DeepSeek key is set. The stored memories come from the user's coding sessions.
- Line 838 on GitHub
- # Get top 30 high-value memories for AI analysis
- rows = db.execute("""SELECT key, content, tags, COALESCE(confidence,0.5) as conf, access_count
- FROM semantic WHERE key != '_schema_version' AND promotion_count >= 0
- AND COALESCE(confidence, 0.3) >= 0.3
- ORDER BY access_count DESC, conf DESC LIMIT 30""").fetchall()
- if len(rows) < 5: return {'merged': 0, 'deprecated': 0, 'workflows': 0}
- catalog = '\n'.join([f"- {r[0]}: {r[1][:120]}" for r in rows])
- prompt = f"""Analyze these memories from an AI coding assistant's long-term memory. Your job is to find improvements.
- ## TASKS
- 1. **Merge**: Find 2-3 groups of memories that say the same thing in different ways. Output merged version.
- 2. **Deprecate**: Find any memories that are clearly outdated/wrong based on newer ones. Mark them.
- 3. **Workflow**: If 3+ memories describe steps of a process, create a structured procedural workflow.
- ## OUTPUT JSON
- {{"merges": [{{"keys":["old1","old2"], "new_key":"merged_name", "new_content":"combined concise fact"}}], "deprecate": ["outdated_key"], "workflows": [{{"name":"workflow-name", "description":"what it does", "steps":["step1","step2","step3"], "trigger":"when to use"}}]}}
- ## MEMORIES
- {catalog}"""
- try:
- body = json.dumps({
- 'model': 'deepseek-v4-pro[1m]',
- 'max_tokens': 16384,
- 'messages': [{'role': 'user', 'content': prompt}]
- }).encode('utf-8')
- req = urllib.request.Request('https://api.deepseek.com/anthropic/v1/messages',
- data=body,
- headers={'Content-Type': 'application/json', 'x-api-key': API_KEY, 'anthropic-version': '2023-06-01'})
- resp = urllib.request.urlopen(req, timeout=300)
- data = json.loads(resp.read())
- text_block = next((c for c in data.get('content', []) if c.get('type') == 'text'), None)
- result = text_block.get('text', '') if text_block else ''
- except Exception as e:
- db.execute("INSERT INTO evolution_log (session_id, action, detail) VALUES ('hermes', 'ai_evolve_error', ?)", (str(e)[:500],))
- db.commit()
- return {'merged': 0, 'deprecated': 0, 'workflows': 0, 'error': str(e)[:100]}
- # Parse result
- try:
- m = re.search(r'\{.*\}', result, re.DOTALL)
- plan = json.loads(m.group(0)) if m else {}
- except Exception:
- plan = {}
- merged = 0; deprecated = 0; workflows = 0
- # Apply merges
- for mg in plan.get('merges', []):
- keys = mg.get('keys', [])
- nk, nc = mg.get('new_key'), mg.get('new_content')
- if not keys or not nk or not nc: continue
- try:
- # Save merged version
- db.execute("INSERT OR REPLACE INTO semantic (key, content, tags, confidence) VALUES (?,?,?,'merged',0.8)", (nk, nc))
- try: db.execute("INSERT OR REPLACE INTO semantic_fts(key, content, tags) VALUES (?,?,?)", (nk, ' '.join(jieba.cut(nc)), 'merged'))
- except Exception: pass
- # Mark old as deprecated
- for k in keys:
- db.execute("UPDATE semantic SET tags = 'deprecated', confidence = 0.1 WHERE key = ?", (k,))
- db.execute("INSERT INTO evolution_log (session_id, action, detail) VALUES ('hermes', 'ai_merge', ?)", (json.dumps({'keys': keys, 'new': nk}),))
- merged += 1
- except Exception: pass
- # Apply deprecations
- for dk in plan.get('deprecate', []):
- try:
- db.execute("UPDATE semantic SET tags = 'deprecated', confidence = 0.1 WHERE key = ?", (dk,))
- db.execute("INSERT INTO evolution_log (session_id, action, detail) VALUES ('hermes', 'ai_deprecate', ?)", (json.dumps({'key': dk}),))
- deprecated += 1
- except Exception: pass
- # Apply workflows
- for wf in plan.get('workflows', []):
- n, d, ss, t = wf.get('name'), wf.get('description'), wf.get('steps', []), wf.get('trigger', '')
- if not n or not ss: continue
- try:
- steps_text = '\n'.join(ss)
- db.execute("INSERT OR REPLACE INTO procedural (name, description, steps, trigger_patterns) VALUES (?,?,?,?)", (n, d or '', steps_text, t))
- try: db.execute("INSERT OR REPLACE INTO procedural_fts(name, description, trigger_patterns) VALUES (?,?,?)", (n, ' '.join(jieba.cut(d or '')), t))
- except Exception: pass
- db.execute("INSERT INTO evolution_log (session_id, action, detail) VALUES ('hermes', 'ai_workflow', ?)", (json.dumps({'name': n, 'steps': len(ss)}),))
- workflows += 1
- except Exception: pass
- db.commit()
- return {'merged': merged, 'deprecated': deprecated, 'workflows': workflows}
- def hermes_fusion():
- """Weekly fusion: decay stale, promote patterns, merge similars, prune skills + graph"""
- decay_stale_memories()
- promos = auto_promote()
- compacted = auto_compact()
- score_skills()
- pruned = prune_low_quality_skills()
- ai = evolve_with_ai()
- graph_pruned = prune_graph()
- ineffective_pruned = prune_ineffective_mems()
- db.execute("INSERT INTO evolution_log (session_id, action, detail) VALUES ('hermes', 'fusion', ?)", (json.dumps({'promotions': len(promos), 'compacted': compacted, 'skills_pruned': pruned, 'ai': ai, 'graph_pruned': graph_pruned, 'ineffective_pruned': ineffective_pruned}),))
- db.commit()
- return {'promotions': len(promos), 'compacted': compacted, 'skills_pruned': pruned, 'ai_evolution': ai, 'graph_pruned': graph_pruned, 'ineffective_pruned': ineffective_pruned}
- def search_warnings(query):
- """Detect dangerous patterns related to query from graph + feedback"""
- # Search graph for dangerous edges matching query
- try:
- rows = gdb.execute("SELECT e.source, e.target, e.relation_type, e.confidence, e.evidence FROM edges_fts f JOIN edges e ON f.rowid = e.id WHERE edges_fts MATCH ? AND e.relation_type IN ('blocked_by','conflicts_with','causes') AND e.confidence >= 0.4 ORDER BY rank LIMIT 10", (query,)).fetchall()
- except Exception:
- like = f'%{query}%'
- rows = gdb.execute("SELECT source, target, relation_type, confidence, evidence FROM edges WHERE (source LIKE ? OR target LIKE ? OR evidence LIKE ?) AND relation_type IN ('blocked_by','conflicts_with','causes') AND confidence >= 0.4 ORDER BY confidence DESC LIMIT 10", (like, like, like)).fetchall()
- warnings = []
- for src, tgt, rtype, conf, ev in rows:
- label = {'blocked_by': '⚠️ 阻塞风险', 'conflicts_with': '⚡ 潜在冲突', 'causes': '🔴 失败模式'}.get(rtype, '⚠️ 风险')
- fb = db.execute('SELECT COALESCE(effectiveness_score,0.5), COALESCE(ineffective_count,0) FROM semantic WHERE key=?', (tgt,)).fetchone()
- eff = fb[0] if fb else 0.5
- nc = fb[1] if fb else 0
- sev = 'high' if (conf >= 0.7 or (rtype == 'causes' and nc >= 2 and eff < 0.4)) else 'medium'
- warnings.append({
- 'source': src, 'target': tgt, 'relation_type': rtype,
- 'danger_type': rtype, 'severity': sev,
- 'reason': (ev or f'{src} {rtype} {tgt}')[:300],
- 'confidence': conf, 'evidence': ev or '',
- 'failure_count': nc, 'effectiveness': round(eff, 2),
- 'label': label
- })
- # Also check low-effectiveness memories matching query
- bad_mems = db.execute("SELECT key, content, COALESCE(effectiveness_score,0.5), COALESCE(ineffective_count,0), COALESCE(injected_count,0) FROM semantic WHERE key != '_schema_version' AND ineffective_count >= 2 AND COALESCE(effectiveness_score,0.5) < 0.3 AND key LIKE ? LIMIT 5", (f'%{query}%',)).fetchall()
- for k, c, eff, nc, inj in bad_mems:
- warnings.append({
- 'source': k, 'target': k, 'relation_type': 'self',
- 'danger_type': 'persistent_failure', 'severity': 'medium',
- 'reason': f'低效记忆: {c[:100]}(失败 {nc} 次,有效率 {eff:.1%})',
- 'confidence': 0.8, 'evidence': '',
- 'failure_count': nc, 'effectiveness': round(eff, 2),
- 'injected_count': inj, 'label': '🟡 低效记忆'
- })
- sev_ord = {'high': 0, 'medium': 1}
- warnings.sort(key=lambda w: (sev_ord.get(w['severity'], 2), -w['confidence']))
- return warnings[:8]
- def get_skill_rankings(limit=20):
- rows = db.execute('''SELECT skill_name,
- SUM(CASE WHEN event_type="injected" THEN 1 ELSE 0 END) as injected,
- SUM(CASE WHEN event_type="invoked" THEN 1 ELSE 0 END) as invoked,
- SUM(CASE WHEN event_type="completed" THEN 1 ELSE 0 END) as completed,
- SUM(CASE WHEN event_type="failed" THEN 1 ELSE 0 END) as failed,
- COALESCE(AVG(effectiveness), 0.5) as avg_eff,
- COUNT(*) as total
- FROM skill_feedback GROUP BY skill_name
- ORDER BY completed DESC, invoked DESC LIMIT ?''', (limit,)).fetchall()
- return [{'skill': r[0], 'injected': r[1], 'invoked': r[2], 'completed': r[3],
- 'failed': r[4], 'avg_effectiveness': round(r[5], 2), 'total': r[6],
- 'hit_rate': f'{int(r[2]/r[1]*100)}%' if r[1] > 0 else 'N/A'} for r in rows]
- def get_skill_prefs(skill_name=None):
- if skill_name:
- rows = db.execute('SELECT * FROM skill_prefs WHERE skill_name=? ORDER BY use_count DESC, effectiveness DESC', (skill_name,)).fetchall()
- else:
- rows = db.execute('SELECT * FROM skill_prefs ORDER BY use_count DESC, effectiveness DESC').fetchall()
- return [{'skill': r[0], 'task_pattern': r[1], 'use_count': r[2],
- 'effectiveness': round(r[3], 2), 'last_used': r[4]} for r in rows]
- # Index existing skills on startup
- try: index_skills()
- except Exception: pass
- # ═══════════════════════════════════════════════════════════
- # z2 HUB: FleetWatcher — scans all CC sessions, broadcasts
- # ═══════════════════════════════════════════════════════════
- class FleetWatcher:
- def __init__(self):
- self.running = True
- self.last_broadcast = ''
- self.lockfile = os.path.join(ROOT, '.fleetwatcher.lock')
- def start(self):
- t = threading.Thread(target=self._loop, daemon=True)
- t.start()
- def _loop(self):
- while self.running:
- try:
- self._scan_and_broadcast()
- self._check_triggers()
- except Exception:
- pass
- time.sleep(SCAN_INTERVAL)
- def _check_triggers(self):
- trigger_inject = os.path.join(ROOT, '.trigger_inject')
- trigger_inject_tmp = trigger_inject + '.tmp'
- trigger_consolidate = os.path.join(ROOT, '.trigger_consolidate')
- trigger_consolidate_tmp = trigger_consolidate + '.tmp'
- try:
- # Atomic consume: rename(tmp→real) before processing
- if os.path.exists(trigger_inject_tmp):
- try: os.rename(trigger_inject_tmp, trigger_inject)
- except OSError: pass
- if os.path.exists(trigger_consolidate_tmp):
- try: os.rename(trigger_consolidate_tmp, trigger_consolidate)
- except OSError: pass
- if os.path.exists(trigger_inject):
- os.remove(trigger_inject)
- subprocess.Popen(
- ['node', os.path.join(ROOT, 'inject.js')],
- creationflags=0x08000000 | 0x00000008,
- stdin=subprocess.DEVNULL, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL,
- close_fds=True)
- if os.path.exists(trigger_consolidate):
- os.remove(trigger_consolidate)
- subprocess.Popen(
- ['node', os.path.join(ROOT, 'consolidate.js')],
- creationflags=0x08000000 | 0x00000008,
- stdin=subprocess.DEVNULL, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL,
- close_fds=True)
- except Exception:
- pass
- def _scan_and_broadcast(self):
- sessions = self._discover_sessions()
- if not sessions:
- return
- for s in sessions:
- self._extract_qa(s)
- md = self._build_broadcast(sessions)
- if md == self.last_broadcast:
- return
- self.last_broadcast = md
- self._write_broadcast(md)
- self._push_event_queue(sessions)
- def _discover_sessions(self):
- sessions = []
- if not os.path.exists(CC_SESSIONS_DIR):
- return sessions
- for fname in os.listdir(CC_SESSIONS_DIR):
- if not fname.endswith('.json'):
- continue
- try:
- with open(os.path.join(CC_SESSIONS_DIR, fname), 'r', encoding='utf-8') as f:
- data = json.load(f)
- sessions.append({
- 'id': data.get('sessionId', fname.replace('.json', '')),
- 'pid': data.get('pid', 0),
- 'cwd': data.get('cwd', ''),
- 'status': 'active' if data.get('status') == 'busy' else 'idle',
- 'started_at': data.get('startedAt'),
- 'updated_at': data.get('updatedAt'),
- 'version': data.get('version', ''),
- })
- except Exception:
- pass
- return sessions
- def _extract_qa(self, s):
- qa_pairs = []
- try:
- # Dynamic: find the project dir containing this session's transcript
- proj_dir = None
- try:
- for dname in os.listdir(CC_PROJECTS_DIR):
- dpath = os.path.join(CC_PROJECTS_DIR, dname)
- if not os.path.isdir(dpath):
- continue
- candidate = os.path.join(dpath, s['id'] + '.jsonl')
- if os.path.exists(candidate):
- proj_dir = dpath
- break
- except Exception:
- pass
- if proj_dir is None:
- return
- jsonl = os.path.join(proj_dir, f"{s['id']}.jsonl")
- if not os.path.exists(jsonl):
- return
- s['transcript_mtime'] = os.path.getmtime(jsonl)
- s['recently_active'] = (time.time() - s['transcript_mtime']) < 300
- with open(jsonl, 'r', encoding='utf-8', errors='ignore') as f:
- lines = f.readlines()
- pairs = []
- current_q = None
- current_a = None
- for line in reversed(lines[-500:]):
- try:
- entry = json.loads(line.strip())
- except Exception:
- continue
- # Extract user text (skip tool results)
- if entry.get('type') == 'user' and not current_q:
- msg = entry.get('message', {})
- if isinstance(msg, dict) and msg.get('role') == 'user':
- content = msg.get('content', '')
- if isinstance(content, str) and content.strip():
- current_q = content.strip()
- elif isinstance(content, list):
- for part in content:
- if isinstance(part, dict) and part.get('text'):
- txt = part['text'].strip()
- if txt and len(txt) > 10:
- current_q = txt
- break
- continue
- # Extract assistant text
- if entry.get('type') == 'assistant' and current_q and not current_a:
- msg = entry.get('message', {})
- if isinstance(msg, dict) and msg.get('role') == 'assistant':
- content = msg.get('content', '')
- if isinstance(content, str) and content.strip():
- current_a = content.strip()
- elif isinstance(content, list):
- texts = []
- for part in content:
- if isinstance(part, dict) and part.get('text'):
- texts.append(part['text'].strip())
- if texts:
- current_a = '\n'.join(texts)
- if current_q and current_a:
- pairs.append({'q': current_q[:200], 'a': current_a[:300]})
- current_q = None
- current_a = None
- if len(pairs) >= 2:
- break
- s['qa_pairs'] = pairs
- # Detect topic from ai-title
- for line in reversed(lines[-100:]):
- try:
- entry = json.loads(line.strip())
- if entry.get('aiTitle'):
- s['topic'] = entry['aiTitle']
- break
- except Exception:
- pass
- if 'topic' not in s:
- s['topic'] = pairs[0]['q'][:80] if pairs else '(idle)'
- except Exception:
- s['qa_pairs'] = []
- s['topic'] = '(idle)'
- s['recently_active'] = False
- def _build_broadcast(self, sessions):
- # Merge with fleet_state (Overmind registrations)
- overmind = {}
- try:
- if os.path.exists(FLEET_STATE_FILE):
- with open(FLEET_STATE_FILE, 'r', encoding='utf-8') as f:
- fs = json.load(f)
- for inst in fs.get('instances', []):
- overmind[inst['id']] = inst
- except Exception:
- pass
- lines = ['📡 舰队广播 z2']
- lines.append(f'> {len(sessions)} 个实例 · {time.strftime("%H:%M:%S")} 刷新\n')
- our_session = None
- try:
- with open(os.path.join(CC_SESSIONS_DIR, [f for f in os.listdir(CC_SESSIONS_DIR) if f.endswith('.json')][0]), 'r') as f:
- pass
- except Exception:
- pass
- for s in sessions:
- sid = s['id'][:8]
- om = overmind.get(s['id'])
- has_om = bool(om)
- status_icon = '🟢' if s.get('recently_active') else '⚪'
- om_icon = '🧠' if has_om else '👁️'
- topic = (s.get('topic') or '(idle)')[:80]
- lines.append(f'### {sid} [{om_icon}] {status_icon} {topic}')
- qa = s.get('qa_pairs', [])
- if qa:
- for pair in qa:
- q_text = pair['q'][:120].replace('\n', ' ')
- a_text = pair['a'][:200].replace('\n', ' ')
- lines.append(f'> ❓ {q_text}')
- lines.append(f'> 💬 {a_text}')
- lines.append('')
- # Conflict check
- active_topics = [s.get('topic', '')[:50] for s in sessions if s.get('recently_active')]
- if len(active_topics) > 1 and len(set(active_topics)) < len(active_topics):
- lines.append('⚠️ 多个实例可能在做同一件事\n')
- # Locks from orchestrator
- active_locks = []
- for om_id, om in overmind.items():
- if om.get('current_topic'):
- active_locks.append(f"{om_id[:8]}=>{om['current_topic'][:40]}")
- if active_locks:
- lines.append('🔒 任务锁: ' + ', '.join(active_locks) + '\n')
- return '\n'.join(lines)
- def _write_broadcast(self, md):
- try:
- with open(FLEET_BROADCAST_FILE, 'w', encoding='utf-8') as f:
- f.write(md)
- except Exception:
- pass
- def _push_event_queue(self, sessions):
- try:
- os.makedirs(EVENT_QUEUE_DIR, exist_ok=True)
- data = {
- 'event': 'fleet:broadcast',
- 'ts': int(time.time() * 1000),
- 'data': {
- 'count': len(sessions),
- 'instances': [{
- 'id': s['id'][:8],
- 'topic': s.get('topic', '')[:120],
- 'status': 'active' if s.get('recently_active') else 'idle',
- 'qa': s.get('qa_pairs', []),
- 'cwd': s.get('cwd', ''),
- } for s in sessions]
- }
- }
- fpath = os.path.join(EVENT_QUEUE_DIR, f"fleet_broadcast_{int(time.time() * 1000)}.json")
- with open(fpath, 'w', encoding='utf-8') as f:
- json.dump(data, f, ensure_ascii=False)
- except Exception:
- pass
- # Start z2 FleetWatcher
- _watcher = FleetWatcher()
- _watcher.start()
- # ═══════════════════════════════════════════════════════════
- # MCP HANDLER
- # ═══════════════════════════════════════════════════════════
- def handle(req):
- mid = req.get('id')
- method = req.get('method')
- params = req.get('params', {})
- args = params.get('arguments', {})
- if method == 'initialize':
- return send_msg({'jsonrpc': '2.0', 'id': mid, 'result': {
- 'protocolVersion': '2024-11-05',
- 'capabilities': {'tools': {}},
- 'serverInfo': {'name': 'ctxproxy', 'version': '1.0.0'}
- }})
- if method == 'notifications/initialized':
- return
- if mid is None:
- return
- if method == 'tools/list':
- return send_msg({'jsonrpc': '2.0', 'id': mid, 'result': {'tools': [
- {'name': 'search_memory', 'description': '检索记忆(FTS5全文,可选AI精选)', 'inputSchema': {
- 'type': 'object', 'properties': {'query': {'type': 'string'}, 'limit': {'type': 'number'}, 'use_ai': {'type': 'boolean', 'description': '启用AI语义精选(FTS5初筛→flash精选)'}}, 'required': ['query']}},
- {'name': 'save_memory', 'description': '保存一条语义记忆', 'inputSchema': {
- 'type': 'object', 'properties': {'key': {'type': 'string'}, 'content': {'type': 'string'}, 'tags': {'type': 'string'}}, 'required': ['key', 'content']}},
- {'name': 'list_skills', 'description': '列出匹配的技能', 'inputSchema': {
- 'type': 'object', 'properties': {'query': {'type': 'string'}}}},
- {'name': 'create_skill', 'description': '创建新 SKILL.md', 'inputSchema': {
- 'type': 'object', 'properties': {'name': {'type': 'string'}, 'description': {'type': 'string'}, 'content': {'type': 'string'}}, 'required': ['name', 'description', 'content']}},
- {'name': 'memory_stats', 'description': '记忆统计', 'inputSchema': {
- 'type': 'object', 'properties': {}}},
- {'name': 'current_context', 'description': '获取当前会话最新上下文(任务+技能+记忆)', 'inputSchema': {
- 'type': 'object', 'properties': {}}},
- {'name': 'hermes_fusion', 'description': 'Hermes 融合:衰减+晋升+去重', 'inputSchema': {'type': 'object', 'properties': {}}},
- {'name': 'search_procedural', 'description': '搜索程序性记忆模板', 'inputSchema': {'type': 'object', 'properties': {'query': {'type': 'string'}}, 'required': ['query']}},
- {'name': 'search_episodes', 'description': '搜索会话历史(情景记忆)', 'inputSchema': {'type': 'object', 'properties': {'query': {'type': 'string'}}, 'required': ['query']}},
- {'name': 'save_episodic', 'description': '保存一个会话记录', 'inputSchema': {'type': 'object', 'properties': {'session_id': {'type': 'string'}, 'summary': {'type': 'string'}, 'task': {'type': 'string'}, 'message_count': {'type': 'number'}, 'project_name': {'type': 'string'}}, 'required': ['session_id']}},
- {'name': 'search_graph', 'description': '搜索记忆图谱(关系网络),返回子图', 'inputSchema': {'type': 'object', 'properties': {'query': {'type': 'string'}, 'depth': {'type': 'number', 'description': '遍历深度(默认1)'}, 'limit': {'type': 'number'}}, 'required': ['query']}},
- {'name': 'expand_keys', 'description': '将记忆key列表通过图谱扩展为连通子图', 'inputSchema': {'type': 'object', 'properties': {'keys': {'type': 'array', 'items': {'type': 'string'}}, 'depth': {'type': 'number', 'description': '遍历深度(默认1)'}}, 'required': ['keys']}},
- {'name': 'create_edge', 'description': '手动创建图谱关系边', 'inputSchema': {'type': 'object', 'properties': {'source': {'type': 'string'}, 'target': {'type': 'string'}, 'relation_type': {'type': 'string', 'description': 'depends_on/part_of/blocked_by/causes/solves/related_to/extends/conflicts_with/alternative_to/triggers'}, 'confidence': {'type': 'number'}, 'evidence': {'type': 'string'}}, 'required': ['source', 'target', 'relation_type']}},
- {'name': 'graph_stats', 'description': '图谱统计信息', 'inputSchema': {'type': 'object', 'properties': {}}},
- {'name': 'record_feedback', 'description': '记录记忆使用反馈(injected/referenced/helped/did_not_help/caused_confusion)', 'inputSchema': {'type': 'object', 'properties': {'memory_key': {'type': 'string'}, 'event_type': {'type': 'string', 'description': 'injected/referenced/helped/did_not_help/caused_confusion'}, 'session_id': {'type': 'string'}, 'detail': {'type': 'string'}}, 'required': ['memory_key', 'event_type']}},
- {'name': 'search_warnings', 'description': '检测当前任务相关的危险信号(阻塞/冲突/失败模式)', 'inputSchema': {'type': 'object', 'properties': {'query': {'type': 'string', 'description': '当前任务描述或关键词'}}, 'required': ['query']}},
- {'name': 'skill_rankings', 'description': '技能效果排行榜(按完成率排序)', 'inputSchema': {'type': 'object', 'properties': {'limit': {'type': 'number'}}}},
- {'name': 'skill_prefs', 'description': '查询技能使用偏好(什么任务用什么技能)', 'inputSchema': {'type': 'object', 'properties': {'skill_name': {'type': 'string', 'description': '可选,查询特定技能偏好'}}}},
- # ── z2 fleet tools ──
- {'name': 'fleet_status', 'description': '查询所有CC实例状态(自动发现,无需注入)', 'inputSchema': {'type': 'object', 'properties': {}}},
- {'name': 'fleet_peek', 'description': '窥探指定CC会话的最新动态(最近5对Q/A)', 'inputSchema': {'type': 'object', 'properties': {'session_id': {'type': 'string', 'description': '会话ID前8位或完整ID'}}, 'required': ['session_id']}}
- ]}})
- if method == 'tools/call':
- tool = params['name']
- if tool == 'search_memory':
- use_ai = args.get('use_ai', True)
- if use_ai:
- r = search_hybrid_ai(args.get('query', ''), args.get('limit', 10))
- else:
- r = search_hybrid(args.get('query', ''), args.get('limit', 10))
- return send_msg({'jsonrpc': '2.0', 'id': mid, 'result': {'content': [{'type': 'text', 'text': json.dumps(r, ensure_ascii=False)}]}})
- if tool == 'save_memory':
- save_semantic(args['key'], args['content'], args.get('tags', ''))
- return send_msg({'jsonrpc': '2.0', 'id': mid, 'result': {'content': [{'type': 'text', 'text': f'saved: {args["key"]}'}]}})
- if tool == 'list_skills':
- sk = search_skills(args.get('query', ''))
- return send_msg({'jsonrpc': '2.0', 'id': mid, 'result': {'content': [{'type': 'text', 'text': json.dumps(sk, ensure_ascii=False)}]}})
- if tool == 'create_skill':
- d = os.path.join(SKILL_DIRS[0], args['name'])
- os.makedirs(d, exist_ok=True)
- with open(os.path.join(d, 'SKILL.md'), 'w', encoding='utf-8') as f:
- f.write(args['content'])
- index_skills()
- return send_msg({'jsonrpc': '2.0', 'id': mid, 'result': {'content': [{'type': 'text', 'text': f'skill created: {args["name"]}'}]}})
- if tool == 'memory_stats':
- st = get_stats()
- return send_msg({'jsonrpc': '2.0', 'id': mid, 'result': {'content': [{'type': 'text', 'text': json.dumps(st)}]}})
- if tool == 'current_context':
- try:
- with open(os.path.join(ROOT, 'injection.md'), encoding='utf-8') as f:
- ctx = f.read()
- # Track skill invocations from injection
- for m in re.finditer(r'^### (\S+)', ctx, re.MULTILINE):
- skill_name = m.group(1)
- db.execute("UPDATE skill_index SET invoke_count = COALESCE(invoke_count, 0) + 1, last_invoked = datetime('now') WHERE name = ?", (skill_name,))
- db.commit()
- except Exception:
- ctx = 'injection.md not found'
- return send_msg({'jsonrpc': '2.0', 'id': mid, 'result': {'content': [{'type': 'text', 'text': ctx}]}})
- if tool == 'hermes_fusion':
- r = hermes_fusion()
- return send_msg({'jsonrpc': '2.0', 'id': mid, 'result': {'content': [{'type': 'text', 'text': json.dumps(r)}]}})
- if tool == 'search_procedural':
- r = search_procedural(args.get('query', ''))
- return send_msg({'jsonrpc': '2.0', 'id': mid, 'result': {'content': [{'type': 'text', 'text': json.dumps(r, ensure_ascii=False)}]}})
- if tool == 'search_episodes':
- r = search_episodes(args.get('query', ''))
- return send_msg({'jsonrpc': '2.0', 'id': mid, 'result': {'content': [{'type': 'text', 'text': json.dumps(r, ensure_ascii=False)}]}})
- if tool == 'save_episodic':
- save_episodic(args['session_id'], args.get('summary',''), args.get('task',''), args.get('message_count', 0), '', args.get('project_name',''))
- sid = args['session_id']
- return send_msg({'jsonrpc': '2.0', 'id': mid, 'result': {'content': [{'type': 'text', 'text': f'episodic saved: {sid}'}]}})
- if tool == 'search_graph':
- query = args.get('query', '')
- depth = int(args.get('depth', 1))
- limit = int(args.get('limit', 10))
- edges = search_graph(query, limit)
- # If edges found, expand to subgraph
- if edges and depth > 0:
- roots = set()
- for e in edges[:5]:
- roots.add(e['source'])
- roots.add(e['target'])
- sub = expand_keys(list(roots), depth)
- else:
- sub = {'keys': [], 'subgraph': {'nodes': [], 'edges': []}}
- return send_msg({'jsonrpc': '2.0', 'id': mid, 'result': {'content': [{'type': 'text', 'text': json.dumps({'matches': edges, 'subgraph': sub['subgraph'], 'root_keys': list(roots) if edges else []}, ensure_ascii=False)}]}})
- if tool == 'expand_keys':
- keys = args.get('keys', [])
- depth = int(args.get('depth', 1))
- result = expand_keys(keys, depth)
- return send_msg({'jsonrpc': '2.0', 'id': mid, 'result': {'content': [{'type': 'text', 'text': json.dumps(result, ensure_ascii=False)}]}})
- if tool == 'create_edge':
- eid = upsert_edge(args['source'], args['target'], args['relation_type'], float(args.get('confidence', 0.7)), args.get('evidence', ''))
- return send_msg({'jsonrpc': '2.0', 'id': mid, 'result': {'content': [{'type': 'text', 'text': f'edge created: {eid}' if eid else 'edge skipped (duplicate or error)'}]}})
- if tool == 'graph_stats':
- gs = get_graph_stats()
- return send_msg({'jsonrpc': '2.0', 'id': mid, 'result': {'content': [{'type': 'text', 'text': json.dumps(gs)}]}})
- if tool == 'record_feedback':
- ok = record_feedback(args['memory_key'], args['event_type'], args.get('session_id', ''), args.get('detail', ''))
- return send_msg({'jsonrpc': '2.0', 'id': mid, 'result': {'content': [{'type': 'text', 'text': f'feedback recorded: {args["memory_key"]} {args["event_type"]}' if ok else 'feedback failed'}]}})
- if tool == 'search_warnings':
- w = search_warnings(args.get('query', ''))
- return send_msg({'jsonrpc': '2.0', 'id': mid, 'result': {'content': [{'type': 'text', 'text': json.dumps(w, ensure_ascii=False)}]}})
- if tool == 'skill_rankings':
- rankings = get_skill_rankings(int(args.get('limit', 20)))
- return send_msg({'jsonrpc': '2.0', 'id': mid, 'result': {'content': [{'type': 'text', 'text': json.dumps(rankings, ensure_ascii=False)}]}})
- if tool == 'skill_prefs':
- prefs = get_skill_prefs(args.get('skill_name'))
- return send_msg({'jsonrpc': '2.0', 'id': mid, 'result': {'content': [{'type': 'text', 'text': json.dumps(prefs, ensure_ascii=False)}]}})
- if tool == 'fleet_status':
- # Reuse FleetWatcher's scan results from broadcast file
- try:
- with open(FLEET_BROADCAST_FILE, 'r', encoding='utf-8') as f:
- text = f.read()
- except Exception:
- text = '舰队广播未就绪,请稍候'
- return send_msg({'jsonrpc': '2.0', 'id': mid, 'result': {'content': [{'type': 'text', 'text': text}]}})
- if tool == 'fleet_peek':
- sid = args.get('session_id', '')
- qa_pairs = []
- try:
- # Dynamic: scan all project dirs for this session
- jsonl = None
- try:
- for dname in os.listdir(CC_PROJECTS_DIR):
- dpath = os.path.join(CC_PROJECTS_DIR, dname)
- if not os.path.isdir(dpath):
- continue
- for fname in os.listdir(dpath):
- if fname.endswith('.jsonl') and sid[:8] in fname:
- jsonl = os.path.join(dpath, fname)
- break
- if jsonl:
- break
- except Exception:
- pass
- if jsonl:
- with open(jsonl, 'r', encoding='utf-8', errors='ignore') as f:
- lines = f.readlines()
- pairs = []
- cur_q = None; cur_a = None
- for line in reversed(lines[-800:]):
- try:
- entry = json.loads(line.strip())
- except Exception:
- continue
- if entry.get('type') == 'user' and not cur_q:
- msg = entry.get('message', {})
- if isinstance(msg, dict):
- c = msg.get('content', '')
- if isinstance(c, str) and c.strip(): cur_q = c.strip()
- elif isinstance(c, list):
- for p in c:
- if isinstance(p, dict) and p.get('text'):
- cur_q = p['text'].strip(); break
- continue
- if entry.get('type') == 'assistant' and cur_q and not cur_a:
- msg = entry.get('message', {})
- if isinstance(msg, dict):
- c = msg.get('content', '')
- if isinstance(c, str) and c.strip(): cur_a = c.strip()
- elif isinstance(c, list):
- txts = [p['text'].strip() for p in c if isinstance(p, dict) and p.get('text')]
- if txts: cur_a = '\n'.join(txts)
- if cur_q and cur_a:
- pairs.append({'q': cur_q[:200], 'a': cur_a[:300]})
- cur_q = None; cur_a = None
- if len(pairs) >= 5: break
- qa_pairs = pairs
- except Exception:
- pass
- return send_msg({'jsonrpc': '2.0', 'id': mid, 'result': {'content': [{'type': 'text', 'text': json.dumps({'session_id': sid, 'qa_pairs': qa_pairs, 'count': len(qa_pairs)}, ensure_ascii=False)}]}})
- if __name__ == '__main__':
- # Main loop — raw JSON lines (matching claude_opus_mcp.py pattern)
- while True:
- req = read_msg()
- if req is None:
- break
- try:
- handle(req)
- except Exception:
- pass
dream.js · Bundled script · 261 lines · 1 finding marked
- const path = require('path')
- const fs = require('fs')
- const https = require('https')
- const ROOT = path.dirname(__filename)
- const { getAPIConfig } = require('./config')
- function callLLM(messages, useFlash = true) {
- return new Promise((resolve, reject) => {
- let cfg; try { cfg = getAPIConfig(useFlash) } catch(e) { reject(e); return }
- const body = cfg.bodyBuilder(Array.isArray(messages) ? messages : [{ role: 'user', content: messages }])
- const req = require('https').request({
- hostname: cfg.hostname, path: cfg.path, method: 'POST',
- headers: cfg.headers, timeout: cfg.timeout
- }, res => {
- let data = ''
- res.on('data', c => data += c)
- res.on('end', () => {
- try {
- const obj = JSON.parse(data)
- if (obj.error) { reject(new Error(obj.error.message || 'API error')); return }
- if (cfg.format === 'openai') { resolve(obj.choices?.[0]?.message?.content || ''); return }
- const textBlock = obj.content?.find(c => c.type === 'text')
- resolve(textBlock ? textBlock.text : (obj.content?.[0]?.text || ''))
- } catch(e) { reject(e) }
- })
- })
- req.on('error', reject)
- req.write(body)
- req.end()
- })
- }
- const DREAM_FILE = path.join(ROOT, '.dream_findings.json')
- function callLLM(messages) {
- return new Promise((resolve, reject) => {
- const body = JSON.stringify({
- model: 'deepseek-v4-flash',
- max_tokens: 8192,
- messages: messages.map(m => ({ role: m.role, content: m.content }))
- })
- const req = https.request({
- Warning: Sends stored memory contents to api.deepseek.com in an HTTPS request authenticated with the user's API key. Your session memories leave the machine for a third-party LLM.
- Line 43 on GitHub
- hostname: 'api.deepseek.com', path: '/v1/chat/completions', method: 'POST',
- headers: { 'Content-Type': 'application/json', 'Authorization': `Bearer ${API_KEY}` },
- timeout: 120000
- }, res => {
- let data = ''
- res.on('data', c => data += c)
- res.on('end', () => {
- try { resolve(JSON.parse(data).choices[0].message.content) }
- catch(e) { reject(e) }
- })
- })
- req.on('error', reject)
- req.write(body)
- req.end()
- })
- }
- function callLLM(messages) {
- return new Promise((resolve, reject) => {
- const body = JSON.stringify({
- model: 'deepseek-v4-pro[1m]',
- max_tokens: 16384,
- messages: messages.map(m => ({ role: m.role, content: m.content }))
- })
- const req = https.request({
- hostname: 'api.deepseek.com', path: '/anthropic/v1/messages', method: 'POST',
- headers: { 'Content-Type': 'application/json', 'x-api-key': API_KEY, 'anthropic-version': '2023-06-01' },
- timeout: 300000
- }, res => {
- let data = ''
- res.on('data', c => data += c)
- res.on('end', () => {
- try {
- const body = JSON.parse(data)
- if (body.error) { reject(new Error(body.error.message)); return }
- const textBlock = body.content?.find(c => c.type === 'text')
- resolve(textBlock ? textBlock.text : (body.content?.[0]?.text || ''))
- } catch(e) { reject(e) }
- })
- })
- req.on('error', reject)
- req.write(body)
- req.end()
- })
- }
- async function dream(index) {
- index.init()
- const db = require('better-sqlite3')(path.join(ROOT, 'memory.db'))
- // Gather memory summary (aggregated, not raw dump)
- const stats = index.getStats()
- // Top 30 most accessed memories
- const topMems = db.prepare(`SELECT key, content, tags, access_count, COALESCE(effectiveness_score,0.5) as eff, COALESCE(injected_count,0) as ic, created_at FROM semantic WHERE key != '_schema_version' ORDER BY access_count DESC LIMIT 30`).all()
- // Top contradictions from graph
- let conflicts = []
- try {
- const gdb = require('better-sqlite3')(path.join(ROOT, 'graph.db'))
- conflicts = gdb.prepare(`SELECT e.source, e.target, e.evidence, e.confidence FROM edges e WHERE e.relation_type = 'conflicts_with' ORDER BY e.confidence DESC LIMIT 10`).all()
- gdb.close()
- } catch(e) {}
- // Recent open issues
- const issues = db.prepare(`SELECT key, content, created_at FROM semantic WHERE key != '_schema_version' AND (key LIKE '%issue_%' OR key LIKE '%blocker%' OR key LIKE '%unresolved%' OR content LIKE '%未解决%' OR content LIKE '%尚未解决%') ORDER BY created_at DESC LIMIT 10`).all()
- // Skill effectiveness summary
- const skillSummary = db.prepare(`SELECT skill_name, SUM(CASE WHEN event_type='completed' THEN 1 ELSE 0 END) as done, SUM(CASE WHEN event_type='failed' THEN 1 ELSE 0 END) as fail FROM skill_feedback GROUP BY skill_name ORDER BY done DESC LIMIT 15`).all()
- // Build context for pro model
- const catalog = [
- `## 记忆状态: ${stats.semanticCount} 条语义记忆, ${stats.skillCount} 个技能`,
- '',
- '### 高访问记忆 (top 30)',
- ...topMems.map(m => `- ${m.key}: ${m.content?.substring(0, 120)} [访问:${m.access_count} 有效率:${(m.eff*100).toFixed(0)}% 注入:${m.ic}次]`),
- '',
- '### 活跃矛盾',
- ...(conflicts.length ? conflicts.map(c => `- ${c.source} ↔ ${c.target}: ${c.evidence?.substring(0, 100)}`) : ['- 无活跃矛盾']),
- '',
- '### 未解决问题',
- ...(issues.length ? issues.map(i => `- ${i.key}: ${i.content?.substring(0, 100)} [${i.created_at}]`) : ['- 无']),
- '',
- '### 技能效果',
- ...(skillSummary.length ? skillSummary.map(s => `- ${s.skill_name}: 成功${s.done} 失败${s.fail}`) : ['- 暂无数据']),
- ].join('\n')
- const prompt = `你是一个 AI 认知引擎的夜间分析模块。分析以下记忆库快照,产出深度研究结论。
- ## 任务
- 1. **记忆融合**: 找到 3-5 组表达同一事实但用不同方式描述的记忆,给出合并后的精确表述
- 2. **矛盾仲裁**: 裁决矛盾,给出确定性结论
- 3. **知识缺口**: 识别 3 个最关键的未解决问题,按严重度排序
- 4. **模式发现**: 从技能效果数据中提取最有效的工解模式
- 5. **预测**: 基于历史数据,预测当前路径最可能的结果
- ## 输出格式
- 返回 JSON:
- {
- "merges": [{"fragment_keys": ["key1","key2"], "merged_fact": "合并后的精确事实"}],
- "arbitrations": [{"conflict": "A vs B", "winner": "A", "verdict": "确定性结论"}],
- "critical_gaps": [{"issue": "问题描述", "severity": "high|medium", "impact": "不解决会导致..."}],
- "patterns": [{"pattern": "模式描述", "evidence": "证据", "recommendation": "建议"}],
- "prediction": {"most_likely": "最可能走向", "confidence": 0.0-1.0, "alternative": "替代路径"},
- "summary": "2-3 句整体总结"
- }
- ## 记忆快照
- ${catalog}`
- try {
- let result = null
- try {
- result = await callLLM([{ role: 'user', content: prompt }])
- } catch(e) {
- // Pro failed — fallback to flash
- }
- // Flash fallback: if pro returned nothing or crashed
- if (!result) {
- const flashPrompt = `简化分析以下记忆库快照,只输出关键发现。返回 JSON: {"merges":[], "critical_gaps":[], "summary":"一句话总结"}
- ${catalog.substring(0, 5000)}`
- try {
- result = await callLLM([{ role: 'user', content: flashPrompt }])
- } catch(e2) {
- return null
- }
- }
- if (!result) return null
- // Parse JSON from response
- let findings = null
- try { findings = JSON.parse(result.trim()) } catch(e) {
- const m = result.match(/\{[\s\S]*\}/)
- if (m) try { findings = JSON.parse(m[0]) } catch(e2) {}
- }
- if (!findings) return null
- findings.dreamed_at = new Date().toISOString()
- findings.stats = stats
- // Save to file
- fs.writeFileSync(DREAM_FILE, JSON.stringify(findings, null, 2), 'utf-8')
- // v4: emit dream complete event
- try { require(path.join(ROOT, 'eventbus')).emit('dream:complete', { findings, stats }) } catch(e) {}
- // Apply merges
- for (const mg of (findings.merges || [])) {
- const keys = mg.fragment_keys
- const mergedKey = keys[0] + '_dream_merged'
- const mergedContent = `[梦境融合 ${keys.length} 条记忆] ${mg.merged_fact}`
- try {
- db.prepare('INSERT OR REPLACE INTO semantic (key, content, tags, confidence) VALUES (?,?,?,?)').run(mergedKey, mergedContent, 'dream-merged', 0.8)
- for (const k of keys) {
- db.prepare("UPDATE semantic SET tags = COALESCE(tags,'') || ',dream-merged', confidence = confidence * 0.5 WHERE key = ?").run(k)
- }
- } catch(e) {}
- }
- db.close()
- return findings
- } catch(e) {
- db.close()
- return null
- }
- }
- function loadDreamFindings() {
- try {
- if (!fs.existsSync(DREAM_FILE)) return null
- const data = JSON.parse(fs.readFileSync(DREAM_FILE, 'utf-8'))
- const age = Date.now() - new Date(data.dreamed_at).getTime()
- if (age > 24 * 60 * 60 * 1000) return null // stale
- return data
- } catch(e) { return null }
- }
- function formatDream(findings) {
- if (!findings) return ''
- let text = `\n## 🌙 梦境研究\n\n超脑昨夜深度分析了记忆库 (${findings.stats?.semanticCount || '?'} 条记忆):\n\n`
- if (findings.summary) {
- text += `**${findings.summary}**\n\n`
- }
- if (findings.merges?.length) {
- text += '### 记忆融合\n' + findings.merges.map(m =>
- `- ${m.fragment_keys.join(' + ')} → ${m.merged_fact?.substring(0, 150)}`
- ).join('\n') + '\n\n'
- }
- if (findings.arbitrations?.length) {
- text += '### 矛盾裁决\n' + findings.arbitrations.map(a =>
- `- **${a.winner}** 胜出: ${a.verdict?.substring(0, 150)}`
- ).join('\n') + '\n\n'
- }
- if (findings.critical_gaps?.length) {
- text += '### 关键缺口\n' + findings.critical_gaps.map(g =>
- `- [${g.severity}] ${g.issue}: ${g.impact?.substring(0, 120)}`
- ).join('\n') + '\n\n'
- }
- if (findings.patterns?.length) {
- text += '### 有效模式\n' + findings.patterns.map(p =>
- `- ${p.pattern}: ${p.recommendation?.substring(0, 120)}`
- ).join('\n') + '\n\n'
- }
- if (findings.prediction) {
- text += `### 预测\n最可能: ${findings.prediction.most_likely} (置信度 ${(findings.prediction.confidence*100).toFixed(0)}%)\n`
- }
- text += `\n> 下一次梦境: 明天凌晨\n`
- return text
- }
- module.exports = { dream, loadDreamFindings, formatDream }
extract_worker.js · Bundled script · 427 lines · 1 finding marked
- const path = require('path')
- const fs = require('fs')
- const https = require('https')
- process.on('unhandledRejection', (reason) => {
- try { fs.appendFileSync(path.join(__dirname, 'worker.log'), `${new Date().toISOString()} [worker] unhandledRejection: ${reason?.message || reason}\n`) } catch(e) {}
- })
- const ROOT = path.dirname(__filename)
- const { shouldSkipExtraction } = require(path.join(ROOT, 'privacy_filter'))
- const EPISODIC_DIR = path.join(ROOT, 'memory', 'episodic')
- const adapter = require(path.join(ROOT, 'adapters')).getAgent()
- const { getAPIConfig } = require('./config')
- function callLLM(messages, useFlash = true) {
- return new Promise((resolve, reject) => {
- let cfg; try { cfg = getAPIConfig(useFlash) } catch(e) { reject(e); return }
- const body = cfg.bodyBuilder(Array.isArray(messages) ? messages : [{ role: 'user', content: messages }])
- const req = require('https').request({
- hostname: cfg.hostname, path: cfg.path, method: 'POST',
- headers: cfg.headers, timeout: cfg.timeout
- }, res => {
- let data = ''
- res.on('data', c => data += c)
- res.on('end', () => {
- try {
- const obj = JSON.parse(data)
- if (obj.error) { reject(new Error(obj.error.message || 'API error')); return }
- if (cfg.format === 'openai') { resolve(obj.choices?.[0]?.message?.content || ''); return }
- const textBlock = obj.content?.find(c => c.type === 'text')
- resolve(textBlock ? textBlock.text : (obj.content?.[0]?.text || ''))
- } catch(e) { reject(e) }
- })
- })
- req.on('error', reject)
- req.write(body)
- req.end()
- })
- }
- function findCurrentTranscript() {
- try {
- if (!fs.existsSync(adapter.transcriptDir())) return null
- const files = fs.readdirSync(adapter.transcriptDir())
- .filter(f => f.endsWith('.jsonl'))
- .map(f => ({ name: f, mtime: fs.statSync(path.join(adapter.transcriptDir(), f)).mtimeMs, path: path.join(adapter.transcriptDir(), f) }))
- .sort((a, b) => b.mtime - a.mtime)
- return files[0] || null
- } catch(e) { return null }
- }
- function loadNewLines(filepath, lastPos) {
- try {
- const stat = fs.statSync(filepath)
- if (stat.size <= lastPos) return { lines: [], newPos: lastPos }
- const fd = fs.openSync(filepath, 'r')
- fs.readSync(fd, Buffer.alloc(0), 0, 0, lastPos)
- const buf = Buffer.alloc(stat.size - lastPos)
- fs.readSync(fd, buf, 0, buf.length, lastPos)
- fs.closeSync(fd)
- const text = buf.toString('utf-8')
- const lines = text.split('\n').filter(Boolean)
- return { lines, newPos: stat.size }
- } catch(e) { return { lines: [], newPos: lastPos } }
- }
- function compressForExtraction(lines, maxChars) {
- const msgs = []
- for (let i = 0; i < lines.length && msgs.length < 100; i++) {
- try {
- const obj = JSON.parse(lines[i])
- const msg = obj.message || {}
- const role = msg.role || 'unknown'
- let content = msg.content || ''
- if (typeof content !== 'string') content = JSON.stringify(content)
- if (content) msgs.push(`[${role}] ${content.substring(0, 800)}`)
- } catch(e) {}
- }
- const text = msgs.join('\n')
- if (text.length <= maxChars) return text
- return text.substring(0, maxChars)
- }
- async function extractAndSave(lines, sessionId) {
- const text = compressForExtraction(lines, 30000)
- if (text.length < 200) return 0
- const prompt = `${getHermesPrompt()}
- ## 当前对话片段
- ${text}
- ## 已存记忆关键词(避免重复)
- ${getExistingKeys()}
- 请按格式输出每条新发现的事实。`
- try {
- const result = await callLLM([{ role: 'system', content: getHermesPrompt() }, { role: 'user', content: prompt }])
- Warning: A background worker sends up to 30,000 characters of the user's Claude Code conversation transcript to a third-party LLM API to extract memories, repeatedly and without any prompt.
- Line 100 on GitHub
- if (!result) return 0
- const index = require(path.join(ROOT, 'index'))
- index.init()
- index.ensureMemoryDirs()
- let saved = 0, skipped = 0, skillDrafts = 0
- const lines = result.split('\n').filter(l => l.trim().startsWith('{'))
- for (const line of lines) {
- try {
- const fact = JSON.parse(line.trim())
- // Check for procedural_candidate / skill draft format
- if (fact.type === 'procedural' && fact.name && fact.steps) {
- const skillPath = path.join(ROOT, 'skills', 'all', `${fact.name}.md`)
- const triggerStr = fact.trigger || ''
- const stepsStr = Array.isArray(fact.steps) ? fact.steps.join('\n') : String(fact.steps)
- const content = `---\nname: ${fact.name}\ndescription: ${fact.trigger || fact.name}\ntriggers: ${triggerStr}\n---\n\n# ${fact.name}\n\n${fact.trigger || ''}\n\n## Steps\n${stepsStr}`
- if (!fs.existsSync(skillPath)) {
- fs.writeFileSync(skillPath, content, 'utf-8')
- index.logEvolution(sessionId, 'skill_draft', { name: fact.name })
- skillDrafts++
- }
- continue
- }
- if (fact.key && fact.content && fact.is_new !== false) {
- if (shouldSkipExtraction(fact.content)) { skipped++; continue }
- const head = index.getGitHead()
- if (index.saveSemantic(fact.key, fact.content, fact.category || 'general', sessionId, fact.dedup_key, fact.confidence || 0.5, head)) {
- saved++
- }
- }
- } catch(e) {}
- }
- if (skipped > 0) log(`privacy filter: skipped ${skipped} facts`)
- if (skillDrafts > 0) {
- log(`skill drafts created: ${skillDrafts}`)
- const { spawn } = require('child_process')
- spawn('pythonw', ['-c', `import sys; sys.path.insert(0,'${ROOT}'); from daemon import index_skills; print(f'Re-indexed: {len(index_skills())} skills')`], { stdio: 'ignore', detached: true, windowsHide: true }).unref()
- }
- // Injection refresh handled by CC hooks — removing redundant spawn
- // that caused triple-fire + lock contention + console flash every 30s
- // ---- GRAPH: Extract relationships from conversation ----
- try {
- const graph = require(path.join(ROOT, 'graph'))
- const allMems = index.getAllMemoryKeys()
- if (allMems.length >= 5) {
- const relPrompt = graph.buildExtractionPrompt(allMems, text)
- const relResult = await callLLM([{ role: 'user', content: relPrompt }])
- if (relResult) {
- const relations = graph.parseExtractionResult(relResult)
- if (relations.length > 0) {
- const added = graph.applyRelations(relations, sessionId)
- log(`graph: extracted ${added} relations from ${relations.length} candidates`)
- }
- }
- }
- } catch(e) {
- log(`graph extraction error: ${e.message}`)
- }
- // ---- SKILL PREFERENCE: Detect what skills were used for what tasks ----
- try {
- const allSkills = index.getAllSkills()
- const validNames = allSkills.map(s => s.name)
- // Read injected skills for implicit detection
- let injectedSkillNames = ''
- try {
- const injContent = fs.readFileSync(path.join(ROOT, 'injection.md'), 'utf-8')
- const re = /^### (\S+)/gm
- let m; const names = []
- while ((m = re.exec(injContent)) !== null) names.push(m[1])
- if (names.length > 0) injectedSkillNames = `\nSKILLS INJECTED THIS SESSION: ${names.join(', ')}\nCheck if these were used — even IMPLICITLY (assistant follows skill rules without naming it).`
- } catch(e) {}
- const skillPrefPrompt = `Analyze this conversation. Detect when a skill was used.${injectedSkillNames}
- IMPORTANT — Implicit Usage: The assistant often follows a skill's rules WITHOUT naming it. Look for:
- - Code style matches: zero comments + zero types + stdlib only = elon-code was used
- - Thinking mentions: "从另一个角度看"/"反证" = lateral-jump was used
- - Multi-path analysis: A/B/C 方案对比 = net-analyze was used
- - The user said "用X" and the assistant's output matches X's style → X was used
- Output format: SKILL_PREF: <skill_name> | <task_scenario> | <effectiveness_0_to_1>
- Rules:
- - skill_name: MUST be one of the injected skills above, or a skill name from the conversation
- - task_scenario: SHORT task category (2-8 Chinese chars). Generalize.
- - effectiveness: 0.9=completed, 0.7=partial, 0.3=failed
- - Output ONLY SKILL_PREF lines. Nothing else.
- - Output nothing if no skill usage detected.
- Conversation:
- ${text.substring(0, 4000)}`
- const prefResult = await callLLM([{ role: 'user', content: skillPrefPrompt }])
- if (prefResult) {
- const prefLines = prefResult.split('\n').filter(l => l.startsWith('SKILL_PREF:'))
- for (const line of prefLines) {
- const m = line.match(/^SKILL_PREF:\s*(\S+)\s*\|\s*(.+?)\s*\|\s*([\d.]+)/)
- if (m) {
- let skillName = m[1].trim()
- let taskPattern = m[2].trim()
- const eff = parseFloat(m[3]) || 0.7
- // Normalize: extract core task category
- const clean = taskPattern.replace(/[的之地得和与了在是]/g, '').replace(/[a-zA-Z0-9_\-\.\/]+/g, '').trim()
- const pair = clean.match(/(逆向|安全|代码|网络|系统|数据|性能|前端|后端|数据库|API|加密|合约|协议|内存|进程|流量|抓包|令牌|分析|编写|生成)(分析|工程|审查|审计|测试|调试|开发|配置|优化|代码|管理|检测|追踪|提取|破解|脚本)/)
- if (pair) {
- taskPattern = pair[0]
- } else if (clean.length > 8) {
- taskPattern = clean.substring(0, 8)
- }
- // Fuzzy match: find the closest real skill name
- const matched = fuzzyMatchSkill(skillName, validNames)
- if (matched) {
- index.upsertSkillPref(matched, taskPattern, eff)
- }
- }
- }
- if (prefLines.length > 0) {
- index.syncSkillPrefsToFile()
- log(`skill_prefs: detected ${prefLines.length} usage patterns`)
- } else {
- log(`skill_prefs: API responded but no SKILL_PREF lines (${prefResult.substring(0, 100).replace(/\n/g, ' ')})`)
- }
- } else {
- log(`skill_prefs: API returned empty response`)
- }
- } catch(e) {
- log(`skill_pref extraction error: ${e.message}`)
- }
- return saved
- } catch(e) {
- log(`extract error: ${e.message}`)
- return 0
- }
- }
- function fuzzyMatchSkill(name, validNames) {
- if (validNames.includes(name)) return name
- const norm = (s) => s.toLowerCase().replace(/[-_\s]/g, '')
- const target = norm(name)
- for (const v of validNames) {
- if (norm(v) === target) return v
- }
- for (const v of validNames) {
- const nv = norm(v)
- const ratio = Math.min(target.length, nv.length) / Math.max(target.length, nv.length)
- if (ratio >= 0.4 && (nv.includes(target) || target.includes(nv))) return v
- }
- const aliases = {
- 'elonmask': 'elon-code', 'elonmusk': 'elon-code', 'elon': 'elon-code',
- 'superpower': 'using-superpowers', 'superpowers': 'using-superpowers',
- 'netanalyze': 'net-analyze', 'lateraljump': 'lateral-jump',
- }
- if (aliases[target]) return aliases[target]
- return null
- }
- function getExistingKeys() {
- try {
- const index = require(path.join(ROOT, 'index'))
- index.init()
- const db = index.db || require('better-sqlite3')(index.DB_PATH)
- const rows = db.prepare('SELECT key FROM semantic LIMIT 200').all()
- return rows.map(r => r.key).join(', ')
- } catch(e) { return '' }
- }
- // Main watch loop
- async function main() {
- // Singleton guard — atomic PID file prevents race between workers
- const pidFile = path.join(ROOT, '.worker.pid')
- let pidFd = null
- for (let attempt = 0; attempt < 5; attempt++) {
- try {
- pidFd = fs.openSync(pidFile, 'wx')
- fs.writeSync(pidFd, String(process.pid))
- break
- } catch(e) {
- if (e.code === 'EEXIST') {
- try {
- const oldPid = parseInt(fs.readFileSync(pidFile, 'utf8'))
- try { process.kill(oldPid, 'SIGTERM') } catch(e) {}
- } catch(e) {}
- try { fs.unlinkSync(pidFile) } catch(e) {}
- require('child_process').execSync(process.platform === 'win32' ? 'ping 127.0.0.1 -n 1 >nul' : 'sleep 0.05', {stdio:'ignore'})
- } else { log(`pid lock error: ${e.message}`); return }
- }
- }
- if (!pidFd) { log('could not acquire PID lock, exiting'); return }
- process.on('exit', () => { try { fs.closeSync(pidFd); fs.unlinkSync(pidFile) } catch(e) {} })
- try { require(path.join(ROOT, 'wiring')).init() } catch(e) {}
- log('worker started (watch mode)')
- const transcript = findCurrentTranscript()
- if (!transcript) { log('no transcript found'); return }
- log(`watching: ${transcript.name}`)
- let lastPos = fs.statSync(transcript.path).size
- let accumulatedLines = []
- let startTime = Date.now()
- let lastContentTime = Date.now()
- let lastConsolidationCheck = 0
- const SESSION_IDLE_TIMEOUT = 15 * 60 * 1000
- const check = async () => {
- try {
- const { lines, newPos } = loadNewLines(transcript.path, lastPos)
- if (lines.length > 0) {
- accumulatedLines.push(...lines)
- lastPos = newPos
- lastContentTime = Date.now()
- }
- const shouldExtract = accumulatedLines.length >= MIN_NEW_LINES ||
- (accumulatedLines.length > 0 && lines.length === 0 && Date.now() - startTime > 60000)
- if (shouldExtract && accumulatedLines.length > 0) {
- const sessionId = 'incr_' + Date.now()
- const lineCount = accumulatedLines.length
- const saved = await extractAndSave(accumulatedLines, sessionId)
- log(`incremental: ${saved} facts from ${lineCount} lines`)
- // v4: emit extraction event
- try { require(path.join(ROOT, 'eventbus')).emit('memory:extracted', { count: saved, lines: lineCount, sessionId }) } catch(e) {}
- accumulatedLines = []
- startTime = Date.now()
- }
- // Autonomous research & dream phase
- const idle = Date.now() - lastContentTime
- const RESEARCH_IDLE = 5 * 60 * 1000
- const DREAM_IDLE = 10 * 60 * 1000
- if (idle > RESEARCH_IDLE && (Date.now() - lastConsolidationCheck) > RESEARCH_IDLE) {
- try {
- const index = require(path.join(ROOT, 'index'))
- // Lightweight research (local SQL)
- try {
- const research = require(path.join(ROOT, 'research'))
- const analysis = research.analyze(index)
- if (analysis && analysis.total_findings > 0) {
- fs.writeFileSync(path.join(ROOT, '.research_findings.json'), JSON.stringify(analysis, null, 2), 'utf-8')
- log(`research: ${analysis.total_findings} patterns found`)
- }
- } catch(e) { log(`research error: ${e.message}`) }
- // Dream phase (pro model) — idle >30min, once per 8 hours
- if (idle > DREAM_IDLE) {
- const dreamFile = path.join(ROOT, '.dream_findings.json')
- const shouldDream = !fs.existsSync(dreamFile) ||
- (Date.now() - fs.statSync(dreamFile).mtimeMs) > 2 * 60 * 60 * 1000
- if (shouldDream) {
- try {
- const dream = require(path.join(ROOT, 'dream'))
- const findings = await dream.dream(index)
- if (findings && findings.summary) log(`dream: ${findings.summary.substring(0, 100)}`)
- } catch(e) { log(`dream error: ${e.message}`) }
- }
- }
- } catch(e) { log(`research/dream error: ${e.message}`) }
- }
- // Session end: transcript idle >15min → auto-consolidate
- if (idle > SESSION_IDLE_TIMEOUT && (Date.now() - lastConsolidationCheck) > SESSION_IDLE_TIMEOUT) {
- lastConsolidationCheck = Date.now()
- const { spawn } = require('child_process')
- const cp = spawn('node', [path.join(ROOT, 'consolidate.js')], { windowsHide: true, detached: true,
- cwd: ROOT, timeout: 120000, stdio: 'pipe'
- })
- let out = ''
- cp.stdout.on('data', c => out += c)
- cp.on('close', code => {
- if (code === 0) log(`session_end(worker): ${out.trim().substring(0, 200)}`)
- else log(`session_end(worker): exit ${code}`)
- })
- cp.on('error', e => log(`session_end(worker) failed: ${e.message}`))
- }
- } catch(e) {
- log(`check error: ${e.message}`)
- }
- }
- // Self-healing: periodic health check
- try {
- const healer = require(path.join(ROOT, 'healer'))
- healer.start(120000) // every 2 minutes
- } catch(e) {}
- // Adaptive interval — dynamic polling
- const adaptive = require(path.join(ROOT, 'adaptive'))
- let timer = setInterval(check, POLL_INTERVAL)
- // Check interval every 5 minutes and adjust
- setInterval(() => {
- const newInterval = adaptive.computeInterval(lastContentTime, accumulatedLines.length)
- if (Math.abs(newInterval - POLL_INTERVAL) > 10000) {
- // Interval would change significantly — restart timer
- clearInterval(timer)
- timer = setInterval(check, newInterval)
- log(`adaptive: interval adjusted to ${newInterval/1000}s`)
- }
- }, 300000)
- setTimeout(() => { clearInterval(timer); log('worker lifetime expired') }, MAX_LIFETIME)
- // Noise self-learner — periodic
- setInterval(() => {
- try {
- const nl = require(path.join(ROOT, 'noiselearner'))
- nl.learn(require(path.join(ROOT, 'index')))
- } catch(e) {}
- }, 600000)
- process.on('SIGTERM', () => {
- clearInterval(timer)
- log('worker graceful shutdown: checkpointing DBs')
- try { require(path.join(ROOT, 'pool')).checkpoint() } catch(e) {}
- process.exit(0)
- })
- }
- main().catch(e => log(`fatal: ${e.message}`))
Ratings and comments
Sign in to rate this skill.
Sign inFree to use. If it’s helped, pass it on.