Skip to content
Back to search

overmind

玄霖超脑 · 无量网络 v4 重构版— 66 模块6通道 AI 认知神经系统 · 装一次,你所有的 AI 工具从此共享一个永远不失忆的大脑。跨会话记忆 · 多 Agent 互通 · 自动代码审查 · 零配置

Safety scan

A static text scan, not a guarantee. Select the badge to see what was scanned and every finding.

Checked today
Badge for your README
findsafeskills safety rating

Markdown

[![findsafeskills safety rating](/api/listings/xuanlinai--overmind/badge.svg)](/listings/xuanlinai--overmind)

HTML

<a href="/listings/xuanlinai--overmind"><img src="/api/listings/xuanlinai--overmind/badge.svg" alt="findsafeskills safety rating" /></a>

Install

i

Install instructions from the README

As written by the repo's author. findsafeskills has not verified these steps.

git clone https://github.com/xuanlinAI/overmind.git && cd overmind && node install.js

一条命令完成全部 6 步:

| 步骤 | 内容 | |:--:|------| | 1 | **环境探测** — OS/CPU/内存/磁盘/Python/Node/AI Agent 全检 | | 2 | **安全快照** — 安装前备份所有关键文件 | | 3 | **完整性校验** — 检测缺失/损坏,自动修复 | | 4 | **依赖安装** — npm + pip 按需安装 | | 5 | **配置生成** — .overmind_env.json + hooks/MCP/CLAUDE.md 自动写入 | | 6 | **冒烟测试** — 6 通道全链验证,失败自动回滚 |

安装后设置 `DEEPSEEK_API_KEY`,重启 Claude Code 即可。

What was scanned

The files the safety scan read, as written by the repository's author and shown as plain text. Lines with a finding are marked.

plugin.json · Manifest · 51 lines

View this file on GitHub (read at the repository's current version: the scan did not record which commit it read)

  1. {
  2. "name": "xuanlin-overmind",
  3. "version": "v4",
  4. "description": "CC 的外挂认知引擎 — 66 模块 · 5 层记忆 · 知识图谱推理 · 红队审查 · 自进化",
  5. "author": "玄霖AI (xuanlinAI)",
  6. "hooks": {
  7. "SessionStart": [
  8. {
  9. "matcher": "",
  10. "hooks": [
  11. {
  12. "type": "command",
  13. "command": "mshta vbscript:Execute(\"CreateObject(\"\"WScript.Shell\"\").Run \"\"node <INSTALL_DIR>/inject.js\"\",0,False:close\")"
  14. }
  15. ]
  16. }
  17. ],
  18. "UserPromptSubmit": [
  19. {
  20. "matcher": "",
  21. "hooks": [
  22. {
  23. "type": "command",
  24. "command": "mshta vbscript:Execute(\"CreateObject(\"\"WScript.Shell\"\").Run \"\"node <INSTALL_DIR>/inject.js\"\",0,False:close\")"
  25. }
  26. ]
  27. }
  28. ],
  29. "SessionEnd": [
  30. {
  31. "matcher": "",
  32. "hooks": [
  33. {
  34. "type": "command",
  35. "command": "mshta vbscript:Execute(\"CreateObject(\"\"WScript.Shell\"\").Run \"\"node <INSTALL_DIR>/consolidate.js\"\",0,False:close\")"
  36. }
  37. ]
  38. }
  39. ]
  40. },
  41. "mcpServers": {
  42. "ctxproxy": {
  43. "type": "stdio",
  44. "command": "python",
  45. "args": [
  46. "<INSTALL_DIR>/daemon.py"
  47. ],
  48. "env": {}
  49. }
  50. }
  51. }
README.md · README · 341 lines

View this file on GitHub (read at the repository's current version: the scan did not record which commit it read)

  1. # 玄霖超脑 · 无量网络 v4 重构版
  2. <p align="center">
  3. <b>66 模块 AI 认知神经系统 · 6 通道架构 · 知识图谱推理 · 红队审查 · 自进化</b>
  4. </p>
  5. <p align="center">
  6. <a href="#中文">🇨🇳 中文</a> | <a href="#english">English</a>
  7. </p>
  8. <p align="center">
  9. <b>不是记忆,是推理。不是工具,是神经系统。不是插件,是认知引擎。</b>
  10. </p>
  11. <p align="center">
  12. 🥇 6 通道架构 — 串并联管道 + z2 中枢 + n2 终端<br>
  13. 🥈 零配置自适应安装 — 探针 + 自修复 + 诚实降级<br>
  14. 🥉 零进程触发 — 根治 Windows 窗口闪烁<br>
  15. 🔮 舰队模式 — 多 CC 自发现 + 跨 CC 技能 boosting + 冲突检测<br>
  16. 🧠 66 模块实时互连 — 知识图谱 4500+ 节点 · 10 种关系 · 被动推理
  17. </p>
  18. ---
  19. <span id="中文"></span>
  20. # 🇨🇳 中文
  21. ## 这是什么
  22. 玄霖超脑 v4(无量网络)是 Claude Code 的外挂认知引擎。66 个模块通过 6 条通道 + 事件总线实时互连,形成 AI 界首个认知神经系统。
  23. **它在你不在的时候仍在思考。你写的每行代码先被红队分身攻击才到你面前。你做的每个决策都有"选了另一个会怎样"的阴影追踪。你开的每个 CC 实例都能看到彼此在做什么。**
  24. ## 重构版 vs 初版 v4
  25. | 维度 | 初版 v4 | 重构版 v4 |
  26. |------|---------|-----------|
  27. | **安装方式** | 手动配置,Windows 特化 | `node install.js` 一键,全平台自适应 |
  28. | **通道架构** | 2 通道 (串+并) | **6 通道** (CH1-CH6) |
  29. | **管道阶段** | 21 阶段 | **37 阶段** + 量子缓存 |
  30. | **广播模块** | 18 模块 | **48 模块** |
  31. | **AI Agent 适配** | 仅 Claude Code | 自动检测 CC/Cursor/Aider/Codex/Gemini |
  32. | **z2 中枢** | 无 | daemon.py FleetWatcher,5s 舰队广播 |
  33. | **n2 终端** | 无 | communicator AI 筛选 + 串并联后处理 |
  34. | **舰队模式** | 无 | 多 CC 自发现 + 跨 CC 协同 |
  35. | **窗口闪烁** | Windows 多次闪现 | **彻底根治**(零进程触发架构) |
  36. | **自修复** | 无 | vault.js 校验 + 快照回滚 |
  37. | **错误处理** | 裸 except 吞信号 | 全部 `except Exception` + error 监听 |
  38. | **execSync 安全** | 14 处 cmd 窗口 | 全替换 execHidden (spawnSync) |
  39. | **自进化** | Hermes Fusion 基础 | 14 轮融合 + 70 次自动晋升 + 4 个 AI 工作流 |
  40. | **紧急恢复** | 无 | 安装失败自动快照回滚 |
  41. ## 版本演进
  42. | 版本 | 代号 | 模块数 | 核心 |
  43. |------|------|--------|------|
  44. | v1 | Context Proxy | 9 | 基础记忆 |
  45. | v2 | Xuanlin Overmind | 9+ | 知识图谱 · 自进化 |
  46. | v3 | (跳过发布) | 30 | 5 层记忆 · 18 MCP 工具 |
  47. | **v4 初版** | **无量网络** | **66** | **事件总线 · 红队 · 反事实 · 心理模型** |
  48. | **v4 重构版** | **无量网络重构** | **66** | **6 通道 · z2 中枢 · n2 终端 · 自适应安装 · 零进程触发** |
  49. ## 性能
  50. | 指标 | 数值 |
  51. |------|------|
  52. | 核心延迟 p50 | 0.001ms (60/66 模块亚毫秒) |
  53. | 管道吞吐 | 270 stages/s (缓存命中 ∞) |
  54. | 广播延迟 p99 | <1ms |
  55. | 事件 I/O | 2,500 write/s, 2,800 drain/s |
  56. | 6 通道并发 | 77ms/周期, 0% 崩溃率 |
  57. | 极限压力测试 | 200 周期 · 50 CC · 10,000 事件, 0 错误 |
  58. | 内存占用 | <500MB (66 模块) |
  59. | 日均 API 成本 | $0.05-0.15 |
  60. | 发布自测 | **42/42 PASSED** |
  61. [完整自测说明 → docs/SELF_TEST_CHECKLIST.md](docs/SELF_TEST_CHECKLIST.md)
  62. ## 安装
  63. ```bash
  64. git clone https://github.com/xuanlinAI/overmind.git && cd overmind && node install.js
  65. ```
  66. 一条命令完成全部 6 步:
  67. | 步骤 | 内容 |
  68. |:--:|------|
  69. | 1 | **环境探测** — OS/CPU/内存/磁盘/Python/Node/AI Agent 全检 |
  70. | 2 | **安全快照** — 安装前备份所有关键文件 |
  71. | 3 | **完整性校验** — 检测缺失/损坏,自动修复 |
  72. | 4 | **依赖安装** — npm + pip 按需安装 |
  73. | 5 | **配置生成** — .overmind_env.json + hooks/MCP/CLAUDE.md 自动写入 |
  74. | 6 | **冒烟测试** — 6 通道全链验证,失败自动回滚 |
  75. 安装后设置 `DEEPSEEK_API_KEY`,重启 Claude Code 即可。
  76. ## 六通道架构
  77. ```
  78. ┌── 核心模块 ──┐ ┌── z2中枢 ──┐ ┌── n2终端 ──┐
  79. │ │ │ │ │ │
  80. 串联 ────┤ CH1 │ │ CH3 │ │ CH5 │
  81. │ 37 stages │ │ !include │ │ 8链补全 │
  82. │ → injection │ │ → LLM自觉 │ │ → enrich │
  83. │ │ │ │ │ │
  84. 并联 ────┤ CH2 │ │ CH4 │ │ CH6 │
  85. │ 48 fire │ │ event→bus │ │ bus→11通知 │
  86. │ → sideFx │ │ → 9模块 │ │ → modules │
  87. └──────────────┘ └────────────┘ └────────────┘
  88. ```
  89. ## 核心能力 (66 模块)
  90. ### 记忆系统 (5层)
  91. - 语义记忆:SQLite + FTS5 + jieba 中文分词,AI 精选
  92. - 程序性记忆:同模式 ≥3 次自动晋升为可复用模板
  93. - 情景记忆:AI 生成会话摘要存档,按项目检索
  94. - 知识图谱:10 种关系类型,BFS 被动推理,因果链追踪
  95. - 反馈回路:4,500+ 事件驱动有效性评分自动调节
  96. ### 知识图谱 (10 种关系)
  97. `depends_on → part_of → blocked_by → causes → solves → related_to → extends → conflicts_with → alternative_to → triggers`
  98. 4,500+ 节点,4,600+ 边。被动推理:搜索一个记忆自动带出因果链。
  99. ### 智能分析 (CH1 37 阶段管道)
  100. 意图预判 · 安全门禁 · 提交门禁 · 输出盾 · 人格画像(8维) · 红队审查 · 异常检测 · 预测调试 · 成本分析 · 跨会话续接 · 技能编排 · 反事实引擎 · 知识验证 · 预测调试 · 检查点 · TDD约束 · 环境预取 · 噪声学习 · 心理模型 · 梦境研究(Pro深度分析) · 创意合成 · 思维链透明 · 自主研究 · 预算拦截 · 跨项目迁移(6领域) · 技能血统 · 记忆预算(5维) · 反流失 · 预判加载 · 会话简报 · 因果可视化 · 时间旅行 · 僵尸检测(90天) · 自愈检查 · 模块链接
  101. ### z2 中枢 (daemon.py)
  102. - FleetWatcher 5 秒扫描所有 CC 会话,提取 Q/A,生成舰队广播
  103. - CH3 直连:`.fleet_broadcast.md` → `!include` → LLM 自动感知同伴
  104. - CH4 中枢广播:`.event_queue/` → wiring.js drain → bus → 9 模块
  105. - 零进程触发:hook 写 `.trigger.tmp` → daemon 原子 rename → spawn inject.js
  106. - 20 个 MCP 工具,含 `fleet_status` `fleet_peek` 舰队查询
  107. ### n2 终端 (communicator.js)
  108. - AI 筛选:flash 模型过滤注入文档,94% 压缩率
  109. - CH5 串联补全:8 模块后处理链
  110. - CH6 并联通知:11 模块并行接收过滤后输出
  111. ### 涌现功能
  112. 自我镜像回路 · 舰队自省记忆 · 跨 CC 技能 boosting · 舰队冲突检测 · 跨 CC 创意碰撞 · CH4 去重 · 技能自动贴标签 (30/30)
  113. ### 技能生态
  114. 双阶段选择(AI精选+激进退底) · 偏好学习 · 技能编排(A→B→C链) · 技能热加载 · 技能血统 · 技能市场协议(manifest) · 30 个自创建技能(全标签)
  115. ### 安全与质量
  116. 红队审查(恶魔分身攻击) · 安全拦截(rm -rf检测) · 输出盾 · 提交门禁 · TDD约束 · 预算拦截(3次止损) · 补丁模式
  117. ### 部署与运维
  118. 自适应安装器(probe/vault/config/ready) · 舰队模式(多CC共享大脑) · 自愈系统(崩→重启,坏→恢复) · 看门狗(每分钟检查) · 日志旋转(5000行) · 跨平台(Win/Mac/Linux) · 多Agent(CC/Cursor/Aider等8个) · 零窗口闪烁
  119. ### MCP 工具 (20 个)
  120. `search_memory` · `save_memory` · `list_skills` · `create_skill` · `memory_stats` · `current_context` · `hermes_fusion` · `search_procedural` · `search_episodes` · `save_episodic` · `search_graph` · `expand_keys` · `create_edge` · `graph_stats` · `search_warnings` · `record_feedback` · `skill_rankings` · `skill_prefs` · **`fleet_status`** · **`fleet_peek`**
  121. ## 与竞品对比
  122. | | Xuanlin Overmind v4 | Mem0 | AgentMemory |
  123. |---|---|---|---|
  124. | 部署 | 100% 本地免费 | 云服务 | 本地 |
  125. | 模块数 | **66** | ~8 | ~12 |
  126. | 通道架构 | **6 通道** | 无 | 无 |
  127. | 图推理 | **10 种关系**,免费 | 锁 $249/月 | 基础 BM25 |
  128. | 认知功能 | 预警/预测/进化/梦境/红队/反事实/心理模型/舰队 | 无 | 无 |
  129. | 舰队协同 | **多 CC 自发现+协同** | 无 | 无 |
  130. | 自适应安装 | **全平台零配置** | 手动 | 手动 |
  131. | 日均成本 | **$0.05-0.15** | $8+/天 (Pro) | $0 (本地) |
  132. | 自测覆盖 | **42 项全绿** | 未公开 | 未公开 |
  133. 他们做的是 AI 的硬盘。Overmind 做的是 AI 的脑。
  134. ---
  135. <span id="english"></span>
  136. # 🇬🇧 English
  137. ## What is this
  138. Xuanlin Overmind v4 (Immeasurable Network) is a cognitive engine for Claude Code. 66 modules interconnected through 6 channels and an event bus form the first cognitive nervous system for AI coding agents.
  139. **It thinks while you're away. Every line of code you write is attacked by a red-team shadow before you see it. Every decision you make has a counterfactual shadow tracking "what if you chose differently." Every CC instance you open can see what the others are doing.**
  140. ## Refactored v4 vs Original v4
  141. | Dimension | Original v4 | Refactored v4 |
  142. |-----------|-------------|---------------|
  143. | **Install** | Manual config, Windows-only | `node install.js` one-click, cross-platform |
  144. | **Channels** | 2 (serial+parallel) | **6 channels** (CH1-CH6) |
  145. | **Pipeline stages** | 21 | **37** + quantum caching |
  146. | **Broadcast modules** | 18 | **48** |
  147. | **Agent support** | Claude Code only | Auto-detect CC/Cursor/Aider/Codex/Gemini |
  148. | **z2 Hub** | None | daemon.py FleetWatcher, 5s fleet broadcast |
  149. | **n2 Terminal** | None | AI filter + serial/parallel post-processing |
  150. | **Fleet mode** | None | Multi-CC auto-discovery + cross-CC collaboration |
  151. | **Window flash** | Frequent Windows console flash | **Completely eliminated** |
  152. | **Self-repair** | None | vault.js verification + snapshot rollback |
  153. | **Error handling** | Naked except swallowing signals | All `except Exception` + error listeners |
  154. | **execSync safety** | 14 cmd.exe window sites | All replaced with execHidden (spawnSync) |
  155. | **Self-evolution** | Basic Hermes Fusion | 14 fusions + 70 auto-promotions + 4 workflows |
  156. | **Emergency recovery** | None | Install failure → auto snapshot rollback |
  157. ## Version History
  158. | Version | Codename | Modules | Highlights |
  159. |---------|----------|---------|------------|
  160. | v1 | Context Proxy | 9 | Basic memory |
  161. | v2 | Xuanlin Overmind | 9+ | Knowledge graph · Self-evolution |
  162. | v3 | (Skipped) | 30 | 5-layer memory · 18 MCP tools |
  163. | **v4 Original** | **Immeasurable Network** | **66** | **Event bus · Red team · Counterfactual · Theory of mind** |
  164. | **v4 Refactored** | **Immeasurable Network R2** | **66** | **6-channel · z2 hub · n2 terminal · Adaptive install** |
  165. ## Performance
  166. | Metric | Value |
  167. |--------|-------|
  168. | Core latency p50 | 0.001ms (60/66 modules sub-ms) |
  169. | Pipeline throughput | 270 stages/s (∞ with cache hit) |
  170. | Broadcast latency p99 | <1ms |
  171. | Event I/O | 2,500 write/s, 2,800 drain/s |
  172. | 6-channel concurrent | 77ms/cycle, 0% crash rate |
  173. | Extreme stress test | 200 cycles · 50 CC · 10,000 events, 0 errors |
  174. | Memory footprint | <500MB (66 modules) |
  175. | Daily API cost | $0.05-0.15 |
  176. | Release self-test | **42/42 PASSED** |
  177. [Full self-test documentation → docs/SELF_TEST_CHECKLIST.md](docs/SELF_TEST_CHECKLIST.md)
  178. ## Installation
  179. ```bash
  180. git clone https://github.com/xuanlinAI/overmind.git && cd overmind && node install.js
  181. ```
  182. One command completes all 6 steps:
  183. | Step | Description |
  184. |:--:|------|
  185. | 1 | **Environment Probe** — OS/CPU/RAM/Disk/Python/Node/AI Agent detection |
  186. | 2 | **Safety Snapshot** — Backup all critical files before changes |
  187. | 3 | **Integrity Check** — Detect missing/corrupted files, auto-repair |
  188. | 4 | **Dependencies** — npm + pip install as needed |
  189. | 5 | **Config Generation** — .overmind_env.json + hooks/MCP/CLAUDE.md auto-written |
  190. | 6 | **Smoke Test** — Full 6-channel verification, auto-rollback on failure |
  191. Set `DEEPSEEK_API_KEY` after install, restart Claude Code.
  192. ## Six-Channel Architecture
  193. ```
  194. ┌── Core Modules ──┐ ┌── z2 Hub ──┐ ┌── n2 Terminal ──┐
  195. │ │ │ │ │ │
  196. Serial ──┤ CH1 │ │ CH3 │ │ CH5 │
  197. │ 37 stages │ │ !include │ │ 8-chain enrich │
  198. │ → injection │ │ → LLM │ │ → enrich doc │
  199. │ │ │ │ │ │
  200. Parallel ┤ CH2 │ │ CH4 │ │ CH6 │
  201. │ 48 fire │ │ event→bus │ │ bus→11 notify │
  202. │ → side effects │ │ → 9 modules│ │ → modules │
  203. └──────────────────┘ └────────────┘ └─────────────────┘
  204. ```
  205. ## Core Capabilities (66 Modules)
  206. ### Memory System (5 Layers)
  207. - **Semantic Memory** — SQLite + FTS5 + jieba Chinese tokenizer, AI-curated selection
  208. - **Procedural Memory** — Patterns occurring ≥3 times auto-promoted to reusable templates
  209. - **Episodic Memory** — AI-generated session summaries, searchable by project
  210. - **Knowledge Graph** — 10 relation types, BFS passive reasoning, causal chain tracking
  211. - **Feedback Loop** — 4,500+ events driving automated effectiveness scoring
  212. ### Knowledge Graph (10 Relations)
  213. `depends_on → part_of → blocked_by → causes → solves → related_to → extends → conflicts_with → alternative_to → triggers`
  214. 4,500+ nodes, 4,600+ edges. Passive reasoning: searching one memory automatically surfaces its causal chain.
  215. ### Intelligent Analysis (CH1 37-Stage Pipeline)
  216. Intent prediction · Security gatekeeper · Commit gate · Output shield · Persona profiling (8-dim) · Red-team adversarial review · Anomaly detection · Forecast debugging · Cost optimization · Cross-session continuity · Skill composition · Counterfactual engine · Knowledge verification · Predictive debugging · Checkpoints · TDD enforcement · Environment prefetch · Noise learning · Theory of mind · Dream research (pro model) · Creative synthesis · Reasoning transparency · Autonomous research · Budget interception · Cross-project transfer (6 domains) · Skill lineage · Memory budgeting (5-dim) · Anti-compaction · Preloading · Session briefing · Causal visualization · Time travel · Zombie detection (90-day) · Self-healing · Module nexus
  217. ### z2 Hub (daemon.py)
  218. - FleetWatcher: 5-second scan of all CC sessions, Q/A extraction, fleet broadcast generation
  219. - CH3 Direct: `.fleet_broadcast.md` → `!include` → LLM auto-awareness of peer CCs
  220. - CH4 Hub Broadcast: `.event_queue/` → wiring.js drain → bus → 9 modules
  221. - Zero-process trigger: hook writes `.trigger.tmp` → daemon atomic rename → spawn inject.js
  222. - 20 MCP tools, including `fleet_status` and `fleet_peek`
  223. ### n2 Terminal (communicator.js)
  224. - AI Filter: flash model filtering, 94% compression ratio
  225. - CH5 Serial Enrichment: 8-module post-processing chain
  226. - CH6 Parallel Notification: 11 modules receive filtered output simultaneously
  227. ### Emergent Capabilities
  228. Self-mirror loop · Fleet self-aware memory · Cross-CC skill boosting · Fleet conflict detection · Cross-CC creative collision · CH4 dedup · Auto skill tagging (30/30)
  229. ### Skill Ecosystem
  230. Dual-stage AI selection with aggressive fallback · Preference learning · Skill chain detection (A→B→C) · Hot-reload · Skill lineage · Marketplace protocol · 30 auto-created skills (fully tagged)
  231. ### Security & Quality
  232. Red-team adversarial review (shadow twin attack) · Security interception (rm -rf detection) · Output shield · Commit gate · TDD enforcement · Budget interceptor (3-failure stop-loss) · Patch mode
  233. ### Deployment & Operations
  234. Adaptive installer (probe/vault/config/ready) · Fleet mode (multi-CC shared brain) · Self-healing (crash→restart, corrupt→recover) · Watchdog (per-minute checks) · Log rotation (5,000 lines) · Cross-platform (Win/Mac/Linux) · Multi-agent (CC/Cursor/Aider + 5 more) · Zero window flash
  235. ### MCP Tools (20)
  236. `search_memory` · `save_memory` · `list_skills` · `create_skill` · `memory_stats` · `current_context` · `hermes_fusion` · `search_procedural` · `search_episodes` · `save_episodic` · `search_graph` · `expand_keys` · `create_edge` · `graph_stats` · `search_warnings` · `record_feedback` · `skill_rankings` · `skill_prefs` · **`fleet_status`** · **`fleet_peek`**
  237. ## vs Competitors
  238. | | Xuanlin Overmind v4 | Mem0 | AgentMemory |
  239. |---|---|---|---|
  240. | Deployment | 100% local, free | Cloud service | Local |
  241. | Modules | **66** | ~8 | ~12 |
  242. | Channel architecture | **6 channels** | None | None |
  243. | Graph reasoning | **10 relations**, free | Locked behind $249/mo | Basic BM25 |
  244. | Cognitive functions | Warning/Prediction/Evolution/Dream/Red-team/Counterfactual/ToM/Fleet | None | None |
  245. | Fleet collaboration | **Multi-CC auto-discovery** | None | None |
  246. | Adaptive install | **Cross-platform, zero-config** | Manual | Manual |
  247. | Daily cost | **$0.05-0.15** | $8+/day (Pro) | $0 (local) |
  248. | Self-test coverage | **42 items all-green** | Unknown | Unknown |
  249. They built AI's hard drive. Overmind is AI's brain.
  250. ## Platform Compatibility
  251. | Agent | Status |
  252. |-------|--------|
  253. | Claude Code | ✅ Native |
  254. | Cursor | ✅ Adapted (auto-detect) |
  255. | Aider | ✅ Adapted (auto-detect) |
  256. | Hermes Agent | ✅ Adapter |
  257. | Codex CLI | ✅ Adapter |
  258. | Gemini CLI | ✅ Adapter |
  259. | OpenClaw | ✅ Adapter |
  260. ---
  261. <p align="center">
  262. Made by <a href="https://github.com/xuanlinAI">玄霖AI (xuanlinAI)</a>
  263. </p>
daemon.py · Bundled script · 1,500 lines · 3 findings marked
  1. """Xuanlin Overmind z2 Hub — MCP server + FleetWatcher broadcast"""
  2. import json, sys, os, sqlite3, hashlib, re, time, glob, jieba, threading, tempfile, subprocess
  3. # pythonw on Windows has sys.stdout = None — redirect to devnull to prevent
  4. # process hang when print() buffer fills (subsystem-level deadlock)
  5. if sys.stdout is None:
  6. sys.stdout = open(os.devnull, 'w')
  7. if sys.stderr is None:
  8. sys.stderr = open(os.devnull, 'w')
  9. ROOT = os.path.dirname(os.path.abspath(__file__))
  10. HOME = os.environ.get('HOME') or os.environ.get('USERPROFILE') or os.path.expanduser('~')
  11. CC_SESSIONS_DIR = os.path.join(HOME, '.claude', 'sessions')
  12. CC_PROJECTS_DIR = os.path.join(HOME, '.claude', 'projects')
  13. FLEET_BROADCAST_FILE = os.path.join(ROOT, '.fleet_broadcast.md')
  14. EVENT_QUEUE_DIR = os.path.join(ROOT, '.event_queue')
  15. FLEET_STATE_FILE = os.path.join(ROOT, '.fleet_state.json')
  16. SCAN_INTERVAL = 5 # seconds
  17. SKILL_DIRS = [
  18. os.path.join(ROOT, 'skills'), # custom skills
  19. os.path.join(HOME, '.claude', 'skills'), # CC skills dir
  20. os.path.join(HOME, '.claude', 'plugins', 'cache'), # installed plugins
  21. ]
  22. DB_PATH = os.path.join(ROOT, 'memory.db')
  23. GRAPH_DB_PATH = os.path.join(ROOT, 'graph.db')
  24. RELATION_TYPES = [
  25. 'depends_on', 'part_of', 'blocked_by', 'causes', 'solves',
  26. 'related_to', 'extends', 'conflicts_with', 'alternative_to', 'triggers'
  27. ]
  28. def init_db():
  29. db = sqlite3.connect(DB_PATH)
  30. db.execute('CREATE TABLE IF NOT EXISTS semantic (id INTEGER PRIMARY KEY AUTOINCREMENT, key TEXT UNIQUE, content TEXT, tags TEXT DEFAULT "", created_at TEXT DEFAULT (datetime("now")), updated_at TEXT DEFAULT (datetime("now")), access_count INTEGER DEFAULT 0)')
  31. for col, ctype in [('confidence', 'REAL DEFAULT 0.5'), ('last_accessed', 'TEXT'), ('promotion_count', 'INTEGER DEFAULT 0'), ('content_hash', 'TEXT')]:
  32. try: db.execute(f'ALTER TABLE semantic ADD COLUMN {col} {ctype}')
  33. except Exception: pass
  34. for col, ctype in [('version', "TEXT DEFAULT '1.0.0'"), ('requires', 'TEXT'), ('provides', 'TEXT'), ('invoke_count', 'INTEGER DEFAULT 0'), ('last_invoked', 'TEXT'), ('quality_score', 'REAL DEFAULT 0.3')]:
  35. try: db.execute(f'ALTER TABLE skill_index ADD COLUMN {col} {ctype}')
  36. except Exception: pass
  37. try: db.execute('CREATE VIRTUAL TABLE IF NOT EXISTS semantic_fts USING fts5(key, content, tags, tokenize="unicode61")')
  38. except Exception: pass
  39. try: db.execute('ALTER TABLE semantic ADD COLUMN source_session TEXT')
  40. except Exception: pass
  41. try: db.execute('ALTER TABLE semantic ADD COLUMN dedup_key TEXT')
  42. except Exception: pass
  43. try: db.execute('ALTER TABLE semantic ADD COLUMN procedural_source TEXT')
  44. except Exception: pass
  45. try: db.execute('CREATE INDEX IF NOT EXISTS idx_semantic_dedup ON semantic(dedup_key)')
  46. except Exception: pass
  47. db.execute('''CREATE TABLE IF NOT EXISTS episodic (
  48. id INTEGER PRIMARY KEY AUTOINCREMENT,
  49. session_id TEXT NOT NULL,
  50. summary TEXT DEFAULT "",
  51. task TEXT DEFAULT "",
  52. message_count INTEGER DEFAULT 0,
  53. transcript_path TEXT DEFAULT "",
  54. project_name TEXT DEFAULT "",
  55. created_at TEXT NOT NULL
  56. )''')
  57. db.execute('CREATE TABLE IF NOT EXISTS skill_index (name TEXT PRIMARY KEY, description TEXT, triggers TEXT DEFAULT "", file_path TEXT, installed_at TEXT DEFAULT (datetime("now")))')
  58. db.execute('CREATE TABLE IF NOT EXISTS evolution_log (id INTEGER PRIMARY KEY AUTOINCREMENT, session_id TEXT, action TEXT, detail TEXT, created_at TEXT DEFAULT (datetime("now")))')
  59. db.execute('''CREATE TABLE IF NOT EXISTS procedural (
  60. id INTEGER PRIMARY KEY AUTOINCREMENT,
  61. name TEXT UNIQUE NOT NULL,
  62. description TEXT NOT NULL,
  63. steps TEXT NOT NULL,
  64. trigger_patterns TEXT DEFAULT '',
  65. use_count INTEGER DEFAULT 0,
  66. status TEXT DEFAULT "active",
  67. created_at TEXT DEFAULT (datetime("now"))
  68. )''')
  69. try: db.execute('CREATE VIRTUAL TABLE IF NOT EXISTS procedural_fts USING fts5(name, description, steps, trigger_patterns, tokenize="unicode61")')
  70. except Exception: pass
  71. db.execute("INSERT OR IGNORE INTO semantic (key, content, tags) VALUES ('_schema_version', '1', 'system')")
  72. db.execute('CREATE INDEX IF NOT EXISTS idx_semantic_stale ON semantic(last_accessed, updated_at)')
  73. db.commit()
  74. return db
  75. db = init_db()
  76. # ---- FEEDBACK LOOP ----
  77. for col, ctype in [('effectiveness_score', 'REAL DEFAULT 0.5'), ('last_effective_at', 'TEXT'), ('ineffective_count', 'INTEGER DEFAULT 0'), ('injected_count', 'INTEGER DEFAULT 0')]:
  78. try: db.execute(f'ALTER TABLE semantic ADD COLUMN {col} {ctype}')
  79. except Exception: pass
  80. db.execute('''CREATE TABLE IF NOT EXISTS feedback_events (
  81. id INTEGER PRIMARY KEY AUTOINCREMENT,
  82. memory_key TEXT NOT NULL,
  83. event_type TEXT NOT NULL,
  84. session_id TEXT,
  85. detail TEXT DEFAULT "",
  86. created_at TEXT DEFAULT (datetime("now")))''')
  87. try: db.execute('CREATE INDEX IF NOT EXISTS idx_feedback_key ON feedback_events(memory_key)')
  88. except Exception: pass
  89. db.execute('''CREATE TABLE IF NOT EXISTS skill_feedback (
  90. id INTEGER PRIMARY KEY AUTOINCREMENT,
  91. skill_name TEXT NOT NULL,
  92. event_type TEXT NOT NULL,
  93. task_context TEXT DEFAULT "",
  94. session_id TEXT,
  95. effectiveness REAL DEFAULT 0.5,
  96. created_at TEXT DEFAULT (datetime("now")))''')
  97. try: db.execute('CREATE INDEX IF NOT EXISTS idx_skillfb_name ON skill_feedback(skill_name)')
  98. except Exception: pass
  99. db.execute('''CREATE TABLE IF NOT EXISTS skill_prefs (
  100. skill_name TEXT NOT NULL,
  101. task_pattern TEXT NOT NULL,
  102. use_count INTEGER DEFAULT 1,
  103. effectiveness REAL DEFAULT 0.5,
  104. last_used TEXT,
  105. UNIQUE(skill_name, task_pattern))''')
  106. def record_feedback(memory_key, event_type, session_id='', detail=''):
  107. valid = {'injected', 'referenced', 'helped', 'did_not_help', 'caused_confusion'}
  108. if event_type not in valid: return False
  109. try:
  110. db.execute('INSERT INTO feedback_events (memory_key, event_type, session_id, detail) VALUES (?,?,?,?)', (memory_key, event_type, session_id, detail))
  111. if event_type == 'injected':
  112. db.execute("UPDATE semantic SET injected_count = COALESCE(injected_count, 0) + 1, last_accessed = datetime('now') WHERE key = ?", (memory_key,))
  113. elif event_type == 'helped':
  114. db.execute("UPDATE semantic SET effectiveness_score = MIN(1.0, COALESCE(effectiveness_score, 0.5) + 0.15), last_effective_at = datetime('now'), confidence = MIN(1.0, COALESCE(confidence, 0.5) + 0.1) WHERE key = ?", (memory_key,))
  115. elif event_type in ('did_not_help', 'caused_confusion'):
  116. db.execute("UPDATE semantic SET effectiveness_score = MAX(0.05, COALESCE(effectiveness_score, 0.5) - 0.2), ineffective_count = COALESCE(ineffective_count, 0) + 1, confidence = MAX(0.1, COALESCE(confidence, 0.5) - 0.1) WHERE key = ?", (memory_key,))
  117. elif event_type == 'referenced':
  118. db.execute("UPDATE semantic SET access_count = COALESCE(access_count, 0) + 1, last_accessed = datetime('now') WHERE key = ?", (memory_key,))
  119. db.commit()
  120. return True
  121. except Exception: return False
  122. def prune_ineffective_mems():
  123. bad = db.execute("SELECT key, content, COALESCE(effectiveness_score,0.5) as eff, COALESCE(ineffective_count,0) as nc FROM semantic WHERE key != '_schema_version' AND ineffective_count >= 3 AND COALESCE(effectiveness_score, 0.5) < 0.2").fetchall()
  124. if not bad: return 0
  125. archive_dir = os.path.join(ROOT, 'memory', 'archive')
  126. os.makedirs(archive_dir, exist_ok=True)
  127. archive_file = os.path.join(archive_dir, f'ineffective_{time.strftime("%Y%m%d_%H%M%S")}.json')
  128. archived = [{'key': r[0], 'content': r[1], 'effectiveness': r[2], 'ineffective_count': r[3]} for r in bad]
  129. with open(archive_file, 'w', encoding='utf-8') as f:
  130. json.dump(archived, f, ensure_ascii=False, indent=2)
  131. for r in bad:
  132. db.execute('DELETE FROM semantic WHERE key = ?', (r[0],))
  133. db.execute("DELETE FROM semantic_fts WHERE rowid = (SELECT rowid FROM semantic_fts WHERE key = ?)", (r[0],))
  134. db.execute('DELETE FROM feedback_events WHERE memory_key = ?', (r[0],))
  135. db.execute("INSERT INTO evolution_log (session_id, action, detail) VALUES ('hermes', 'prune_ineffective', ?)", (json.dumps({'count': len(bad)}),))
  136. db.commit()
  137. return len(bad)
  138. # ---- GRAPH DATABASE ----
  139. def init_graph():
  140. gdb = sqlite3.connect(GRAPH_DB_PATH)
  141. gdb.execute('PRAGMA journal_mode = WAL')
  142. gdb.execute('PRAGMA wal_autocheckpoint = 200')
  143. gdb.execute('''CREATE TABLE IF NOT EXISTS nodes (
  144. key TEXT PRIMARY KEY, label TEXT DEFAULT '',
  145. node_type TEXT DEFAULT "memory", importance REAL DEFAULT 0.5,
  146. created_at TEXT DEFAULT (datetime("now")), updated_at TEXT DEFAULT (datetime("now")))''')
  147. gdb.execute('''CREATE TABLE IF NOT EXISTS edges (
  148. id INTEGER PRIMARY KEY AUTOINCREMENT,
  149. source TEXT NOT NULL, target TEXT NOT NULL, relation_type TEXT NOT NULL,
  150. confidence REAL DEFAULT 0.5, evidence TEXT DEFAULT "",
  151. source_session TEXT DEFAULT "",
  152. created_at TEXT DEFAULT (datetime("now")),
  153. access_count INTEGER DEFAULT 0,
  154. UNIQUE(source, target, relation_type))''')
  155. gdb.execute('CREATE INDEX IF NOT EXISTS idx_gedges_src ON edges(source)')
  156. gdb.execute('CREATE INDEX IF NOT EXISTS idx_gedges_tgt ON edges(target)')
  157. gdb.execute('CREATE INDEX IF NOT EXISTS idx_gedges_type ON edges(relation_type)')
  158. try: gdb.execute("ALTER TABLE edges ADD COLUMN updated_at TEXT")
  159. except Exception: pass
  160. try: gdb.execute("CREATE VIRTUAL TABLE IF NOT EXISTS edges_fts USING fts5(source, target, relation_type, evidence, tokenize='unicode61')")
  161. except Exception: pass
  162. gdb.commit()
  163. return gdb
  164. gdb = init_graph()
  165. def upsert_edge(source, target, relation_type, confidence=0.5, evidence='', session_id=''):
  166. if relation_type not in RELATION_TYPES:
  167. relation_type = 'related_to'
  168. if source == target: return None
  169. row = gdb.execute('SELECT id, confidence FROM edges WHERE source=? AND target=? AND relation_type=?', (source, target, relation_type)).fetchone()
  170. if row:
  171. new_conf = min(1.0, max(row[1], confidence) + 0.05)
  172. gdb.execute('UPDATE edges SET confidence=?, evidence=CASE WHEN ?!="" AND (evidence IS NULL OR evidence NOT LIKE "%"||?||"%") THEN COALESCE(evidence,"") || " | " || ? ELSE evidence END, access_count=access_count+1 WHERE id=?', (new_conf, evidence, evidence, evidence, row[0]))
  173. return row[0]
  174. try:
  175. cur = gdb.execute('INSERT OR IGNORE INTO edges (source, target, relation_type, confidence, evidence, source_session) VALUES (?,?,?,?,?,?)', (source, target, relation_type, confidence, evidence, session_id))
  176. gdb.commit()
  177. if cur.lastrowid:
  178. try: gdb.execute('INSERT INTO edges_fts(rowid, source, target, relation_type, evidence) VALUES (?,?,?,?,?)', (cur.lastrowid, source, target, relation_type, evidence))
  179. except Exception: pass
  180. gdb.execute('INSERT OR IGNORE INTO nodes (key) VALUES (?)', (source,))
  181. gdb.execute('INSERT OR IGNORE INTO nodes (key) VALUES (?)', (target,))
  182. gdb.commit()
  183. return cur.lastrowid
  184. except Exception: pass
  185. return None
  186. def get_neighbors(key, depth=2):
  187. visited = {key}
  188. nodes_map = {}
  189. edges = []
  190. frontier = [key]
  191. for d in range(depth):
  192. next_frontier = []
  193. for nk in frontier:
  194. rows = gdb.execute('SELECT source, target, relation_type, confidence, evidence FROM edges WHERE source=? OR target=? ORDER BY confidence DESC', (nk, nk)).fetchall()
  195. for src, tgt, rtype, conf, ev in rows:
  196. edges.append({'source': src, 'target': tgt, 'relation_type': rtype, 'confidence': conf, 'evidence': ev or ''})
  197. nb = tgt if src == nk else src
  198. if nb not in visited:
  199. visited.add(nb)
  200. next_frontier.append(nb)
  201. node = gdb.execute('SELECT key, label, node_type, importance FROM nodes WHERE key=?', (nb,)).fetchone()
  202. if node: nodes_map[nb] = {'key': node[0], 'label': node[1], 'node_type': node[2], 'importance': node[3]}
  203. else: nodes_map[nb] = {'key': nb, 'label': '', 'node_type': 'memory', 'importance': 0.5}
  204. frontier = next_frontier
  205. return {'nodes': list(nodes_map.values()), 'edges': edges, 'root': key, 'depth': depth}
  206. def expand_keys(memory_keys, depth=1):
  207. if not memory_keys: return {'keys': [], 'subgraph': {'nodes': [], 'edges': []}}
  208. all_edges = []
  209. all_nodes = {}
  210. expanded = set(memory_keys)
  211. for k in memory_keys:
  212. sub = get_neighbors(k, depth)
  213. for e in sub['edges']: all_edges.append(e)
  214. for n in sub['nodes']:
  215. all_nodes[n['key']] = n
  216. expanded.add(n['key'])
  217. seen = set()
  218. deduped = []
  219. for e in all_edges:
  220. sig = (e['source'], e['target'], e['relation_type'])
  221. if sig not in seen:
  222. seen.add(sig)
  223. deduped.append(e)
  224. return {'keys': list(expanded), 'subgraph': {'nodes': list(all_nodes.values()), 'edges': deduped}}
  225. def search_graph(query, limit=10):
  226. if not query:
  227. rows = gdb.execute('SELECT source, target, relation_type, confidence, evidence FROM edges ORDER BY confidence DESC LIMIT ?', (limit,)).fetchall()
  228. return [{'source': r[0], 'target': r[1], 'relation_type': r[2], 'confidence': r[3], 'evidence': r[4] or ''} for r in rows]
  229. try:
  230. rows = gdb.execute("SELECT e.source, e.target, e.relation_type, e.confidence, e.evidence FROM edges_fts f JOIN edges e ON f.rowid = e.id WHERE edges_fts MATCH ? ORDER BY rank LIMIT ?", (query, limit)).fetchall()
  231. except Exception:
  232. like = f'%{query}%'
  233. rows = gdb.execute('SELECT source, target, relation_type, confidence, evidence FROM edges WHERE source LIKE ? OR target LIKE ? OR relation_type LIKE ? OR evidence LIKE ? ORDER BY confidence DESC LIMIT ?', (like, like, like, like, limit)).fetchall()
  234. return [{'source': r[0], 'target': r[1], 'relation_type': r[2], 'confidence': r[3], 'evidence': r[4] or ''} for r in rows]
  235. def get_graph_stats():
  236. nodes = gdb.execute('SELECT COUNT(*) FROM nodes').fetchone()[0]
  237. edges = gdb.execute('SELECT COUNT(*) FROM edges').fetchone()[0]
  238. types = gdb.execute('SELECT relation_type, COUNT(*) as c FROM edges GROUP BY relation_type ORDER BY c DESC').fetchall()
  239. return {'nodes': nodes, 'edges': edges, 'relation_types': [{'type': r[0], 'count': r[1]} for r in types]}
  240. # Graph maintenance: prune low-confidence edges, merge duplicates
  241. def prune_graph():
  242. deleted = gdb.execute('DELETE FROM edges WHERE confidence < 0.15').rowcount
  243. if deleted:
  244. gdb.execute("INSERT INTO evolution_log (session_id, action, detail) VALUES ('hermes', 'graph_prune', ?)", (json.dumps({'deleted': deleted}),))
  245. db.commit()
  246. return deleted
  247. def read_msg():
  248. line = sys.stdin.readline()
  249. if not line:
  250. return None
  251. return json.loads(line)
  252. def send_msg(obj):
  253. sys.stdout.write(json.dumps(obj, ensure_ascii=False) + '\n')
  254. sys.stdout.flush()
  255. def search_skills(query='', limit=3):
  256. if not query:
  257. rows = db.execute('SELECT * FROM skill_index ORDER BY installed_at DESC LIMIT ?', (limit,)).fetchall()
  258. return [{'name': r[0], 'description': r[1], 'triggers': r[2], 'file_path': r[3]} for r in rows]
  259. words = [w.lower() for w in query.split() if len(w) > 1]
  260. if not words:
  261. rows = db.execute('SELECT * FROM skill_index LIMIT ?', (limit,)).fetchall()
  262. return [{'name': r[0], 'description': r[1], 'triggers': r[2], 'file_path': r[3]} for r in rows]
  263. # Get all skills, score each by word hit count
  264. rows = db.execute('SELECT * FROM skill_index').fetchall()
  265. scored = []
  266. for r in rows:
  267. name = (r[0] or '').lower()
  268. desc = (r[1] or '').lower()
  269. triggers = (r[2] or '').lower()
  270. combined = name + ' ' + desc + ' ' + triggers
  271. score = 0
  272. for w in words:
  273. if w in name: score += 3
  274. elif w in triggers: score += 3
  275. elif w in desc: score += 1
  276. # Partial match (substring)
  277. elif any(w in part for part in combined.split()): score += 0.5
  278. if score > 0:
  279. scored.append((score, r))
  280. scored.sort(key=lambda x: x[0], reverse=True)
  281. return [{'name': r[0], 'description': r[1], 'triggers': r[2], 'file_path': r[3]} for _, r in scored[:limit]]
  282. def char_bigrams(text):
  283. """Generate character bigrams for CJK text matching"""
  284. chars = re.sub(r'[\s\d\W]', '', text)
  285. bigrams = set()
  286. for i in range(len(chars) - 1):
  287. bigrams.add(chars[i:i+2])
  288. return list(bigrams)
  289. def jieba_segment(text):
  290. """Segment Chinese text with jieba, preserve non-Chinese tokens"""
  291. words = jieba.cut(text)
  292. return ' '.join(w for w in words if w.strip())
  293. def search_hybrid(query, limit=10):
  294. q = jieba_segment(query).split()
  295. fts_query = ' OR '.join(w for w in q if w and len(w) > 1)
  296. rows = []
  297. if fts_query:
  298. try:
  299. rows = db.execute(
  300. "SELECT s.rowid, s.key, s.content, s.tags, rank, COALESCE(m.effectiveness_score, 0.5), COALESCE(m.injected_count, 0), COALESCE(m.ineffective_count, 0) FROM semantic_fts s JOIN semantic m ON s.key = m.key WHERE semantic_fts MATCH ? AND s.key != '_schema_version' ORDER BY rank LIMIT ?",
  301. (fts_query, limit)).fetchall()
  302. except Exception:
  303. rows = db.execute(
  304. "SELECT id, key, content, tags, COALESCE(effectiveness_score, 0.5), COALESCE(injected_count, 0), COALESCE(ineffective_count, 0) FROM semantic WHERE key != '_schema_version' ORDER BY updated_at DESC LIMIT ?",
  305. (limit,)).fetchall()
  306. else:
  307. rows = db.execute(
  308. "SELECT id, key, content, tags, COALESCE(effectiveness_score, 0.5), COALESCE(injected_count, 0), COALESCE(ineffective_count, 0) FROM semantic WHERE key != '_schema_version' ORDER BY updated_at DESC LIMIT ?",
  309. (limit,)).fetchall()
  310. seen = set()
  311. result = []
  312. for r in rows:
  313. k = r[1]
  314. if k not in seen:
  315. seen.add(k)
  316. update_confidence(k, 0.03)
  317. # FTS row: (rowid, key, content, tags, rank, eff, inj_cnt, ineff_cnt) = 8 cols
  318. # Non-FTS row: (id, key, content, tags, eff, inj_cnt, ineff_cnt) = 7 cols
  319. n = len(r)
  320. eff = round(r[n-3], 2) if n >= 7 and r[n-3] is not None else 0.5
  321. inj = int(r[n-2] or 0) if n >= 7 else 0
  322. ineff = int(r[n-1] or 0) if n >= 7 else 0
  323. result.append({'key': k, 'content': r[2], 'tags': r[3] if r[3] else '', 'effectiveness': eff, 'injected_count': inj, 'ineffective_count': ineff})
  324. auto_promote()
  325. return result[:limit]
  326. def call_ai_api(messages, max_tokens=16384, timeout=120):
  327. import urllib.request
  328. API_KEY = os.environ.get('DEEPSEEK_API_KEY') or os.environ.get('ANTHROPIC_AUTH_TOKEN') or 'YOUR_LLM_API_KEY'
    • Warning: If no DeepSeek key is set, it falls back to the user's ANTHROPIC_AUTH_TOKEN and sends it as a Bearer token to api.deepseek.com, together with memory and session text.
    • Line 363 on GitHub
  329. body = json.dumps({
  330. 'model': 'deepseek-v4-flash',
  331. 'max_tokens': max_tokens,
  332. 'messages': messages
  333. }).encode('utf-8')
  334. req = urllib.request.Request('https://api.deepseek.com/v1/chat/completions',
  335. data=body,
  336. headers={'Content-Type': 'application/json', 'Authorization': f'Bearer {API_KEY}'})
  337. resp = urllib.request.urlopen(req, timeout=timeout)
  338. data = json.loads(resp.read())
  339. return data['choices'][0]['message']['content']
  340. def select_memories_ai(query, all_mems, limit=5):
  341. """FTS5 pre-filter → flash model picks best memories. Same pattern as inject.js selectMemoriesAI."""
  342. if not all_mems:
  343. return []
  344. # Pre-filter: sliding bigram keyword scoring
  345. raw = (query or '').lower()
  346. keywords = []
  347. for i in range(len(raw) - 1):
  348. ch = raw[i:i+2]
  349. if ch and len(ch) == 2:
  350. keywords.append(ch)
  351. eng_words = re.findall(r'[a-z0-9]{3,}', raw)
  352. keywords.extend(eng_words)
  353. has_cjk = bool(re.search(r'[一-鿿]', query or ''))
  354. scored = []
  355. for m in all_mems:
  356. key_low = (m.get('key') or '').lower()
  357. content_low = (m.get('content') or '').lower()
  358. txt = key_low + ' ' + content_low
  359. s = 0
  360. for k in keywords:
  361. if k in key_low: s += 2
  362. elif k in content_low: s += 1
  363. if has_cjk and len(content_low) > 20:
  364. s += 0.5
  365. eff = float(m.get('effectiveness', 0.5) or 0.5)
  366. inj = int(m.get('injected_count', 0) or 0)
  367. # Effectiveness boost: proven memories get up to +1.5, unproven get baseline
  368. eff_boost = eff * 1.5 + (0.1 if inj > 0 else 0)
  369. scored.append((s + eff_boost, m))
  370. scored.sort(key=lambda x: x[0], reverse=True)
  371. shortlist = [m for _, m in scored[:40]]
  372. if len(shortlist) < 3:
  373. return shortlist[:limit]
  374. catalog = '\n'.join([f"- {m['key']}: {(m.get('content') or '')[:60]}" for m in shortlist])
  375. prompt = f"Pick the {limit} BEST memories for this task. Prefer specific technical facts. Skip plugin-internal noise. You MUST pick at least 2. Return ONLY JSON array: [\"key1\",\"key2\",\"key3\"]\n\nTask: {(query or '')[:300]}\n\n{catalog}"
  376. try:
  377. result = call_ai_api([{'role': 'user', 'content': prompt}], max_tokens=16384, timeout=120)
  378. if not result:
  379. return shortlist[:limit]
  380. names = []
  381. try: names = json.loads(result.strip())
  382. except Exception:
  383. m = re.search(r'\[.*\]', result, re.DOTALL)
  384. if m:
  385. try: names = json.loads(m.group(0))
  386. except Exception: pass
  387. if not isinstance(names, list):
  388. return shortlist[:limit]
  389. selected = []
  390. for n in names:
  391. for m in shortlist:
  392. if m.get('key') == n:
  393. selected.append(m)
  394. break
  395. return selected[:limit] if selected else shortlist[:limit]
  396. except Exception:
  397. return shortlist[:limit]
  398. def search_hybrid_ai(query, limit=10):
  399. """FTS5 pre-filter → AI selection. Fall back to pure FTS5 on API failure."""
  400. pre = search_hybrid(query, 30)
  401. if not pre:
  402. return []
  403. all_mems = [{'key': r['key'], 'content': r['content'], 'tags': r.get('tags', '')} for r in pre]
  404. result = select_memories_ai(query, all_mems, limit)
  405. for r in result:
  406. update_confidence(r['key'], 0.02)
  407. return result
  408. def auto_tag_skill(name, desc, content=''):
  409. """Extract matching keywords from skill name + description + body"""
  410. tags = set()
  411. text = (name + ' ' + desc + ' ' + (content or '')[:1000]).lower()
  412. # 1. Extract English technical terms (only camelCase, snake_case, dot.case — not plain words)
  413. for m in re.finditer(r'\b([a-z]+(?:[A-Z][a-z]+)+|[a-z]+_[a-z]+|[a-z]+\.[a-z]+)\b', text):
  414. word = m.group(1)
  415. if len(word) >= 4:
  416. tags.add(word.lower())
  417. # Also extract ALL_CAPS acronyms (FTS5, JSON, API, etc) — 2-6 uppercase letters
  418. for m in re.finditer(r'\b([A-Z]{2,6})\b', text):
  419. tags.add(m.group(1))
  420. # 2. Chinese technical bigrams (2-4 chars, filter common words)
  421. cjk = re.findall(r'[一-鿿]{2,4}', text)
  422. stop_cjk = {'使用','可以','需要','一个','这个','进行','通过','所有','什么','怎么','如何','如果','或者','但是','因为','所以','已经','没有','不是','只是','还是','并且','然后','其中','之后','之前','包含','以及'}
  423. for w in cjk:
  424. if w not in stop_cjk:
  425. tags.add(w)
  426. # 3. Tech domain keywords from name
  427. name_low = name.lower()
  428. domain_map = {
  429. 'reverse': ['逆向', 'reverse', '反汇编', 'decompile'],
  430. 'encrypt': ['加密', 'encrypt', '签名', 'sign', 'AES', 'RSA', 'SM2', 'SM4', 'HMAC', 'MD5'],
  431. 'frida': ['frida', 'hook', '注入', 'inject'],
  432. 'js': ['javascript', 'JS', 'AST', '反混淆', 'deobfuscate'],
  433. 'binary': ['二进制', 'binary', 'native', 'so文件', 'elf'],
  434. 'api': ['API', 'http', 'fetch', '接口', '请求'],
  435. 'wechat': ['微信', 'wechat', '小程序', 'miniprogram'],
  436. 'proxy': ['代理', 'proxy', 'mitmproxy', '抓包'],
  437. 'ticket': ['抢票', 'ticket', '预约', 'booking'],
  438. 'skill': ['技能', 'skill', '自动', 'auto'],
  439. 'review': ['审查', 'review', '代码质量'],
  440. 'test': ['测试', 'test', 'benchmark'],
  441. 'debug': ['调试', 'debug', '日志', 'log'],
  442. 'clean': ['清理', 'clean', '删除', 'remove'],
  443. 'inject': ['注入', 'inject', 'injection'],
  444. 'evolve': ['进化', 'evolve', '自进化', 'self'],
  445. }
  446. for key, terms in domain_map.items():
  447. if key in name_low:
  448. for t in terms:
  449. tags.add(t)
  450. # 4. Fallback: extract keywords from name itself (split on hyphen/underscore)
  451. if not tags:
  452. for part in re.split(r'[-_]', name):
  453. if len(part) >= 3:
  454. tags.add(part.lower())
  455. # 5. Merge and return
  456. result = ','.join(sorted(tags)[:80]) # cap at 80 tags
  457. return result
  458. def index_skills():
  459. skills = []
  460. for skills_dir in SKILL_DIRS:
  461. if not os.path.exists(skills_dir):
  462. continue
  463. for root, dirs, files in os.walk(skills_dir):
  464. depth = root.count(os.sep) - skills_dir.count(os.sep)
  465. if depth > 7:
  466. continue
  467. if 'SKILL.md' not in files:
  468. continue
  469. try:
  470. with open(os.path.join(root, 'SKILL.md'), encoding='utf-8') as f:
  471. content = f.read()
  472. fm_match = re.match(r'^---\n(.*?)\n---', content, re.DOTALL)
  473. if fm_match:
  474. meta = {}
  475. for line in fm_match.group(1).split('\n'):
  476. m = re.match(r'(\w+):\s*(.+)', line)
  477. if m:
  478. meta[m.group(1)] = m.group(2).strip()
  479. if 'name' in meta:
  480. desc = meta.get('description', '')
  481. triggers = re.search(r'TRIGGERS?:' + '\s*(.+)', desc, re.IGNORECASE)
  482. trigger_str = triggers.group(1) if triggers else ''
  483. db.execute('INSERT OR REPLACE INTO skill_index (name, description, triggers, file_path) VALUES (?,?,?,?)', (meta['name'], desc, trigger_str, os.path.join(root, 'SKILL.md')))
  484. skills.append({'name': meta['name'], 'description': desc, 'triggers': trigger_str})
  485. except Exception:
  486. pass
  487. # Also index flat .md files from skills/all/ (auto-created by Worker)
  488. all_dir = os.path.join(ROOT, 'skills', 'all')
  489. if os.path.exists(all_dir):
  490. for fname in os.listdir(all_dir):
  491. if not fname.endswith('.md'):
  492. continue
  493. fpath = os.path.join(all_dir, fname)
  494. try:
  495. with open(fpath, encoding='utf-8') as f:
  496. content = f.read()
  497. fm_match = re.match(r'^---\n(.*?)\n---', content, re.DOTALL)
  498. if fm_match:
  499. meta = {}
  500. for line in fm_match.group(1).split('\n'):
  501. m = re.match(r'(\w+):\s*(.+)', line)
  502. if m:
  503. meta[m.group(1)] = m.group(2).strip()
  504. if 'name' in meta:
  505. desc = meta.get('description', '')
  506. triggers = re.search(r'TRIGGERS?:' + '\s*(.+)', desc, re.IGNORECASE)
  507. trigger_str = triggers.group(1) if triggers else ''
  508. name = meta['name']
  509. db.execute('INSERT OR REPLACE INTO skill_index (name, description, triggers, file_path) VALUES (?,?,?,?)', (name, desc, trigger_str, fpath))
  510. # Auto-tag: generate matching_tags from name+desc+content if empty
  511. existing_tags = db.execute('SELECT matching_tags FROM skill_index WHERE name=?', (name,)).fetchone()
  512. if not existing_tags or not (existing_tags[0] or '').strip():
  513. # Use full content as fallback if desc is empty (YAML block scalars)
  514. tag_text = desc if desc and len(desc) > 10 else content
  515. tags = auto_tag_skill(name, tag_text, content)
  516. if tags:
  517. db.execute('UPDATE skill_index SET matching_tags=? WHERE name=?', (tags, name))
  518. skills.append({'name': name, 'description': desc, 'triggers': trigger_str})
  519. except Exception:
  520. pass
  521. db.commit()
  522. return skills
  523. def search_procedural(query, limit=5):
  524. q = re.sub(r'[^\w\s]', ' ', query).split()
  525. q = ' OR '.join(w for w in q if w)
  526. if not q:
  527. rows = db.execute("SELECT id, name, description, steps, trigger_patterns FROM procedural WHERE status='active' ORDER BY use_count DESC LIMIT ?", (limit,)).fetchall()
  528. else:
  529. try:
  530. rows = db.execute("SELECT rowid, name, description, steps, trigger_patterns, rank FROM procedural_fts WHERE procedural_fts MATCH ? ORDER BY rank LIMIT ?", (q, limit)).fetchall()
  531. except Exception:
  532. rows = db.execute("SELECT id, name, description, steps, trigger_patterns FROM procedural WHERE (description LIKE ? OR name LIKE ?) AND status='active' LIMIT ?", (f'%{query.split()[0]}%', f'%{query.split()[0]}%', limit)).fetchall()
  533. return [{'name': r[1], 'description': r[2], 'steps': r[3], 'triggers': r[4]} for r in rows]
  534. def save_procedural(name, description, steps, triggers=''):
  535. existing = db.execute('SELECT id FROM procedural WHERE name = ?', (name,)).fetchone()
  536. if existing:
  537. db.execute('UPDATE procedural SET description=?, steps=?, trigger_patterns=? WHERE name=?', (description, steps, triggers, name))
  538. else:
  539. db.execute('INSERT INTO procedural (name, description, steps, trigger_patterns) VALUES (?,?,?,?)', (name, description, steps, triggers))
  540. try:
  541. db.execute('DELETE FROM procedural_fts WHERE name = ?', (name,))
  542. db.execute('INSERT INTO procedural_fts(name, description, steps, trigger_patterns) VALUES (?,?,?,?)', (name, description, steps, triggers))
  543. except Exception: pass
  544. db.commit()
  545. def score_skills():
  546. """Calculate quality score for each skill based on invoke_count and recency."""
  547. db.execute("""
  548. UPDATE skill_index SET quality_score =
  549. CASE
  550. WHEN invoke_count >= 5 THEN 1.0
  551. WHEN invoke_count >= 2 THEN 0.8
  552. WHEN invoke_count = 1 THEN 0.5
  553. WHEN last_invoked IS NULL AND julianday('now') - julianday(installed_at) > 7 THEN 0.1
  554. ELSE 0.3
  555. END
  556. WHERE name IS NOT NULL
  557. """)
  558. db.commit()
  559. def prune_low_quality_skills():
  560. """Archive skills with quality_score < 0.2 and >7 days without invocation."""
  561. rows = db.execute("SELECT name, description, file_path, COALESCE(quality_score, 0) as qs FROM skill_index WHERE COALESCE(quality_score, 0) < 0.2 AND julianday('now') - julianday(installed_at) > 7").fetchall()
  562. if rows:
  563. archive_dir = os.path.join(ROOT, 'memory', 'archive')
  564. os.makedirs(archive_dir, exist_ok=True)
  565. archive_file = os.path.join(archive_dir, f'skills_pruned_{time.strftime("%Y%m%d")}.json')
  566. pruned = [{'name': r[0], 'description': r[1], 'file_path': r[2], 'quality_score': r[3]} for r in rows]
  567. with open(archive_file, 'w', encoding='utf-8') as f:
  568. json.dump(pruned, f, ensure_ascii=False, indent=2)
  569. for r in rows:
  570. db.execute('DELETE FROM skill_index WHERE name = ?', (r[0],))
  571. try: os.remove(r[2]) if r[2] and os.path.exists(r[2]) else None
  572. except Exception: pass
  573. db.execute("INSERT INTO evolution_log (session_id, action, detail) VALUES ('hermes', 'prune_skills', ?)", (json.dumps({'count': len(rows)}),))
  574. db.commit()
  575. return len(rows)
  576. return 0
  577. def auto_compact():
  578. """Merge similar memories by key prefix AND content similarity"""
  579. merged = 0
  580. # 1. Key-prefix merge
  581. rows = db.execute("SELECT key, COUNT(*) as cnt, GROUP_CONCAT(id) as ids FROM semantic WHERE key != '_schema_version' GROUP BY substr(key, 1, instr(key||'_','_')-1) HAVING cnt > 1").fetchall()
  582. for r in rows:
  583. ids = [int(x) for x in r[2].split(',')]
  584. if len(ids) > 1:
  585. for rid in ids[1:]:
  586. db.execute('DELETE FROM semantic WHERE id = ?', (rid,))
  587. merged += len(ids) - 1
  588. # 2. Content similarity: hash-bucket dedup (first 50 chars)
  589. buckets = {}
  590. all_mems = db.execute("SELECT id, key, content FROM semantic WHERE key != '_schema_version' AND LENGTH(content) > 30 ORDER BY id").fetchall()
  591. for mem in all_mems:
  592. h = hashlib.md5((mem[2] or '')[:50].encode()).hexdigest()[:8]
  593. if h not in buckets: buckets[h] = []
  594. buckets[h].append(mem)
  595. to_delete = set()
  596. for h, mems in buckets.items():
  597. if len(mems) < 2: continue
  598. for i in range(1, len(mems)):
  599. to_delete.add(mems[i][0])
  600. merged += 1
  601. for rid in to_delete:
  602. db.execute('DELETE FROM semantic WHERE id = ?', (rid,))
  603. if merged:
  604. db.commit()
  605. return merged
  606. def get_stats():
  607. sem = db.execute('SELECT COUNT(*) FROM semantic').fetchone()[0]
  608. proc = db.execute('SELECT COUNT(*) FROM procedural').fetchone()[0]
  609. skill = db.execute('SELECT COUNT(*) FROM skill_index').fetchone()[0]
  610. evo = db.execute('SELECT COUNT(*) FROM evolution_log').fetchone()[0]
  611. epi = get_episodic_count()
  612. return {'semanticCount': sem, 'proceduralCount': proc, 'skillCount': skill, 'evoCount': evo, 'episodeCount': epi}
  613. def save_semantic(key, content, tags='', source_session=None):
  614. existing = db.execute('SELECT id FROM semantic WHERE key = ?', (key,)).fetchone()
  615. seg = jieba_segment(content)
  616. if existing:
  617. db.execute('UPDATE semantic SET content=?, tags=?, updated_at=datetime("now") WHERE key=?', (content, tags, key))
  618. try: db.execute('DELETE FROM semantic_fts WHERE key = ?', (key,))
  619. except Exception: pass
  620. try: db.execute('INSERT INTO semantic_fts(key, content, tags) VALUES (?,?,?)', (key, seg, tags))
  621. except Exception: pass
  622. else:
  623. db.execute('INSERT INTO semantic (key, content, tags, source_session) VALUES (?,?,?,?)', (key, content, tags, source_session))
  624. try: db.execute('INSERT INTO semantic_fts(key, content, tags) VALUES (?,?,?)', (key, seg, tags))
  625. except Exception: pass
  626. db.commit()
  627. def save_episodic(session_id, summary='', task='', message_count=0, transcript_path='', project_name=''):
  628. db.execute("INSERT INTO episodic (session_id, summary, task, message_count, transcript_path, project_name, created_at) VALUES (?,?,?,?,?,?,datetime('now'))",
  629. (session_id, summary, task, message_count, transcript_path, project_name))
  630. db.commit()
  631. def search_episodes(query, limit=10):
  632. q = f'%{query}%'
  633. return [{'session_id': r[1], 'summary': r[2], 'task': r[3], 'message_count': r[4], 'project_name': r[6], 'created_at': r[7]}
  634. for r in db.execute('SELECT * FROM episodic WHERE summary LIKE ? OR task LIKE ? OR project_name LIKE ? ORDER BY created_at DESC LIMIT ?',
  635. (q, q, q, limit)).fetchall()]
  636. def get_episodic_count():
  637. return db.execute('SELECT COUNT(*) FROM episodic').fetchone()[0]
  638. # ─── Hermes Memory Engine ───
  639. def update_confidence(key, delta=0.05):
  640. """Boost confidence on access, cap at 1.0"""
  641. db.execute("UPDATE semantic SET confidence = MIN(1.0, COALESCE(confidence, 0.5) + ?), last_accessed = datetime('now'), access_count = access_count + 1 WHERE key = ?", (delta, key))
  642. db.commit()
  643. def decay_stale_memories():
  644. """Confidence decay: 30+ days no access → confidence *= 0.9. Archive if confidence < 0.2."""
  645. db.execute("UPDATE semantic SET confidence = MAX(0.1, COALESCE(confidence, 0.5) * 0.9) WHERE key != '_schema_version' AND (last_accessed < datetime('now', '-30 days') OR (last_accessed IS NULL AND updated_at < datetime('now', '-30 days')))")
  646. # Archive memories that decayed below 0.2
  647. stale = db.execute("SELECT key, content, tags, COALESCE(confidence,0.5) as conf, updated_at FROM semantic WHERE key != '_schema_version' AND COALESCE(confidence, 0.5) < 0.2").fetchall()
  648. if stale:
  649. archive_dir = os.path.join(ROOT, 'memory', 'archive')
  650. os.makedirs(archive_dir, exist_ok=True)
  651. archive_file = os.path.join(archive_dir, f'decay_{time.strftime("%Y%m%d")}.json')
  652. archived = []
  653. for r in stale:
  654. archived.append({'key': r[0], 'content': r[1], 'tags': r[2], 'confidence': r[3], 'updated_at': r[4]})
  655. db.execute('DELETE FROM semantic WHERE key = ?', (r[0],))
  656. db.execute("DELETE FROM semantic_fts WHERE rowid = (SELECT rowid FROM semantic_fts WHERE key = ?)", (r[0],))
  657. with open(archive_file, 'w', encoding='utf-8') as f:
  658. json.dump(archived, f, ensure_ascii=False, indent=2)
  659. db.execute("INSERT INTO evolution_log (session_id, action, detail) VALUES ('hermes', 'archive_decay', ?)", (json.dumps({'count': len(stale)}),))
  660. db.commit()
  661. def auto_promote():
  662. """Episodic→Semantic promotion check, Semantic→Procedural pattern creation"""
  663. events = []
  664. # Lower threshold: access_count >= 1, confidence >= 0.3
  665. rows = db.execute("""SELECT key, content, access_count, COALESCE(confidence, 0.3) as conf, tags
  666. FROM semantic WHERE key != '_schema_version' AND access_count >= 1 AND COALESCE(confidence, 0.3) >= 0.3
  667. AND COALESCE(promotion_count, 0) = 0
  668. ORDER BY access_count DESC, conf DESC LIMIT 5""").fetchall()
  669. # Meta/noise patterns — skip these for procedural promotion
  670. META_PATTERNS = [
  671. 'test_question', 'memory.*contain', 'knowledge.*in.*memory',
  672. 'losing.*cc.*history', 'cc.*history.*loses',
  673. 'inject_log', 'worker_log', 'mem_size', 'catalog.*chars',
  674. 'selectSkillsAI', 'selectMemoriesAI', 'getAllMemory',
  675. 'restarted.*session', 'user_says_restart', 'user_agrees',
  676. 'semanticCount', 'skillCount', 'promotion_count'
  677. ]
  678. for r in rows:
  679. key, content, ac, conf, tags = r
  680. # Skip meta/noise — self-referential or plugin-internal trivia
  681. txt = (key + ' ' + (content or '')).lower()
  682. if any(re.search(p, txt) for p in META_PATTERNS):
  683. db.execute("UPDATE semantic SET promotion_count = -1 WHERE key = ?", (key,))
  684. continue
  685. try:
  686. db.execute("""INSERT OR IGNORE INTO procedural (name, description, steps, trigger_patterns)
  687. VALUES (?, ?, ?, ?)""",
  688. (key, content, content, tags or ''))
  689. try:
  690. seg = ' '.join(jieba.cut(content))
  691. db.execute("INSERT OR IGNORE INTO procedural_fts(name, description, trigger_patterns) VALUES (?,?,?)", (key, seg, tags or ''))
  692. except Exception: pass
  693. except Exception: pass
  694. db.execute("UPDATE semantic SET promotion_count = 1, confidence = MIN(1.0, ?), procedural_source = ? WHERE key = ?", (ac * 0.1 + conf, key, key))
  695. db.execute("INSERT INTO evolution_log (session_id, action, detail) VALUES ('hermes', 'auto_promote', ?)", (json.dumps({'key': key, 'reason': 'procedural_pattern'}),))
  696. events.append({'key': key, 'action': 'promoted_to_procedural'})
  697. db.commit()
  698. return events
  699. def analyze_skills_with_pro():
  700. """Offline: pro reads all SKILL.md files, generates matching_tags for each"""
  701. import urllib.request
  702. API_KEY = os.environ.get('DEEPSEEK_API_KEY') or os.environ.get('ANTHROPIC_AUTH_TOKEN') or 'YOUR_LLM_API_KEY'
  703. # Add matching_tags column if needed
  704. try: db.execute('ALTER TABLE skill_index ADD COLUMN matching_tags TEXT DEFAULT \"\"')
  705. except Exception: pass
  706. rows = db.execute('SELECT name, file_path, description FROM skill_index WHERE 1=1').fetchall()
  707. if not rows: return {'analyzed': 0}
  708. # Build compact catalog
  709. catalog = ''
  710. for r in rows:
  711. name, fp, desc = r
  712. try:
  713. with open(fp, encoding='utf-8') as f:
  714. body = f.read()[:600]
  715. except Exception:
  716. body = desc or ''
  717. catalog += f'### {name}\n{body}\n\n'
  718. prompt = f"""Analyze each skill below. For each, output one line: SKILL_NAME: comma-separated Chinese+English trigger keywords that describe WHEN to use this skill.
  719. Keywords should cover: task types, domains, technologies, verbs. Be GENEROUS — include broad and specific matches.
  720. Example format:
  721. binary-analysis: 逆向,二进制,反汇编,加密算法,AES,SM4,IDA Pro,Ghidra,x64dbg,reverse engineering,binary RE
  722. my-review: 审查,review,代码质量,安全审计,code quality,bug finding
  723. Skills:
  724. {catalog}"""
  725. try:
  726. body = json.dumps({
  727. 'model': 'deepseek-v4-pro[1m]',
  728. 'max_tokens': 16384,
  729. 'messages': [{'role': 'user', 'content': prompt}]
  730. }).encode('utf-8')
  731. req = urllib.request.Request('https://api.deepseek.com/anthropic/v1/messages',
    • Warning: Uploads the first 600 characters of every indexed skill file to api.deepseek.com, authenticated with a key taken from the environment. This sends local skill content to a third-party service.
    • Line 806 on GitHub
  732. data=body,
  733. headers={'Content-Type': 'application/json', 'x-api-key': API_KEY, 'anthropic-version': '2023-06-01'})
  734. resp = urllib.request.urlopen(req, timeout=300)
  735. data = json.loads(resp.read())
  736. text_block = next((c for c in data.get('content', []) if c.get('type') == 'text'), None)
  737. result = text_block.get('text', '') if text_block else ''
  738. except Exception as e:
  739. db.execute("INSERT INTO evolution_log (session_id, action, detail) VALUES ('hermes', 'analyze_skills_error', ?)", (str(e)[:500],))
  740. db.commit()
  741. return {'analyzed': 0, 'error': str(e)[:100]}
  742. # Parse and apply
  743. analyzed = 0
  744. for line in result.split('\n'):
  745. line = line.strip()
  746. if ':' not in line: continue
  747. m = re.match(r'^([\w_-]+):\s*(.+)', line)
  748. if not m: continue
  749. name, tags = m.group(1), m.group(2)
  750. try:
  751. db.execute('UPDATE skill_index SET matching_tags = ? WHERE name = ?', (tags, name))
  752. analyzed += 1
  753. except Exception: pass
  754. db.commit()
  755. db.execute("INSERT INTO evolution_log (session_id, action, detail) VALUES ('hermes', 'analyze_skills', ?)", (json.dumps({'count': analyzed}),))
  756. db.commit()
  757. return {'analyzed': analyzed}
  758. def evolve_with_ai():
  759. """AI-driven evolution: merge similar, detect contradictions, create structured workflows"""
  760. import urllib.request
  761. API_KEY = os.environ.get('DEEPSEEK_API_KEY') or os.environ.get('ANTHROPIC_AUTH_TOKEN') or 'YOUR_LLM_API_KEY'
    • Warning: Sends stored memory contents to api.deepseek.com, using ANTHROPIC_AUTH_TOKEN as the API key if no DeepSeek key is set. The stored memories come from the user's coding sessions.
    • Line 838 on GitHub
  762. # Get top 30 high-value memories for AI analysis
  763. rows = db.execute("""SELECT key, content, tags, COALESCE(confidence,0.5) as conf, access_count
  764. FROM semantic WHERE key != '_schema_version' AND promotion_count >= 0
  765. AND COALESCE(confidence, 0.3) >= 0.3
  766. ORDER BY access_count DESC, conf DESC LIMIT 30""").fetchall()
  767. if len(rows) < 5: return {'merged': 0, 'deprecated': 0, 'workflows': 0}
  768. catalog = '\n'.join([f"- {r[0]}: {r[1][:120]}" for r in rows])
  769. prompt = f"""Analyze these memories from an AI coding assistant's long-term memory. Your job is to find improvements.
  770. ## TASKS
  771. 1. **Merge**: Find 2-3 groups of memories that say the same thing in different ways. Output merged version.
  772. 2. **Deprecate**: Find any memories that are clearly outdated/wrong based on newer ones. Mark them.
  773. 3. **Workflow**: If 3+ memories describe steps of a process, create a structured procedural workflow.
  774. ## OUTPUT JSON
  775. {{"merges": [{{"keys":["old1","old2"], "new_key":"merged_name", "new_content":"combined concise fact"}}], "deprecate": ["outdated_key"], "workflows": [{{"name":"workflow-name", "description":"what it does", "steps":["step1","step2","step3"], "trigger":"when to use"}}]}}
  776. ## MEMORIES
  777. {catalog}"""
  778. try:
  779. body = json.dumps({
  780. 'model': 'deepseek-v4-pro[1m]',
  781. 'max_tokens': 16384,
  782. 'messages': [{'role': 'user', 'content': prompt}]
  783. }).encode('utf-8')
  784. req = urllib.request.Request('https://api.deepseek.com/anthropic/v1/messages',
  785. data=body,
  786. headers={'Content-Type': 'application/json', 'x-api-key': API_KEY, 'anthropic-version': '2023-06-01'})
  787. resp = urllib.request.urlopen(req, timeout=300)
  788. data = json.loads(resp.read())
  789. text_block = next((c for c in data.get('content', []) if c.get('type') == 'text'), None)
  790. result = text_block.get('text', '') if text_block else ''
  791. except Exception as e:
  792. db.execute("INSERT INTO evolution_log (session_id, action, detail) VALUES ('hermes', 'ai_evolve_error', ?)", (str(e)[:500],))
  793. db.commit()
  794. return {'merged': 0, 'deprecated': 0, 'workflows': 0, 'error': str(e)[:100]}
  795. # Parse result
  796. try:
  797. m = re.search(r'\{.*\}', result, re.DOTALL)
  798. plan = json.loads(m.group(0)) if m else {}
  799. except Exception:
  800. plan = {}
  801. merged = 0; deprecated = 0; workflows = 0
  802. # Apply merges
  803. for mg in plan.get('merges', []):
  804. keys = mg.get('keys', [])
  805. nk, nc = mg.get('new_key'), mg.get('new_content')
  806. if not keys or not nk or not nc: continue
  807. try:
  808. # Save merged version
  809. db.execute("INSERT OR REPLACE INTO semantic (key, content, tags, confidence) VALUES (?,?,?,'merged',0.8)", (nk, nc))
  810. try: db.execute("INSERT OR REPLACE INTO semantic_fts(key, content, tags) VALUES (?,?,?)", (nk, ' '.join(jieba.cut(nc)), 'merged'))
  811. except Exception: pass
  812. # Mark old as deprecated
  813. for k in keys:
  814. db.execute("UPDATE semantic SET tags = 'deprecated', confidence = 0.1 WHERE key = ?", (k,))
  815. db.execute("INSERT INTO evolution_log (session_id, action, detail) VALUES ('hermes', 'ai_merge', ?)", (json.dumps({'keys': keys, 'new': nk}),))
  816. merged += 1
  817. except Exception: pass
  818. # Apply deprecations
  819. for dk in plan.get('deprecate', []):
  820. try:
  821. db.execute("UPDATE semantic SET tags = 'deprecated', confidence = 0.1 WHERE key = ?", (dk,))
  822. db.execute("INSERT INTO evolution_log (session_id, action, detail) VALUES ('hermes', 'ai_deprecate', ?)", (json.dumps({'key': dk}),))
  823. deprecated += 1
  824. except Exception: pass
  825. # Apply workflows
  826. for wf in plan.get('workflows', []):
  827. n, d, ss, t = wf.get('name'), wf.get('description'), wf.get('steps', []), wf.get('trigger', '')
  828. if not n or not ss: continue
  829. try:
  830. steps_text = '\n'.join(ss)
  831. db.execute("INSERT OR REPLACE INTO procedural (name, description, steps, trigger_patterns) VALUES (?,?,?,?)", (n, d or '', steps_text, t))
  832. try: db.execute("INSERT OR REPLACE INTO procedural_fts(name, description, trigger_patterns) VALUES (?,?,?)", (n, ' '.join(jieba.cut(d or '')), t))
  833. except Exception: pass
  834. db.execute("INSERT INTO evolution_log (session_id, action, detail) VALUES ('hermes', 'ai_workflow', ?)", (json.dumps({'name': n, 'steps': len(ss)}),))
  835. workflows += 1
  836. except Exception: pass
  837. db.commit()
  838. return {'merged': merged, 'deprecated': deprecated, 'workflows': workflows}
  839. def hermes_fusion():
  840. """Weekly fusion: decay stale, promote patterns, merge similars, prune skills + graph"""
  841. decay_stale_memories()
  842. promos = auto_promote()
  843. compacted = auto_compact()
  844. score_skills()
  845. pruned = prune_low_quality_skills()
  846. ai = evolve_with_ai()
  847. graph_pruned = prune_graph()
  848. ineffective_pruned = prune_ineffective_mems()
  849. db.execute("INSERT INTO evolution_log (session_id, action, detail) VALUES ('hermes', 'fusion', ?)", (json.dumps({'promotions': len(promos), 'compacted': compacted, 'skills_pruned': pruned, 'ai': ai, 'graph_pruned': graph_pruned, 'ineffective_pruned': ineffective_pruned}),))
  850. db.commit()
  851. return {'promotions': len(promos), 'compacted': compacted, 'skills_pruned': pruned, 'ai_evolution': ai, 'graph_pruned': graph_pruned, 'ineffective_pruned': ineffective_pruned}
  852. def search_warnings(query):
  853. """Detect dangerous patterns related to query from graph + feedback"""
  854. # Search graph for dangerous edges matching query
  855. try:
  856. rows = gdb.execute("SELECT e.source, e.target, e.relation_type, e.confidence, e.evidence FROM edges_fts f JOIN edges e ON f.rowid = e.id WHERE edges_fts MATCH ? AND e.relation_type IN ('blocked_by','conflicts_with','causes') AND e.confidence >= 0.4 ORDER BY rank LIMIT 10", (query,)).fetchall()
  857. except Exception:
  858. like = f'%{query}%'
  859. rows = gdb.execute("SELECT source, target, relation_type, confidence, evidence FROM edges WHERE (source LIKE ? OR target LIKE ? OR evidence LIKE ?) AND relation_type IN ('blocked_by','conflicts_with','causes') AND confidence >= 0.4 ORDER BY confidence DESC LIMIT 10", (like, like, like)).fetchall()
  860. warnings = []
  861. for src, tgt, rtype, conf, ev in rows:
  862. label = {'blocked_by': '⚠️ 阻塞风险', 'conflicts_with': '⚡ 潜在冲突', 'causes': '🔴 失败模式'}.get(rtype, '⚠️ 风险')
  863. fb = db.execute('SELECT COALESCE(effectiveness_score,0.5), COALESCE(ineffective_count,0) FROM semantic WHERE key=?', (tgt,)).fetchone()
  864. eff = fb[0] if fb else 0.5
  865. nc = fb[1] if fb else 0
  866. sev = 'high' if (conf >= 0.7 or (rtype == 'causes' and nc >= 2 and eff < 0.4)) else 'medium'
  867. warnings.append({
  868. 'source': src, 'target': tgt, 'relation_type': rtype,
  869. 'danger_type': rtype, 'severity': sev,
  870. 'reason': (ev or f'{src} {rtype} {tgt}')[:300],
  871. 'confidence': conf, 'evidence': ev or '',
  872. 'failure_count': nc, 'effectiveness': round(eff, 2),
  873. 'label': label
  874. })
  875. # Also check low-effectiveness memories matching query
  876. bad_mems = db.execute("SELECT key, content, COALESCE(effectiveness_score,0.5), COALESCE(ineffective_count,0), COALESCE(injected_count,0) FROM semantic WHERE key != '_schema_version' AND ineffective_count >= 2 AND COALESCE(effectiveness_score,0.5) < 0.3 AND key LIKE ? LIMIT 5", (f'%{query}%',)).fetchall()
  877. for k, c, eff, nc, inj in bad_mems:
  878. warnings.append({
  879. 'source': k, 'target': k, 'relation_type': 'self',
  880. 'danger_type': 'persistent_failure', 'severity': 'medium',
  881. 'reason': f'低效记忆: {c[:100]}(失败 {nc} 次,有效率 {eff:.1%})',
  882. 'confidence': 0.8, 'evidence': '',
  883. 'failure_count': nc, 'effectiveness': round(eff, 2),
  884. 'injected_count': inj, 'label': '🟡 低效记忆'
  885. })
  886. sev_ord = {'high': 0, 'medium': 1}
  887. warnings.sort(key=lambda w: (sev_ord.get(w['severity'], 2), -w['confidence']))
  888. return warnings[:8]
  889. def get_skill_rankings(limit=20):
  890. rows = db.execute('''SELECT skill_name,
  891. SUM(CASE WHEN event_type="injected" THEN 1 ELSE 0 END) as injected,
  892. SUM(CASE WHEN event_type="invoked" THEN 1 ELSE 0 END) as invoked,
  893. SUM(CASE WHEN event_type="completed" THEN 1 ELSE 0 END) as completed,
  894. SUM(CASE WHEN event_type="failed" THEN 1 ELSE 0 END) as failed,
  895. COALESCE(AVG(effectiveness), 0.5) as avg_eff,
  896. COUNT(*) as total
  897. FROM skill_feedback GROUP BY skill_name
  898. ORDER BY completed DESC, invoked DESC LIMIT ?''', (limit,)).fetchall()
  899. return [{'skill': r[0], 'injected': r[1], 'invoked': r[2], 'completed': r[3],
  900. 'failed': r[4], 'avg_effectiveness': round(r[5], 2), 'total': r[6],
  901. 'hit_rate': f'{int(r[2]/r[1]*100)}%' if r[1] > 0 else 'N/A'} for r in rows]
  902. def get_skill_prefs(skill_name=None):
  903. if skill_name:
  904. rows = db.execute('SELECT * FROM skill_prefs WHERE skill_name=? ORDER BY use_count DESC, effectiveness DESC', (skill_name,)).fetchall()
  905. else:
  906. rows = db.execute('SELECT * FROM skill_prefs ORDER BY use_count DESC, effectiveness DESC').fetchall()
  907. return [{'skill': r[0], 'task_pattern': r[1], 'use_count': r[2],
  908. 'effectiveness': round(r[3], 2), 'last_used': r[4]} for r in rows]
  909. # Index existing skills on startup
  910. try: index_skills()
  911. except Exception: pass
  912. # ═══════════════════════════════════════════════════════════
  913. # z2 HUB: FleetWatcher — scans all CC sessions, broadcasts
  914. # ═══════════════════════════════════════════════════════════
  915. class FleetWatcher:
  916. def __init__(self):
  917. self.running = True
  918. self.last_broadcast = ''
  919. self.lockfile = os.path.join(ROOT, '.fleetwatcher.lock')
  920. def start(self):
  921. t = threading.Thread(target=self._loop, daemon=True)
  922. t.start()
  923. def _loop(self):
  924. while self.running:
  925. try:
  926. self._scan_and_broadcast()
  927. self._check_triggers()
  928. except Exception:
  929. pass
  930. time.sleep(SCAN_INTERVAL)
  931. def _check_triggers(self):
  932. trigger_inject = os.path.join(ROOT, '.trigger_inject')
  933. trigger_inject_tmp = trigger_inject + '.tmp'
  934. trigger_consolidate = os.path.join(ROOT, '.trigger_consolidate')
  935. trigger_consolidate_tmp = trigger_consolidate + '.tmp'
  936. try:
  937. # Atomic consume: rename(tmp→real) before processing
  938. if os.path.exists(trigger_inject_tmp):
  939. try: os.rename(trigger_inject_tmp, trigger_inject)
  940. except OSError: pass
  941. if os.path.exists(trigger_consolidate_tmp):
  942. try: os.rename(trigger_consolidate_tmp, trigger_consolidate)
  943. except OSError: pass
  944. if os.path.exists(trigger_inject):
  945. os.remove(trigger_inject)
  946. subprocess.Popen(
  947. ['node', os.path.join(ROOT, 'inject.js')],
  948. creationflags=0x08000000 | 0x00000008,
  949. stdin=subprocess.DEVNULL, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL,
  950. close_fds=True)
  951. if os.path.exists(trigger_consolidate):
  952. os.remove(trigger_consolidate)
  953. subprocess.Popen(
  954. ['node', os.path.join(ROOT, 'consolidate.js')],
  955. creationflags=0x08000000 | 0x00000008,
  956. stdin=subprocess.DEVNULL, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL,
  957. close_fds=True)
  958. except Exception:
  959. pass
  960. def _scan_and_broadcast(self):
  961. sessions = self._discover_sessions()
  962. if not sessions:
  963. return
  964. for s in sessions:
  965. self._extract_qa(s)
  966. md = self._build_broadcast(sessions)
  967. if md == self.last_broadcast:
  968. return
  969. self.last_broadcast = md
  970. self._write_broadcast(md)
  971. self._push_event_queue(sessions)
  972. def _discover_sessions(self):
  973. sessions = []
  974. if not os.path.exists(CC_SESSIONS_DIR):
  975. return sessions
  976. for fname in os.listdir(CC_SESSIONS_DIR):
  977. if not fname.endswith('.json'):
  978. continue
  979. try:
  980. with open(os.path.join(CC_SESSIONS_DIR, fname), 'r', encoding='utf-8') as f:
  981. data = json.load(f)
  982. sessions.append({
  983. 'id': data.get('sessionId', fname.replace('.json', '')),
  984. 'pid': data.get('pid', 0),
  985. 'cwd': data.get('cwd', ''),
  986. 'status': 'active' if data.get('status') == 'busy' else 'idle',
  987. 'started_at': data.get('startedAt'),
  988. 'updated_at': data.get('updatedAt'),
  989. 'version': data.get('version', ''),
  990. })
  991. except Exception:
  992. pass
  993. return sessions
  994. def _extract_qa(self, s):
  995. qa_pairs = []
  996. try:
  997. # Dynamic: find the project dir containing this session's transcript
  998. proj_dir = None
  999. try:
  1000. for dname in os.listdir(CC_PROJECTS_DIR):
  1001. dpath = os.path.join(CC_PROJECTS_DIR, dname)
  1002. if not os.path.isdir(dpath):
  1003. continue
  1004. candidate = os.path.join(dpath, s['id'] + '.jsonl')
  1005. if os.path.exists(candidate):
  1006. proj_dir = dpath
  1007. break
  1008. except Exception:
  1009. pass
  1010. if proj_dir is None:
  1011. return
  1012. jsonl = os.path.join(proj_dir, f"{s['id']}.jsonl")
  1013. if not os.path.exists(jsonl):
  1014. return
  1015. s['transcript_mtime'] = os.path.getmtime(jsonl)
  1016. s['recently_active'] = (time.time() - s['transcript_mtime']) < 300
  1017. with open(jsonl, 'r', encoding='utf-8', errors='ignore') as f:
  1018. lines = f.readlines()
  1019. pairs = []
  1020. current_q = None
  1021. current_a = None
  1022. for line in reversed(lines[-500:]):
  1023. try:
  1024. entry = json.loads(line.strip())
  1025. except Exception:
  1026. continue
  1027. # Extract user text (skip tool results)
  1028. if entry.get('type') == 'user' and not current_q:
  1029. msg = entry.get('message', {})
  1030. if isinstance(msg, dict) and msg.get('role') == 'user':
  1031. content = msg.get('content', '')
  1032. if isinstance(content, str) and content.strip():
  1033. current_q = content.strip()
  1034. elif isinstance(content, list):
  1035. for part in content:
  1036. if isinstance(part, dict) and part.get('text'):
  1037. txt = part['text'].strip()
  1038. if txt and len(txt) > 10:
  1039. current_q = txt
  1040. break
  1041. continue
  1042. # Extract assistant text
  1043. if entry.get('type') == 'assistant' and current_q and not current_a:
  1044. msg = entry.get('message', {})
  1045. if isinstance(msg, dict) and msg.get('role') == 'assistant':
  1046. content = msg.get('content', '')
  1047. if isinstance(content, str) and content.strip():
  1048. current_a = content.strip()
  1049. elif isinstance(content, list):
  1050. texts = []
  1051. for part in content:
  1052. if isinstance(part, dict) and part.get('text'):
  1053. texts.append(part['text'].strip())
  1054. if texts:
  1055. current_a = '\n'.join(texts)
  1056. if current_q and current_a:
  1057. pairs.append({'q': current_q[:200], 'a': current_a[:300]})
  1058. current_q = None
  1059. current_a = None
  1060. if len(pairs) >= 2:
  1061. break
  1062. s['qa_pairs'] = pairs
  1063. # Detect topic from ai-title
  1064. for line in reversed(lines[-100:]):
  1065. try:
  1066. entry = json.loads(line.strip())
  1067. if entry.get('aiTitle'):
  1068. s['topic'] = entry['aiTitle']
  1069. break
  1070. except Exception:
  1071. pass
  1072. if 'topic' not in s:
  1073. s['topic'] = pairs[0]['q'][:80] if pairs else '(idle)'
  1074. except Exception:
  1075. s['qa_pairs'] = []
  1076. s['topic'] = '(idle)'
  1077. s['recently_active'] = False
  1078. def _build_broadcast(self, sessions):
  1079. # Merge with fleet_state (Overmind registrations)
  1080. overmind = {}
  1081. try:
  1082. if os.path.exists(FLEET_STATE_FILE):
  1083. with open(FLEET_STATE_FILE, 'r', encoding='utf-8') as f:
  1084. fs = json.load(f)
  1085. for inst in fs.get('instances', []):
  1086. overmind[inst['id']] = inst
  1087. except Exception:
  1088. pass
  1089. lines = ['📡 舰队广播 z2']
  1090. lines.append(f'> {len(sessions)} 个实例 · {time.strftime("%H:%M:%S")} 刷新\n')
  1091. our_session = None
  1092. try:
  1093. with open(os.path.join(CC_SESSIONS_DIR, [f for f in os.listdir(CC_SESSIONS_DIR) if f.endswith('.json')][0]), 'r') as f:
  1094. pass
  1095. except Exception:
  1096. pass
  1097. for s in sessions:
  1098. sid = s['id'][:8]
  1099. om = overmind.get(s['id'])
  1100. has_om = bool(om)
  1101. status_icon = '🟢' if s.get('recently_active') else '⚪'
  1102. om_icon = '🧠' if has_om else '👁️'
  1103. topic = (s.get('topic') or '(idle)')[:80]
  1104. lines.append(f'### {sid} [{om_icon}] {status_icon} {topic}')
  1105. qa = s.get('qa_pairs', [])
  1106. if qa:
  1107. for pair in qa:
  1108. q_text = pair['q'][:120].replace('\n', ' ')
  1109. a_text = pair['a'][:200].replace('\n', ' ')
  1110. lines.append(f'> ❓ {q_text}')
  1111. lines.append(f'> 💬 {a_text}')
  1112. lines.append('')
  1113. # Conflict check
  1114. active_topics = [s.get('topic', '')[:50] for s in sessions if s.get('recently_active')]
  1115. if len(active_topics) > 1 and len(set(active_topics)) < len(active_topics):
  1116. lines.append('⚠️ 多个实例可能在做同一件事\n')
  1117. # Locks from orchestrator
  1118. active_locks = []
  1119. for om_id, om in overmind.items():
  1120. if om.get('current_topic'):
  1121. active_locks.append(f"{om_id[:8]}=>{om['current_topic'][:40]}")
  1122. if active_locks:
  1123. lines.append('🔒 任务锁: ' + ', '.join(active_locks) + '\n')
  1124. return '\n'.join(lines)
  1125. def _write_broadcast(self, md):
  1126. try:
  1127. with open(FLEET_BROADCAST_FILE, 'w', encoding='utf-8') as f:
  1128. f.write(md)
  1129. except Exception:
  1130. pass
  1131. def _push_event_queue(self, sessions):
  1132. try:
  1133. os.makedirs(EVENT_QUEUE_DIR, exist_ok=True)
  1134. data = {
  1135. 'event': 'fleet:broadcast',
  1136. 'ts': int(time.time() * 1000),
  1137. 'data': {
  1138. 'count': len(sessions),
  1139. 'instances': [{
  1140. 'id': s['id'][:8],
  1141. 'topic': s.get('topic', '')[:120],
  1142. 'status': 'active' if s.get('recently_active') else 'idle',
  1143. 'qa': s.get('qa_pairs', []),
  1144. 'cwd': s.get('cwd', ''),
  1145. } for s in sessions]
  1146. }
  1147. }
  1148. fpath = os.path.join(EVENT_QUEUE_DIR, f"fleet_broadcast_{int(time.time() * 1000)}.json")
  1149. with open(fpath, 'w', encoding='utf-8') as f:
  1150. json.dump(data, f, ensure_ascii=False)
  1151. except Exception:
  1152. pass
  1153. # Start z2 FleetWatcher
  1154. _watcher = FleetWatcher()
  1155. _watcher.start()
  1156. # ═══════════════════════════════════════════════════════════
  1157. # MCP HANDLER
  1158. # ═══════════════════════════════════════════════════════════
  1159. def handle(req):
  1160. mid = req.get('id')
  1161. method = req.get('method')
  1162. params = req.get('params', {})
  1163. args = params.get('arguments', {})
  1164. if method == 'initialize':
  1165. return send_msg({'jsonrpc': '2.0', 'id': mid, 'result': {
  1166. 'protocolVersion': '2024-11-05',
  1167. 'capabilities': {'tools': {}},
  1168. 'serverInfo': {'name': 'ctxproxy', 'version': '1.0.0'}
  1169. }})
  1170. if method == 'notifications/initialized':
  1171. return
  1172. if mid is None:
  1173. return
  1174. if method == 'tools/list':
  1175. return send_msg({'jsonrpc': '2.0', 'id': mid, 'result': {'tools': [
  1176. {'name': 'search_memory', 'description': '检索记忆(FTS5全文,可选AI精选)', 'inputSchema': {
  1177. 'type': 'object', 'properties': {'query': {'type': 'string'}, 'limit': {'type': 'number'}, 'use_ai': {'type': 'boolean', 'description': '启用AI语义精选(FTS5初筛→flash精选)'}}, 'required': ['query']}},
  1178. {'name': 'save_memory', 'description': '保存一条语义记忆', 'inputSchema': {
  1179. 'type': 'object', 'properties': {'key': {'type': 'string'}, 'content': {'type': 'string'}, 'tags': {'type': 'string'}}, 'required': ['key', 'content']}},
  1180. {'name': 'list_skills', 'description': '列出匹配的技能', 'inputSchema': {
  1181. 'type': 'object', 'properties': {'query': {'type': 'string'}}}},
  1182. {'name': 'create_skill', 'description': '创建新 SKILL.md', 'inputSchema': {
  1183. 'type': 'object', 'properties': {'name': {'type': 'string'}, 'description': {'type': 'string'}, 'content': {'type': 'string'}}, 'required': ['name', 'description', 'content']}},
  1184. {'name': 'memory_stats', 'description': '记忆统计', 'inputSchema': {
  1185. 'type': 'object', 'properties': {}}},
  1186. {'name': 'current_context', 'description': '获取当前会话最新上下文(任务+技能+记忆)', 'inputSchema': {
  1187. 'type': 'object', 'properties': {}}},
  1188. {'name': 'hermes_fusion', 'description': 'Hermes 融合:衰减+晋升+去重', 'inputSchema': {'type': 'object', 'properties': {}}},
  1189. {'name': 'search_procedural', 'description': '搜索程序性记忆模板', 'inputSchema': {'type': 'object', 'properties': {'query': {'type': 'string'}}, 'required': ['query']}},
  1190. {'name': 'search_episodes', 'description': '搜索会话历史(情景记忆)', 'inputSchema': {'type': 'object', 'properties': {'query': {'type': 'string'}}, 'required': ['query']}},
  1191. {'name': 'save_episodic', 'description': '保存一个会话记录', 'inputSchema': {'type': 'object', 'properties': {'session_id': {'type': 'string'}, 'summary': {'type': 'string'}, 'task': {'type': 'string'}, 'message_count': {'type': 'number'}, 'project_name': {'type': 'string'}}, 'required': ['session_id']}},
  1192. {'name': 'search_graph', 'description': '搜索记忆图谱(关系网络),返回子图', 'inputSchema': {'type': 'object', 'properties': {'query': {'type': 'string'}, 'depth': {'type': 'number', 'description': '遍历深度(默认1)'}, 'limit': {'type': 'number'}}, 'required': ['query']}},
  1193. {'name': 'expand_keys', 'description': '将记忆key列表通过图谱扩展为连通子图', 'inputSchema': {'type': 'object', 'properties': {'keys': {'type': 'array', 'items': {'type': 'string'}}, 'depth': {'type': 'number', 'description': '遍历深度(默认1)'}}, 'required': ['keys']}},
  1194. {'name': 'create_edge', 'description': '手动创建图谱关系边', 'inputSchema': {'type': 'object', 'properties': {'source': {'type': 'string'}, 'target': {'type': 'string'}, 'relation_type': {'type': 'string', 'description': 'depends_on/part_of/blocked_by/causes/solves/related_to/extends/conflicts_with/alternative_to/triggers'}, 'confidence': {'type': 'number'}, 'evidence': {'type': 'string'}}, 'required': ['source', 'target', 'relation_type']}},
  1195. {'name': 'graph_stats', 'description': '图谱统计信息', 'inputSchema': {'type': 'object', 'properties': {}}},
  1196. {'name': 'record_feedback', 'description': '记录记忆使用反馈(injected/referenced/helped/did_not_help/caused_confusion)', 'inputSchema': {'type': 'object', 'properties': {'memory_key': {'type': 'string'}, 'event_type': {'type': 'string', 'description': 'injected/referenced/helped/did_not_help/caused_confusion'}, 'session_id': {'type': 'string'}, 'detail': {'type': 'string'}}, 'required': ['memory_key', 'event_type']}},
  1197. {'name': 'search_warnings', 'description': '检测当前任务相关的危险信号(阻塞/冲突/失败模式)', 'inputSchema': {'type': 'object', 'properties': {'query': {'type': 'string', 'description': '当前任务描述或关键词'}}, 'required': ['query']}},
  1198. {'name': 'skill_rankings', 'description': '技能效果排行榜(按完成率排序)', 'inputSchema': {'type': 'object', 'properties': {'limit': {'type': 'number'}}}},
  1199. {'name': 'skill_prefs', 'description': '查询技能使用偏好(什么任务用什么技能)', 'inputSchema': {'type': 'object', 'properties': {'skill_name': {'type': 'string', 'description': '可选,查询特定技能偏好'}}}},
  1200. # ── z2 fleet tools ──
  1201. {'name': 'fleet_status', 'description': '查询所有CC实例状态(自动发现,无需注入)', 'inputSchema': {'type': 'object', 'properties': {}}},
  1202. {'name': 'fleet_peek', 'description': '窥探指定CC会话的最新动态(最近5对Q/A)', 'inputSchema': {'type': 'object', 'properties': {'session_id': {'type': 'string', 'description': '会话ID前8位或完整ID'}}, 'required': ['session_id']}}
  1203. ]}})
  1204. if method == 'tools/call':
  1205. tool = params['name']
  1206. if tool == 'search_memory':
  1207. use_ai = args.get('use_ai', True)
  1208. if use_ai:
  1209. r = search_hybrid_ai(args.get('query', ''), args.get('limit', 10))
  1210. else:
  1211. r = search_hybrid(args.get('query', ''), args.get('limit', 10))
  1212. return send_msg({'jsonrpc': '2.0', 'id': mid, 'result': {'content': [{'type': 'text', 'text': json.dumps(r, ensure_ascii=False)}]}})
  1213. if tool == 'save_memory':
  1214. save_semantic(args['key'], args['content'], args.get('tags', ''))
  1215. return send_msg({'jsonrpc': '2.0', 'id': mid, 'result': {'content': [{'type': 'text', 'text': f'saved: {args["key"]}'}]}})
  1216. if tool == 'list_skills':
  1217. sk = search_skills(args.get('query', ''))
  1218. return send_msg({'jsonrpc': '2.0', 'id': mid, 'result': {'content': [{'type': 'text', 'text': json.dumps(sk, ensure_ascii=False)}]}})
  1219. if tool == 'create_skill':
  1220. d = os.path.join(SKILL_DIRS[0], args['name'])
  1221. os.makedirs(d, exist_ok=True)
  1222. with open(os.path.join(d, 'SKILL.md'), 'w', encoding='utf-8') as f:
  1223. f.write(args['content'])
  1224. index_skills()
  1225. return send_msg({'jsonrpc': '2.0', 'id': mid, 'result': {'content': [{'type': 'text', 'text': f'skill created: {args["name"]}'}]}})
  1226. if tool == 'memory_stats':
  1227. st = get_stats()
  1228. return send_msg({'jsonrpc': '2.0', 'id': mid, 'result': {'content': [{'type': 'text', 'text': json.dumps(st)}]}})
  1229. if tool == 'current_context':
  1230. try:
  1231. with open(os.path.join(ROOT, 'injection.md'), encoding='utf-8') as f:
  1232. ctx = f.read()
  1233. # Track skill invocations from injection
  1234. for m in re.finditer(r'^### (\S+)', ctx, re.MULTILINE):
  1235. skill_name = m.group(1)
  1236. db.execute("UPDATE skill_index SET invoke_count = COALESCE(invoke_count, 0) + 1, last_invoked = datetime('now') WHERE name = ?", (skill_name,))
  1237. db.commit()
  1238. except Exception:
  1239. ctx = 'injection.md not found'
  1240. return send_msg({'jsonrpc': '2.0', 'id': mid, 'result': {'content': [{'type': 'text', 'text': ctx}]}})
  1241. if tool == 'hermes_fusion':
  1242. r = hermes_fusion()
  1243. return send_msg({'jsonrpc': '2.0', 'id': mid, 'result': {'content': [{'type': 'text', 'text': json.dumps(r)}]}})
  1244. if tool == 'search_procedural':
  1245. r = search_procedural(args.get('query', ''))
  1246. return send_msg({'jsonrpc': '2.0', 'id': mid, 'result': {'content': [{'type': 'text', 'text': json.dumps(r, ensure_ascii=False)}]}})
  1247. if tool == 'search_episodes':
  1248. r = search_episodes(args.get('query', ''))
  1249. return send_msg({'jsonrpc': '2.0', 'id': mid, 'result': {'content': [{'type': 'text', 'text': json.dumps(r, ensure_ascii=False)}]}})
  1250. if tool == 'save_episodic':
  1251. save_episodic(args['session_id'], args.get('summary',''), args.get('task',''), args.get('message_count', 0), '', args.get('project_name',''))
  1252. sid = args['session_id']
  1253. return send_msg({'jsonrpc': '2.0', 'id': mid, 'result': {'content': [{'type': 'text', 'text': f'episodic saved: {sid}'}]}})
  1254. if tool == 'search_graph':
  1255. query = args.get('query', '')
  1256. depth = int(args.get('depth', 1))
  1257. limit = int(args.get('limit', 10))
  1258. edges = search_graph(query, limit)
  1259. # If edges found, expand to subgraph
  1260. if edges and depth > 0:
  1261. roots = set()
  1262. for e in edges[:5]:
  1263. roots.add(e['source'])
  1264. roots.add(e['target'])
  1265. sub = expand_keys(list(roots), depth)
  1266. else:
  1267. sub = {'keys': [], 'subgraph': {'nodes': [], 'edges': []}}
  1268. return send_msg({'jsonrpc': '2.0', 'id': mid, 'result': {'content': [{'type': 'text', 'text': json.dumps({'matches': edges, 'subgraph': sub['subgraph'], 'root_keys': list(roots) if edges else []}, ensure_ascii=False)}]}})
  1269. if tool == 'expand_keys':
  1270. keys = args.get('keys', [])
  1271. depth = int(args.get('depth', 1))
  1272. result = expand_keys(keys, depth)
  1273. return send_msg({'jsonrpc': '2.0', 'id': mid, 'result': {'content': [{'type': 'text', 'text': json.dumps(result, ensure_ascii=False)}]}})
  1274. if tool == 'create_edge':
  1275. eid = upsert_edge(args['source'], args['target'], args['relation_type'], float(args.get('confidence', 0.7)), args.get('evidence', ''))
  1276. return send_msg({'jsonrpc': '2.0', 'id': mid, 'result': {'content': [{'type': 'text', 'text': f'edge created: {eid}' if eid else 'edge skipped (duplicate or error)'}]}})
  1277. if tool == 'graph_stats':
  1278. gs = get_graph_stats()
  1279. return send_msg({'jsonrpc': '2.0', 'id': mid, 'result': {'content': [{'type': 'text', 'text': json.dumps(gs)}]}})
  1280. if tool == 'record_feedback':
  1281. ok = record_feedback(args['memory_key'], args['event_type'], args.get('session_id', ''), args.get('detail', ''))
  1282. return send_msg({'jsonrpc': '2.0', 'id': mid, 'result': {'content': [{'type': 'text', 'text': f'feedback recorded: {args["memory_key"]} {args["event_type"]}' if ok else 'feedback failed'}]}})
  1283. if tool == 'search_warnings':
  1284. w = search_warnings(args.get('query', ''))
  1285. return send_msg({'jsonrpc': '2.0', 'id': mid, 'result': {'content': [{'type': 'text', 'text': json.dumps(w, ensure_ascii=False)}]}})
  1286. if tool == 'skill_rankings':
  1287. rankings = get_skill_rankings(int(args.get('limit', 20)))
  1288. return send_msg({'jsonrpc': '2.0', 'id': mid, 'result': {'content': [{'type': 'text', 'text': json.dumps(rankings, ensure_ascii=False)}]}})
  1289. if tool == 'skill_prefs':
  1290. prefs = get_skill_prefs(args.get('skill_name'))
  1291. return send_msg({'jsonrpc': '2.0', 'id': mid, 'result': {'content': [{'type': 'text', 'text': json.dumps(prefs, ensure_ascii=False)}]}})
  1292. if tool == 'fleet_status':
  1293. # Reuse FleetWatcher's scan results from broadcast file
  1294. try:
  1295. with open(FLEET_BROADCAST_FILE, 'r', encoding='utf-8') as f:
  1296. text = f.read()
  1297. except Exception:
  1298. text = '舰队广播未就绪,请稍候'
  1299. return send_msg({'jsonrpc': '2.0', 'id': mid, 'result': {'content': [{'type': 'text', 'text': text}]}})
  1300. if tool == 'fleet_peek':
  1301. sid = args.get('session_id', '')
  1302. qa_pairs = []
  1303. try:
  1304. # Dynamic: scan all project dirs for this session
  1305. jsonl = None
  1306. try:
  1307. for dname in os.listdir(CC_PROJECTS_DIR):
  1308. dpath = os.path.join(CC_PROJECTS_DIR, dname)
  1309. if not os.path.isdir(dpath):
  1310. continue
  1311. for fname in os.listdir(dpath):
  1312. if fname.endswith('.jsonl') and sid[:8] in fname:
  1313. jsonl = os.path.join(dpath, fname)
  1314. break
  1315. if jsonl:
  1316. break
  1317. except Exception:
  1318. pass
  1319. if jsonl:
  1320. with open(jsonl, 'r', encoding='utf-8', errors='ignore') as f:
  1321. lines = f.readlines()
  1322. pairs = []
  1323. cur_q = None; cur_a = None
  1324. for line in reversed(lines[-800:]):
  1325. try:
  1326. entry = json.loads(line.strip())
  1327. except Exception:
  1328. continue
  1329. if entry.get('type') == 'user' and not cur_q:
  1330. msg = entry.get('message', {})
  1331. if isinstance(msg, dict):
  1332. c = msg.get('content', '')
  1333. if isinstance(c, str) and c.strip(): cur_q = c.strip()
  1334. elif isinstance(c, list):
  1335. for p in c:
  1336. if isinstance(p, dict) and p.get('text'):
  1337. cur_q = p['text'].strip(); break
  1338. continue
  1339. if entry.get('type') == 'assistant' and cur_q and not cur_a:
  1340. msg = entry.get('message', {})
  1341. if isinstance(msg, dict):
  1342. c = msg.get('content', '')
  1343. if isinstance(c, str) and c.strip(): cur_a = c.strip()
  1344. elif isinstance(c, list):
  1345. txts = [p['text'].strip() for p in c if isinstance(p, dict) and p.get('text')]
  1346. if txts: cur_a = '\n'.join(txts)
  1347. if cur_q and cur_a:
  1348. pairs.append({'q': cur_q[:200], 'a': cur_a[:300]})
  1349. cur_q = None; cur_a = None
  1350. if len(pairs) >= 5: break
  1351. qa_pairs = pairs
  1352. except Exception:
  1353. pass
  1354. return send_msg({'jsonrpc': '2.0', 'id': mid, 'result': {'content': [{'type': 'text', 'text': json.dumps({'session_id': sid, 'qa_pairs': qa_pairs, 'count': len(qa_pairs)}, ensure_ascii=False)}]}})
  1355. if __name__ == '__main__':
  1356. # Main loop — raw JSON lines (matching claude_opus_mcp.py pattern)
  1357. while True:
  1358. req = read_msg()
  1359. if req is None:
  1360. break
  1361. try:
  1362. handle(req)
  1363. except Exception:
  1364. pass
dream.js · Bundled script · 261 lines · 1 finding marked
  1. const path = require('path')
  2. const fs = require('fs')
  3. const https = require('https')
  4. const ROOT = path.dirname(__filename)
  5. const { getAPIConfig } = require('./config')
  6. function callLLM(messages, useFlash = true) {
  7. return new Promise((resolve, reject) => {
  8. let cfg; try { cfg = getAPIConfig(useFlash) } catch(e) { reject(e); return }
  9. const body = cfg.bodyBuilder(Array.isArray(messages) ? messages : [{ role: 'user', content: messages }])
  10. const req = require('https').request({
  11. hostname: cfg.hostname, path: cfg.path, method: 'POST',
  12. headers: cfg.headers, timeout: cfg.timeout
  13. }, res => {
  14. let data = ''
  15. res.on('data', c => data += c)
  16. res.on('end', () => {
  17. try {
  18. const obj = JSON.parse(data)
  19. if (obj.error) { reject(new Error(obj.error.message || 'API error')); return }
  20. if (cfg.format === 'openai') { resolve(obj.choices?.[0]?.message?.content || ''); return }
  21. const textBlock = obj.content?.find(c => c.type === 'text')
  22. resolve(textBlock ? textBlock.text : (obj.content?.[0]?.text || ''))
  23. } catch(e) { reject(e) }
  24. })
  25. })
  26. req.on('error', reject)
  27. req.write(body)
  28. req.end()
  29. })
  30. }
  31. const DREAM_FILE = path.join(ROOT, '.dream_findings.json')
  32. function callLLM(messages) {
  33. return new Promise((resolve, reject) => {
  34. const body = JSON.stringify({
  35. model: 'deepseek-v4-flash',
  36. max_tokens: 8192,
  37. messages: messages.map(m => ({ role: m.role, content: m.content }))
  38. })
  39. const req = https.request({
    • Warning: Sends stored memory contents to api.deepseek.com in an HTTPS request authenticated with the user's API key. Your session memories leave the machine for a third-party LLM.
    • Line 43 on GitHub
  40. hostname: 'api.deepseek.com', path: '/v1/chat/completions', method: 'POST',
  41. headers: { 'Content-Type': 'application/json', 'Authorization': `Bearer ${API_KEY}` },
  42. timeout: 120000
  43. }, res => {
  44. let data = ''
  45. res.on('data', c => data += c)
  46. res.on('end', () => {
  47. try { resolve(JSON.parse(data).choices[0].message.content) }
  48. catch(e) { reject(e) }
  49. })
  50. })
  51. req.on('error', reject)
  52. req.write(body)
  53. req.end()
  54. })
  55. }
  56. function callLLM(messages) {
  57. return new Promise((resolve, reject) => {
  58. const body = JSON.stringify({
  59. model: 'deepseek-v4-pro[1m]',
  60. max_tokens: 16384,
  61. messages: messages.map(m => ({ role: m.role, content: m.content }))
  62. })
  63. const req = https.request({
  64. hostname: 'api.deepseek.com', path: '/anthropic/v1/messages', method: 'POST',
  65. headers: { 'Content-Type': 'application/json', 'x-api-key': API_KEY, 'anthropic-version': '2023-06-01' },
  66. timeout: 300000
  67. }, res => {
  68. let data = ''
  69. res.on('data', c => data += c)
  70. res.on('end', () => {
  71. try {
  72. const body = JSON.parse(data)
  73. if (body.error) { reject(new Error(body.error.message)); return }
  74. const textBlock = body.content?.find(c => c.type === 'text')
  75. resolve(textBlock ? textBlock.text : (body.content?.[0]?.text || ''))
  76. } catch(e) { reject(e) }
  77. })
  78. })
  79. req.on('error', reject)
  80. req.write(body)
  81. req.end()
  82. })
  83. }
  84. async function dream(index) {
  85. index.init()
  86. const db = require('better-sqlite3')(path.join(ROOT, 'memory.db'))
  87. // Gather memory summary (aggregated, not raw dump)
  88. const stats = index.getStats()
  89. // Top 30 most accessed memories
  90. const topMems = db.prepare(`SELECT key, content, tags, access_count, COALESCE(effectiveness_score,0.5) as eff, COALESCE(injected_count,0) as ic, created_at FROM semantic WHERE key != '_schema_version' ORDER BY access_count DESC LIMIT 30`).all()
  91. // Top contradictions from graph
  92. let conflicts = []
  93. try {
  94. const gdb = require('better-sqlite3')(path.join(ROOT, 'graph.db'))
  95. conflicts = gdb.prepare(`SELECT e.source, e.target, e.evidence, e.confidence FROM edges e WHERE e.relation_type = 'conflicts_with' ORDER BY e.confidence DESC LIMIT 10`).all()
  96. gdb.close()
  97. } catch(e) {}
  98. // Recent open issues
  99. const issues = db.prepare(`SELECT key, content, created_at FROM semantic WHERE key != '_schema_version' AND (key LIKE '%issue_%' OR key LIKE '%blocker%' OR key LIKE '%unresolved%' OR content LIKE '%未解决%' OR content LIKE '%尚未解决%') ORDER BY created_at DESC LIMIT 10`).all()
  100. // Skill effectiveness summary
  101. const skillSummary = db.prepare(`SELECT skill_name, SUM(CASE WHEN event_type='completed' THEN 1 ELSE 0 END) as done, SUM(CASE WHEN event_type='failed' THEN 1 ELSE 0 END) as fail FROM skill_feedback GROUP BY skill_name ORDER BY done DESC LIMIT 15`).all()
  102. // Build context for pro model
  103. const catalog = [
  104. `## 记忆状态: ${stats.semanticCount} 条语义记忆, ${stats.skillCount} 个技能`,
  105. '',
  106. '### 高访问记忆 (top 30)',
  107. ...topMems.map(m => `- ${m.key}: ${m.content?.substring(0, 120)} [访问:${m.access_count} 有效率:${(m.eff*100).toFixed(0)}% 注入:${m.ic}次]`),
  108. '',
  109. '### 活跃矛盾',
  110. ...(conflicts.length ? conflicts.map(c => `- ${c.source} ↔ ${c.target}: ${c.evidence?.substring(0, 100)}`) : ['- 无活跃矛盾']),
  111. '',
  112. '### 未解决问题',
  113. ...(issues.length ? issues.map(i => `- ${i.key}: ${i.content?.substring(0, 100)} [${i.created_at}]`) : ['- 无']),
  114. '',
  115. '### 技能效果',
  116. ...(skillSummary.length ? skillSummary.map(s => `- ${s.skill_name}: 成功${s.done} 失败${s.fail}`) : ['- 暂无数据']),
  117. ].join('\n')
  118. const prompt = `你是一个 AI 认知引擎的夜间分析模块。分析以下记忆库快照,产出深度研究结论。
  119. ## 任务
  120. 1. **记忆融合**: 找到 3-5 组表达同一事实但用不同方式描述的记忆,给出合并后的精确表述
  121. 2. **矛盾仲裁**: 裁决矛盾,给出确定性结论
  122. 3. **知识缺口**: 识别 3 个最关键的未解决问题,按严重度排序
  123. 4. **模式发现**: 从技能效果数据中提取最有效的工解模式
  124. 5. **预测**: 基于历史数据,预测当前路径最可能的结果
  125. ## 输出格式
  126. 返回 JSON:
  127. {
  128. "merges": [{"fragment_keys": ["key1","key2"], "merged_fact": "合并后的精确事实"}],
  129. "arbitrations": [{"conflict": "A vs B", "winner": "A", "verdict": "确定性结论"}],
  130. "critical_gaps": [{"issue": "问题描述", "severity": "high|medium", "impact": "不解决会导致..."}],
  131. "patterns": [{"pattern": "模式描述", "evidence": "证据", "recommendation": "建议"}],
  132. "prediction": {"most_likely": "最可能走向", "confidence": 0.0-1.0, "alternative": "替代路径"},
  133. "summary": "2-3 句整体总结"
  134. }
  135. ## 记忆快照
  136. ${catalog}`
  137. try {
  138. let result = null
  139. try {
  140. result = await callLLM([{ role: 'user', content: prompt }])
  141. } catch(e) {
  142. // Pro failed — fallback to flash
  143. }
  144. // Flash fallback: if pro returned nothing or crashed
  145. if (!result) {
  146. const flashPrompt = `简化分析以下记忆库快照,只输出关键发现。返回 JSON: {"merges":[], "critical_gaps":[], "summary":"一句话总结"}
  147. ${catalog.substring(0, 5000)}`
  148. try {
  149. result = await callLLM([{ role: 'user', content: flashPrompt }])
  150. } catch(e2) {
  151. return null
  152. }
  153. }
  154. if (!result) return null
  155. // Parse JSON from response
  156. let findings = null
  157. try { findings = JSON.parse(result.trim()) } catch(e) {
  158. const m = result.match(/\{[\s\S]*\}/)
  159. if (m) try { findings = JSON.parse(m[0]) } catch(e2) {}
  160. }
  161. if (!findings) return null
  162. findings.dreamed_at = new Date().toISOString()
  163. findings.stats = stats
  164. // Save to file
  165. fs.writeFileSync(DREAM_FILE, JSON.stringify(findings, null, 2), 'utf-8')
  166. // v4: emit dream complete event
  167. try { require(path.join(ROOT, 'eventbus')).emit('dream:complete', { findings, stats }) } catch(e) {}
  168. // Apply merges
  169. for (const mg of (findings.merges || [])) {
  170. const keys = mg.fragment_keys
  171. const mergedKey = keys[0] + '_dream_merged'
  172. const mergedContent = `[梦境融合 ${keys.length} 条记忆] ${mg.merged_fact}`
  173. try {
  174. db.prepare('INSERT OR REPLACE INTO semantic (key, content, tags, confidence) VALUES (?,?,?,?)').run(mergedKey, mergedContent, 'dream-merged', 0.8)
  175. for (const k of keys) {
  176. db.prepare("UPDATE semantic SET tags = COALESCE(tags,'') || ',dream-merged', confidence = confidence * 0.5 WHERE key = ?").run(k)
  177. }
  178. } catch(e) {}
  179. }
  180. db.close()
  181. return findings
  182. } catch(e) {
  183. db.close()
  184. return null
  185. }
  186. }
  187. function loadDreamFindings() {
  188. try {
  189. if (!fs.existsSync(DREAM_FILE)) return null
  190. const data = JSON.parse(fs.readFileSync(DREAM_FILE, 'utf-8'))
  191. const age = Date.now() - new Date(data.dreamed_at).getTime()
  192. if (age > 24 * 60 * 60 * 1000) return null // stale
  193. return data
  194. } catch(e) { return null }
  195. }
  196. function formatDream(findings) {
  197. if (!findings) return ''
  198. let text = `\n## 🌙 梦境研究\n\n超脑昨夜深度分析了记忆库 (${findings.stats?.semanticCount || '?'} 条记忆):\n\n`
  199. if (findings.summary) {
  200. text += `**${findings.summary}**\n\n`
  201. }
  202. if (findings.merges?.length) {
  203. text += '### 记忆融合\n' + findings.merges.map(m =>
  204. `- ${m.fragment_keys.join(' + ')} → ${m.merged_fact?.substring(0, 150)}`
  205. ).join('\n') + '\n\n'
  206. }
  207. if (findings.arbitrations?.length) {
  208. text += '### 矛盾裁决\n' + findings.arbitrations.map(a =>
  209. `- **${a.winner}** 胜出: ${a.verdict?.substring(0, 150)}`
  210. ).join('\n') + '\n\n'
  211. }
  212. if (findings.critical_gaps?.length) {
  213. text += '### 关键缺口\n' + findings.critical_gaps.map(g =>
  214. `- [${g.severity}] ${g.issue}: ${g.impact?.substring(0, 120)}`
  215. ).join('\n') + '\n\n'
  216. }
  217. if (findings.patterns?.length) {
  218. text += '### 有效模式\n' + findings.patterns.map(p =>
  219. `- ${p.pattern}: ${p.recommendation?.substring(0, 120)}`
  220. ).join('\n') + '\n\n'
  221. }
  222. if (findings.prediction) {
  223. text += `### 预测\n最可能: ${findings.prediction.most_likely} (置信度 ${(findings.prediction.confidence*100).toFixed(0)}%)\n`
  224. }
  225. text += `\n> 下一次梦境: 明天凌晨\n`
  226. return text
  227. }
  228. module.exports = { dream, loadDreamFindings, formatDream }
extract_worker.js · Bundled script · 427 lines · 1 finding marked
  1. const path = require('path')
  2. const fs = require('fs')
  3. const https = require('https')
  4. process.on('unhandledRejection', (reason) => {
  5. try { fs.appendFileSync(path.join(__dirname, 'worker.log'), `${new Date().toISOString()} [worker] unhandledRejection: ${reason?.message || reason}\n`) } catch(e) {}
  6. })
  7. const ROOT = path.dirname(__filename)
  8. const { shouldSkipExtraction } = require(path.join(ROOT, 'privacy_filter'))
  9. const EPISODIC_DIR = path.join(ROOT, 'memory', 'episodic')
  10. const adapter = require(path.join(ROOT, 'adapters')).getAgent()
  11. const { getAPIConfig } = require('./config')
  12. function callLLM(messages, useFlash = true) {
  13. return new Promise((resolve, reject) => {
  14. let cfg; try { cfg = getAPIConfig(useFlash) } catch(e) { reject(e); return }
  15. const body = cfg.bodyBuilder(Array.isArray(messages) ? messages : [{ role: 'user', content: messages }])
  16. const req = require('https').request({
  17. hostname: cfg.hostname, path: cfg.path, method: 'POST',
  18. headers: cfg.headers, timeout: cfg.timeout
  19. }, res => {
  20. let data = ''
  21. res.on('data', c => data += c)
  22. res.on('end', () => {
  23. try {
  24. const obj = JSON.parse(data)
  25. if (obj.error) { reject(new Error(obj.error.message || 'API error')); return }
  26. if (cfg.format === 'openai') { resolve(obj.choices?.[0]?.message?.content || ''); return }
  27. const textBlock = obj.content?.find(c => c.type === 'text')
  28. resolve(textBlock ? textBlock.text : (obj.content?.[0]?.text || ''))
  29. } catch(e) { reject(e) }
  30. })
  31. })
  32. req.on('error', reject)
  33. req.write(body)
  34. req.end()
  35. })
  36. }
  37. function findCurrentTranscript() {
  38. try {
  39. if (!fs.existsSync(adapter.transcriptDir())) return null
  40. const files = fs.readdirSync(adapter.transcriptDir())
  41. .filter(f => f.endsWith('.jsonl'))
  42. .map(f => ({ name: f, mtime: fs.statSync(path.join(adapter.transcriptDir(), f)).mtimeMs, path: path.join(adapter.transcriptDir(), f) }))
  43. .sort((a, b) => b.mtime - a.mtime)
  44. return files[0] || null
  45. } catch(e) { return null }
  46. }
  47. function loadNewLines(filepath, lastPos) {
  48. try {
  49. const stat = fs.statSync(filepath)
  50. if (stat.size <= lastPos) return { lines: [], newPos: lastPos }
  51. const fd = fs.openSync(filepath, 'r')
  52. fs.readSync(fd, Buffer.alloc(0), 0, 0, lastPos)
  53. const buf = Buffer.alloc(stat.size - lastPos)
  54. fs.readSync(fd, buf, 0, buf.length, lastPos)
  55. fs.closeSync(fd)
  56. const text = buf.toString('utf-8')
  57. const lines = text.split('\n').filter(Boolean)
  58. return { lines, newPos: stat.size }
  59. } catch(e) { return { lines: [], newPos: lastPos } }
  60. }
  61. function compressForExtraction(lines, maxChars) {
  62. const msgs = []
  63. for (let i = 0; i < lines.length && msgs.length < 100; i++) {
  64. try {
  65. const obj = JSON.parse(lines[i])
  66. const msg = obj.message || {}
  67. const role = msg.role || 'unknown'
  68. let content = msg.content || ''
  69. if (typeof content !== 'string') content = JSON.stringify(content)
  70. if (content) msgs.push(`[${role}] ${content.substring(0, 800)}`)
  71. } catch(e) {}
  72. }
  73. const text = msgs.join('\n')
  74. if (text.length <= maxChars) return text
  75. return text.substring(0, maxChars)
  76. }
  77. async function extractAndSave(lines, sessionId) {
  78. const text = compressForExtraction(lines, 30000)
  79. if (text.length < 200) return 0
  80. const prompt = `${getHermesPrompt()}
  81. ## 当前对话片段
  82. ${text}
  83. ## 已存记忆关键词(避免重复)
  84. ${getExistingKeys()}
  85. 请按格式输出每条新发现的事实。`
  86. try {
  87. const result = await callLLM([{ role: 'system', content: getHermesPrompt() }, { role: 'user', content: prompt }])
    • Warning: A background worker sends up to 30,000 characters of the user's Claude Code conversation transcript to a third-party LLM API to extract memories, repeatedly and without any prompt.
    • Line 100 on GitHub
  88. if (!result) return 0
  89. const index = require(path.join(ROOT, 'index'))
  90. index.init()
  91. index.ensureMemoryDirs()
  92. let saved = 0, skipped = 0, skillDrafts = 0
  93. const lines = result.split('\n').filter(l => l.trim().startsWith('{'))
  94. for (const line of lines) {
  95. try {
  96. const fact = JSON.parse(line.trim())
  97. // Check for procedural_candidate / skill draft format
  98. if (fact.type === 'procedural' && fact.name && fact.steps) {
  99. const skillPath = path.join(ROOT, 'skills', 'all', `${fact.name}.md`)
  100. const triggerStr = fact.trigger || ''
  101. const stepsStr = Array.isArray(fact.steps) ? fact.steps.join('\n') : String(fact.steps)
  102. const content = `---\nname: ${fact.name}\ndescription: ${fact.trigger || fact.name}\ntriggers: ${triggerStr}\n---\n\n# ${fact.name}\n\n${fact.trigger || ''}\n\n## Steps\n${stepsStr}`
  103. if (!fs.existsSync(skillPath)) {
  104. fs.writeFileSync(skillPath, content, 'utf-8')
  105. index.logEvolution(sessionId, 'skill_draft', { name: fact.name })
  106. skillDrafts++
  107. }
  108. continue
  109. }
  110. if (fact.key && fact.content && fact.is_new !== false) {
  111. if (shouldSkipExtraction(fact.content)) { skipped++; continue }
  112. const head = index.getGitHead()
  113. if (index.saveSemantic(fact.key, fact.content, fact.category || 'general', sessionId, fact.dedup_key, fact.confidence || 0.5, head)) {
  114. saved++
  115. }
  116. }
  117. } catch(e) {}
  118. }
  119. if (skipped > 0) log(`privacy filter: skipped ${skipped} facts`)
  120. if (skillDrafts > 0) {
  121. log(`skill drafts created: ${skillDrafts}`)
  122. const { spawn } = require('child_process')
  123. spawn('pythonw', ['-c', `import sys; sys.path.insert(0,'${ROOT}'); from daemon import index_skills; print(f'Re-indexed: {len(index_skills())} skills')`], { stdio: 'ignore', detached: true, windowsHide: true }).unref()
  124. }
  125. // Injection refresh handled by CC hooks — removing redundant spawn
  126. // that caused triple-fire + lock contention + console flash every 30s
  127. // ---- GRAPH: Extract relationships from conversation ----
  128. try {
  129. const graph = require(path.join(ROOT, 'graph'))
  130. const allMems = index.getAllMemoryKeys()
  131. if (allMems.length >= 5) {
  132. const relPrompt = graph.buildExtractionPrompt(allMems, text)
  133. const relResult = await callLLM([{ role: 'user', content: relPrompt }])
  134. if (relResult) {
  135. const relations = graph.parseExtractionResult(relResult)
  136. if (relations.length > 0) {
  137. const added = graph.applyRelations(relations, sessionId)
  138. log(`graph: extracted ${added} relations from ${relations.length} candidates`)
  139. }
  140. }
  141. }
  142. } catch(e) {
  143. log(`graph extraction error: ${e.message}`)
  144. }
  145. // ---- SKILL PREFERENCE: Detect what skills were used for what tasks ----
  146. try {
  147. const allSkills = index.getAllSkills()
  148. const validNames = allSkills.map(s => s.name)
  149. // Read injected skills for implicit detection
  150. let injectedSkillNames = ''
  151. try {
  152. const injContent = fs.readFileSync(path.join(ROOT, 'injection.md'), 'utf-8')
  153. const re = /^### (\S+)/gm
  154. let m; const names = []
  155. while ((m = re.exec(injContent)) !== null) names.push(m[1])
  156. if (names.length > 0) injectedSkillNames = `\nSKILLS INJECTED THIS SESSION: ${names.join(', ')}\nCheck if these were used — even IMPLICITLY (assistant follows skill rules without naming it).`
  157. } catch(e) {}
  158. const skillPrefPrompt = `Analyze this conversation. Detect when a skill was used.${injectedSkillNames}
  159. IMPORTANT — Implicit Usage: The assistant often follows a skill's rules WITHOUT naming it. Look for:
  160. - Code style matches: zero comments + zero types + stdlib only = elon-code was used
  161. - Thinking mentions: "从另一个角度看"/"反证" = lateral-jump was used
  162. - Multi-path analysis: A/B/C 方案对比 = net-analyze was used
  163. - The user said "用X" and the assistant's output matches X's style → X was used
  164. Output format: SKILL_PREF: <skill_name> | <task_scenario> | <effectiveness_0_to_1>
  165. Rules:
  166. - skill_name: MUST be one of the injected skills above, or a skill name from the conversation
  167. - task_scenario: SHORT task category (2-8 Chinese chars). Generalize.
  168. - effectiveness: 0.9=completed, 0.7=partial, 0.3=failed
  169. - Output ONLY SKILL_PREF lines. Nothing else.
  170. - Output nothing if no skill usage detected.
  171. Conversation:
  172. ${text.substring(0, 4000)}`
  173. const prefResult = await callLLM([{ role: 'user', content: skillPrefPrompt }])
  174. if (prefResult) {
  175. const prefLines = prefResult.split('\n').filter(l => l.startsWith('SKILL_PREF:'))
  176. for (const line of prefLines) {
  177. const m = line.match(/^SKILL_PREF:\s*(\S+)\s*\|\s*(.+?)\s*\|\s*([\d.]+)/)
  178. if (m) {
  179. let skillName = m[1].trim()
  180. let taskPattern = m[2].trim()
  181. const eff = parseFloat(m[3]) || 0.7
  182. // Normalize: extract core task category
  183. const clean = taskPattern.replace(/[的之地得和与了在是]/g, '').replace(/[a-zA-Z0-9_\-\.\/]+/g, '').trim()
  184. const pair = clean.match(/(逆向|安全|代码|网络|系统|数据|性能|前端|后端|数据库|API|加密|合约|协议|内存|进程|流量|抓包|令牌|分析|编写|生成)(分析|工程|审查|审计|测试|调试|开发|配置|优化|代码|管理|检测|追踪|提取|破解|脚本)/)
  185. if (pair) {
  186. taskPattern = pair[0]
  187. } else if (clean.length > 8) {
  188. taskPattern = clean.substring(0, 8)
  189. }
  190. // Fuzzy match: find the closest real skill name
  191. const matched = fuzzyMatchSkill(skillName, validNames)
  192. if (matched) {
  193. index.upsertSkillPref(matched, taskPattern, eff)
  194. }
  195. }
  196. }
  197. if (prefLines.length > 0) {
  198. index.syncSkillPrefsToFile()
  199. log(`skill_prefs: detected ${prefLines.length} usage patterns`)
  200. } else {
  201. log(`skill_prefs: API responded but no SKILL_PREF lines (${prefResult.substring(0, 100).replace(/\n/g, ' ')})`)
  202. }
  203. } else {
  204. log(`skill_prefs: API returned empty response`)
  205. }
  206. } catch(e) {
  207. log(`skill_pref extraction error: ${e.message}`)
  208. }
  209. return saved
  210. } catch(e) {
  211. log(`extract error: ${e.message}`)
  212. return 0
  213. }
  214. }
  215. function fuzzyMatchSkill(name, validNames) {
  216. if (validNames.includes(name)) return name
  217. const norm = (s) => s.toLowerCase().replace(/[-_\s]/g, '')
  218. const target = norm(name)
  219. for (const v of validNames) {
  220. if (norm(v) === target) return v
  221. }
  222. for (const v of validNames) {
  223. const nv = norm(v)
  224. const ratio = Math.min(target.length, nv.length) / Math.max(target.length, nv.length)
  225. if (ratio >= 0.4 && (nv.includes(target) || target.includes(nv))) return v
  226. }
  227. const aliases = {
  228. 'elonmask': 'elon-code', 'elonmusk': 'elon-code', 'elon': 'elon-code',
  229. 'superpower': 'using-superpowers', 'superpowers': 'using-superpowers',
  230. 'netanalyze': 'net-analyze', 'lateraljump': 'lateral-jump',
  231. }
  232. if (aliases[target]) return aliases[target]
  233. return null
  234. }
  235. function getExistingKeys() {
  236. try {
  237. const index = require(path.join(ROOT, 'index'))
  238. index.init()
  239. const db = index.db || require('better-sqlite3')(index.DB_PATH)
  240. const rows = db.prepare('SELECT key FROM semantic LIMIT 200').all()
  241. return rows.map(r => r.key).join(', ')
  242. } catch(e) { return '' }
  243. }
  244. // Main watch loop
  245. async function main() {
  246. // Singleton guard — atomic PID file prevents race between workers
  247. const pidFile = path.join(ROOT, '.worker.pid')
  248. let pidFd = null
  249. for (let attempt = 0; attempt < 5; attempt++) {
  250. try {
  251. pidFd = fs.openSync(pidFile, 'wx')
  252. fs.writeSync(pidFd, String(process.pid))
  253. break
  254. } catch(e) {
  255. if (e.code === 'EEXIST') {
  256. try {
  257. const oldPid = parseInt(fs.readFileSync(pidFile, 'utf8'))
  258. try { process.kill(oldPid, 'SIGTERM') } catch(e) {}
  259. } catch(e) {}
  260. try { fs.unlinkSync(pidFile) } catch(e) {}
  261. require('child_process').execSync(process.platform === 'win32' ? 'ping 127.0.0.1 -n 1 >nul' : 'sleep 0.05', {stdio:'ignore'})
  262. } else { log(`pid lock error: ${e.message}`); return }
  263. }
  264. }
  265. if (!pidFd) { log('could not acquire PID lock, exiting'); return }
  266. process.on('exit', () => { try { fs.closeSync(pidFd); fs.unlinkSync(pidFile) } catch(e) {} })
  267. try { require(path.join(ROOT, 'wiring')).init() } catch(e) {}
  268. log('worker started (watch mode)')
  269. const transcript = findCurrentTranscript()
  270. if (!transcript) { log('no transcript found'); return }
  271. log(`watching: ${transcript.name}`)
  272. let lastPos = fs.statSync(transcript.path).size
  273. let accumulatedLines = []
  274. let startTime = Date.now()
  275. let lastContentTime = Date.now()
  276. let lastConsolidationCheck = 0
  277. const SESSION_IDLE_TIMEOUT = 15 * 60 * 1000
  278. const check = async () => {
  279. try {
  280. const { lines, newPos } = loadNewLines(transcript.path, lastPos)
  281. if (lines.length > 0) {
  282. accumulatedLines.push(...lines)
  283. lastPos = newPos
  284. lastContentTime = Date.now()
  285. }
  286. const shouldExtract = accumulatedLines.length >= MIN_NEW_LINES ||
  287. (accumulatedLines.length > 0 && lines.length === 0 && Date.now() - startTime > 60000)
  288. if (shouldExtract && accumulatedLines.length > 0) {
  289. const sessionId = 'incr_' + Date.now()
  290. const lineCount = accumulatedLines.length
  291. const saved = await extractAndSave(accumulatedLines, sessionId)
  292. log(`incremental: ${saved} facts from ${lineCount} lines`)
  293. // v4: emit extraction event
  294. try { require(path.join(ROOT, 'eventbus')).emit('memory:extracted', { count: saved, lines: lineCount, sessionId }) } catch(e) {}
  295. accumulatedLines = []
  296. startTime = Date.now()
  297. }
  298. // Autonomous research & dream phase
  299. const idle = Date.now() - lastContentTime
  300. const RESEARCH_IDLE = 5 * 60 * 1000
  301. const DREAM_IDLE = 10 * 60 * 1000
  302. if (idle > RESEARCH_IDLE && (Date.now() - lastConsolidationCheck) > RESEARCH_IDLE) {
  303. try {
  304. const index = require(path.join(ROOT, 'index'))
  305. // Lightweight research (local SQL)
  306. try {
  307. const research = require(path.join(ROOT, 'research'))
  308. const analysis = research.analyze(index)
  309. if (analysis && analysis.total_findings > 0) {
  310. fs.writeFileSync(path.join(ROOT, '.research_findings.json'), JSON.stringify(analysis, null, 2), 'utf-8')
  311. log(`research: ${analysis.total_findings} patterns found`)
  312. }
  313. } catch(e) { log(`research error: ${e.message}`) }
  314. // Dream phase (pro model) — idle >30min, once per 8 hours
  315. if (idle > DREAM_IDLE) {
  316. const dreamFile = path.join(ROOT, '.dream_findings.json')
  317. const shouldDream = !fs.existsSync(dreamFile) ||
  318. (Date.now() - fs.statSync(dreamFile).mtimeMs) > 2 * 60 * 60 * 1000
  319. if (shouldDream) {
  320. try {
  321. const dream = require(path.join(ROOT, 'dream'))
  322. const findings = await dream.dream(index)
  323. if (findings && findings.summary) log(`dream: ${findings.summary.substring(0, 100)}`)
  324. } catch(e) { log(`dream error: ${e.message}`) }
  325. }
  326. }
  327. } catch(e) { log(`research/dream error: ${e.message}`) }
  328. }
  329. // Session end: transcript idle >15min → auto-consolidate
  330. if (idle > SESSION_IDLE_TIMEOUT && (Date.now() - lastConsolidationCheck) > SESSION_IDLE_TIMEOUT) {
  331. lastConsolidationCheck = Date.now()
  332. const { spawn } = require('child_process')
  333. const cp = spawn('node', [path.join(ROOT, 'consolidate.js')], { windowsHide: true, detached: true,
  334. cwd: ROOT, timeout: 120000, stdio: 'pipe'
  335. })
  336. let out = ''
  337. cp.stdout.on('data', c => out += c)
  338. cp.on('close', code => {
  339. if (code === 0) log(`session_end(worker): ${out.trim().substring(0, 200)}`)
  340. else log(`session_end(worker): exit ${code}`)
  341. })
  342. cp.on('error', e => log(`session_end(worker) failed: ${e.message}`))
  343. }
  344. } catch(e) {
  345. log(`check error: ${e.message}`)
  346. }
  347. }
  348. // Self-healing: periodic health check
  349. try {
  350. const healer = require(path.join(ROOT, 'healer'))
  351. healer.start(120000) // every 2 minutes
  352. } catch(e) {}
  353. // Adaptive interval — dynamic polling
  354. const adaptive = require(path.join(ROOT, 'adaptive'))
  355. let timer = setInterval(check, POLL_INTERVAL)
  356. // Check interval every 5 minutes and adjust
  357. setInterval(() => {
  358. const newInterval = adaptive.computeInterval(lastContentTime, accumulatedLines.length)
  359. if (Math.abs(newInterval - POLL_INTERVAL) > 10000) {
  360. // Interval would change significantly — restart timer
  361. clearInterval(timer)
  362. timer = setInterval(check, newInterval)
  363. log(`adaptive: interval adjusted to ${newInterval/1000}s`)
  364. }
  365. }, 300000)
  366. setTimeout(() => { clearInterval(timer); log('worker lifetime expired') }, MAX_LIFETIME)
  367. // Noise self-learner — periodic
  368. setInterval(() => {
  369. try {
  370. const nl = require(path.join(ROOT, 'noiselearner'))
  371. nl.learn(require(path.join(ROOT, 'index')))
  372. } catch(e) {}
  373. }, 600000)
  374. process.on('SIGTERM', () => {
  375. clearInterval(timer)
  376. log('worker graceful shutdown: checkpointing DBs')
  377. try { require(path.join(ROOT, 'pool')).checkpoint() } catch(e) {}
  378. process.exit(0)
  379. })
  380. }
  381. main().catch(e => log(`fatal: ${e.message}`))

Ratings and comments

Sign in to rate this skill.

Sign in

Free to use. If it’s helped, pass it on.