Skip to content
Back to search

alexgreensh-token-optimizer

MarketplaceWorks with Claude CodeGitHubIncomplete

Safety scan

A static text scan, not a guarantee. Select the badge to see what was scanned and every finding.

Checked today

Check it deeper on your machine

Its scripts were only partly scanned.

As a Claude Code plugin, install it and then ask Claude Code:

/plugin marketplace add keithmackay/mackayi/plugin install scanskillsafety@mackayiDo a deep scan of https://github.com/alexgreensh/token-optimizer with scanskillsafety

Or from a terminal, clone it and run it:

git clone https://github.com/keithmackay/scanskillsafety-skill ~/.claude/skills/scanskillsafetynode ~/.claude/skills/scanskillsafety/cli.cjs --deep 'https://github.com/alexgreensh/token-optimizer'

It checks the exact commit: every file, its packages against OSV (api.osv.dev), how it compares with this page's scan (findsafeskills.com receives the repository's address), and, if you agree, Claude reading every file with your own Claude subscription. It never runs the code.

Badge for your README
findsafeskills safety rating

Markdown

[![findsafeskills safety rating](/api/listings/alexgreensh-token-optimizer/badge.svg)](/listings/alexgreensh-token-optimizer)

HTML

<a href="/listings/alexgreensh-token-optimizer"><img src="/api/listings/alexgreensh-token-optimizer/badge.svg" alt="findsafeskills safety rating" /></a>

Install

Install instructions from the README

As written by the repo's author. findsafeskills has not verified these steps.

**Claude Code (recommended):**

/plugin marketplace add alexgreensh/token-optimizer
/plugin install token-optimizer@alexgreensh-token-optimizer

Then in Claude Code: `/token-optimizer`

> **Enable auto-update after installing.** Claude Code ships third-party marketplaces with auto-update off by default. `/plugin` → **Marketplaces** tab → select `alexgreensh-token-optimizer` → **Enable auto-update**. One-time, 10 seconds. > > After install, run `/token-optimizer` once to set up hooks. From there, everything runs automatically: compression, checkpoints, quality scoring, dashboard updates. You don't need to run any command again unless you want an audit.

> **Claude Code cloud sessions (claude.ai/code).** Cloud sessions load no plugins, not yours and not ones a repo enables, so the commands above don't reach them. Paste one setup script into your cloud environment instead and every session in it, any repo, runs the real Token Optimizer: > > 1. Run `python3 measure.py cloud-setup-script` (or ask Claude for "the Token Optimizer cloud setup script"). > 2. Paste the output into claude.ai/code → your environment → **Setup script**. > > The script fetches the `@alexgreensh/token-optimizer` npm package (the plugin itself, same version), checks its sha512 before running anything, and wires the same hooks as `hooks/hooks.json` into the VM. Nothing is committed to your repos, and it never fails your setup. State lives in the VM, so each cloud session starts fresh. [Details](https://alexgreensh.github.io/token-optimizer/install/claude-code-cloud/)

<details> <summary><b>Other platforms and install methods</b></summary>

**Codex:**

codex plugin marketplace add alexgreensh/token-optimizer

Then in the Codex TUI: `/plugins` and install Token Optimizer. See [`docs/codex.md`](docs/codex.md).

**OpenCode V2:** add `token-optimizer-opencode` to the `plugins` array in your `opencode.json` (V1 1.18.29+ uses `plugin`):

{ "$schema": "https://opencode.ai/config.json", "plugins": ["token-optimizer-opencode"] }

See [`opencode/README.md`](opencode/README.md).

**OpenClaw:**

openclaw plugins install github:alexgreensh/token-optimizer

See [`openclaw/README.md`](openclaw/README.md).

**Hermes:**

git clone https://github.com/alexgreensh/token-optimizer.git
token-optimizer/install.sh --hermes

Terminal output compression, cheaper re-reads, loop warnings, checkpoints that survive compaction, prior-session hints, and the dashboard, through Hermes's own plugin hooks. See [`hermes/README.md`](hermes/README.md).

**Pi Coding Agent:**

pi install git:github.com/alexgreensh/token-optimizer

Then run `/token-optimizer enable` to opt in. Native model costs are reported on the active branch; local tool archives and continuity markers require separate opt-in. See [`pi/README.md`](pi/README.md) for commands, privacy, update, uninstall, and limits.

**GitHub Copilot:**

git clone --depth 1 https://github.com/alexgreensh/token-optimizer.git
cd token-optimizer
bash install.sh --copilot

See [`docs/copilot.md`](docs/copilot.md).

**Cursor:**

git clone --depth 1 https://github.com/alexgreensh/token-optimizer.git
cd token-optimizer
bash install.sh --cursor

See [`docs/cursor.md`](docs/cursor.md).

**Google Antigravity:**

git clone --depth 1 https://github.com/alexgreensh/token-optimizer.git
cd token-optimizer
bash install.sh --antigravity

See [`docs/antigravity.md`](docs/antigravity.md).

**Grok Build (beta, contract-only):**

git clone --depth 1 https://github.com/alexgreensh/token-optimizer.git
cd token-optimizer
bash install.sh --grok

See [`docs/grok.md`](docs/grok.md).

**macOS/Linux script install (alternative to plugin):**

tmp="$(mktemp -d)"
release_json="$(curl -fsSL https://api.github.com/repos/alexgreensh/token-optimizer/releases/latest)"
tag="$(python3 -c 'import json,sys; print(json.load(sys.stdin)["tag_name"])' <<<"$release_json")"

What was scanned

The files the safety scan read, as written by the repository's author and shown as plain text. Lines with a finding are marked.

.claude-plugin/marketplace.json · Manifest · 76 lines
  1. {
  2. "$schema": "https://anthropic.com/claude-code/marketplace.schema.json",
  3. "name": "alexgreensh-token-optimizer",
  4. "owner": {
  5. "name": "Alex Greenshpun",
  6. "email": "alexgreensh@users.noreply.github.com"
  7. },
  8. "metadata": {
  9. "description": "Token optimization tools for Claude Code"
  10. },
  11. "plugins": [
  12. {
  13. "name": "token-optimizer",
  14. "source": "./",
  15. "description": "Audit, fix, and monitor Claude Code context window usage. Find the ghost tokens.",
  16. "version": "5.13.73",
  17. "author": {
  18. "name": "Alex Greenshpun",
  19. "url": "https://linkedin.com/in/alexgreensh"
  20. },
  21. "homepage": "https://github.com/alexgreensh/token-optimizer",
  22. "repository": "https://github.com/alexgreensh/token-optimizer",
  23. "license": "PolyForm-Noncommercial-1.0.0",
  24. "keywords": [
  25. "token",
  26. "optimization",
  27. "context",
  28. "audit",
  29. "cost",
  30. "coach"
  31. ],
  32. "category": "productivity"
  33. },
  34. {
  35. "name": "to-hook-probe",
  36. "source": "./cowork/to-hook-probe",
  37. "description": "[BETA] Token Optimizer Cowork hook-firing diagnostic. Install this FIRST in Claude Cowork to see which hooks actually fire on your build. Writes a marker + env dump you can read in-session. Part of the beta Cowork adapter.",
  38. "version": "0.1.0",
  39. "author": {
  40. "name": "Alex Greenshpun",
  41. "url": "https://linkedin.com/in/alexgreensh"
  42. },
  43. "homepage": "https://github.com/alexgreensh/token-optimizer",
  44. "repository": "https://github.com/alexgreensh/token-optimizer",
  45. "license": "PolyForm-Noncommercial-1.0.0",
  46. "keywords": [
  47. "cowork",
  48. "beta",
  49. "diagnostic",
  50. "hooks"
  51. ],
  52. "category": "productivity"
  53. },
  54. {
  55. "name": "token-optimizer-cowork",
  56. "source": "./cowork/token-optimizer",
  57. "description": "Token Optimizer for Claude Cowork \u2014 full skills + Cowork-native hooks (beta). Install THIS in Cowork; the standard token-optimizer entry is for desktop Claude Code.",
  58. "version": "5.13.73",
  59. "author": {
  60. "name": "Alex Greenshpun",
  61. "url": "https://linkedin.com/in/alexgreensh"
  62. },
  63. "homepage": "https://github.com/alexgreensh/token-optimizer",
  64. "repository": "https://github.com/alexgreensh/token-optimizer",
  65. "license": "PolyForm-Noncommercial-1.0.0",
  66. "keywords": [
  67. "token",
  68. "optimization",
  69. "context",
  70. "cowork",
  71. "beta"
  72. ],
  73. "category": "productivity"
  74. }
  75. ]
  76. }
README.md · README · 831 lines
  1. <p align="center">
  2. <b>English</b> ·
  3. <a href="README.ko.md">한국어</a> ·
  4. <a href="README.zh-CN.md">简体中文</a> ·
  5. <a href="README.ja.md">日本語</a>
  6. </p>
  7. <p align="center">
  8. <img src="skills/token-optimizer/assets/logo.svg" alt="Token Optimizer" width="780">
  9. </p>
  10. <p align="center">
  11. <a href="https://github.com/alexgreensh/token-optimizer/releases/latest"><img src="https://badgen.net/github/release/alexgreensh/token-optimizer?label=version&color=green" alt="Latest stable version"></a>
  12. <a href="https://github.com/alexgreensh/token-optimizer/commits/main"><img src="https://badgen.net/github/last-commit/alexgreensh/token-optimizer?label=last%20commit" alt="Last commit"></a>
  13. </p>
  14. <p align="center">
  15. <img src="https://img.shields.io/badge/cuts%20context%20waste-3fb950" alt="Cuts context waste">
  16. <img src="https://img.shields.io/badge/survives%20compaction-checkpoint%20%2B%20restore-58a6ff" alt="Survives compaction">
  17. <img src="https://img.shields.io/badge/saves%20real%20%24-every%20session-2ea043" alt="Saves real dollars every session">
  18. <img src="https://img.shields.io/badge/live%20dashboard-tokens%20%2B%20%24%20%2B%20turns-8B5CF6?logo=chartdotjs&logoColor=white" alt="Live dashboard">
  19. <img src="https://img.shields.io/badge/context%20quality-live%20score-blue" alt="Live context quality score">
  20. <img src="https://img.shields.io/badge/tests-passing-brightgreen" alt="Tests passing">
  21. </p>
  22. <p align="center">
  23. <img src="https://img.shields.io/badge/dependencies-zero-brightgreen" alt="Zero Dependencies">
  24. <img src="https://img.shields.io/badge/telemetry-none-brightgreen" alt="Zero Telemetry">
  25. <img src="https://img.shields.io/badge/python-3.9+-blue" alt="Python 3.9+">
  26. <img src="https://img.shields.io/badge/platform-macOS%20%7C%20Linux%20%7C%20Windows-lightgrey" alt="Platform">
  27. <a href="https://github.com/alexgreensh/token-optimizer/blob/main/LICENSE"><img src="https://img.shields.io/badge/license-PolyForm%20Noncommercial-blue.svg" alt="License: PolyForm Noncommercial"></a>
  28. <a href="https://github.com/alexgreensh/token-optimizer/stargazers"><img src="https://badgen.net/github/stars/alexgreensh/token-optimizer" alt="GitHub Stars"></a>
  29. <a href="https://github.com/alexgreensh/token-optimizer/commits/main"><img src="https://badgen.net/github/commits/alexgreensh/token-optimizer?label=commits" alt="Commits"></a>
  30. <a href="https://linkedin.com/in/alexgreensh"><img src="https://img.shields.io/badge/LinkedIn-Connect-0A66C2?logo=linkedin&logoColor=white" alt="Connect on LinkedIn"></a>
  31. </p>
  32. <p align="center">
  33. <a href="https://trendshift.io/repositories/25856?utm_source=repository-badge&amp;utm_medium=badge&amp;utm_campaign=badge-repository-25856"><img src="https://trendshift.io/api/badge/repositories/25856" alt="alexgreensh/token-optimizer | Trendshift" width="250" height="55"></a>
  34. </p>
  35. <p align="center">
  36. <a href="https://github.com/sponsors/alexgreensh"><img src="https://img.shields.io/badge/%E2%99%A5%20Support%20this%20project%20to%20keep%20it%20open%20source-ea4aaa?style=for-the-badge&logo=githubsponsors&logoColor=white" alt="Support this project to keep it open source"></a>
  37. </p>
  38. <h2 align="center">Cut the tokens you waste. Keep the work you'd lose.</h2>
  39. <p align="center"><em>It runs automatically in the background. You keep working as usual. Run the audit when you want the full picture.</em></p>
  40. <p align="center">
  41. <a href="https://alexgreensh.github.io/token-optimizer/"><img src="https://img.shields.io/badge/%F0%9F%93%96%20Read%20the%20Docs-alexgreensh.github.io%2Ftoken--optimizer-e85329?style=for-the-badge&logoColor=white" alt="Read the documentation"></a>
  42. <a href="https://alexgreensh.github.io/token-optimizer/start/quickstart/"><img src="https://img.shields.io/badge/Quickstart-2%20minutes-3fb950?style=for-the-badge" alt="Quickstart in 2 minutes"></a>
  43. </p>
  44. ## The 30-Second Version
  45. Token Optimizer cuts the tokens your AI coding assistant wastes, keeps your work alive across sessions and compactions, and shows you where every dollar went on a live dashboard. **Most of it runs automatically. You install it, run the audit once, and the hooks do the rest.**
  46. **Why not just use Headroom or RTK?** They compress command output, which covers 15-25% of your context. Token Optimizer covers that plus the other 75%: bloated configs, unused skills, stale memory, compaction loss, model misrouting, behavioral waste. Every saving is cache-safe and measured. The dashboard updates after every session, automatically.
  47. Works on the AI coding tools below, with a [Devin output-filter beta](https://github.com/alexgreensh/token-optimizer/blob/main/devin/README.md) (manual skill, or `install.sh --devin --auto` for automatic compression):
  48. <p align="center">
  49. <a href="https://alexgreensh.github.io/token-optimizer/platforms/claude-code/"><img src="skills/token-optimizer/assets/platforms/claude-code.svg" width="96" alt="Claude Code"></a>
  50. <a href="https://alexgreensh.github.io/token-optimizer/platforms/codex/"><img src="skills/token-optimizer/assets/platforms/codex.svg" width="96" alt="Codex"></a>
  51. <a href="https://alexgreensh.github.io/token-optimizer/platforms/opencode/"><img src="skills/token-optimizer/assets/platforms/opencode.svg" width="96" alt="OpenCode"></a>
  52. <a href="https://alexgreensh.github.io/token-optimizer/platforms/openclaw/"><img src="skills/token-optimizer/assets/platforms/openclaw.svg" width="96" alt="OpenClaw"></a>
  53. <a href="https://alexgreensh.github.io/token-optimizer/platforms/hermes/"><img src="skills/token-optimizer/assets/platforms/hermes.svg" width="96" alt="Hermes"></a>
  54. <a href="https://alexgreensh.github.io/token-optimizer/platforms/copilot/"><img src="skills/token-optimizer/assets/platforms/copilot.svg" width="96" alt="GitHub Copilot"></a>
  55. <picture><source media="(max-width: 767px)" srcset="skills/token-optimizer/assets/platforms/row-break-none.svg"><img src="skills/token-optimizer/assets/platforms/row-break.svg" alt=""></picture>
  56. <a href="https://alexgreensh.github.io/token-optimizer/platforms/pi/"><img src="skills/token-optimizer/assets/platforms/pi.svg" width="96" alt="Pi Coding Agent"></a>
  57. <a href="https://alexgreensh.github.io/token-optimizer/platforms/cursor/"><img src="skills/token-optimizer/assets/platforms/cursor.svg" width="96" alt="Cursor"></a>
  58. <a href="https://alexgreensh.github.io/token-optimizer/platforms/antigravity/"><img src="skills/token-optimizer/assets/platforms/antigravity.svg" width="96" alt="Google Antigravity (beta)"></a>
  59. <a href="https://alexgreensh.github.io/token-optimizer/platforms/grok/"><img src="skills/token-optimizer/assets/platforms/grok.svg" width="96" alt="Grok Build (beta)"></a>
  60. <a href="https://alexgreensh.github.io/token-optimizer/platforms/devin/"><img src="skills/token-optimizer/assets/platforms/devin.svg" width="96" alt="Devin (beta)"></a>
  61. </p>
  62. ## What You Get
  63. **Runs automatically, every session, you do nothing:**
  64. - 🔄 **Smart Compaction**: checkpoints before auto-compact, restores after
  65. - 🗄️ **Session Continuity**: cross-session hints, cold-resume, checkpoint scoring
  66. - 📦 **Active Compression**: 9 features, all on by default (delta diffs, skeletons, bash/search compression, lean-output nudges, quality nudges, loop detection, activity mode, decision extraction)
  67. - 📊 **Quality Scoring**: 7 signals, real-time, letter grades S–F
  68. - 🗃️ **Session Database**: SQLite, 15 tables, full audit trail, zero network
  69. - 🔍 **Progressive Disclosure**: large outputs archived, expand on demand
  70. - 🧠 **Context Intel Digest**: post-compaction re-orientation without re-reads
  71. - 🔀 **Model Routing Nudges**: steers to the right tier for the task
  72. **When you ask for it:**
  73. - 🩺 `/token-optimizer`: full audit with guided fixes
  74. - 📈 `/token-coach`: 30-day trend analysis with specific fixes
  75. - ⚡ `quick`: 10-second health check
  76. - 🎯 `route`: the model and effort this task actually needs, before you spend
  77. - 🔧 `doctor`: installation check
  78. - 💰 `savings`: dollar savings report
  79. - 📋 `report`: per-component token breakdown
  80. - 🌐 `dashboard`: open the full dashboard
  81. - 📝 `memory-review`: MEMORY.md structural audit
  82. - 📂 `expand`: retrieve archived tool result
  83. - 🔙 `resume-lean`: reopen a cold session
  84. Install, run `/token-optimizer` once, everything else runs automatically.
  85. ## A Status Bar Inside the Claude Desktop App
  86. <p align="center">
  87. <a href="https://token-optimizer.dev/media/token-optimizer-desktop-status-bar.mp4"><img src="https://raw.githubusercontent.com/alexgreensh/token-optimizer/main/docs-site/public/media/token-optimizer-desktop-status-bar.gif" alt="The Token Optimizer status bar inside the Claude desktop app: Clawd reacts to the session while quality, context, cache and usage limits update live" width="720"></a>
  88. <br><sub>A 30-second preview. <a href="https://token-optimizer.dev/media/token-optimizer-desktop-status-bar.mp4">Watch the full film with sound</a> (50 seconds).</sub>
  89. </p>
  90. Claude Code now lets a plugin add its own piece of interface, called a mod. Token Optimizer uses one to put your session's health right above the prompt: a quality grade, how full the context is, how long the cache stays warm, and both usage limits. One plain sentence tells you what needs attention, and one button fixes it. Clawd, the small orange mascot, acts out what the session is doing, so you can tell at a glance whether Claude is reading, editing, waiting for your OK, or done.
  91. It comes with the plugin, nothing extra to install. [Everything the bar does](#desktop-status-bar).
  92. ## Install
  93. **Claude Code (recommended):**
  94. ```
  95. /plugin marketplace add alexgreensh/token-optimizer
  96. /plugin install token-optimizer@alexgreensh-token-optimizer
  97. ```
  98. Then in Claude Code: `/token-optimizer`
  99. > **Enable auto-update after installing.** Claude Code ships third-party marketplaces with auto-update off by default. `/plugin` → **Marketplaces** tab → select `alexgreensh-token-optimizer` → **Enable auto-update**. One-time, 10 seconds.
  100. >
  101. > After install, run `/token-optimizer` once to set up hooks. From there, everything runs automatically: compression, checkpoints, quality scoring, dashboard updates. You don't need to run any command again unless you want an audit.
  102. > **Claude Code cloud sessions (claude.ai/code).** Cloud sessions load no plugins, not yours and not ones a repo enables, so the commands above don't reach them. Paste one setup script into your cloud environment instead and every session in it, any repo, runs the real Token Optimizer:
  103. >
  104. > 1. Run `python3 measure.py cloud-setup-script` (or ask Claude for "the Token Optimizer cloud setup script").
  105. > 2. Paste the output into claude.ai/code → your environment → **Setup script**.
  106. >
  107. > The script fetches the `@alexgreensh/token-optimizer` npm package (the plugin itself, same version), checks its sha512 before running anything, and wires the same hooks as `hooks/hooks.json` into the VM. Nothing is committed to your repos, and it never fails your setup. State lives in the VM, so each cloud session starts fresh. [Details](https://alexgreensh.github.io/token-optimizer/install/claude-code-cloud/)
  108. <details>
  109. <summary><b>Other platforms and install methods</b></summary>
  110. **Codex:**
  111. ```bash
  112. codex plugin marketplace add alexgreensh/token-optimizer
  113. ```
  114. Then in the Codex TUI: `/plugins` and install Token Optimizer. See [`docs/codex.md`](docs/codex.md).
  115. **OpenCode V2:** add `token-optimizer-opencode` to the `plugins` array in your `opencode.json` (V1 1.18.29+ uses `plugin`):
  116. ```jsonc
  117. { "$schema": "https://opencode.ai/config.json", "plugins": ["token-optimizer-opencode"] }
  118. ```
  119. See [`opencode/README.md`](opencode/README.md).
  120. **OpenClaw:**
  121. ```bash
  122. openclaw plugins install github:alexgreensh/token-optimizer
  123. ```
  124. See [`openclaw/README.md`](openclaw/README.md).
  125. **Hermes:**
  126. ```bash
  127. git clone https://github.com/alexgreensh/token-optimizer.git
  128. token-optimizer/install.sh --hermes
  129. ```
  130. Terminal output compression, cheaper re-reads, loop warnings, checkpoints that survive compaction, prior-session hints, and the dashboard, through Hermes's own plugin hooks. See [`hermes/README.md`](hermes/README.md).
  131. **Pi Coding Agent:**
  132. ```bash
  133. pi install git:github.com/alexgreensh/token-optimizer
  134. ```
  135. Then run `/token-optimizer enable` to opt in. Native model costs are reported on the active branch; local tool archives and continuity markers require separate opt-in. See [`pi/README.md`](pi/README.md) for commands, privacy, update, uninstall, and limits.
  136. **GitHub Copilot:**
  137. ```bash
  138. git clone --depth 1 https://github.com/alexgreensh/token-optimizer.git
  139. cd token-optimizer
  140. bash install.sh --copilot
  141. ```
  142. See [`docs/copilot.md`](docs/copilot.md).
  143. **Cursor:**
  144. ```bash
  145. git clone --depth 1 https://github.com/alexgreensh/token-optimizer.git
  146. cd token-optimizer
  147. bash install.sh --cursor
  148. ```
  149. See [`docs/cursor.md`](docs/cursor.md).
  150. **Google Antigravity:**
  151. ```bash
  152. git clone --depth 1 https://github.com/alexgreensh/token-optimizer.git
  153. cd token-optimizer
  154. bash install.sh --antigravity
  155. ```
  156. See [`docs/antigravity.md`](docs/antigravity.md).
  157. **Grok Build (beta, contract-only):**
  158. ```bash
  159. git clone --depth 1 https://github.com/alexgreensh/token-optimizer.git
  160. cd token-optimizer
  161. bash install.sh --grok
  162. ```
  163. See [`docs/grok.md`](docs/grok.md).
  164. **macOS/Linux script install (alternative to plugin):**
  165. ```bash
  166. tmp="$(mktemp -d)"
  167. release_json="$(curl -fsSL https://api.github.com/repos/alexgreensh/token-optimizer/releases/latest)"
  168. tag="$(python3 -c 'import json,sys; print(json.load(sys.stdin)["tag_name"])' <<<"$release_json")"
  169. git clone --branch "$tag" --depth 1 https://github.com/alexgreensh/token-optimizer.git ~/.claude/token-optimizer
  170. bash ~/.claude/token-optimizer/install.sh
  171. rm -rf "$tmp"
  172. ```
  173. **Windows users:** Use the plugin install only. Do not run `install.sh` on Windows. If you hit `EBUSY` errors, close all Claude Code and Git Bash windows, kill lingering `git.exe` processes, delete `C:\Users\<you>\.claude\token-optimizer` and `C:\Users\<you>\.claude\plugins\marketplaces\alexgreensh-token-optimizer`, then retry.
  174. **If `install.sh` fails with `$'\r': command not found`** (a clone made before LF line endings were enforced converted the script to CRLF), strip the carriage returns once and re-run — the repo now ships a `.gitattributes` that prevents this on fresh clones:
  175. ```bash
  176. sed -i 's/\r$//' ~/.claude/token-optimizer/install.sh
  177. # already have the repo? re-normalize line endings in place:
  178. git -C ~/.claude/token-optimizer add --renormalize . && git -C ~/.claude/token-optimizer checkout -- .
  179. ```
  180. </details>
  181. <details>
  182. <summary>Uninstall</summary>
  183. Token Optimizer is additive and reversible. Every runtime has a clean uninstall
  184. that removes only what we installed, leaving your own hooks, config, and session
  185. data intact. Full per-runtime steps live in **[docs/uninstall.md](docs/uninstall.md)**.
  186. Quickest path (Claude Code plugin install):
  187. ```
  188. /plugin uninstall token-optimizer@alexgreensh-token-optimizer
  189. ```
  190. </details>
  191. ## How It's Different
  192. Most token tools compress command output. That covers 15-25% of your context. The other 75% goes untouched.
  193. **Compression coverage.** Headroom, RTK, and JFrog Boost compress bash and command output. Token Optimizer compresses eight surfaces of the output stack. Status: 🟢 supported, 🟡 partial, 🔴 not supported.
  194. | Compression surface | Token Optimizer | Headroom | RTK | Boost |
  195. |---|---|---|---|---|
  196. | **Bash / command output** (git, tests, lint, build, logs) | 🟢 89 built-in allowlist entries and 25 registered output handlers; 564 → 115 tokens on a pytest run | 🟢 SmartCrusher, CodeCompressor, Kompress-v2 | 🟢 100+ filters | 🟢 Command-aware filters |
  197. | **Search / grep output** | 🟢 Top hits plus a count; 500 lines → 20 | 🔴 | 🔴 | — |
  198. | **Tabular / JSON output** (jq, yq, csvtool, mlr) | 🟢 Value-preserving columnar | 🟢 SmartCrusher | 🔴 | — |
  199. | **File re-reads, delta mode** | 🟢 Diff only; 2,000-token re-read → ~50 | 🔴 | 🔴 | — |
  200. | **File re-reads, structure map** | 🟢 Skeleton of signatures and imports; 720KB → 250 tokens | 🔴 | 🔴 | — |
  201. | **Large tool results** (over 4K chars) | 🟢 Archived to disk, expandable on demand | 🔴 | 🔴 | — |
  202. | **Model output verbosity** | 🟢 Lean-output nudge, cache-safe (savings estimated, not metered) | 🔴 | 🔴 | 🔴 |
  203. | **Structural context** (configs, skills, MCP, memory) | 🟢 Per-component audit, each source scored | 🔴 | 🔴 | 🔴 |
  204. RTK and Boost reach the first surface. Headroom reaches the first and the third. Token Optimizer covers all eight, then keeps going into what happens around compression:
  205. - **Three kinds of waste, not one.** Structural (bloated configs, unused skills, stale memory), runtime (verbose output, re-reads), and behavioral (model misrouting, cache expiry, retry loops). [How each works →](https://alexgreensh.github.io/token-optimizer/features/active-compression/)
  206. - **Savings survive compaction.** Checkpoints before auto-compact, restores after. Without this, compression savings vanish the moment compaction fires.
  207. - **Measures whether it helped.** Before/after token deltas, dollar savings across four pricing tiers, quality scores that track degradation. Not just "we compressed stuff."
  208. - **Zero baseline overhead.** External process, no always-on instructions in your context, no MCP server, no dependencies, no telemetry.
  209. <p align="center">
  210. <img src="skills/token-optimizer/assets/automated-flow.svg" alt="How Token Optimizer works automatically every session" width="900">
  211. </p>
  212. These ten rows are the ones where Token Optimizer is the only 🟢 in the row: what survives beyond compression, and the waste no compressor looks for. Compression mechanics, cost and safety, and the two rows where we lose are behind the fold.
  213. | | Token Optimizer | Headroom | RTK | Boost | context-mode | `/context` |
  214. |---|---|---|---|---|---|---|
  215. | Session continuity | 🟢 Progressive checkpoints before compaction + restore after, cross-session hints, cold-resume, tool-output digest | 🔴 | 🔴 | — | 🟡 Session guide only | 🔴 |
  216. | Structural waste audit | 🟢 Deep per-component (CLAUDE.md, skills, MCP, memory) | 🔴 | 🔴 | 🔴 | 🔴 | 🟡 Summary only |
  217. | CLAUDE.md and MEMORY.md health | 🟢 8 auditors + attention-curve scoring | 🔴 | 🔴 | 🔴 | 🔴 | 🔴 |
  218. | Model routing and behavioral coaching | 🟢 12 detectors, subagent cost breakdown, anti-patterns | 🔴 | 🔴 | — | 🔴 | 🟡 Basic suggestions |
  219. | Historical trend analysis | 🟢 30-day trends, quality/cost/cache/duration correlation, model-switch detection | 🔴 | 🔴 | — | 🔴 | 🔴 |
  220. | Context quality scoring | 🟢 7-signal quality score with grades | 🔴 | 🔴 | — | 🔴 | 🟡 Capacity % only |
  221. | Loop and spin detection | 🟢 Catches behavioral loops before they burn | 🔴 | 🔴 | — | 🔴 | 🔴 |
  222. | Measures if compression helped | 🟢 Local telemetry, before/after tokens, dollar savings | 🔴 | 🟡 `rtk gain` (token counts only) | 🟡 `boost report`, vendor-side | 🔴 | 🔴 |
  223. | Fleet-level cross-agent analysis | 🟢 | 🔴 | 🔴 | — | 🔴 | 🔴 |
  224. <details>
  225. <summary><b>Show the remaining 13 rows</b> (compression mechanics; cost, safety and reach; tuning, proof and install)</summary>
  226. | | Token Optimizer | Headroom | RTK | Boost | context-mode | `/context` |
  227. |---|---|---|---|---|---|---|
  228. | No command rewriting required | 🟢 Hook-wired, fires automatically | 🟢 Transparent proxy | 🟢 Hook-based rewrite | 🟢 `boost init` wires supported agents; terminal use can be prefixed | 🟡 Automatic on hook-capable platforms | N/A Native command |
  229. | Full original recoverable | 🟢 Raw archived before compression, `expand` retrieves it; failures never compressed | 🟢 Reversible retrieval | 🟡 Full output saved on command failure | 🟢 Vendor documents command-output recovery | — | N/A Does not compress |
  230. | Register a custom command filter | 🟢 Custom TOML filters (`command-filters.toml`, additive + exclude, safety-gated) | — | 🟢 Custom TOML filters | 🟢 TOML filters | — | N/A |
  231. | User-tunable configuration | 🟢 137 code-referenced `TOKEN_OPTIMIZER_*` names, explicitly split into user-facing and internal controls; additive allowlist; `.contextignore` | 🟢 Documented configuration | 🟢 `config.toml` and environment controls | 🟢 Filter TOML | 🟢 Documented configuration | N/A |
  232. | Cache-safe | 🟢 Never modifies existing context prefix | 🟡 Proxy mode rewrites in-flight | 🟢 Pre-shell only | 🟢 Pre-shell only | 🟡 MCP overhead | 🟢 |
  233. | Zero baseline context overhead | 🟢 External process, no context injection | 🔴 Injects instructions | 🟢 Shell-level only | 🟢 Shell-level only | 🔴 MCP server overhead | 🟢 Native |
  234. | Zero runtime dependencies | 🟢 Pure stdlib (Python/TypeScript) | 🟡 Python + Rust + optional model | 🟢 Single Rust binary | 🟢 Single binary | 🟡 SQLite adapter required | 🟢 N/A |
  235. | Zero telemetry | 🟢 Nothing leaves the machine | 🟡 `HEADROOM_TELEMETRY` opt-in, off by default | 🟡 Opt-in | 🔴 Collects commands invoked, command arguments, exit codes, duration, CI attributes, IP | 🟡 Varies | 🟢 |
  236. | Multi-platform | 🟢 Claude Code, VS Code, Codex, OpenClaw, OpenCode, Hermes, Copilot, Cursor, Antigravity; Cowork experimental (unverified on a live build) | 🟢 Claude Code, Cursor, Codex, Aider, Copilot | 🟢 15 integrations | 🟡 Cursor, Claude Code, Copilot, Codex CLI | 🟢 17 integrations | 🔴 Claude Code only |
  237. | Per-task model and effort advice | 🟢 `route` sizes the task before you spend | — | — | — | — | — |
  238. | Keep-Warm (cache TTL refresh) | 🟡 Automatic pings paused; manual desktop refresh available | 🔴 | 🔴 | — | 🔴 | 🔴 |
  239. | End-to-end task-outcome benchmark | 🟡 Controlled A/B on 7 real tasks (output tokens) + measured real-session with/without savings; pass-rate study not yet run | — | — | 🟢 Vendor reports Terminal-Bench 2.0 with the same pass rate and ~12% lower cost | — | N/A |
  240. | Signed and checksum-verified install | 🟢 `CHECKSUMS.sha256` per release, verified at install, CI-enforced | — | — | 🔴 Installer verifies neither a checksum nor a signature | — | N/A |
  241. </details>
  242. An em dash means the capability was not verified from first-party material in the 2026-07-26 source audit (Boost's first-party material re-verified 2026-08-26); it is not a claim that the capability is absent. This page scores command-output compression only; JFrog Boost's separately-shipped BoostGraph, a local code-map its agent queries on request, is a different capability and out of scope here. Boost's "pre-shell" and "shell-level" cells are sourced from its documented wrapper form (`boost <command>`); the mechanism `boost init` uses to wire an agent is not described in its first-party material. Token Optimizer's compression claims are tested against real sessions and an 87-fixture suite you can run yourself. **[Full benchmark methodology and results →](BENCHMARK.md)** Compaction timing now has a separate, session-grouped evaluator for advisory precision, future context need, and checkpoint/cold-resume coverage. **[Run the compaction timing evaluator →](docs/evals/README.md)**
  243. <p align="center">
  244. <a href="https://alexgreensh.github.io/token-optimizer/reference/comparison/"><img src="https://img.shields.io/badge/Read%20the%20official%20full%20comparison-in%20our%20docs-0b7285?style=for-the-badge" alt="Read the official full comparison page in our documentation"></a>
  245. </p>
  246. ## The Dashboard
  247. ![Token Optimizer Dashboard](skills/token-optimizer/assets/dashboard-demo.gif)
  248. One HTML page, auto-regenerates after every session via the SessionEnd hook, no manual trigger needed. The bookmarkable URL `http://localhost:24842/token-optimizer` is opt-in: it only works after you run `python3 skills/token-optimizer/scripts/measure.py setup-daemon` to start the local server. Until then, open the file path that `measure.py dashboard --user` prints on the ` Dashboard: ` line.
  249. Per-turn token breakdowns, cost across four pricing tiers, cache analysis with TTL mix and hit rate, quality scores overlaid on every session, subagent cost breakdown, savings tracker with metered and estimated tiers kept apart. Zero setup after install. [Full dashboard docs →](https://alexgreensh.github.io/token-optimizer/features/dashboard/)
  250. ## What It Saves
  251. **About $597.72 in all-in API-rate-equivalent value, a priced subtotal for the 30-day snapshot ending 2026-11-01.** The components keep their evidence labels:
  252. - **Logged actions: $78.59.** Metered token changes across 4,945 events, valued at API rates.
  253. - **Model routing: $422.88.** Estimated comparison using a frozen pre-tool mix and fixed reference model cards.
  254. - **Repeat reads avoided: $60.69.** Modeled from observed turns until the next real compaction, using known per-turn rates.
  255. - **Other estimates: $35.56.** All eight estimate categories are listed in the [methodology](BENCHMARK.md), including zero-valued categories.
  256. **Pricing is incomplete.** Unpriced repeat-read credits and debits are excluded; the complete net dollar value is unavailable. The four components add up to the total.
  257. The same snapshot's token card shows **24.80M tokens saved** (9,831,072 metered + 14,966,284 estimated), about **8% of spent + saved**: 286.22M spent + 24.80M saved = 311.02M. Its scope is logged tokens plus MCP caps, hint-served reads, verbosity nudges and lean resumes (snapshot taken under v5.13.63 rules; checkpoint restores have since moved to the estimated tier). Modeled repeat-read tokens and routing are outside that token-card total; the dollar subtotal has the broader scope above.
  258. <p align="center">
  259. <img src="skills/token-optimizer/assets/real-savings.svg" alt="Snapshot 2026-11-01: $597.72 priced API-rate-equivalent subtotal with incomplete pricing; 24.80M tokens saved, about 8% of spent plus saved. Logged actions $78.59, estimated routing $422.88, modeled rereads $60.69, other estimates $35.56." width="900">
  260. </p>
  261. Based on 1,644 main work sessions and 3 delegated sessions. Your dashboard computes the report from your own history; results differ. [Full methodology →](BENCHMARK.md)
  262. <p align="center">
  263. <img src="skills/token-optimizer/assets/user-profiles.svg" alt="What happens inside a 1M session" width="800">
  264. </p>
  265. ## Active Compression
  266. Nine features that actively reduce context, all on by default, all automatic, all toggleable from the dashboard or CLI.
  267. Under the hood, **PreToolUse hooks** intercept every Read call before it enters your context. If the file was already read, only the diff comes back. If it's a code file re-read, a structural skeleton replaces the full content. **PostToolUse hooks** compress CLI command output, archive the full original to disk, and log a compression event to SQLite. Nothing is lost, and everything is retrievable. **You do nothing. The hooks handle it all.**
  268. ![Active Compression overview](skills/token-optimizer/assets/active-compression-hero.svg)
  269. | Feature | What it does | Savings |
  270. |---|---|---|
  271. | Delta Mode | Re-reads return only what changed | ~20% on re-reads |
  272. | Structure Map | Unchanged file re-reads return a structural skeleton | ~30% (up to 99% per file) |
  273. | Bash Compression | CLI output condensed to essentials | ~10% |
  274. | Search Compression | grep/web results condensed to top hits + counts | ~15% |
  275. | Lean-Output Nudges | Steers model to concise output when context fills | Estimated 10-15%; not counterfactually measured |
  276. | Quality Nudges | Warns when context quality drops | Prevents compaction loss |
  277. | Loop Detection | Catches retry loops before they burn tokens | Measured per loop |
  278. | Activity Mode | Adapts compaction to your session phase | Prevents decision loss |
  279. | Decision Extraction | Preserves decisions across compactions | Prevents decision drift |
  280. Toggle from the dashboard Manage tab, CLI (`measure.py v5 enable|disable <feature>`), or env vars. The `v5` verb is a legacy command name that controls current features.
  281. [Read how each feature works →](https://alexgreensh.github.io/token-optimizer/features/active-compression/)
  282. <details>
  283. <summary><b>Per-feature details</b></summary>
  284. ### Delta Mode
  285. When the AI re-reads a file after editing it, the Read call returns only the diff. 65%+ of Read calls in real sessions are re-reads. A 2,000-token file re-read becomes a 50-token diff.
  286. ![Delta Mode: smart re-reads](skills/token-optimizer/assets/delta-mode.svg)
  287. Disable: `TOKEN_OPTIMIZER_READ_CACHE_DELTA=0`
  288. ### Structure Map
  289. When Claude re-reads a code file it already saw, the Read call is blocked and replaced with a structural summary: function signatures, class hierarchies, imports. A 720KB Python file (180,000 tokens) becomes a 250-token skeleton.
  290. Disable: `TOKEN_OPTIMIZER_READ_CACHE_MODE=shadow`
  291. ### First-Read Skeleton
  292. As of v5.12, the target file is always served in full on the first read; savings come from periphery skeletons (one-hop relationships, PageRank-ordered) injected alongside the target, not from withholding the target itself. This eliminates the re-read flip-flop that demoted cohorts under the old tripwire.
  293. This applies to **code only**. Markdown and other prose are **not** skeletoned on first read (as of v5.11.27): a headings-only outline drops load-bearing prose, so docs always come back complete. A runtime tripwire auto-demotes a code cohort back to measure-only if its live edit-rate climbs.
  294. Disable serving (keep measurement): `TOKEN_OPTIMIZER_FIRST_READ_ACTIVE=0`. Disable entirely: `TOKEN_OPTIMIZER_FIRST_READ_SHADOW=0`. Both are also visible and toggleable via `measure.py v5 status` and the dashboard Manage tab.
  295. ### Bash Output Compression
  296. Rewrites common CLI commands to return compressed summaries. Covers lint, log tails, tree, docker pull, long listings, build output, and test runners. A 564-token pytest output becomes 115 tokens.
  297. Build and test commands (gcc, cargo, npm, make, go, mvn, and more) are compressed PostToolUse even when they are not read-only: the command already ran, so the hook collapses repetition in the captured output while preserving every distinct error line, exit status, and summary. The full original is archived behind an expand pointer.
  298. ![Bash Output Compression](skills/token-optimizer/assets/bash-compression.svg)
  299. Disable: `TOKEN_OPTIMIZER_BASH_COMPRESS=0`
  300. ### Custom Command Filters (TOML)
  301. You can extend the bash-compression allowlist with your own read-only commands
  302. and exclude commands you don't want compressed, via a TOML file at
  303. `<runtime-home>/token-optimizer/command-filters.toml` (override the path with
  304. `TOKEN_OPTIMIZER_COMMAND_FILTERS`).
  305. ```toml
  306. # Add a read-only command to an existing compression handler.
  307. # handler must be one of the built-in pattern names (pytest, lint, ls, etc.).
  308. [filters.add]
  309. my-tests = { command = "cargo test", handler = "pytest" }
  310. my-lint = { command = "ruff check", handler = "lint" }
  311. # Exclude commands from compression (glob patterns supported).
  312. [filters.exclude]
  313. commands = ["git status", "ls -la"]
  314. ```
  315. Custom additions are checked when loaded and again when matched. The guard rejects known destructive commands, dangerous wildcard matches, shell substitutions and redirects. Its lists are bounded, so it is not a general shell-safety proof. Built-in detection runs first, so a user `add` extends the set rather than replacing a built-in handler.
  316. ### Search Result Compression
  317. Long grep and rg results show up to three matches per file for the first 20 files, with omissions identified. If no verified archive is available, search output stays whole.
  318. Disable: `TOKEN_OPTIMIZER_BASH_COMPRESS=0` (search compression is part of bash compression; it has no separate switch)
  319. ### Lean-Output Nudges
  320. At 25–74% fill, a lean-output nudge also requires session efficiency below 75. At 75–89% fill, the stronger nudge uses fill alone. At 90% or above, no nudge is injected. Savings are estimated from later output when available, not measured against an observable counterfactual. The nudge is cache-safe: it does not modify the existing prefix.
  321. No on/off switch today (not a `v5` feature). Tune the trigger point with `TOKEN_OPTIMIZER_VERBOSITY_MIN_FILL` (default `25`).
  322. ### Quality Nudges
  323. Watches context quality in real time. When the score drops more than 15 points or crosses below 60, an inline note enters the context. Claude sees it on the next turn and surfaces the warning or adjusts behavior. Cooldown of 5 minutes, max 3 per session.
  324. Disable: `TOKEN_OPTIMIZER_QUALITY_NUDGES=0`
  325. ### Loop Detection
  326. Catches the AI getting stuck on a retry loop. Compares the last 4 user messages and last 5 tool results for similarity. Fires at confidence ≥0.7, session cap of 2 notes. Savings measured from actual loop turn content.
  327. Also catches the edit-compile-fail cycle: when the same command fails 3 times in a row with different output, a nudge suggests changing approach instead of re-running. Tunable with `TOKEN_OPTIMIZER_FAIL_STREAK_THRESHOLD` (default `3`).
  328. Also catches inline-script repeats: when a command with a heredoc body >= 300 chars has been run 8 times in a session, a nudge suggests saving the script to a file and running that instead, so the body is not re-sent as input tokens every turn. Tunable with `TOKEN_OPTIMIZER_INLINE_SCRIPT_THRESHOLD` (default `8`).
  329. Disable: `TOKEN_OPTIMIZER_LOOP_DETECTION=0`
  330. ### UserPromptSubmit Hook
  331. Every prompt fires the `UserPromptSubmit` hook, which runs the per-turn work: prompt-continuity hint, verbosity steer, quality-cache warn tick, and (in remote/container/Cowork and Codex sessions) the once-per-session ensure-health, forced cache warm, and compact-restore pointer. These six subcommands share one `measure.py` import inside a single dispatcher (`hooks/userpromptsubmit_runner.py`), so one prompt spawns three processes, not eighteen. The dispatcher uses one shared deadline (18s, 2s margin under the 20s hooks.json timeout) with fair-share budgeting across subcommands, and buffers all stdout through a single emitter for controlled, host-consumable output.
  332. Disable the entire `UserPromptSubmit` path: `TOKEN_OPTIMIZER_HOOKS_USERPROMPTSUBMIT=0`. Checked before `measure.py` is imported, so the opt-out costs zero per prompt. The other hook events (PreToolUse, PostToolUse, SessionStart, Stop, etc.) are unaffected.
  333. ### Activity Mode Detection
  334. Classifies your session into one of five modes (code, debug, review, infra, general) using the last 10 tool calls. The mode feeds into compaction guidance so PRESERVE/DROP priorities adapt to what you're doing.
  335. ### Decision Extraction
  336. Detects decision statements in real-time from tool outputs and stores them in the session database. At compaction time, these decisions are injected as CRITICAL DECISIONS that the compaction summary must preserve verbatim. Capped at 10 per session.
  337. ### Measuring real savings
  338. All compression features log to a local SQLite table. Nothing leaves your machine.
  339. ```bash
  340. python3 measure.py compression-stats --days 30
  341. ```
  342. </details>
  343. ## The Session Database
  344. Everything Token Optimizer does is backed by two local SQLite databases. Nothing leaves your machine. Zero network calls.
  345. <p align="center">
  346. <img src="skills/token-optimizer/assets/session-database-flow.svg" alt="Session database flow: tool calls compressed, archived, logged to SQLite, retrievable" width="900">
  347. </p>
  348. **Per-session DB** (`~/.claude/token-optimizer/snapshots/session-store/<session>.db`) holds 8 tables tracking file reads, tool outputs, command outputs, cached content, context intel events, activity log, decision log, and hint serves. WAL mode for concurrent read/write from hook processes. 50MB cap per session.
  349. **Trends DB** (`~/.claude/token-optimizer/snapshots/trends.db`) holds 7 tables tracking session history, daily aggregates, skill/model/subagent usage, savings events, and compression events. Indexed by session UUID for O(log n) joins. This is what powers the dashboard, coach mode, and 30-day trend analysis.
  350. Every compression event, every saving, every quality measurement is a row you can query. The dashboard is a read-only view of this data. `measure.py compression-stats` is a SQL query. Your data is your data.
  351. ## Progressive Disclosure
  352. Large tool results (>4KB) are archived to disk and replaced with a short preview plus a retrieval pointer. The full output survives compaction. When the model needs it, it pulls the original via `expand`, with no command re-run and no lost output.
  353. Recovery is not compressed a second time. The optimizer's own `measure.py expand <id>` command (including `--session <id>`) and known original-recovery tools pass through without a new archive, dedup replacement or re-fetch denial. Ordinary memory search and document retrieval keep their existing behavior.
  354. This isn't just storage. The system tracks how many results were archived vs re-expanded, so you can see the net tokens that stayed collapsed. Re-expansions are netted out of the savings total, so you only count what actually stayed compressed.
  355. ```bash
  356. python3 measure.py expand --list # List archived tool results
  357. python3 measure.py expand <tool-use-id> # Retrieve a specific result
  358. python3 measure.py expand --search <query> # Search archived results by keyword/lineage
  359. python3 measure.py expand <id> --lines 50:120 # Just lines 50-120 of the archive
  360. python3 measure.py expand <id> --start 4000 --chars 2000 # A 2KB char window
  361. python3 measure.py expand <id> --match "ERROR" --context 3 # Literal-substring context windows
  362. ```
  363. `--match` is a literal substring (`grep -F` semantics), not a regex, and line
  364. numbers count `\n` breaks the way `grep -n`/`sed -n` do.
  365. `expand --search` runs a full-text search (FTS5 with a `LIKE` fallback) over
  366. the entire archived response body — not just a prefix — plus tool name,
  367. command, source file path, language, and archive origin. Each hit prints the
  368. key, lineage, capture time, content hash, total length, truncation state, and
  369. a snippet. `expand <key>` still returns the full stored output; the bounded
  370. flags above return only the requested window, print a command for the next
  371. window, and debit re-expansion savings only for the tokens actually returned.
  372. Some MCP tools are *documented* to return large verbatim payloads (source fetchers, doc retrievers) where a metadata preview defeats the point. Allowlist those so their full result reaches context instead of being replaced (it's still archived, so `expand` works). Set a comma-separated list of `fnmatch` globs matched against the full tool name:
  373. `TOKEN_OPTIMIZER_ARCHIVE_EXEMPT_TOOLS="mcp__context7__*,*github_repository_file"`
  374. Default is empty — nothing is exempt unless you opt a tool in.
  375. ## Session Continuity
  376. Compression matters, but the most important thing Token Optimizer does is keep your work alive across sessions and compactions, automatically.
  377. When you end a session, Token Optimizer checkpoints your state to SQLite: active task, key decisions, modified files, git branch, recent reads. When you start a new session, it scores all recent checkpoints against your prompt and surfaces the most relevant one as a hint. You resume with context, not from zero.
  378. **What happens automatically:**
  379. - **Cross-session hints**: relevance-scored checkpoints surface when you start a new session. The hint includes the prior session's task, decisions, files, and branch. All fenced as RECOVERED DATA, never instructions.
  380. - **Cold-resume-lean**: reopen a stale session without paying the full transcript cost. Token Optimizer reconstructs a lean context from its checkpoint. No LLM call, no full-transcript cold-resume. Token-free reconstruction from SQLite.
  381. - **Hint-follow estimate**: when a continuity hint surfaces file paths and the model subsequently reads one, Token Optimizer estimates an avoided exploratory search. The hint->read pair is observed; the avoided-search size is a fixed estimate, reported in the estimated tier, not the measured total.
  382. ```bash
  383. python3 measure.py resume-lean # list reopenable cold sessions
  384. python3 measure.py resume-lean <#|session_id> --print # emit lean context block
  385. ```
  386. Compression savings only stick if your session survives compaction. Session continuity is what makes that happen.
  387. ## Smart Compaction
  388. Auto-compact rewrites the conversation into a summary. How much useful detail survives depends on the compaction and the restore, not on a fixed percentage: the paired-window event study in `research/arxiv-2611.05306/` measured real sessions and found compaction count alone does not predict recovery quality. What Token Optimizer measures instead is what happens *after* each boundary: whether the restore was delivered, whether tool errors spike, whether the session re-reads unchanged files, repeats failed actions, or falls into loops. This release logs that evidence for calibration and shows a neutral `observing` status on every surface. Missing evidence stays unknown, and compaction count carries no quality penalty.
  389. Token Optimizer checkpoints your session before compaction and restores what the summary dropped, automatically, via hooks. It also injects a **context intel digest**: heuristic summaries of large tool outputs the model already processed (file paths touched, errors seen, line counts). After compaction, the model knows what it saw without re-reading everything.
  390. **Activity mode detection** classifies your session in real time (code, debug, review, infra, general) using the last 10 tool calls. The mode feeds into compaction guidance so PRESERVE/DROP priorities adapt to what you're doing right now, not a generic heuristic.
  391. **Decision extraction** captures decision statements from tool outputs as they happen and stores them in the session DB. At compaction time, these are injected as CRITICAL DECISIONS the summary must preserve verbatim. Capped at 10 per session.
  392. Compression savings only stick if your session survives compaction. Saving tokens on `git status` doesn't help if the next auto-compact wipes out the decision that made you run it.
  393. <p align="center">
  394. <img src="skills/token-optimizer/assets/quality-nudges-loops.svg" alt="Quality Nudges and Loop Detection in action" width="800">
  395. </p>
  396. <details>
  397. <summary><b>How Smart Compaction works</b></summary>
  398. ### Progressive Checkpoints
  399. Captures session state at multiple thresholds: 20%, 35%, 50%, 65%, 80% context fill, plus quality drops below 80, 70, 50, and 40. Also snapshots before agent fan-out and after large edit batches. On restore, picks the richest eligible checkpoint.
  400. ### Context Intel Digest
  401. After compaction, Token Optimizer injects a digest of the session's largest tool outputs: file paths touched, errors detected, line counts. Generated heuristically in <30ms, no LLM call. The model re-orients without re-reading everything.
  402. ```bash
  403. python3 measure.py setup-smart-compact # checkpoint + restore hooks
  404. ```
  405. </details>
  406. ## Output Tokens: Lean vs Verbose
  407. Output tokens are the most expensive part of your session. They cost 5x more than input tokens on Opus and are billed per generation, not per cache read. A verbose response to a simple question burns dollars you never needed to spend.
  408. Token Optimizer handles this with **lean-output nudges**. At 25–74% fill they also require session efficiency below 75; at 75–89% fill they use fill alone. At 90% or above no nudge is injected. Savings are estimated from later output when available.
  409. **How it works:**
  410. - The nudge is injected as `additionalContext`, never modifying the existing prefix, so your cache stays intact
  411. - It only fires when context is filling up, not when you have plenty of room
  412. - The model still thinks through the problem; it just produces a more concise visible answer
  413. - No on/off switch today; tune the trigger point with `TOKEN_OPTIMIZER_VERBOSITY_MIN_FILL` (default `25`)
  414. This is one of the 9 active compression features, and it's the one that saves on the output side, where tokens cost the most.
  415. ## Quality Scoring
  416. The quality score tracks two things: **Resource Health** (how close you are to the degradation cliff) and **Session Efficiency** (whether your tokens are doing useful work). Letter grades from S to F make triage instant.
  417. As context fills, quality drops. MRCR falls from 93% to 76% between 256K and 1M context. Your AI gets measurably dumber as the window fills. The quality score shows you exactly when that happens.
  418. ![Real session quality breakdown](skills/token-optimizer/assets/quality-example.svg)
  419. The status bar shifts color as quality degrades: green, yellow, orange, red. When quality drops below 75, session duration appears as a warning. Running subagents show with their model and elapsed time.
  420. ![Status Bar Degradation](skills/token-optimizer/assets/status-bar.svg)
  421. ```bash
  422. python3 measure.py setup-quality-bar # one-time install
  423. ```
  424. [Read how scoring works →](https://alexgreensh.github.io/token-optimizer/features/quality-signals/)
  425. <details>
  426. <summary><b>Quality score details</b></summary>
  427. | Score | Signals | What it means |
  428. |--------|--------|----------------|
  429. | **Resource Health** | Context fill, absolute waste tokens | How close you are to the degradation cliff |
  430. | **Session Efficiency** | Stale reads, bloated results, decision density, agent efficiency | Whether the session is using tokens well right now |
  431. | **Compaction recovery** | Restore delivery, post-boundary errors, re-reads, repeats, loops (matched 20-call windows) | Whether the session actually recovered after each compaction; count alone carries no weight |
  432. | Grade | Range | Meaning |
  433. |-------|-------|---------|
  434. | **S** | 90-100 | Peak efficiency |
  435. | **A** | 80-89 | Healthy, minor optimization possible |
  436. | **B** | 70-79 | Degradation starting |
  437. | **C** | 55-69 | Significant waste |
  438. | **D** | 40-54 | Serious problems |
  439. | **F** | 0-39 | Context is rotting, immediate action needed |
  440. **Quality bar disappeared?** Running Claude Code's `/statusline` overwrites Token Optimizer's entry. SessionStart auto-restores it. Just start a new session and it's back.
  441. **Want it off for good?**
  442. ```bash
  443. python3 measure.py setup-quality-bar --uninstall
  444. ```
  445. </details>
  446. ## Desktop Status Bar
  447. Using the Claude desktop app? The status bar comes with the Token Optimizer plugin: once it is installed, the bar appears above your prompt. Nothing extra to install.
  448. It needs Claude Code 2.1.287 or newer and Anthropic's mods feature. Anthropic can switch mods off remotely; if that happens nothing breaks, the bar just doesn't appear. The terminal keeps its existing status line. VS Code is not supported. To hide it, add `"TOKEN_OPTIMIZER_STATUS_BAR": "0"` to the `env` block of `~/.claude/settings.json`; `"TOKEN_OPTIMIZER_STATUS_BAR_ANIMATE": "0"` keeps Clawd still.
  449. - **One sentence, one button**: the most urgent thing about your session, with at most one button to act on it.
  450. - **Five marks**: quality grade, context fill, cache countdown, 5-hour limit and weekly limit, each with its label. Hover any mark for a card with the detail.
  451. - **The arrow beside Clawd** unfolds one more line: branch, session time, tool calls, compactions (when there are any), when the last checkpoint was saved (any kind, or a relevant one from an earlier session), and how many tokens Token Optimizer saved you in the last 30 days, the same total the dashboard shows. The last thing on that line is a **Full dashboard** link that regenerates the dashboard and opens it in your browser.
  452. - **Clawd acts out the session**: thinking, reading, typing, subagents, waiting for your permission, writing, compacting, done, stopped, API error, cold cache, napping.
  453. - **Clean up**: compacts with Token Optimizer's guidance.
  454. - **Start fresh**: asks for a second click, then saves a checkpoint, clears, and hands the checkpoint to your first message.
  455. - **Keep warm**: a manual, one-click cache refresh. It is offered only while the cache is still warm, never runs on its own, and uses a small amount of your usage. Afterwards it tells you how many tokens it re-read from the cache.
  456. - **Two sizes**: the `–` button beside the arrow shrinks the band to one line (tiny Clawd, the marks, and whatever needs attention) and `+` brings it back. The arrow still unfolds the details row under it. The choice is remembered across sessions and restarts; `TOKEN_OPTIMIZER_STATUS_BAR_SIZE=slim` sets the starting size and makes the terminal status line print one line too.
  457. Point at the bar and Clawd looks over. The cache countdown is an estimate, because Claude does not publish the exact expiry: 1 hour on Claude plans, 5 minutes on the API, measured from the session itself when possible.
  458. ## Coach Mode
  459. ```
  460. > /token-coach
  461. ```
  462. Tell it your goal. Get back specific, prioritized fixes with exact token savings. It reads 30 days of your session data and surfaces what no single session can show: quality drifting down, sessions getting longer, cache hit rates falling, cost per session climbing.
  463. Every insight is grounded in your actual numbers. "Your short sessions score 68 vs 60 for long ones" hits differently than "consider shorter sessions." Coach Mode also identifies project-level optimization opportunities (skills you never use, MCP servers that load eagerly, CLAUDE.md patterns that break your cache) and teaches you how to fix them so future sessions start leaner.
  464. [Read about Coach Mode →](https://alexgreensh.github.io/token-optimizer/features/token-coach/)
  465. <details>
  466. <summary><b>Coach Mode details</b></summary>
  467. ### Historical patterns detected
  468. | Pattern | What it detects |
  469. |---|---|
  470. | Quality drift | Average quality dropping week over week |
  471. | Session duration creep | Sessions getting longer, filling context faster |
  472. | Cache degradation | Cache hit rate falling (and whether model switches cause it) |
  473. | Grade distribution | Too many D-grade sessions piling up |
  474. | Cost awareness | Cost per session climbing, with routing advice |
  475. | Duration-quality correlation | Short sessions scoring higher, suggesting you break up long ones |
  476. | Compression gap | Shadow-only savings far exceeding active compression |
  477. | Model switching | Frequent mid-session model switches invalidating the prompt cache |
  478. ### Waste detectors
  479. 11 automated detectors analyze your session patterns:
  480. | Detector | What it catches |
  481. |---|---|
  482. | PDF/binary ingestion | Large files consuming context |
  483. | Web search overhead | Too many web results dumped into context |
  484. | Retry churn | Same tool retried 3+ times with errors |
  485. | Tool cascade | 3+ consecutive tool errors |
  486. | Looping | Repeated similar messages |
  487. | Overpowered model | Opus used for simple edits |
  488. | Weak model | Haiku on complex tasks |
  489. | Bad decomposition | Monolithic 500+ word prompts |
  490. | Wasteful thinking | Extended thinking >2x output for small edits |
  491. | Output waste | Verbose responses to simple operations |
  492. | Cache instability | CLAUDE.md patterns that break the prompt cache |
  493. ### Keep-Warm
  494. The opt-in scheduler models cache refreshes for API-billed sessions. Automatic production pings are paused. The manual desktop button remains available separately.
  495. ```bash
  496. python3 measure.py keepwarm-enable # opt in (API billing only)
  497. python3 measure.py keepwarm-report # net savings, spend, tripwire state
  498. python3 measure.py keepwarm-disable # opt out any time
  499. ```
  500. Prefer a button? The [desktop status bar](#desktop-status-bar) has a manual **Keep warm** refresh. It is a separate one-click action, never automatic, and does not use the API-billed daemon above.
  501. ### Subagent prompt cache
  502. Claude Code only. Subagents get a 5-minute prompt cache even on a subscription, so a subagent returned to after five minutes rewrites its whole prefix. On Claude Code 2.1.243+, Token Optimizer measures whether `subagentPromptCacheTtl: "1h"` would pay on your own transcripts -- at least 200 subagent requests in the last 30 days and an estimated saving of at least 1.15x the 1-hour write premium -- and **recommends** it; it never sets the key for you. The recommendation carries your numbers and the exact command, in `status`, `doctor`, `quick`, `coach`, and the audit. Anyone whose subagents run once, or too rarely, gets a neutral verdict instead. The scan reads your transcripts in a background process, never inside session start. `enable` never overrides a value you set, never fights a `CLAUDE_CODE_SUBAGENT_PROMPT_CACHE_TTL` or `FORCE_PROMPT_CACHING_5M` override, and `disable` is final. Under the `=1` opt-in (the only automatic write), a 14-day tripwire (needs at least 200 subagent requests, else "not enough data") reverts a Token Optimizer set if your own transcripts show the 1h write premium costing more than the rewrites it avoids, counting reuse across spawns as well as within one agent.
  503. ```bash
  504. python3 measure.py subagent-cache status # state, the verdict and why, the advice, estimated net
  505. python3 measure.py subagent-cache enable # take the recommendation, whatever the estimate says
  506. python3 measure.py subagent-cache disable # undo (only what Token Optimizer set; final)
  507. TOKEN_OPTIMIZER_SUBAGENT_CACHE_1H=1 # opt in to the automatic write + tripwire
  508. TOKEN_OPTIMIZER_SUBAGENT_CACHE_1H=0 # never: opt out permanently
  509. ```
  510. ### Usage recommendations
  511. Claude Code only. Once a day, Token Optimizer re-measures the two settings your own history can answer -- the auto-compact window and the subagent cache lifetime -- against your last 30 days and writes one `usage_recommendations.json` record in its own data directory. The measurement runs as a detached background process (spawned at session start or when the dashboard collects its data), never inside a hook, and a partial run retries the next day. The record shows in `quick`, `doctor`, `status`, `coach`, the audit, and the dashboard's Coach view under "From your own usage". When you act on a recommendation and the measured numbers move, the record says so as a "since you changed it" win -- observational, never causal. Token Optimizer never writes either setting: every command it prints is one you run yourself.
  512. ```bash
  513. python3 measure.py recommendations # the daily record: state, numbers, the exact command
  514. python3 measure.py recommendations --json # machine-readable
  515. python3 measure.py recommendations refresh # re-measure now (bounded; locks against a second run)
  516. ```
  517. ### Fleet Auditor
  518. Scans across Claude Code, Codex, and custom transcript setups to find idle burns, model misrouting, and config bloat with dollar savings per finding.
  519. ### CLAUDE.md Routing Injection
  520. Generate model routing instructions from your actual usage data and inject them into CLAUDE.md. 48-hour staleness guard auto-removes stale advice.
  521. ```bash
  522. python3 measure.py inject-routing --dry-run # Preview
  523. python3 measure.py inject-routing # Inject
  524. ```
  525. </details>
  526. ## FAQ
  527. <details>
  528. <summary><b>🔒 Can it degrade my context quality?</b></summary>
  529. No. Structural optimization only removes genuinely unused components. Active Compression controls can be disabled with a single command or env var. The quality scoring system tracks degradation in real time.
  530. </details>
  531. <details>
  532. <summary><b>💾 Does it break the prompt cache?</b></summary>
  533. No. Token Optimizer never touches content already in your context. It works on new content entering your window and on the compaction boundary. Your cache prefix stays intact, which means it saves you money twice: less input per turn, and cheaper cache reads on every turn forward.
  534. </details>
  535. <details>
  536. <summary><b>📡 Does it send any data anywhere?</b></summary>
  537. No analytics, no telemetry endpoint, no product data leaves your machine. Measurement events are local SQLite rows you own. Zero network calls.
  538. </details>
  539. <details>
  540. <summary><b>⚠️ Can it hurt my session?</b></summary>
  541. No. All hooks are non-blocking with fail-open design. If a Token Optimizer script errors, your command runs normally.
  542. </details>
  543. <details>
  544. <summary><b>📦 Any runtime dependencies?</b></summary>
  545. No. Pure Python stdlib on Claude Code and Codex. TypeScript with zero runtime deps on OpenCode and OpenClaw.
  546. </details>
  547. <details>
  548. <summary><b>🔐 How does install.sh verify file integrity?</b></summary>
  549. Resolves the latest GitHub Release tag, checks out that tag, fetches CHECKSUMS.sha256 from the same release, and verifies every script file. Out-of-band verification means a compromised commit cannot swap both code and checksums simultaneously.
  550. </details>
  551. ## All Commands
  552. <details>
  553. <summary><b>Show all commands</b></summary>
  554. | Command | What it does | Docs |
  555. |---|---|---|
  556. | `/token-optimizer` | Full audit with 6 parallel agents, guided fixes | [→](https://alexgreensh.github.io/token-optimizer/start/quickstart/) |
  557. | `/token-coach` | 30-day trend analysis, prioritized fixes | [→](https://alexgreensh.github.io/token-optimizer/features/token-coach/) |
  558. | `quick` | 10-second health check | [→](https://alexgreensh.github.io/token-optimizer/reference/cli/) |
  559. | `doctor` | Installation check, score out of 10 | [→](https://alexgreensh.github.io/token-optimizer/reference/cli/) |
  560. | `dashboard` | Open the HTML dashboard | [→](https://alexgreensh.github.io/token-optimizer/features/dashboard/) |
  561. | `savings` | Dollar savings report | [→](https://alexgreensh.github.io/token-optimizer/reference/cli/) |
  562. | `report` | Per-component token breakdown | [→](https://alexgreensh.github.io/token-optimizer/reference/cli/) |
  563. | `quality` | Context-quality analysis of live session | [→](https://alexgreensh.github.io/token-optimizer/features/quality-signals/) |
  564. | `trends` | Skill adoption, model mix, overhead over time | [→](https://alexgreensh.github.io/token-optimizer/reference/cli/) |
  565. | `compression-stats` | Measured savings from active compression | [→](https://alexgreensh.github.io/token-optimizer/features/active-compression/) |
  566. | `memory-review` | MEMORY.md structural audit | [→](https://alexgreensh.github.io/token-optimizer/features/memory-health/) |
  567. | `git-context` | Suggest files for your current diff | [→](https://alexgreensh.github.io/token-optimizer/reference/cli/) |
  568. | `drift` | Side-by-side comparison vs your last snapshot | [→](https://alexgreensh.github.io/token-optimizer/reference/cli/) |
  569. | `conversation` | Per-message token and cost breakdown | [→](https://alexgreensh.github.io/token-optimizer/reference/cli/) |
  570. | `pricing-tier` | View or switch pricing tiers | [→](https://alexgreensh.github.io/token-optimizer/reference/cli/) |
  571. | `expand` | Retrieve an archived tool result (progressive disclosure) | [→](https://alexgreensh.github.io/token-optimizer/reference/cli/) |
  572. | `resume-lean` | Reopen a cold session with token-free reconstruction | [→](https://alexgreensh.github.io/token-optimizer/reference/cli/) |
  573. [Full CLI reference →](https://alexgreensh.github.io/token-optimizer/reference/cli/)
  574. </details>
  575. ## License
  576. **PolyForm Noncommercial 1.0.0**, plus a written [small business permission](LICENSE-SMALL-BUSINESS.md). Source-available.
  577. ### Personal / hobby / research / education?
  578. Go for it. Full source, runs locally, no license purchase needed. That includes your own open-source side projects. What counts is who the work is for, not where it runs: work for your employer is your employer's use, even from home.
  579. ### Small business (fewer than 5 people AND under $20k/month revenue)?
  580. No-cost commercial license, in writing: [LICENSE-SMALL-BUSINESS.md](LICENSE-SMALL-BUSINESS.md). Both limits apply to your **whole company** (including any parent or sister companies), not to the team or the number of people using Token Optimizer. A small team inside a larger company needs a commercial license.
  581. ### Started personal, now it's turning into a business?
  582. Your past use is totally fine. You get 32 days after you stop qualifying to sort out a commercial license.
  583. ### Larger company / commercial use?
  584. Contact [Alex Greenshpun](https://linkedin.com/in/alexgreensh) or me@alexgreenshpun.com. Terms are reasonable and scale with company size.
  585. ---
  586. Created by [Alex Greenshpun](https://linkedin.com/in/alexgreensh).
cowork/token-optimizer/skills/token-optimizer/scripts/_measure/daemon.py · Bundled script · 6,710 lines · 1 finding marked
  1. """Definition-only implementations; mutable state belongs to measure."""
  2. from __future__ import annotations
  3. from . import _compat
  4. def _parse_elapsed_time(elapsed_str):
  5. """Parse ps elapsed time format (dd-HH:MM:SS or HH:MM:SS or MM:SS) to seconds."""
  6. _measure_facade = _compat.facade()
  7. elapsed_str = elapsed_str.strip()
  8. days = 0
  9. if "-" in elapsed_str:
  10. parts = elapsed_str.split("-", 1)
  11. days = int(parts[0])
  12. elapsed_str = parts[1]
  13. parts = elapsed_str.split(":")
  14. if len(parts) == 3:
  15. hours, minutes, seconds = int(parts[0]), int(parts[1]), int(parts[2])
  16. elif len(parts) == 2:
  17. hours = 0
  18. minutes, seconds = int(parts[0]), int(parts[1])
  19. else:
  20. return 0
  21. return days * 86400 + hours * 3600 + minutes * 60 + seconds
  22. def _format_elapsed(seconds):
  23. """Format seconds into a human-readable elapsed string."""
  24. _measure_facade = _compat.facade()
  25. if seconds < 3600:
  26. return f"{seconds // 60}m"
  27. hours = seconds // 3600
  28. if hours < 24:
  29. mins = (seconds % 3600) // 60
  30. return f"{hours}h {mins}m"
  31. d = hours // 24
  32. h = hours % 24
  33. return f"{d}d {h}h"
  34. def _scan_project_jsonl_stats():
  35. """Return [(path, birth_time, mtime), ...] for every project JSONL, cached.
  36. The full-tree stat walk of ~/.claude/projects is the dominant cost of version
  37. detection on a large history (tens of thousands of transcripts): a cold stat
  38. of every file. _find_session_version_for_pid runs once per live Claude process,
  39. so a dashboard gen with several running sessions repeated the identical walk N
  40. times -- an N-file stat storm that blows past the 45s regen-step budget on a
  41. cold cache. Compute it once and share it across those calls. A 30s TTL keeps
  42. the long-lived daemon fresh without re-walking on every call.
  43. """
  44. _measure_facade = _compat.facade()
  45. pass
  46. # monotonic() so NTP skew can't stretch or collapse the TTL window.
  47. if (_measure_facade._PROJECT_JSONL_STAT_CACHE is not None
  48. and (_measure_facade.time.monotonic() - _measure_facade._PROJECT_JSONL_STAT_CACHE_TS) < _measure_facade._PROJECT_JSONL_STAT_TTL):
  49. return _measure_facade._PROJECT_JSONL_STAT_CACHE
  50. out = []
  51. projects_base = _measure_facade.CLAUDE_DIR / "projects"
  52. if projects_base.exists():
  53. for project_dir in projects_base.iterdir():
  54. if not project_dir.is_dir():
  55. continue
  56. for jf in project_dir.glob("*.jsonl"):
  57. try:
  58. st = jf.stat()
  59. birth = getattr(st, "st_birthtime", st.st_ctime)
  60. out.append((jf, birth, st.st_mtime))
  61. except (PermissionError, OSError):
  62. continue
  63. _measure_facade._PROJECT_JSONL_STAT_CACHE = out
  64. # Stamp AFTER the walk finishes, not before: on a large cold history the walk
  65. # itself can exceed the TTL. Stamping the pre-walk time would make the cache
  66. # born already-expired, so the very next per-PID call re-walks and the memoization
  67. # is a no-op in exactly the cold-cache case this exists to fix.
  68. _measure_facade._PROJECT_JSONL_STAT_CACHE_TS = _measure_facade.time.monotonic()
  69. return out
  70. def _find_session_version_for_pid(pid):
  71. """Try to find the Claude Code version for a running process by matching its session JSONL.
  72. We look for JSONL files whose first message timestamp is close to the
  73. process start time. For long-running sessions, we also check file birth
  74. time (macOS) or creation time as a secondary signal.
  75. """
  76. _measure_facade = _compat.facade()
  77. projects_base = _measure_facade.CLAUDE_DIR / "projects"
  78. if not projects_base.exists():
  79. return None
  80. # Get process start time for correlation
  81. try:
  82. result = _measure_facade.subprocess.run(
  83. ["ps", "-o", "lstart=", "-p", str(pid)],
  84. capture_output=True, text=True, timeout=5,
  85. # Force C locale so ps emits English lstart regardless of host
  86. # locale (e.g. he_IL.UTF-8 emits Hebrew month names).
  87. env={**_measure_facade.os.environ, "LC_ALL": "C", "LC_TIME": "C"}, creationflags=_measure_facade._NO_WINDOW,
  88. )
  89. if result.returncode != 0:
  90. return None
  91. lstart_str = result.stdout.strip()
  92. # Parse "Fri Feb 27 10:18:43 2026"
  93. proc_start = _measure_facade.datetime.strptime(lstart_str, "%a %b %d %H:%M:%S %Y")
  94. proc_start_ts = proc_start.timestamp()
  95. except (_measure_facade.subprocess.SubprocessError, ValueError, OSError):
  96. return None
  97. # Find JSONL files whose creation or first-record timestamp matches process start
  98. best_match = None
  99. best_diff = float("inf")
  100. for jf, birth_time, mtime in _measure_facade._scan_project_jsonl_stats():
  101. try:
  102. # Skip files created well before or well after the process
  103. if birth_time < proc_start_ts - 60 and mtime < proc_start_ts - 60:
  104. continue
  105. # Read first 10 lines for version and timestamp
  106. version_found = None
  107. with open(jf, "r", encoding="utf-8", errors="replace") as f:
  108. for line_num, line in enumerate(f):
  109. if line_num > 10:
  110. break
  111. try:
  112. record = _measure_facade.json.loads(line)
  113. v = record.get("version")
  114. if v and not version_found:
  115. version_found = v
  116. ts_str = record.get("timestamp")
  117. if not ts_str:
  118. continue
  119. ts = _measure_facade.datetime.fromisoformat(ts_str.replace("Z", "+00:00")).replace(tzinfo=None)
  120. diff = abs((ts - proc_start).total_seconds())
  121. if diff < best_diff and version_found:
  122. best_diff = diff
  123. best_match = version_found
  124. except (_measure_facade.json.JSONDecodeError, ValueError):
  125. continue
  126. # Also try correlating birth time to process start
  127. birth_diff = abs(birth_time - proc_start_ts)
  128. if birth_diff < best_diff and version_found:
  129. best_diff = birth_diff
  130. best_match = version_found
  131. except (PermissionError, OSError):
  132. continue
  133. # Return if we found a reasonable match (within 10 minutes of start)
  134. if best_match and best_diff < 600:
  135. return best_match
  136. return None # No confident match; don't guess (causes false OUTDATED flags)
  137. def _command_matches_process(command, process_name):
  138. """True if a `ps` COMMAND field denotes a `process_name` CLI session.
  139. Matches a bare ``argv[0]`` (``claude`` / ``claude --resume ...``) and
  140. also absolute-path or wrapper launches -- e.g.
  141. ``/usr/local/.../@anthropic-ai/claude-code/bin/claude`` and the patched
  142. ``.../bin/claude.exe ...`` form -- by comparing the executable basename
  143. with a trailing ``.exe`` stripped. The bare-name fast path is preserved
  144. for backward compatibility. Matching is case-sensitive so the macOS
  145. desktop app (``/Applications/Claude.app/Contents/MacOS/Claude``) is not
  146. mistaken for the ``claude`` CLI, and ``claude`` appearing only as an
  147. argument (``vim claude.py``) does not match.
  148. Claude Desktop's "Code" tab launches Claude Code through a remote
  149. launcher whose ``argv[0]`` is a VERSIONED binary under
  150. ``~/.claude/remote/ccd-cli/<version>`` (e.g.
  151. ``~/.claude/remote/ccd-cli/2.1.271 --output-format stream-json ...``),
  152. notably under WSL 2 where ``claude`` is not on ``PATH``. The executable
  153. basename is then the bare version string, so the basename comparison can
  154. never match. Recognise that launcher by an anchored, version-shaped
  155. ``argv[0]`` match (``.../.claude/remote/ccd-cli/<version>``): a bare
  156. substring test was too loose and also swept in bundled binaries under the
  157. version dir (``.../ccd-cli/<version>/rg``), helper scripts
  158. (``.../ccd-cli/helper.sh``), ``.download`` partials, and the bare directory,
  159. inflating the session count. Restricted to ``process_name == "claude"``
  160. (Codex is not launched this way) and to ``argv[0]`` (so the path appearing
  161. only as an argument does not match), which keeps the match from catching
  162. unrelated processes. See issue #192.
  163. """
  164. _measure_facade = _compat.facade()
  165. command = (command or "").strip()
  166. if not command:
  167. return False
  168. if command == process_name or command.startswith(process_name + " "):
  169. return True
  170. argv0 = command.split()[0]
  171. if process_name == "claude" and _measure_facade._CCD_CLI_LAUNCHER_RE.search(argv0):
  172. return True
  173. exe_base = _measure_facade.os.path.basename(argv0)
  174. if exe_base.endswith(".exe"):
  175. exe_base = exe_base[:-4]
  176. return exe_base == process_name
  177. def _posix_comm_name(comm):
  178. """Lower-case executable name from a ps COMM field ("-zsh" -> "zsh",
  179. "tmux: server" -> "tmux", "/bin/bash" -> "bash"). Never raises."""
  180. _measure_facade = _compat.facade()
  181. text = (comm or "").strip()
  182. if not text:
  183. return ""
  184. text = text.split(": ", 1)[0].rsplit("/", 1)[-1]
  185. return text.lstrip("-").strip().lower()
  186. def _posix_comm_is_claude(comm):
  187. """True when a COMM field names a claude CLI executable (basename `claude`,
  188. `.exe` stripped, or the versioned ccd-cli launcher). Matches desktop-hosted
  189. sessions installed under a path with spaces (`.../Application Support/...`),
  190. which the whitespace-split `ps` command column cannot show."""
  191. _measure_facade = _compat.facade()
  192. text = (comm or "").strip()
  193. if not text:
  194. return False
  195. base = text.rsplit("/", 1)[-1]
  196. if base.endswith(".exe"):
  197. base = base[:-4]
  198. return base == "claude" or bool(_measure_facade._CCD_CLI_LAUNCHER_RE.search(text))
  199. def _posix_is_claude_host(comm, args):
  200. """True when a process is the Claude desktop app (main, helper, disclaimer
  201. wrapper, Electron build) or another claude process: a claude below it is hosted."""
  202. _measure_facade = _compat.facade()
  203. if _measure_facade._posix_comm_name(comm).startswith("claude"):
  204. return True
  205. low = f"{comm or ''}\n{args or ''}".lower()
  206. if "claude.app/" in low or "/.claude/remote/ccd-cli/" in low or "@anthropic-ai/claude-code" in low:
  207. return True
  208. if "app.asar" in low and "claude" in low:
  209. return True
  210. return _measure_facade._command_matches_process(args or "", "claude")
  211. def _posix_option_args(argv):
  212. """Arguments that can be options: everything after argv[0] up to a bare --."""
  213. _measure_facade = _compat.facade()
  214. args = list(argv[1:])
  215. return args[:args.index("--")] if "--" in args else args
  216. def _posix_is_headless(options):
  217. """True for SDK/headless/IDE switches and for subcommand processes."""
  218. _measure_facade = _compat.facade()
  219. for raw in options:
  220. a = raw.lower()
  221. if len(a) > 1 and a[0] == "-" and a[1] != "-" and "p" in a.split("=")[0]:
  222. return True # -p, -pc ...
  223. if any(a == f or a.startswith(f + "=") for f in _measure_facade._WIN_HEADLESS_FLAGS):
  224. return True
  225. if a in _measure_facade._WIN_HEADLESS_SUBCOMMANDS:
  226. return True
  227. return False
  228. def _posix_read_argv(pid):
  229. """Exact argv of a process, or None when the OS will not say.
  230. Linux reads NUL-separated /proc/<pid>/cmdline; macOS asks the kernel
  231. (sysctl KERN_PROCARGS2). Unlike the `ps` command column, both keep an
  232. argument that contains spaces as ONE token, so prompt text can neither
  233. masquerade as a switch nor hide one. Callers fall back to the `ps` column.
  234. """
  235. _measure_facade = _compat.facade()
  236. try:
  237. pid = int(pid)
  238. if _measure_facade.sys.platform.startswith("linux"):
  239. with open(f"/proc/{pid}/cmdline", "rb") as fh:
  240. data = fh.read()
  241. if not data:
  242. return None
  243. parts = data.split(b"\0")
  244. while parts and parts[-1] == b"":
  245. parts.pop() # trailing NUL, and the padding left by a rewritten title
  246. return [p.decode("utf-8", "replace") for p in parts] or None
  247. if _measure_facade.sys.platform == "darwin":
  248. import ctypes
  249. import ctypes.util
  250. libc = ctypes.CDLL(ctypes.util.find_library("c"), use_errno=True)
  251. mib = (ctypes.c_int * 3)(1, 49, pid) # CTL_KERN, KERN_PROCARGS2, pid
  252. size = ctypes.c_size_t(0)
  253. if libc.sysctl(mib, 3, None, ctypes.byref(size), None, 0) != 0 or size.value < 8:
  254. return None
  255. buf = ctypes.create_string_buffer(size.value)
  256. if libc.sysctl(mib, 3, buf, ctypes.byref(size), None, 0) != 0:
  257. return None
  258. data = buf.raw[:size.value]
  259. argc = int.from_bytes(data[:4], _measure_facade.sys.byteorder)
  260. rest = data[4:]
  261. i = rest.index(b"\0") # exec_path
  262. while i < len(rest) and rest[i] == 0:
  263. i += 1
  264. parts = rest[i:].split(b"\0")[:argc]
  265. if argc < 1 or len(parts) < argc:
  266. return None
  267. return [p.decode("utf-8", "replace") for p in parts]
  268. except Exception:
  269. return None
  270. return None
  271. def _posix_exe_path(pid, comm=None):
  272. """Real executable path of a process, or None.
  273. /proc/<pid>/exe on Linux (a replaced binary reads "<path> (deleted)", whose
  274. directory is still the right one to probe), proc_pidpath on macOS. A native
  275. install deletes the previous version's binary after an update, so a
  276. long-running session can legitimately have no resolvable path; callers treat
  277. None as "no directory evidence", not as a reason to fail.
  278. """
  279. _measure_facade = _compat.facade()
  280. try:
  281. pid = int(pid)
  282. if _measure_facade.sys.platform.startswith("linux"):
  283. target = _measure_facade.os.readlink(f"/proc/{pid}/exe")
  284. return target[:-len(" (deleted)")] if target.endswith(" (deleted)") else target
  285. if _measure_facade.sys.platform == "darwin":
  286. import ctypes
  287. import ctypes.util
  288. lib = ctypes.CDLL(ctypes.util.find_library("proc") or "/usr/lib/libproc.dylib", use_errno=True)
  289. buf = ctypes.create_string_buffer(4096)
  290. if lib.proc_pidpath(pid, buf, 4096) > 0:
  291. return buf.value.decode("utf-8", "replace")
  292. except Exception:
  293. pass
  294. return comm if comm and comm.startswith("/") else None
  295. def _posix_dir_is_electron_app(exe_path):
  296. """True/False when the executable's directory is/is not an Electron app
  297. directory; None when that cannot be determined (fail closed upstream)."""
  298. _measure_facade = _compat.facade()
  299. if not exe_path:
  300. return None
  301. base = _measure_facade.os.path.dirname(exe_path)
  302. # os.path.exists swallows every OSError and reports False, which would turn
  303. # an unreadable directory into "not Electron". Probe with os.stat so that
  304. # anything other than "definitely absent" is indeterminate.
  305. try:
  306. _measure_facade.os.stat(base or ".")
  307. except (OSError, ValueError):
  308. return None
  309. probes = [_measure_facade.os.path.join(base, *marker) for marker in _measure_facade._WIN_ELECTRON_MARKERS]
  310. if _measure_facade.os.path.basename(base) == "MacOS" and _measure_facade.os.path.basename(_measure_facade.os.path.dirname(base)) == "Contents":
  311. contents = _measure_facade.os.path.dirname(base) # <App>.app/Contents
  312. probes.append(_measure_facade.os.path.join(contents, "Resources", "app.asar"))
  313. probes.append(_measure_facade.os.path.join(contents, "Frameworks", "Electron Framework.framework"))
  314. found = False
  315. for probe in probes:
  316. try:
  317. _measure_facade.os.stat(probe)
  318. found = True
  319. except (FileNotFoundError, NotADirectoryError):
  320. continue
  321. except (OSError, ValueError):
  322. return None
  323. return found
  324. def _posix_is_init_parent(ppid, names, args_by_pid):
  325. """True when ppid is the init process an orphan is reparented to: pid 1
  326. (launchd, systemd, init) or a per-user `systemd --user` subreaper."""
  327. _measure_facade = _compat.facade()
  328. if ppid == 1:
  329. return True
  330. entry = names.get(ppid)
  331. if not entry or _measure_facade._posix_comm_name(entry[1]) != "systemd":
  332. return False
  333. return "--user" in (args_by_pid.get(ppid) or "").split()
  334. def _posix_ancestry_state(pid, names, args_by_pid, electron_parent_pids=None):
  335. """Walk parents of pid. Returns "hosted", "complete" or "incomplete".
  336. "complete" means the chain provably reached init/launchd (or a pid-namespace
  337. root) without meeting the desktop app or another claude process. Missing
  338. rows, cycles and over-long chains are "incomplete": absence of evidence is
  339. never a negative host check.
  340. """
  341. _measure_facade = _compat.facade()
  342. seen = {pid}
  343. cur = pid
  344. for _ in range(_measure_facade._POSIX_ANCESTRY_MAX_HOPS):
  345. entry = names.get(cur)
  346. if not entry:
  347. return "incomplete"
  348. ppid = entry[0]
  349. if ppid <= 0:
  350. return "complete"
  351. if ppid in seen:
  352. return "incomplete"
  353. parent = names.get(ppid)
  354. if not parent:
  355. return "incomplete"
  356. if ppid in electron_parent_pids or _measure_facade._posix_is_claude_host(parent[1], args_by_pid.get(ppid)):
  357. return "hosted"
  358. if ppid == 1:
  359. return "complete"
  360. seen.add(ppid)
  361. cur = ppid
  362. return "incomplete"
  363. def _classify_posix_claude_process(detail, names, args_by_pid, electron_parent_pids=None):
  364. """Classify one claude process on macOS/Linux by identity.
  365. Same classes as `_classify_windows_claude_process`:
  366. - "helper": Electron child (--type=...). Never a session.
  367. - "desktop_app": The Electron main process of the desktop app (it has
  368. --type= children, or an Electron directory layout).
  369. Not a session; never terminable.
  370. - "embedded_session": A Claude Code process hosted by another program
  371. (desktop app, IDE, SDK, a parent claude) or running
  372. headless (--print, stream-json, mcp ...). A real
  373. session owned by its host: never terminated by age.
  374. - "terminal_cli": Positively identified interactive terminal process:
  375. readable argv, no host/headless markers, a complete
  376. parent chain, a controlling TTY and a parent that is a
  377. shell, multiplexer, sshd or terminal. The only identity
  378. kill_stale_sessions may terminate.
  379. - "orphan_cli": An interactive session whose terminal died. ALL of:
  380. exact argv was read (KERN_PROCARGS2 or /proc cmdline,
  381. never the ps-split fallback), no --type=/headless
  382. flag/subcommand, no controlling TTY, the parent is the
  383. init process (pid 1 or `systemd --user`), and the
  384. executable is positively not an Electron app. Listed
  385. as ORPHAN; only `kill-stale --include-orphans` ends it.
  386. Anything short of that stays "unknown". Windows has no
  387. such class (parent-pid semantics differ).
  388. - "unknown": Anything else, including every case where identity
  389. evidence could not be read (unreadable ps, undecodable
  390. text, an orphan short of the conditions above). Listed,
  391. never terminated.
  392. detail: pid, ppid, tty, argv (list), argv_exact (bool), exe (path or None).
  393. names: {pid: (ppid, comm)} or None; args_by_pid: {pid: ps args string}.
  394. """
  395. _measure_facade = _compat.facade()
  396. if not detail:
  397. return "unknown"
  398. argv = detail.get("argv")
  399. if not argv:
  400. return "unknown"
  401. exe = detail.get("exe") or ""
  402. if "�" in exe or any("�" in a for a in argv):
  403. return "unknown" # undecodable text is never affirmative evidence
  404. # One token holding spaces is either a rewritten process title or a spaced
  405. # install path: the two cannot be told apart, so it is not exact argv.
  406. argv_exact = bool(detail.get("argv_exact")) and (len(argv) > 1 or len(argv[0].split()) == 1)
  407. if len(argv) == 1:
  408. # A process that rewrote its title carries its whole command line in
  409. # argv[0]; split it so a switch inside cannot hide.
  410. argv = argv[0].split() or argv
  411. options = _measure_facade._posix_option_args(argv) # tokens after a bare -- are prompt text
  412. if any(a.lower().startswith("--type=") for a in options):
  413. return "helper"
  414. pid = detail.get("pid")
  415. if pid in electron_parent_pids:
  416. return "desktop_app"
  417. electron_dir = None
  418. if exe:
  419. electron_dir = _measure_facade._posix_dir_is_electron_app(exe)
  420. if electron_dir is None:
  421. # Positive headless evidence (-p, --print, mcp ...) is never
  422. # killable either way, so it names the session even when the
  423. # executable's directory cannot be inspected.
  424. return "embedded_session" if _measure_facade._posix_is_headless(options) else "unknown"
  425. if electron_dir:
  426. return "desktop_app"
  427. if not names:
  428. return "unknown"
  429. ancestry = _measure_facade._posix_ancestry_state(pid, names, args_by_pid, electron_parent_pids)
  430. if ancestry == "hosted":
  431. return "embedded_session"
  432. if ancestry != "complete":
  433. return "unknown"
  434. if _measure_facade._posix_is_headless(options):
  435. return "embedded_session"
  436. entry = names.get(pid)
  437. if not entry or detail.get("ppid") != entry[0]:
  438. return "unknown" # the two snapshots disagree about the parent
  439. if (detail.get("tty") or "?") in _measure_facade._POSIX_NO_TTY:
  440. # No controlling terminal. Under a shell that is just a process that
  441. # lost its tty (unknown). Under init it is the zombie `health` exists
  442. # for, but only on exact argv and a binary positively not Electron
  443. # (an unresolvable executable gives no such evidence).
  444. if argv_exact and electron_dir is False and _measure_facade._posix_is_init_parent(entry[0], names, args_by_pid):
  445. return "orphan_cli"
  446. return "unknown"
  447. parent = names.get(entry[0])
  448. if parent and _measure_facade._posix_comm_name(parent[1]) in _measure_facade._POSIX_TERMINAL_PARENTS:
  449. return "terminal_cli"
  450. return "unknown"
  451. def _posix_process_names():
  452. """Return {pid: (ppid, comm)} for every process, or None when unreadable.
  453. COMM is the last column because it may contain spaces ("Claude Helper
  454. (Renderer)"); on macOS it is the full executable path. None means the
  455. parent chain cannot be established: callers treat every process as
  456. unverified, never as a confirmed terminal session.
  457. """
  458. _measure_facade = _compat.facade()
  459. try:
  460. result = _measure_facade.subprocess.run(
  461. ["ps", "-ww", "-eo", "pid,ppid,comm"],
  462. capture_output=True, text=True, encoding="utf-8", errors="replace", timeout=10,
  463. env={**_measure_facade.os.environ, "LC_ALL": "C", "LC_TIME": "C"}, creationflags=_measure_facade._NO_WINDOW,
  464. )
  465. except (_measure_facade.subprocess.SubprocessError, OSError, ValueError):
  466. return None
  467. if result.returncode != 0:
  468. return None
  469. names = {}
  470. for line in result.stdout.splitlines():
  471. m = _measure_facade._POSIX_NAME_ROW_RE.match(line)
  472. if m:
  473. names[int(m.group(1))] = (int(m.group(2)), m.group(3))
  474. return names or None
  475. def _posix_ancestor_pids(pid):
  476. """Pids of every ancestor of `pid`, or None when the chain cannot be established.
  477. kill-stale runs inside the conversation it was asked from; that claude is an
  478. ancestor of the command, not its direct parent, so it must be excluded too.
  479. None (not an empty set) when `ps` is unreadable or its table lacks `pid`:
  480. "unknown" is a distinct answer from "no ancestors", and kill-stale must not
  481. act without proof that a candidate is not the conversation it runs inside of.
  482. """
  483. _measure_facade = _compat.facade()
  484. names = _measure_facade._posix_process_names()
  485. if not names or pid not in names:
  486. return None
  487. out = set()
  488. cur = pid
  489. for _ in range(64):
  490. entry = names.get(cur)
  491. if not entry or entry[0] <= 0 or entry[0] in out:
  492. break
  493. out.add(entry[0])
  494. cur = entry[0]
  495. return out
  496. def _collect_posix_claude_sessions(process_name=None):
  497. """Collect running Claude/Codex CLI sessions via `ps` on macOS/Linux.
  498. Returns a list of session dicts. Returns None if `ps` itself raises a
  499. subprocess or OS error -- the caller treats None as "health check
  500. unavailable" to preserve the historical POSIX contract. A non-zero
  501. `ps` exit with no sessions found returns `[]`.
  502. For `claude`, every session carries ``identity`` ("terminal_cli",
  503. "orphan_cli", "embedded_session" or "unknown") and ``identity_source: "ps"``; Electron
  504. helpers and the desktop app's own process are dropped. See
  505. `_classify_posix_claude_process`. Codex inventories are untagged: the Codex
  506. path never terminates by age (see `_collect_health_data`/`kill_stale_sessions`).
  507. """
  508. _measure_facade = _compat.facade()
  509. try:
  510. result = _measure_facade.subprocess.run(
  511. ["ps", "-ww", "-eo", "pid,ppid,uid,tty,lstart,etime,command"],
  512. capture_output=True, text=True, encoding="utf-8", errors="replace", timeout=10,
  513. # Force C locale so lstart is always English 5-field format. Under
  514. # non-English locales (e.g. he_IL.UTF-8) ps emits localized dates
  515. # with a different field count, breaking the positional parse below
  516. # and dropping every session.
  517. env={**_measure_facade.os.environ, "LC_ALL": "C", "LC_TIME": "C"}, creationflags=_measure_facade._NO_WINDOW,
  518. )
  519. except (_measure_facade.subprocess.SubprocessError, OSError, ValueError):
  520. return None
  521. if result.returncode != 0:
  522. return []
  523. rows = []
  524. for line in result.stdout.splitlines():
  525. m = _measure_facade._POSIX_PS_ROW_RE.match(line)
  526. if not m:
  527. continue # header and anything that is not a process row
  528. rows.append({
  529. "pid": int(m.group(1)), "ppid": int(m.group(2)),
  530. "uid": int(m.group(3)) if m.group(3) is not None else None, "tty": m.group(4),
  531. "lstart": " ".join(m.group(5).split()), "etime": m.group(6), "args": m.group(7),
  532. })
  533. claude = process_name == "claude"
  534. names = _measure_facade._posix_process_names() if claude else None
  535. args_by_pid = {r["pid"]: r["args"] for r in rows}
  536. # Parents of claude-named Electron children (--type=...) are the desktop
  537. # app's main process, whatever their own name or directory layout.
  538. electron_parent_pids = set()
  539. if claude and names:
  540. for r in rows:
  541. if r["ppid"] > 0 and any(t.lower().startswith("--type=") for t in _measure_facade._posix_option_args(r["args"].split())):
  542. comm = (names.get(r["pid"]) or (0, ""))[1]
  543. if _measure_facade._posix_is_claude_host(comm, r["args"]):
  544. electron_parent_pids.add(r["ppid"])
  545. sessions = []
  546. for r in rows:
  547. comm = (names.get(r["pid"]) or (0, ""))[1] if names else ""
  548. command = r["args"]
  549. if not (_measure_facade._command_matches_process(command, process_name)
  550. or (claude and _measure_facade._posix_comm_is_claude(comm))):
  551. continue
  552. pid = r["pid"]
  553. tty = r["tty"]
  554. elapsed_seconds = _measure_facade._parse_elapsed_time(r["etime"])
  555. has_terminal = tty not in _measure_facade._POSIX_NO_TTY
  556. identity = None
  557. if claude:
  558. argv = _measure_facade._posix_read_argv(pid)
  559. argv_exact = bool(argv)
  560. if not argv:
  561. # ps joins argv with spaces: still safe (a switch can never be
  562. # hidden by splitting), merely coarser. On macOS COMM is argv[0]
  563. # exactly, which keeps a spaced install path in one token.
  564. if comm.startswith("/") and command.startswith(comm):
  565. argv = [comm] + command[len(comm):].split()
  566. else:
  567. argv = command.split()
  568. exe = _measure_facade._posix_exe_path(pid, comm)
  569. detail = {"pid": pid, "ppid": r["ppid"], "tty": tty, "argv": argv,
  570. "argv_exact": argv_exact, "exe": exe}
  571. identity = _measure_facade._classify_posix_claude_process(detail, names, args_by_pid, electron_parent_pids)
  572. if identity in ("helper", "desktop_app"):
  573. continue
  574. session = {
  575. "pid": pid,
  576. "started": r["lstart"],
  577. "elapsed_seconds": elapsed_seconds,
  578. "elapsed_human": _measure_facade._format_elapsed(elapsed_seconds),
  579. "command": command,
  580. "has_terminal": has_terminal,
  581. "tty": tty if has_terminal else None,
  582. "uid": r["uid"],
  583. }
  584. if identity is not None:
  585. session["identity"] = identity
  586. session["identity_source"] = "ps"
  587. sessions.append(session)
  588. return sessions
  589. def _local_start_label(started):
  590. """Format a process start time in local time, as the POSIX collector does.
  591. A timezone-aware value (the Windows collectors report UTC or an offset) is
  592. converted to the local zone first; a naive value is already local.
  593. """
  594. _measure_facade = _compat.facade()
  595. if started.tzinfo is not None:
  596. try:
  597. started = started.astimezone()
  598. except (ValueError, OverflowError, OSError):
  599. pass
  600. return started.strftime("%a %b %d %H:%M:%S %Y")
  601. def _parse_wmi_datetime(wmi_ts):
  602. """Parse a WMI CIM_DATETIME to elapsed seconds since process start.
  603. Format: ``YYYYMMDDHHMMSS.ffffff<sign><MMM>`` where sign is + or - and
  604. MMM is minutes offset from UTC. Parses the timezone when present and
  605. uses timezone-aware math so DST transitions can't flip elapsed
  606. negative. Falls back to naive math if the offset is missing/garbage.
  607. Returns {"started": human, "elapsed_seconds": int} or None.
  608. """
  609. _measure_facade = _compat.facade()
  610. if not wmi_ts or len(wmi_ts) < 14:
  611. return None
  612. try:
  613. year = int(wmi_ts[0:4])
  614. month = int(wmi_ts[4:6])
  615. day = int(wmi_ts[6:8])
  616. hour = int(wmi_ts[8:10])
  617. minute = int(wmi_ts[10:12])
  618. second = int(wmi_ts[12:14])
  619. except (ValueError, TypeError):
  620. return None
  621. # Offset: position 21 is sign char, 22-25 is MMM (may be absent on short strings).
  622. tz_info = None
  623. if len(wmi_ts) >= 25 and wmi_ts[21] in ("+", "-"):
  624. try:
  625. offset_minutes = int(wmi_ts[22:25])
  626. if wmi_ts[21] == "-":
  627. offset_minutes = -offset_minutes
  628. tz_info = _measure_facade.timezone(_measure_facade.timedelta(minutes=offset_minutes))
  629. except (ValueError, TypeError):
  630. tz_info = None
  631. try:
  632. if tz_info is not None:
  633. started = _measure_facade.datetime(year, month, day, hour, minute, second, tzinfo=tz_info)
  634. elapsed = (_measure_facade.datetime.now(_measure_facade.timezone.utc) - started).total_seconds()
  635. else:
  636. started = _measure_facade.datetime(year, month, day, hour, minute, second)
  637. elapsed = (_measure_facade.datetime.now() - started).total_seconds()
  638. except (ValueError, TypeError, OverflowError):
  639. return None
  640. return {
  641. "started": _measure_facade._local_start_label(started),
  642. "elapsed_seconds": max(0, int(elapsed)),
  643. }
  644. def _normalize_ps_iso(iso_ts):
  645. """Return a string datetime.fromisoformat accepts (3.9+), or "" when empty."""
  646. _measure_facade = _compat.facade()
  647. s = (iso_ts or "").strip()
  648. if not s:
  649. return ""
  650. s = _measure_facade._PS_DOTTED_TIME_RE.sub(r"\1:\2:\3", s)
  651. s = _measure_facade._PS_FRACTION_RE.sub(lambda m: ((_measure_facade := _compat.facade()), "." + m.group(1)[:6].ljust(6, "0"))[1], s)
  652. if s.endswith(("Z", "z")):
  653. s = s[:-1] + "+00:00"
  654. return s
  655. def _parse_iso_process_datetime(iso_ts):
  656. """Parse an ISO 8601 timestamp from PowerShell to elapsed seconds."""
  657. _measure_facade = _compat.facade()
  658. if not iso_ts:
  659. return None
  660. s = _measure_facade._normalize_ps_iso(iso_ts)
  661. try:
  662. started = _measure_facade.datetime.fromisoformat(s)
  663. except (ValueError, TypeError):
  664. return None
  665. if started.tzinfo is None:
  666. elapsed = (_measure_facade.datetime.now() - started).total_seconds()
  667. else:
  668. elapsed = (_measure_facade.datetime.now(_measure_facade.timezone.utc) - started).total_seconds()
  669. return {
  670. "started": _measure_facade._local_start_label(started),
  671. "elapsed_seconds": max(0, int(elapsed)),
  672. }
  673. def _windows_process_creation(pid):
  674. """Return {"started", "elapsed_seconds"} for a Windows PID, or {}.
  675. Tries `wmic` first (built-in back to Win7). Falls back to PowerShell
  676. `Get-CimInstance` when wmic is absent (deprecated in Windows 11 24H2+).
  677. All subprocess calls use errors='replace' so non-ASCII output on
  678. localized Windows (ja-JP, zh-CN, ru-RU) can't raise UnicodeDecodeError.
  679. """
  680. _measure_facade = _compat.facade()
  681. try:
  682. result = _measure_facade.subprocess.run(
  683. ["wmic", "process", "where", f"ProcessId={int(pid)}",
  684. "get", "CreationDate", "/format:list"],
  685. capture_output=True, text=True, timeout=10, errors="replace", creationflags=_measure_facade._NO_WINDOW,
  686. )
  687. if result.returncode == 0:
  688. for line in result.stdout.splitlines():
  689. line = line.strip()
  690. if line.startswith("CreationDate="):
  691. parsed = _measure_facade._parse_wmi_datetime(line[len("CreationDate="):])
  692. if parsed is not None:
  693. return parsed
  694. except (_measure_facade.subprocess.SubprocessError, OSError, FileNotFoundError):
  695. pass
  696. try:
  697. ps_cmd = (
  698. "(Get-CimInstance Win32_Process -Filter "
  699. f"'ProcessId={int(pid)}').CreationDate "
  700. "| ForEach-Object { $_.ToString('o') }"
  701. )
  702. result = _measure_facade.subprocess.run(
  703. # -ExecutionPolicy Bypass for parity with the other PowerShell
  704. # spawns: an AllSigned host would otherwise block this per-PID
  705. # fallback and leave the session flagged UNKNOWN_AGE.
  706. ["powershell", "-NoProfile", "-ExecutionPolicy", "Bypass", "-Command", ps_cmd],
  707. capture_output=True, text=True, timeout=10, errors="replace", creationflags=_measure_facade._NO_WINDOW,
  708. )
  709. if result.returncode == 0 and result.stdout.strip():
  710. parsed = _measure_facade._parse_iso_process_datetime(result.stdout.strip())
  711. if parsed is not None:
  712. return parsed
  713. except (_measure_facade.subprocess.SubprocessError, OSError, FileNotFoundError):
  714. pass
  715. return {}
  716. def _windows_start_times_agree(process_start, cim_creation, tolerance_seconds=None):
  717. """True only when Get-Process StartTime and CIM CreationDate match.
  718. Both come from separate queries joined by PID; a PID reused between them
  719. would otherwise borrow the old process's start time. Missing or
  720. unparseable values on either side are not agreement.
  721. """
  722. _measure_facade = _compat.facade()
  723. try:
  724. a = _measure_facade.datetime.fromisoformat(_measure_facade._normalize_ps_iso(process_start))
  725. b = _measure_facade.datetime.fromisoformat(_measure_facade._normalize_ps_iso(cim_creation))
  726. return abs((a - b).total_seconds()) <= tolerance_seconds
  727. except (ValueError, TypeError, AttributeError):
  728. return False
  729. def _windows_cim_process_details():
  730. """Return {pid: {"ppid", "path", "cmdline", "creation"}} for claude* processes, or None.
  731. Get-Process does not expose the command line on Windows PowerShell 5, so
  732. this asks Win32_Process. None means identity could not be established
  733. (PowerShell locked down, CIM unavailable): callers must treat every
  734. process as unverified, never as a confirmed terminal session.
  735. """
  736. _measure_facade = _compat.facade()
  737. import csv as _csv
  738. import io as _io
  739. ps_cmd = (
  740. "[Console]::OutputEncoding = [System.Text.Encoding]::UTF8; "
  741. "Get-CimInstance Win32_Process -Filter 'Name LIKE ''claude%''' "
  742. "-ErrorAction SilentlyContinue | "
  743. "Select-Object ProcessId, ParentProcessId, ExecutablePath, CommandLine, "
  744. "@{N='CreationDate';E={try { $_.CreationDate.ToUniversalTime().ToString('yyyy-MM-ddTHH:mm:ssZ', [System.Globalization.CultureInfo]::InvariantCulture) } catch { '' }}} | "
  745. "ConvertTo-Csv -NoTypeInformation"
  746. )
  747. try:
  748. result = _measure_facade.subprocess.run(
  749. ["powershell", "-NoProfile", "-ExecutionPolicy", "Bypass", "-Command", ps_cmd],
  750. capture_output=True, text=True, encoding="utf-8", timeout=10, errors="replace", creationflags=_measure_facade._NO_WINDOW,
  751. )
  752. except (_measure_facade.subprocess.SubprocessError, OSError, FileNotFoundError):
  753. return None
  754. if result.returncode != 0:
  755. return None
  756. rows = _measure_facade._read_strict_csv(result.stdout, ("ProcessId", "ParentProcessId", "ExecutablePath", "CommandLine", "CreationDate"))
  757. if rows is None:
  758. return None
  759. details = {}
  760. for row in rows:
  761. try:
  762. pid = int((row.get("ProcessId") or "").strip())
  763. except ValueError:
  764. continue
  765. try:
  766. ppid = int((row.get("ParentProcessId") or "").strip())
  767. except ValueError:
  768. ppid = None
  769. details[pid] = {
  770. "ppid": ppid,
  771. "path": (row.get("ExecutablePath") or "").strip(),
  772. "cmdline": (row.get("CommandLine") or "").strip(),
  773. "creation": (row.get("CreationDate") or "").strip(),
  774. }
  775. return details
  776. def _windows_process_names():
  777. """Return {pid: (ppid, image_name_lower)} for all processes, or None.
  778. Used for ancestry: a claude.exe is only treated as a terminal session when
  779. its parent is a known shell/terminal host. None = unavailable.
  780. """
  781. _measure_facade = _compat.facade()
  782. import csv as _csv
  783. import io as _io
  784. ps_cmd = (
  785. "[Console]::OutputEncoding = [System.Text.Encoding]::UTF8; "
  786. "Get-CimInstance Win32_Process -ErrorAction SilentlyContinue | "
  787. "Select-Object ProcessId, ParentProcessId, Name | "
  788. "ConvertTo-Csv -NoTypeInformation"
  789. )
  790. try:
  791. result = _measure_facade.subprocess.run(
  792. ["powershell", "-NoProfile", "-ExecutionPolicy", "Bypass", "-Command", ps_cmd],
  793. capture_output=True, text=True, encoding="utf-8", timeout=10, errors="replace", creationflags=_measure_facade._NO_WINDOW,
  794. )
  795. except (_measure_facade.subprocess.SubprocessError, OSError, FileNotFoundError):
  796. return None
  797. if result.returncode != 0:
  798. return None
  799. rows = _measure_facade._read_strict_csv(result.stdout, ("ProcessId", "ParentProcessId", "Name"))
  800. if rows is None:
  801. return None
  802. names = {}
  803. for row in rows:
  804. try:
  805. pid = int((row.get("ProcessId") or "").strip())
  806. ppid = int((row.get("ParentProcessId") or "").strip())
  807. except ValueError:
  808. continue
  809. names[pid] = (ppid, (row.get("Name") or "").strip().lower())
  810. return names or None
  811. def _windows_ancestor_pids(pid, names=None):
  812. """Pids of every ancestor of `pid` from the Windows process table, or None.
  813. Same walk as `_posix_ancestor_pids`, but None (not an empty set) when the table
  814. is unreadable, does not contain `pid`, or the chain runs into a parent pid the
  815. table lacks below a system root: kill-stale must not act without proof that a
  816. candidate is not the conversation it runs inside of (claude.exe -> shell ->
  817. python), so "unknown" is a distinct answer from "no ancestors".
  818. """
  819. _measure_facade = _compat.facade()
  820. if names is None:
  821. names = _measure_facade._windows_process_names()
  822. if not names or pid not in names:
  823. return None
  824. out = set()
  825. cur = pid
  826. for _ in range(64):
  827. entry = names.get(cur)
  828. if not entry:
  829. # `cur` is a parent pid the table does not list: its process exited
  830. # and Windows never reparents, so the link above it is gone. That is
  831. # a normal end only for a system root (explorer.exe's parent,
  832. # userinit.exe, is always gone). Anywhere else an ancestor such as
  833. # the claude.exe this runs inside of may sit above the gap.
  834. top = names.get(last)
  835. return out if top and top[1] in _measure_facade._WINDOWS_ANCESTRY_ROOTS else None
  836. if entry[0] <= 0 or entry[0] in out:
  837. break
  838. out.add(entry[0])
  839. last = cur
  840. cur = entry[0]
  841. return out
  842. def _windows_dir_is_electron_app(exe_path):
  843. """True/False when the executable's directory is/is not an Electron app
  844. directory; None when that cannot be determined (fail closed upstream)."""
  845. _measure_facade = _compat.facade()
  846. if not exe_path:
  847. return False
  848. import ntpath as _ntpath
  849. base = _ntpath.dirname(exe_path)
  850. # os.path.exists swallows every OSError and reports False, which would turn
  851. # an unreadable directory into "not Electron". Probe with os.stat so that
  852. # anything other than "definitely absent" is indeterminate.
  853. try:
  854. _measure_facade.os.stat(base or ".")
  855. except (OSError, ValueError):
  856. return None
  857. found = False
  858. for marker in _measure_facade._WIN_ELECTRON_MARKERS:
  859. try:
  860. _measure_facade.os.stat(_measure_facade.os.path.join(base, *marker))
  861. found = True
  862. except (FileNotFoundError, NotADirectoryError):
  863. continue
  864. except (OSError, ValueError):
  865. return None
  866. return found
  867. def _windows_ancestry_state(pid, names, electron_parent_pids):
  868. """Walk parents of pid. Returns "hosted", "complete" or "incomplete".
  869. "complete" means the chain provably reached a root without meeting a
  870. desktop app or another claude process. Missing rows, cycles and over-long
  871. chains are "incomplete": absence of evidence is never a negative host check.
  872. """
  873. _measure_facade = _compat.facade()
  874. seen = {pid}
  875. cur = pid
  876. for _ in range(_measure_facade._WIN_ANCESTRY_MAX_HOPS):
  877. entry = names.get(cur)
  878. if not entry:
  879. return "incomplete"
  880. ppid = entry[0]
  881. if ppid == 0:
  882. return "complete"
  883. if ppid in seen:
  884. return "incomplete"
  885. if ppid in electron_parent_pids:
  886. return "hosted"
  887. parent = names.get(ppid)
  888. if not parent:
  889. return "incomplete"
  890. if parent[1].startswith("claude"):
  891. return "hosted"
  892. if parent[1] in _measure_facade._WIN_SERVICE_HOSTS:
  893. return "incomplete"
  894. if parent[1] in _measure_facade._WIN_STABLE_ROOTS:
  895. return "complete"
  896. seen.add(ppid)
  897. cur = ppid
  898. return "incomplete"
  899. def _windows_cmdline_tokens(cmdline):
  900. """Split a Windows command line the way the MSVC runtime does.
  901. Quotes toggle grouping and are removed (so --"print" is --print),
  902. backslashes before a quote follow the 2n / 2n+1 rule, and a doubled quote
  903. inside a quoted run is a literal quote. Quoted prompt text therefore stays
  904. one token and can never masquerade as a switch.
  905. """
  906. _measure_facade = _compat.facade()
  907. text = cmdline or ""
  908. n = len(text)
  909. pos = 0
  910. tokens = []
  911. # Program name: quotes toggle grouping and are removed; no escape processing.
  912. while pos < n and text[pos] in " \t":
  913. pos += 1
  914. if pos < n:
  915. buf = []
  916. in_quote = False
  917. while pos < n and (in_quote or text[pos] not in " \t"):
  918. if text[pos] == '"':
  919. in_quote = not in_quote
  920. else:
  921. buf.append(text[pos])
  922. pos += 1
  923. tokens.append("".join(buf))
  924. while True:
  925. while pos < n and text[pos] in " \t":
  926. pos += 1
  927. if pos >= n:
  928. break
  929. buf = []
  930. in_quote = False
  931. started = False
  932. while pos < n:
  933. ch = text[pos]
  934. if ch == "\\":
  935. k = pos
  936. while k < n and text[k] == "\\":
  937. k += 1
  938. slashes = k - pos
  939. if k < n and text[k] == '"':
  940. buf.append("\\" * (slashes // 2))
  941. if slashes % 2:
  942. buf.append('"')
  943. pos = k + 1
  944. else:
  945. pos = k # quote handled by the next iteration
  946. else:
  947. buf.append("\\" * slashes)
  948. pos = k
  949. started = True
  950. continue
  951. if ch == '"':
  952. if in_quote and pos + 1 < n and text[pos + 1] == '"':
  953. buf.append('"')
  954. pos += 2
  955. else:
  956. in_quote = not in_quote
  957. pos += 1
  958. started = True
  959. continue
  960. if ch in " \t" and not in_quote:
  961. break
  962. buf.append(ch)
  963. started = True
  964. pos += 1
  965. if started:
  966. tokens.append("".join(buf))
  967. return tokens
  968. def _windows_option_args(cmdline):
  969. """Arguments that can be options: tokens after the program, up to a bare --."""
  970. _measure_facade = _compat.facade()
  971. args = _measure_facade._windows_cmdline_tokens(cmdline)[1:]
  972. return args[:args.index("--")] if "--" in args else args
  973. def _read_strict_csv(text, required):
  974. """Parse PowerShell CSV output, or None when it is malformed or partial.
  975. Unterminated quotes, short/long rows and a missing header all mean the
  976. output may have been truncated, which must never be read as evidence.
  977. """
  978. _measure_facade = _compat.facade()
  979. import csv as _csv
  980. import io as _io
  981. try:
  982. rows = list(_csv.reader(_io.StringIO((text or "").lstrip("\ufeff")), strict=True))
  983. except (_csv.Error, ValueError):
  984. return None
  985. if not rows:
  986. return []
  987. header = rows[0]
  988. if not all(col in header for col in required):
  989. return None
  990. out = []
  991. for row in rows[1:]:
  992. if not row:
  993. continue
  994. if len(row) != len(header):
  995. return None
  996. out.append(dict(zip(header, row)))
  997. return out
  998. def _classify_windows_claude_process(image_name, detail, electron_parent_pids, names=None):
  999. """Classify one claude* Windows process by identity.
  1000. Returns one of:
  1001. - "helper": Electron child (--type=...), SSH broker or any other
  1002. claude-adjacent image. Never a session.
  1003. - "desktop_app": The Electron main process of the desktop app. Not a
  1004. session; never terminable.
  1005. - "embedded_session": A Claude Code process hosted by another program
  1006. (desktop app, IDE, SDK, a parent claude) or running
  1007. headless (--print, stream-json, mcp). A real session
  1008. owned by its host: never terminated by age.
  1009. - "terminal_cli": Positively identified interactive terminal process:
  1010. readable command line, no host/headless markers and
  1011. a parent that is a known shell or terminal host. The
  1012. only identity kill_stale_sessions may terminate.
  1013. - "unknown": Anything else, including every case where identity
  1014. evidence could not be read. Listed, never terminated.
  1015. There is deliberately no "orphan_cli" here (macOS/Linux only): a Windows
  1016. child keeps the pid of a parent that has exited and is not reparented to a
  1017. stable init process, so "the parent is gone" is not evidence of an orphan.
  1018. """
  1019. _measure_facade = _compat.facade()
  1020. image = (image_name or "").strip().lower()
  1021. if image not in ("claude", "claude.exe"):
  1022. return "helper"
  1023. if not detail:
  1024. return "unknown"
  1025. cmdline = detail.get("cmdline") or ""
  1026. if not cmdline:
  1027. return "unknown"
  1028. if "\ufffd" in cmdline or "\ufffd" in (detail.get("path") or ""):
  1029. return "unknown" # undecodable text is never affirmative evidence
  1030. args = _measure_facade._windows_option_args(cmdline) # tokens after a bare -- are prompt text
  1031. if any(a.lower().startswith("--type=") for a in args):
  1032. return "helper"
  1033. if detail.get("pid") in electron_parent_pids:
  1034. return "desktop_app"
  1035. electron_dir = _measure_facade._windows_dir_is_electron_app(detail.get("path"))
  1036. if electron_dir is None:
  1037. return "unknown"
  1038. if electron_dir:
  1039. return "desktop_app"
  1040. if not detail.get("path") or not names:
  1041. return "unknown"
  1042. # Ancestry: hosted by the desktop app or by another claude process.
  1043. ancestry = _measure_facade._windows_ancestry_state(detail.get("pid"), names, electron_parent_pids)
  1044. if ancestry == "hosted":
  1045. return "embedded_session"
  1046. if ancestry != "complete":
  1047. return "unknown"
  1048. lowered = [a.lower() for a in args]
  1049. if any((len(a) > 1 and a[0] == "-" and a[1] != "-" and "p" in a.split("=")[0]) or any(a == f or a.startswith(f + "=") for f in _measure_facade._WIN_HEADLESS_FLAGS) for a in lowered):
  1050. return "embedded_session"
  1051. if any(a in _measure_facade._WIN_HEADLESS_SUBCOMMANDS for a in lowered):
  1052. return "embedded_session"
  1053. parent_entry = names.get(detail.get("pid"))
  1054. if parent_entry and detail.get("ppid") != parent_entry[0]:
  1055. return "unknown" # the two snapshots disagree about the parent
  1056. parent_name = names.get(parent_entry[0], (None, ""))[1] if parent_entry else ""
  1057. if parent_name in _measure_facade._WIN_TERMINAL_PARENTS:
  1058. return "terminal_cli"
  1059. return "unknown"
  1060. def _collect_windows_claude_sessions(process_name=None, creation_fallback=None):
  1061. """Collect runtime processes on Windows via PowerShell Get-Process.
  1062. Safety invariants:
  1063. - Only matches on the process image name (claude / claude-*).
  1064. Matching on Window Title would catch Chrome tabs viewing claude.ai,
  1065. editors with 'claude' in the filename, etc. kill_stale_sessions
  1066. would then TerminateProcess those apps -> unsaved-work data loss.
  1067. POSIX parity: ps-based match requires command == 'claude'; Windows
  1068. requires the same strictness. The PowerShell-side wildcard pre-filter
  1069. is a performance optimization only; the strict matcher below is the
  1070. security layer.
  1071. - Process identity comes from the command line (Win32_Process), not the
  1072. image name: Electron children (--type=), the desktop app main process
  1073. and claude-ssh-broker are dropped, desktop/SDK-hosted sessions are
  1074. tagged "embedded_session", and anything without positive evidence of an
  1075. interactive terminal parent is tagged "unknown". Only "terminal_cli" is
  1076. ever terminable.
  1077. - Uses SessionId (numeric) to detect service-hosted processes.
  1078. Services run in session 0; unlike the literal 'Services' string,
  1079. SessionId never localizes.
  1080. - subprocess calls use errors='replace' so non-ASCII PowerShell output
  1081. on localized Windows can't raise UnicodeDecodeError.
  1082. The previous `tasklist /v` enumeration was pathologically slow for
  1083. standard (non-elevated) users on Win11 -- /v queries verbose info (incl.
  1084. window titles) for EVERY process, hitting access-denied retries on
  1085. protected and other-user processes. Get-Process -Name 'claude*' filters
  1086. server-side, so only candidate processes are materialized, and StartTime
  1087. is dereferenced inside a try/catch so protected processes can't fail the
  1088. query.
  1089. Returns a list of session dicts. On subprocess failure returns an
  1090. empty list (not None) so the Health tab still renders.
  1091. """
  1092. _measure_facade = _compat.facade()
  1093. import csv as _csv
  1094. import io as _io
  1095. if process_name not in ("claude", "codex"):
  1096. raise ValueError("Unsupported runtime process name")
  1097. sessions = []
  1098. ps_cmd = (
  1099. # -Name 'claude*' filters server-side (only candidate processes are
  1100. # ever materialized), so the "touches only candidates" claim is real,
  1101. # not a post-enumeration Where-Object. -ErrorAction SilentlyContinue
  1102. # keeps a zero-match run from erroring.
  1103. f"Get-Process -Name '{process_name}*' -ErrorAction SilentlyContinue | "
  1104. "Select-Object Id, ProcessName, SessionId, "
  1105. "@{N='StartTime';E={try { $_.StartTime.ToUniversalTime().ToString('yyyy-MM-ddTHH:mm:ssZ', [System.Globalization.CultureInfo]::InvariantCulture) } catch { '' }}} | "
  1106. "ConvertTo-Csv -NoTypeInformation"
  1107. )
  1108. try:
  1109. result = _measure_facade.subprocess.run(
  1110. # -ExecutionPolicy Bypass so a locked-down host (AllSigned policy)
  1111. # can still run this inline -Command (tasklist needed no policy).
  1112. ["powershell", "-NoProfile", "-ExecutionPolicy", "Bypass", "-Command", ps_cmd],
  1113. capture_output=True, text=True, timeout=10, errors="replace", creationflags=_measure_facade._NO_WINDOW,
  1114. )
  1115. except (_measure_facade.subprocess.SubprocessError, OSError, FileNotFoundError):
  1116. return sessions
  1117. if result.returncode != 0:
  1118. return sessions
  1119. try:
  1120. reader = list(_csv.DictReader(_io.StringIO(result.stdout)))
  1121. except (_csv.Error, ValueError):
  1122. return sessions
  1123. cim_details = _measure_facade._windows_cim_process_details() if process_name == "claude" else None
  1124. proc_names = _measure_facade._windows_process_names() if cim_details else None
  1125. electron_parent_pids = set()
  1126. if cim_details:
  1127. for detail in cim_details.values():
  1128. if (any(a.lower().startswith("--type=") for a in _measure_facade._windows_option_args(detail.get("cmdline")))
  1129. and detail.get("ppid")):
  1130. electron_parent_pids.add(detail["ppid"])
  1131. for row in reader:
  1132. image_name = (row.get("ProcessName") or "").strip()
  1133. pid_str = row.get("Id") or ""
  1134. session_id_str = (row.get("SessionId") or "").strip()
  1135. start_time = (row.get("StartTime") or "").strip()
  1136. image_lower = image_name.lower()
  1137. # Strict image-name match only. See docstring invariants.
  1138. matches = image_lower in ("codex", "codex.exe") if process_name == "codex" else (
  1139. image_lower == "claude.exe"
  1140. or image_lower == "claude"
  1141. or image_lower.startswith("claude.")
  1142. or image_lower.startswith("claude-"))
  1143. if not matches:
  1144. continue
  1145. try:
  1146. pid = int(pid_str.replace(",", "").strip())
  1147. except (ValueError, AttributeError):
  1148. continue
  1149. if pid <= 0:
  1150. continue
  1151. identity = None
  1152. if process_name == "claude":
  1153. detail = dict(cim_details.get(pid) or {}) if cim_details else {}
  1154. detail["pid"] = pid
  1155. identity = _measure_facade._classify_windows_claude_process(image_name, detail, electron_parent_pids, proc_names)
  1156. if identity == "terminal_cli" and not _measure_facade._windows_start_times_agree(start_time, detail.get("creation")):
  1157. identity = "unknown" # the two queries saw different processes under this PID
  1158. if identity in ("helper", "desktop_app"):
  1159. continue
  1160. creation = _measure_facade._parse_iso_process_datetime(start_time) if start_time else None
  1161. if creation is None:
  1162. # StartTime unreadable (protected process) or unparseable: fall
  1163. # back to the per-PID wmic/CIM lookup.
  1164. creation = _measure_facade._windows_process_creation(pid) if creation_fallback else {}
  1165. elapsed_seconds = int(creation.get("elapsed_seconds") or 0)
  1166. # SessionId 0 is the Services session (language-independent); any
  1167. # other value indicates a user session. A missing/unparseable
  1168. # SessionId assumes a user session (services always report 0).
  1169. try:
  1170. session_id = int(session_id_str)
  1171. except ValueError:
  1172. session_id = -1
  1173. has_terminal = session_id != 0
  1174. session = {
  1175. "pid": pid,
  1176. "started": creation.get("started", "unknown"),
  1177. "elapsed_seconds": elapsed_seconds,
  1178. "elapsed_human": _measure_facade._format_elapsed(elapsed_seconds) if elapsed_seconds else "unknown",
  1179. "command": image_name if image_lower.endswith(".exe") else image_name + ".exe",
  1180. "has_terminal": has_terminal,
  1181. "tty": f"session-{session_id}" if has_terminal and session_id > 0 else None,
  1182. }
  1183. if identity is not None:
  1184. session["identity"] = identity
  1185. sessions.append(session)
  1186. return sessions
  1187. def _resolve_runtime_bin(process_name):
  1188. """Resolve a runtime CLI (`claude`/`codex`) to an absolute path, or None.
  1189. LIVE BUG: the dashboard's "installed version" showed `unknown` whenever
  1190. `_collect_health_data` ran from a minimal-PATH context. The auto-refresh
  1191. daemon (launchd / scheduled task) and SessionStart hooks inherit a
  1192. near-empty PATH that omits the modern native-installer dir
  1193. (`~/.local/bin`), so a bare `["claude", "--version"]` raised
  1194. FileNotFoundError and the version silently became None.
  1195. We resolve exactly like `_keepwarm_resolve_claude_bin`: `shutil.which`
  1196. first (honours the real PATH when run from a foreground shell), then the
  1197. common install locations launchd cannot see. Nothing is persisted (a
  1198. resolved path goes stale across npm/Homebrew/native updates); this runs
  1199. once per health collection. Returns the bare name as a last resort so the
  1200. historical `subprocess.run` behaviour is preserved when nothing resolves.
  1201. """
  1202. _measure_facade = _compat.facade()
  1203. import shutil as _shutil
  1204. which = _shutil.which(process_name)
  1205. if which:
  1206. return which
  1207. home = _measure_facade.Path.home()
  1208. candidates = [
  1209. home / ".local" / "bin" / process_name,
  1210. home / ".claude" / "local" / process_name,
  1211. _measure_facade.Path("/opt/homebrew/bin") / process_name,
  1212. _measure_facade.Path("/usr/local/bin") / process_name,
  1213. home / ".bun" / "bin" / process_name,
  1214. ]
  1215. # Claude Code's local installer lives under its config dir, so a relocated
  1216. # CLAUDE_CONFIG_DIR moves it off ~/.claude/local.
  1217. if process_name == "claude" and _measure_facade.CLAUDE_DIR / "local" / process_name not in candidates:
  1218. candidates.insert(2, _measure_facade.CLAUDE_DIR / "local" / process_name)
  1219. for cand in candidates:
  1220. try:
  1221. if cand.is_file() and _measure_facade.os.access(str(cand), _measure_facade.os.X_OK):
  1222. return str(cand)
  1223. except OSError:
  1224. continue
  1225. return process_name
  1226. def _semver_key(version):
  1227. """Return a SemVer precedence key, or None when ordering is unknown."""
  1228. _measure_facade = _compat.facade()
  1229. if not isinstance(version, str):
  1230. return None
  1231. match = _measure_facade.re.fullmatch(
  1232. r"(0|[1-9][0-9]*)\.(0|[1-9][0-9]*)\.(0|[1-9][0-9]*)"
  1233. r"(?:-([0-9A-Za-z-]+(?:\.[0-9A-Za-z-]+)*))?"
  1234. r"(?:\+[0-9A-Za-z-]+(?:\.[0-9A-Za-z-]+)*)?", version)
  1235. if not match:
  1236. return None
  1237. identifiers = []
  1238. prerelease = match.group(4)
  1239. if prerelease is not None:
  1240. for identifier in prerelease.split("."):
  1241. if identifier.isdigit():
  1242. if len(identifier) > 1 and identifier.startswith("0"):
  1243. return None
  1244. identifiers.append((0, (len(identifier), identifier)))
  1245. else:
  1246. identifiers.append((1, identifier))
  1247. return (*((len(part), part) for part in match.group(1, 2, 3)),
  1248. prerelease is None, tuple(identifiers))
  1249. def _collect_health_data():
  1250. """Collect session health data.
  1251. Returns a dict on macOS, Linux, and Windows. Returns None only if the
  1252. POSIX `ps` probe itself errors (historical contract preserved so
  1253. callers that short-circuit on None keep working).
  1254. """
  1255. _measure_facade = _compat.facade()
  1256. system = _measure_facade.platform.system()
  1257. runtime = _measure_facade.detect_runtime()
  1258. # Hermes: Hermes sessions are not `claude` processes.
  1259. # Probing `claude --version` and `ps | grep claude` under hermes is
  1260. # semantically wrong and would report stale/empty data. Return a
  1261. # hermes-mode minimal dict so the dashboard's health row renders
  1262. # without probing the Claude CLI.
  1263. if runtime == "hermes":
  1264. return {
  1265. "installed_version": None,
  1266. "running_sessions": [],
  1267. "automated": [],
  1268. "runtime": "hermes",
  1269. }
  1270. # Cursor: sessions are not `claude` processes either; probing the Claude
  1271. # CLI and `ps | grep claude` under a Cursor pin reports wrong data.
  1272. if runtime == "cursor":
  1273. return {
  1274. "installed_version": None,
  1275. "running_sessions": [],
  1276. "automated": [],
  1277. "runtime": "cursor",
  1278. }
  1279. process_name = "codex" if runtime == "codex" else "claude"
  1280. installed_version = None
  1281. try:
  1282. result = _measure_facade.subprocess.run(
  1283. [_measure_facade._resolve_runtime_bin(process_name), "--version"],
  1284. capture_output=True, text=True, timeout=10, creationflags=_measure_facade._NO_WINDOW,
  1285. )
  1286. if result.returncode == 0:
  1287. raw_version = result.stdout.strip()
  1288. installed_version = raw_version if runtime == "codex" else (raw_version.split()[0] if raw_version else None)
  1289. except (_measure_facade.subprocess.SubprocessError, FileNotFoundError, OSError):
  1290. pass
  1291. if system == "Windows":
  1292. running_sessions = (_measure_facade._collect_windows_claude_sessions(process_name="codex")
  1293. if runtime == "codex" else _measure_facade._collect_windows_claude_sessions())
  1294. else:
  1295. running_sessions = _measure_facade._collect_posix_claude_sessions(process_name=process_name)
  1296. if running_sessions is None:
  1297. return None
  1298. # Version enrichment per session. _find_session_version_for_pid relies
  1299. # on POSIX /proc-style discovery, so on Windows we set version=None.
  1300. # Future: Windows version probe via Get-Process ProductVersion.
  1301. if system == "Windows" or runtime == "codex":
  1302. for session in running_sessions:
  1303. session["version"] = None
  1304. else:
  1305. for session in running_sessions:
  1306. session["version"] = _measure_facade._find_session_version_for_pid(session["pid"])
  1307. # Zero-subprocess fallback: if `claude --version` could not run (locked-down
  1308. # PATH, sandbox, transient spawn failure), the running session JSONLs still
  1309. # carry the exact version that wrote them -- which IS the installed CLI.
  1310. # Use the highest detected session version so the dashboard never shows
  1311. # "unknown" while a Claude session is demonstrably running.
  1312. if not installed_version and runtime != "codex":
  1313. detected = [(key, session["version"]) for session in running_sessions
  1314. if (key := _measure_facade._semver_key(session.get("version"))) is not None]
  1315. if detected:
  1316. installed_version = max(detected, key=lambda item: ((_measure_facade := _compat.facade()), item[0])[1])[1]
  1317. installed_key = _measure_facade._semver_key(installed_version)
  1318. # Flag sessions
  1319. for s in running_sessions:
  1320. if runtime == "codex":
  1321. # Desktop app-server lifetime is not task age or evidence of an
  1322. # abandoned session. Never recommend killing shared Codex hosts.
  1323. s["flags"] = ["RUNNING"]
  1324. continue
  1325. flags = []
  1326. session_key = _measure_facade._semver_key(s.get("version"))
  1327. if session_key is not None and installed_key is not None and session_key < installed_key:
  1328. flags.append("OUTDATED")
  1329. if s["elapsed_seconds"] > 172800:
  1330. flags.append("ZOMBIE")
  1331. elif s["elapsed_seconds"] > 86400:
  1332. flags.append("STALE")
  1333. elif system == "Windows" and s["elapsed_seconds"] == 0:
  1334. # Windows without a creation-time source (wmic gone, PowerShell locked
  1335. # down) can't threshold STALE/ZOMBIE. Surface explicitly so the user
  1336. # isn't fooled into thinking all sessions are fresh.
  1337. flags.append("UNKNOWN_AGE")
  1338. identity = s.get("identity")
  1339. if identity == "embedded_session":
  1340. flags.append("DESKTOP")
  1341. elif identity == "unknown":
  1342. flags.append("UNVERIFIED")
  1343. elif identity == "orphan_cli":
  1344. flags.append("ORPHAN")
  1345. elif s.get("has_terminal"):
  1346. flags.append("TERMINAL")
  1347. else:
  1348. flags.append("HEADLESS")
  1349. s["flags"] = flags
  1350. automated = []
  1351. if system == "Darwin":
  1352. try:
  1353. result = _measure_facade.subprocess.run(
  1354. ["launchctl", "list"],
  1355. capture_output=True, text=True, timeout=10, creationflags=_measure_facade._NO_WINDOW,
  1356. )
  1357. if result.returncode == 0:
  1358. for line in result.stdout.strip().split("\n"):
  1359. needles = ("codex", "token-optimizer.codex", "tokenoptimizer.codex") if runtime == "codex" else ("claude", "anthropic")
  1360. if any(needle in line.lower() for needle in needles):
  1361. automated.append(line.strip())
  1362. except (_measure_facade.subprocess.SubprocessError, OSError):
  1363. pass
  1364. elif system == "Windows":
  1365. # Windows parity: surface scheduled tasks that touch claude/token-optimizer.
  1366. try:
  1367. import csv as _csv
  1368. import io as _io
  1369. result = _measure_facade.subprocess.run(
  1370. ["schtasks", "/Query", "/FO", "CSV", "/NH"],
  1371. capture_output=True, text=True, timeout=10, errors="replace", creationflags=_measure_facade._NO_WINDOW,
  1372. )
  1373. if result.returncode == 0:
  1374. for row in _csv.reader(_io.StringIO(result.stdout)):
  1375. if not row:
  1376. continue
  1377. name = row[0]
  1378. lname = name.lower()
  1379. needles = ("codex", "token-optimizer.codex", "tokenoptimizer.codex") if runtime == "codex" else ("claude", "token-optimizer", "tokenoptimizer", "anthropic")
  1380. if any(tok in lname for tok in needles):
  1381. automated.append(name.strip())
  1382. except (_measure_facade.subprocess.SubprocessError, OSError, FileNotFoundError):
  1383. pass
  1384. # Build recommendations
  1385. recommendations = []
  1386. # Sessions hosted by another app (DESKTOP) or with unreadable identity
  1387. # (UNVERIFIED) are not terminals the user can close and reopen.
  1388. # ORPHAN sessions have no terminal to close and reopen; they get their own
  1389. # recommendation below.
  1390. _own = [s for s in running_sessions if not any(f in s.get("flags", []) for f in ("DESKTOP", "UNVERIFIED", "ORPHAN"))]
  1391. outdated_count = sum(1 for s in _own if "OUTDATED" in s.get("flags", []))
  1392. stale_count = sum(1 for s in _own if any(f in s.get("flags", []) for f in ("STALE", "ZOMBIE")))
  1393. if outdated_count > 0 and installed_version:
  1394. recommendations.append(
  1395. f"{outdated_count} session{'s' if outdated_count != 1 else ''} running "
  1396. f"older version (installed: {installed_version}). "
  1397. f"Restart to get latest fixes: close and reopen these terminals."
  1398. )
  1399. if stale_count > 0:
  1400. recommendations.append(
  1401. f"{stale_count} session{'s' if stale_count != 1 else ''} running "
  1402. f"24+ hours. Check if still needed, long sessions accumulate context bloat."
  1403. )
  1404. orphan_count = sum(1 for s in running_sessions if "ORPHAN" in s.get("flags", []))
  1405. if orphan_count > 0:
  1406. recommendations.append(
  1407. f"{orphan_count} session{'s' if orphan_count != 1 else ''} with the terminal gone "
  1408. f"(ORPHAN). Never ended automatically. Review, then run "
  1409. f"`{_measure_facade._measure_cli('kill-stale', '--include-orphans', '--dry-run')}` to preview ending them."
  1410. )
  1411. unknown_age_count = sum(1 for s in running_sessions if "UNKNOWN_AGE" in s.get("flags", []))
  1412. if unknown_age_count > 0 and system == "Windows":
  1413. recommendations.append(
  1414. f"{unknown_age_count} session{'s' if unknown_age_count != 1 else ''} missing "
  1415. "start time (wmic and PowerShell Get-CimInstance both unavailable). "
  1416. "Install PowerShell 7+ (winget install Microsoft.PowerShell) or use Task "
  1417. "Manager to identify orphaned processes; STALE/ZOMBIE detection needs a "
  1418. "start-time source to work."
  1419. )
  1420. # Version-specific warnings
  1421. if installed_version:
  1422. try:
  1423. version_parts = tuple(int(x) for x in installed_version.split(".")[:3])
  1424. if version_parts < (2, 1, 70):
  1425. recommendations.append(
  1426. "Upgrade to Claude Code 2.1.70+ to fix skill listing re-injection on resume (~600 tokens/resume)."
  1427. )
  1428. except (ValueError, TypeError):
  1429. pass
  1430. return {
  1431. "installed_version": installed_version,
  1432. "running_sessions": running_sessions,
  1433. "automated": automated,
  1434. "recommendations": recommendations,
  1435. "cli": _measure_facade._measure_cli(),
  1436. "cli_windows": _measure_facade._windows_hints(),
  1437. }
  1438. def health_selfcheck():
  1439. """Probe the current platform's process-listing commands and parse their real output.
  1440. Designed as a lightweight smoke test for the Session Health code path.
  1441. Prints PASS/FAIL per probe; exits non-zero if any critical check fails.
  1442. Intended to be run by users (especially Windows users) after installing
  1443. the plugin: `python3 measure.py health-selfcheck`.
  1444. """
  1445. _measure_facade = _compat.facade()
  1446. system = _measure_facade.platform.system()
  1447. print(f"\nSESSION HEALTH SELF-CHECK (platform: {system})")
  1448. print("=" * 60)
  1449. passed = 0
  1450. failed = 0
  1451. def check(name, ok, detail=""):
  1452. _measure_facade = _compat.facade()
  1453. nonlocal passed, failed
  1454. mark = "PASS" if ok else "FAIL"
  1455. print(f" [{mark}] {name}")
  1456. if detail:
  1457. print(f" {detail}")
  1458. if ok:
  1459. passed += 1
  1460. else:
  1461. failed += 1
  1462. # Parser self-tests (platform-agnostic)
  1463. wmi_probe = _measure_facade._parse_wmi_datetime("20260101120000.000000+000")
  1464. check("WMI datetime parser", wmi_probe is not None and wmi_probe["elapsed_seconds"] > 0)
  1465. iso_probe = _measure_facade._parse_iso_process_datetime("2026-01-01T12:00:00Z")
  1466. check("ISO-8601 datetime parser", iso_probe is not None and iso_probe["elapsed_seconds"] > 0)
  1467. # Process-name matcher: bare argv0, absolute-path and ".exe" wrapper
  1468. # launches all match; the case-different desktop app, an arg-only
  1469. # mention, and unrelated processes must not.
  1470. _m = _measure_facade._command_matches_process
  1471. matcher_ok = (
  1472. _m("claude", "claude")
  1473. and _m("claude --resume abc", "claude")
  1474. and _m("/usr/local/lib/node_modules/@anthropic-ai/claude-code/bin/claude", "claude")
  1475. and _m("/usr/local/lib/node_modules/@anthropic-ai/claude-code/bin/claude.exe --resume abc", "claude")
  1476. and _m("/opt/homebrew/bin/codex serve", "codex")
  1477. # Claude Desktop / WSL 2 launcher: versioned ccd-cli binary (issue #192)
  1478. and _m("/home/user/.claude/remote/ccd-cli/2.1.271 --output-format stream-json --verbose", "claude")
  1479. and not _m("/Applications/Claude.app/Contents/MacOS/Claude", "claude")
  1480. and not _m("/usr/bin/vim claude.py", "claude")
  1481. and not _m("node /path/to/app.js", "claude")
  1482. # ccd-cli launcher must NOT satisfy a codex probe, and must NOT match
  1483. # when the path appears only as an argument.
  1484. and not _m("/home/user/.claude/remote/ccd-cli/2.1.271", "codex")
  1485. and not _m("/usr/bin/vim /home/user/.claude/remote/ccd-cli/notes.txt", "claude")
  1486. and not _m("", "claude")
  1487. )
  1488. check("process-name matcher (basename + .exe + ccd-cli)", matcher_ok)
  1489. # Live process-listing command
  1490. if system == "Windows":
  1491. # Get-Process probe (same pipeline shape as
  1492. # _collect_windows_claude_sessions; tasklist /v was retired, which is
  1493. # pathologically slow for standard users on Win11)
  1494. try:
  1495. res = _measure_facade.subprocess.run(
  1496. ["powershell", "-NoProfile", "-ExecutionPolicy", "Bypass", "-Command",
  1497. "Get-Process -ErrorAction SilentlyContinue | "
  1498. "Select-Object -First 1 Id, ProcessName, SessionId | "
  1499. "ConvertTo-Csv -NoTypeInformation"],
  1500. capture_output=True, text=True, timeout=10, errors="replace", creationflags=_measure_facade._NO_WINDOW,
  1501. )
  1502. ok = res.returncode == 0 and "ProcessName" in res.stdout
  1503. check("Get-Process csv probe", ok,
  1504. f"exit={res.returncode}, bytes={len(res.stdout)}")
  1505. except (_measure_facade.subprocess.SubprocessError, OSError, FileNotFoundError) as e:
  1506. check("Get-Process csv probe", False, f"exception: {e!r}")
  1507. # _collect_windows_claude_sessions end-to-end
  1508. try:
  1509. sessions = _measure_facade._collect_windows_claude_sessions()
  1510. check("_collect_windows_claude_sessions end-to-end", True,
  1511. f"returned {len(sessions)} session(s)")
  1512. except Exception as e:
  1513. check("_collect_windows_claude_sessions end-to-end", False, f"exception: {e!r}")
  1514. # wmic probe against our own pid
  1515. own_pid = _measure_facade.os.getpid()
  1516. try:
  1517. res = _measure_facade.subprocess.run(
  1518. ["wmic", "process", "where", f"ProcessId={own_pid}",
  1519. "get", "CreationDate", "/format:list"],
  1520. capture_output=True, text=True, timeout=10, creationflags=_measure_facade._NO_WINDOW,
  1521. )
  1522. wmic_ok = res.returncode == 0 and "CreationDate=" in res.stdout
  1523. check("wmic CreationDate probe (self-pid)", wmic_ok,
  1524. f"exit={res.returncode}")
  1525. except (_measure_facade.subprocess.SubprocessError, OSError, FileNotFoundError) as e:
  1526. check("wmic CreationDate probe (self-pid)", False,
  1527. f"exception: {e!r} (Win11 24H2 removes wmic; PowerShell fallback will be used)")
  1528. else:
  1529. try:
  1530. res = _measure_facade.subprocess.run(
  1531. ["ps", "-ww", "-eo", "pid,ppid,uid,tty,lstart,etime,command"],
  1532. capture_output=True, text=True, encoding="utf-8", errors="replace", timeout=10,
  1533. # Match the production collectors: force C locale so this
  1534. # diagnostic mirrors what _collect_posix_claude_sessions sees.
  1535. env={**_measure_facade.os.environ, "LC_ALL": "C", "LC_TIME": "C"}, creationflags=_measure_facade._NO_WINDOW,
  1536. )
  1537. ok = res.returncode == 0 and len(res.stdout.strip().split("\n")) > 1
  1538. check("ps -ww -eo pid,ppid,uid,tty,lstart,etime,command", ok,
  1539. f"exit={res.returncode}, lines={len(res.stdout.strip().split(chr(10)))}")
  1540. except (_measure_facade.subprocess.SubprocessError, OSError) as e:
  1541. check("ps -ww -eo pid,ppid,uid,tty,lstart,etime,command", False, f"exception: {e!r}")
  1542. try:
  1543. sessions = _measure_facade._collect_posix_claude_sessions()
  1544. ok = sessions is not None
  1545. detail = f"returned {len(sessions) if sessions else 0} session(s)" if ok else "returned None"
  1546. check("_collect_posix_claude_sessions end-to-end", ok, detail)
  1547. except Exception as e:
  1548. check("_collect_posix_claude_sessions end-to-end", False, f"exception: {e!r}")
  1549. # Dispatch end-to-end
  1550. try:
  1551. health = _measure_facade._collect_health_data()
  1552. if system == "Windows":
  1553. # Windows must return a dict, not None, after this fix
  1554. ok = isinstance(health, dict) and "running_sessions" in health
  1555. check("_collect_health_data returns dict on Windows",
  1556. ok, "this is the Bug-1 regression guard")
  1557. else:
  1558. ok = health is None or (isinstance(health, dict) and "running_sessions" in health)
  1559. check("_collect_health_data POSIX contract", ok)
  1560. except Exception as e:
  1561. check("_collect_health_data dispatch", False, f"exception: {e!r}")
  1562. print()
  1563. print(f" Results: {passed} passed, {failed} failed")
  1564. print()
  1565. if failed > 0:
  1566. print(" Paste this output into the issue to help us diagnose.")
  1567. _measure_facade.sys.exit(1)
  1568. def session_health():
  1569. """Check health of running Claude Code sessions."""
  1570. _measure_facade = _compat.facade()
  1571. health = _measure_facade._collect_health_data()
  1572. if health is None:
  1573. print("\nSession health check is not supported on this platform.")
  1574. return
  1575. installed_version, running_sessions, automated, recommendations = (
  1576. health[k] for k in ("installed_version", "running_sessions", "automated", "recommendations"))
  1577. print("\nSESSION HEALTH CHECK")
  1578. print("=" * 55)
  1579. print(f"\n Installed version: {installed_version or 'unknown (could not run ' + repr('claude --version') + ')'}")
  1580. if not running_sessions:
  1581. print("\n No running Claude Code CLI sessions found.")
  1582. else:
  1583. print(f"\nRUNNING SESSIONS ({len(running_sessions)})")
  1584. for s in running_sessions:
  1585. flags = s.get("flags", [])
  1586. version_str = s["version"] or "unknown"
  1587. flag_str = f" {' '.join(flags)}" if flags else ""
  1588. print(f" PID {s['pid']:<7d} Started: {s['started']} ({s['elapsed_human']} ago)")
  1589. print(f" Version: {version_str}{flag_str}")
  1590. if "ORPHAN" in flags:
  1591. print(f" terminal gone; started {s['elapsed_human']} ago")
  1592. if recommendations:
  1593. print("\nRECOMMENDATIONS")
  1594. for rec in recommendations:
  1595. print(f" - {rec}")
  1596. if automated:
  1597. print("\nAUTOMATED PROCESSES")
  1598. for proc in automated:
  1599. print(f" {proc}")
  1600. try: # hooks the malformed-hook healer left alone (heal_report.py)
  1601. for _line in __import__("heal_report").health_lines(_measure_facade):
  1602. print(_line)
  1603. except Exception:
  1604. pass # health must never fail on this optional section
  1605. print()
  1606. def _windows_revalidate_terminal_cli(session, fresh_inventory=None):
  1607. """Re-check a Windows session right before termination.
  1608. The inventory is a snapshot; a PID can be reused or reclassified between
  1609. that snapshot and the kill. Re-collect (without per-PID fallback probes for
  1610. unrelated processes) and require the same PID with the same start time to
  1611. still classify as terminal_cli. A race between that snapshot and
  1612. TerminateProcess remains: PID-based termination has no process handle to
  1613. bind to, so a PID reused inside that interval could be terminated.
  1614. """
  1615. _measure_facade = _compat.facade()
  1616. if session.get("started") in (None, "", "unknown"):
  1617. return False
  1618. if fresh_inventory is None:
  1619. try:
  1620. fresh_inventory = {x["pid"]: x for x in _measure_facade._collect_windows_claude_sessions(creation_fallback=False)}
  1621. except Exception:
  1622. return False
  1623. now = fresh_inventory.get(session["pid"])
  1624. return bool(
  1625. now
  1626. and now.get("identity") == "terminal_cli"
  1627. and now.get("started") == session.get("started")
  1628. )
  1629. def _posix_revalidate_terminal_cli(session, fresh_inventory=None, identity=None):
  1630. """Re-check a POSIX session right before termination.
  1631. The inventory is a snapshot; a PID can be reused or reclassified between
  1632. that snapshot and the kill. Re-collect and require the same PID with the
  1633. same start time and command line to still classify as `identity`
  1634. (terminal_cli, or orphan_cli for `kill-stale --include-orphans`). A race
  1635. between that re-check and os.kill remains: a PID-based signal has no handle
  1636. to bind to, so a PID reused inside that interval could be signalled.
  1637. """
  1638. _measure_facade = _compat.facade()
  1639. if session.get("started") in (None, "", "unknown"):
  1640. return False
  1641. if fresh_inventory is None:
  1642. try:
  1643. fresh = _measure_facade._collect_posix_claude_sessions("claude")
  1644. except Exception:
  1645. return False
  1646. if fresh is None:
  1647. return False
  1648. fresh_inventory = {x["pid"]: x for x in fresh}
  1649. now = fresh_inventory.get(session["pid"])
  1650. return bool(
  1651. now
  1652. and now.get("identity") == identity
  1653. and now.get("started") == session.get("started")
  1654. and now.get("command") == session.get("command")
  1655. and now.get("uid") == session.get("uid")
  1656. )
  1657. def _revalidate_terminal_cli(session, identity=None):
  1658. """Re-verify one identity-tagged session with the collector that tagged it.
  1659. `identity` is the class the session must still have (orphan_cli exists on
  1660. POSIX only; Windows sessions are always terminal_cli)."""
  1661. _measure_facade = _compat.facade()
  1662. if session.get("identity_source") == "ps":
  1663. return _measure_facade._posix_revalidate_terminal_cli(session, identity=identity)
  1664. return identity == "terminal_cli" and _measure_facade._windows_revalidate_terminal_cli(session)
  1665. def _parse_kill_stale_args(args):
  1666. """(hours, dry_run, include_orphans) from the `kill-stale` argv tail."""
  1667. _measure_facade = _compat.facade()
  1668. hours = 12
  1669. for i, a in enumerate(args):
  1670. if a == "--hours" and i + 1 < len(args):
  1671. try:
  1672. hours = int(args[i + 1])
  1673. except ValueError:
  1674. pass
  1675. return hours, "--dry-run" in args, "--include-orphans" in args
  1676. def _kill_stale_owns(session):
  1677. """True when kill-stale may consider `session`: POSIX sessions must be owned by
  1678. the effective uid of this process (uid unknown means not provably ours)."""
  1679. _measure_facade = _compat.facade()
  1680. if session.get("identity_source") != "ps" or not hasattr(_measure_facade.os, "geteuid"):
  1681. return True
  1682. return session.get("uid") == _measure_facade.os.geteuid()
  1683. def kill_stale_sessions(threshold_hours=None, dry_run=None, include_orphans=None):
  1684. """Kill Claude Code sessions that have been running longer than threshold_hours.
  1685. Targets abandoned terminal sessions only: a session is terminable when its
  1686. collector positively identified it as a terminal CLI process. Orphans
  1687. (orphan_cli: terminal gone, reparented to init, macOS/Linux) are listed but
  1688. only terminated with include_orphans. Skips the current process's own PID
  1689. and ancestors to avoid self-termination.
  1690. """
  1691. _measure_facade = _compat.facade()
  1692. import signal
  1693. if _measure_facade.detect_runtime() == "codex":
  1694. print("\n Codex processes can host multiple active tasks. Process age cannot identify stale tasks; automatic termination is disabled.")
  1695. return
  1696. health = _measure_facade._collect_health_data()
  1697. if health is None:
  1698. print("\n Session health check is not supported on this platform.")
  1699. return
  1700. # Only ever act on this user's own processes. `ps -e` lists everyone's; as root on
  1701. # a shared box that would otherwise reach another user's conversation. A POSIX
  1702. # session whose owner is unknown is treated as not ours. The Windows collectors
  1703. # carry no owner (see the closing message below).
  1704. running = [s for s in health["running_sessions"] if _measure_facade._kill_stale_owns(s)]
  1705. threshold_seconds = threshold_hours * 3600
  1706. my_pid = _measure_facade.os.getpid()
  1707. my_ppid = _measure_facade.os.getppid()
  1708. # Never terminate the session this command is running inside of: it is an
  1709. # ancestor of this process (claude -> shell -> python), not its direct parent.
  1710. ancestry_unknown = False
  1711. if _measure_facade.os.name == "nt":
  1712. my_ancestors = _measure_facade._windows_ancestor_pids(my_pid)
  1713. if my_ancestors is None:
  1714. ancestry_unknown = True
  1715. my_ancestors = set()
  1716. else:
  1717. my_ancestors = _measure_facade._posix_ancestor_pids(my_pid)
  1718. if my_ancestors is None:
  1719. ancestry_unknown = True
  1720. my_ancestors = set()
  1721. # Fail closed: process age is not evidence that a conversation is
  1722. # abandoned. Sessions whose identity is known to belong to a host app
  1723. # (desktop/SDK/IDE/headless) or could not be established are never
  1724. # terminated, and neither is a session no collector tagged: the default
  1725. # below is "unknown", not "terminal_cli".
  1726. killable = ("terminal_cli", "orphan_cli") if include_orphans else ("terminal_cli",)
  1727. protected = [s for s in running
  1728. if s.get("identity", "unknown") not in ("terminal_cli", "orphan_cli")
  1729. and s["elapsed_seconds"] > threshold_seconds]
  1730. stale = [s for s in running
  1731. if s["elapsed_seconds"] > threshold_seconds
  1732. and s["pid"] != my_pid
  1733. and s["pid"] != my_ppid
  1734. and s["pid"] not in my_ancestors
  1735. and s.get("identity", "unknown") in killable]
  1736. held_orphans = [] if include_orphans else [
  1737. s for s in running
  1738. if s.get("identity") == "orphan_cli"
  1739. and s["elapsed_seconds"] > threshold_seconds
  1740. and s["pid"] not in (my_pid, my_ppid)
  1741. and s["pid"] not in my_ancestors]
  1742. if protected:
  1743. print(f"\n Skipping {len(protected)} long-running session{'s' if len(protected) != 1 else ''} "
  1744. "hosted by the Claude desktop app, an IDE or the SDK, or whose identity could not be verified.")
  1745. print(" Process age alone is not evidence that these are abandoned; they are never auto-terminated.")
  1746. if held_orphans:
  1747. n = len(held_orphans)
  1748. print(f"\n {n} orphaned session{'s' if n != 1 else ''} (terminal gone, running >{threshold_hours}h) "
  1749. f"{'were' if n != 1 else 'was'} left alone. Orphans are only ended on request:")
  1750. print(f" {_measure_facade._measure_cli('kill-stale', '--include-orphans', '--hours', str(threshold_hours))} (add --dry-run to preview)")
  1751. if not stale:
  1752. if protected or held_orphans:
  1753. print(f"\n No terminable stale sessions found (threshold: {threshold_hours}h).")
  1754. return
  1755. print(f"\n No stale sessions found (threshold: {threshold_hours}h).")
  1756. print(f" {len(running)} active session{'s' if len(running) != 1 else ''}, all within threshold.")
  1757. return
  1758. print(f"\n Found {len(stale)} stale session{'s' if len(stale) != 1 else ''} (running >{threshold_hours}h):\n")
  1759. for s in stale:
  1760. flags = " ".join(s.get("flags", []))
  1761. print(f" PID {s['pid']:<7d} {s['elapsed_human']:>10s} v{s.get('version') or '?':<10s} {flags}")
  1762. if dry_run:
  1763. print(f"\n Dry run. Would kill {len(stale)} process{'es' if len(stale) != 1 else ''}.")
  1764. print(" Run without --dry-run to terminate them.\n")
  1765. return
  1766. if ancestry_unknown:
  1767. print("\n Could not read this process's ancestry, so a candidate cannot be ruled out as the")
  1768. print(" conversation this command is running inside of. Nothing was terminated.\n")
  1769. return
  1770. killed = 0
  1771. for s in stale:
  1772. # Re-collect per candidate, immediately before its termination: the
  1773. # window between verification and the signal is as small as a
  1774. # PID-based kill allows (no process handle is retained).
  1775. if not _measure_facade._revalidate_terminal_cli(s, identity=s.get("identity", "unknown")):
  1776. print(f" PID {s['pid']} skipped: identity changed or could not be re-verified.")
  1777. continue
  1778. try:
  1779. _measure_facade.os.kill(s["pid"], signal.SIGTERM)
  1780. killed += 1
  1781. except ProcessLookupError:
  1782. print(f" PID {s['pid']} already gone.")
  1783. except PermissionError:
  1784. print(f" PID {s['pid']} permission denied (owned by another user).")
  1785. except OSError as e:
  1786. # Windows raises a plain OSError (WinError 87), not ProcessLookupError,
  1787. # for a pid that exited between the re-check and the signal. One
  1788. # candidate failing must not skip the rest.
  1789. print(f" PID {s['pid']} could not be signalled (it may have just exited): {e}")
  1790. print(f"\n Terminated {killed} stale session{'s' if killed != 1 else ''}.")
  1791. if killed > 0:
  1792. print(f" These were Claude Code processes running >{threshold_hours}h.")
  1793. if _measure_facade.os.name == "nt":
  1794. # No owner column in the Windows process inventory (a GetOwner call per
  1795. # process is a new slow CIM round trip), so do not claim an account filter.
  1796. print(" Only sessions older than the threshold were targeted. The Windows process list")
  1797. print(" does not record the owning account, so from an elevated shell this can include")
  1798. print(" other accounts' sessions.\n")
  1799. else:
  1800. print(" Your active terminal sessions are unaffected.\n")
  1801. def _read_plugin_version(default=None):
  1802. """Read the shipped version from the plugin manifest, once, at import.
  1803. This was a hand-maintained string carrying the comment "keep in sync with
  1804. plugin.json + marketplace.json". It was not kept in sync: it sat four
  1805. releases behind the manifests, and had drifted across at least two earlier
  1806. releases as well, so a fresh install rendered a stale version in the
  1807. dashboard header. A constant that must be updated by hand on every release
  1808. will eventually be missed, so the manifest is now the single source.
  1809. Falls back to the default when the manifest is absent, which is the case for
  1810. a skill-only install with no plugin directory. Never raises: a version
  1811. string is cosmetic and must not prevent the tool from running.
  1812. """
  1813. _measure_facade = _compat.facade()
  1814. here = _measure_facade.Path(_measure_facade.__file__).resolve()
  1815. for base in here.parents:
  1816. candidate = base / ".claude-plugin" / "plugin.json"
  1817. try:
  1818. if candidate.is_file():
  1819. value = _measure_facade.json.loads(candidate.read_text(encoding="utf-8")).get("version")
  1820. if isinstance(value, str) and value.strip():
  1821. return value.strip()
  1822. except (OSError, ValueError):
  1823. return default
  1824. return default
  1825. def _scheduler_names_to_sweep(this_install_only: bool, all_names, active_name):
  1826. """Scheduler identifiers to unregister.
  1827. ``this_install_only`` scoped the per-identity FILE sweep but not the
  1828. scheduler loops, so a "just this install" uninstall still booted out every
  1829. runtime's LaunchAgent/unit/task -- leaving siblings half-dead (files
  1830. intact, registration gone) and silently tearing down a co-installed
  1831. Codex/Hermes dashboard. Honor the flag on both halves.
  1832. """
  1833. _measure_facade = _compat.facade()
  1834. return (active_name,) if this_install_only else tuple(all_names)
  1835. def _sweep_identity_daemon_files(snap_dir: Path, keys) -> tuple[list[str], list[str]]:
  1836. """Delete the daemon artifacts named by ``keys`` from one identity.
  1837. Returns ``(removed, failed)`` as path strings. Failures are reported
  1838. rather than swallowed so the caller can refuse to claim
  1839. a clean sweep while a 0600 daemon-token is still on disk.
  1840. """
  1841. _measure_facade = _compat.facade()
  1842. files = _measure_facade._daemon_per_identity_files(snap_dir)
  1843. removed: list[str] = []
  1844. failed: list[str] = []
  1845. for key in keys:
  1846. was_removed, did_fail = _measure_facade._unlink_reporting(files[key])
  1847. if was_removed:
  1848. removed.append(str(files[key]))
  1849. elif did_fail:
  1850. failed.append(str(files[key]))
  1851. return removed, failed
  1852. def _print_identity_sweep_report(removed, per_identity_removed, failed, this_install_only):
  1853. """Shared uninstall reporting for all three platforms.
  1854. Honesty rules preserved and extended: never print a
  1855. "Deleted" line for a survivor, keep "Nothing to remove" when nothing
  1856. existed, and NEVER print the all-identities banner when any artifact
  1857. resisted deletion -- a surviving token means the sweep was not clean.
  1858. """
  1859. _measure_facade = _compat.facade()
  1860. if removed:
  1861. print("[Token Optimizer] Dashboard daemon removed.")
  1862. for path in removed:
  1863. print(f" Deleted: {path}")
  1864. if per_identity_removed and not this_install_only and not failed:
  1865. print(" Swept all token-optimizer-* identities:")
  1866. for snap_dir, items in per_identity_removed:
  1867. print(f" {snap_dir}: {len(items)} file(s)")
  1868. elif not failed:
  1869. print("[Token Optimizer] No daemon artifacts found. Nothing to remove.")
  1870. if failed:
  1871. print(
  1872. f" WARNING: {len(failed)} daemon artifact(s) could NOT be removed "
  1873. "(permission denied or in use)."
  1874. )
  1875. for path in failed:
  1876. print(f" Survived: {path}")
  1877. print(
  1878. " A surviving daemon-token is a live CSRF secret -- delete these "
  1879. "manually, then re-run."
  1880. )
  1881. def _daemon_per_identity_files(snapshot_dir: Path) -> dict:
  1882. """Per-identity daemon artifact paths rooted at ``snapshot_dir``.
  1883. These are the files that live INSIDE a single plugin-data identity's data
  1884. dir (one per ``token-optimizer-*`` install): the generated daemon script,
  1885. the 0600 CSRF token, the persisted bind-host, the tombstone, and
  1886. the platform launcher/XML. The OS scheduler REGISTRATION (LaunchAgent
  1887. plist / scheduled task / systemd unit) is per-RUNTIME and shared across
  1888. identities of the same runtime, so it is removed once, not per-identity.
  1889. """
  1890. _measure_facade = _compat.facade()
  1891. return {
  1892. "daemon_script": snapshot_dir / "dashboard-server.py",
  1893. "daemon_token": snapshot_dir / "daemon-token",
  1894. "daemon_host": snapshot_dir / "dashboard-host",
  1895. "thrash_breadcrumb": snapshot_dir / ".daemon-thrash",
  1896. "windows_launcher": snapshot_dir / _measure_facade.WINDOWS_LAUNCHER_NAME,
  1897. "linux_launcher": snapshot_dir / _measure_facade.LINUX_LAUNCHER_NAME,
  1898. }
  1899. def _daemon_identity_snapshot_dirs(this_install_only: bool) -> list[Path]:
  1900. """Snapshot dirs to sweep during daemon uninstall.
  1901. Root cause: daemon paths derive from one module-level ``SNAPSHOT_DIR``
  1902. (the resolved identity), but multiple installs create multiple
  1903. ``token-optimizer-*`` identities. An identity-scoped uninstall leaves the
  1904. sibling's ``dashboard-server.py`` + ``0600 daemon-token`` (a live local
  1905. HTTP daemon plus its CSRF secret) on disk.
  1906. By default (sweep-all): every ``token-optimizer-*`` plugin-data identity's
  1907. ``data`` dir, so no sibling outlives the uninstall. With
  1908. ``this_install_only``: just the resolved ``SNAPSHOT_DIR`` (the pre-fix
  1909. behavior), for a user intentionally running side-by-side installs who only
  1910. wants this one gone. Reuses ``plugin_env._all_plugin_data_dirs`` rather
  1911. than a fourth glob, and always includes the resolved ``SNAPSHOT_DIR`` even
  1912. when it is not under the standard base (e.g. a
  1913. ``TOKEN_OPTIMIZER_SNAPSHOT_DIR`` sandbox override).
  1914. """
  1915. _measure_facade = _compat.facade()
  1916. if this_install_only:
  1917. return [_measure_facade.SNAPSHOT_DIR]
  1918. dirs: list[Path] = []
  1919. try:
  1920. for ident in _measure_facade._all_plugin_data_dirs():
  1921. data_dir = ident / "data"
  1922. # _all_plugin_data_dirs vets the IDENTITY dir, but
  1923. # the dir we actually delete from is its `data` child. A real
  1924. # identity dir whose `data` is a SYMLINK pointed at, say, $HOME
  1925. # made the sweep unlink daemon-token/dashboard-server.py/etc from
  1926. # the symlink target -- outside the plugin-data base entirely.
  1927. # Vet the deletion target itself, not just its parent.
  1928. if _measure_facade._daemon_sweep_dir_is_safe(data_dir):
  1929. dirs.append(data_dir)
  1930. except Exception: # noqa: BLE001
  1931. pass
  1932. if _measure_facade.SNAPSHOT_DIR not in dirs:
  1933. dirs.append(_measure_facade.SNAPSHOT_DIR)
  1934. # Deterministic order, de-duplicated by resolved path.
  1935. seen: set[Path] = set()
  1936. ordered: list[Path] = []
  1937. for d in sorted(dirs, key=lambda p: ((_measure_facade := _compat.facade()), str(p))[1]):
  1938. try:
  1939. r = d.resolve(strict=False)
  1940. except (OSError, ValueError):
  1941. r = d
  1942. if r in seen:
  1943. continue
  1944. seen.add(r)
  1945. ordered.append(d)
  1946. return ordered
  1947. def _daemon_sweep_dir_is_safe(snap_dir: Path) -> bool:
  1948. """True when ``snap_dir`` is a real directory we may delete daemon files from.
  1949. The sweep deletes a fixed allow-list of filenames out of
  1950. every identity's ``data`` dir. That dir must therefore be a REAL directory
  1951. whose realpath lands under the plugin-data base -- otherwise a symlinked
  1952. ``data`` child redirects the unlinks outside the base (the escape found in
  1953. review: ``token-optimizer-evil/data -> $HOME`` deleted files from $HOME).
  1954. The resolved ``SNAPSHOT_DIR`` is always allowed even when it sits outside
  1955. the base, because ``TOKEN_OPTIMIZER_SNAPSHOT_DIR`` (tests, sandboxes) and
  1956. the legacy script-install layout legitimately point elsewhere. That is the
  1957. identity we were told to operate on, not one we discovered by globbing.
  1958. """
  1959. _measure_facade = _compat.facade()
  1960. try:
  1961. if snap_dir.is_symlink() or not snap_dir.is_dir():
  1962. return False
  1963. resolved = snap_dir.resolve(strict=True)
  1964. except (OSError, ValueError):
  1965. return False
  1966. try:
  1967. if resolved == _measure_facade.SNAPSHOT_DIR.resolve(strict=False):
  1968. return True
  1969. except (OSError, ValueError):
  1970. pass
  1971. try:
  1972. # Read the base off the module (not a from-import binding) so the
  1973. # check honors a runtime-repointed plugin-data root.
  1974. base = _measure_facade.plugin_env._PLUGIN_DATA_BASE
  1975. return resolved.is_relative_to(base.resolve(strict=False))
  1976. except (OSError, ValueError, AttributeError):
  1977. return False
  1978. def _unlink_if_exists(path: Path) -> bool:
  1979. """Delete ``path`` if present. Returns True iff a file was removed.
  1980. Never follows a symlink to delete its target: a planted symlink named
  1981. ``daemon-token`` must be removed as the link itself, not as whatever it
  1982. points at (defense in depth behind
  1983. ``_daemon_sweep_dir_is_safe``).
  1984. """
  1985. _measure_facade = _compat.facade()
  1986. try:
  1987. if path.is_symlink():
  1988. path.unlink()
  1989. return True
  1990. if path.exists():
  1991. path.unlink()
  1992. return True
  1993. except OSError:
  1994. pass
  1995. return False
  1996. def _unlink_reporting(path: Path) -> tuple[bool, bool]:
  1997. """Delete ``path``; return ``(removed, failed)``.
  1998. ``_unlink_if_exists`` swallows OSError and returns False,
  1999. which is indistinguishable from "was not there". The sweep then printed
  2000. "Swept all token-optimizer-* identities" while a 0600 daemon-token it
  2001. could not delete (read-only dir, EPERM) was still on disk. This variant
  2002. separates the two so the report can be honest about survivors.
  2003. """
  2004. _measure_facade = _compat.facade()
  2005. try:
  2006. if not path.is_symlink() and not path.exists():
  2007. return (False, False)
  2008. except OSError:
  2009. return (False, True)
  2010. try:
  2011. path.unlink()
  2012. return (True, False)
  2013. except OSError:
  2014. return (False, True)
  2015. def _get_or_create_daemon_token():
  2016. """Return the per-install daemon auth token, creating it on first use.
  2017. v5.4.19 (security H-2/M-4 fix): all state-mutating POST endpoints
  2018. (/api/v5/toggle, /api/skill/archive, /api/mcp/disable, etc.) require
  2019. a matching X-TO-Token header. The token is generated at daemon
  2020. install time, persisted at DAEMON_TOKEN_PATH with 0600 perms, and
  2021. exposed to the dashboard via a same-origin /api/token endpoint.
  2022. Any local same-user process can still read the token file (standard
  2023. Unix permissions), so this is NOT a defense against a same-user
  2024. attacker — it's defense-in-depth against (a) cross-origin drive-by
  2025. POSTs that bypass the empty-Origin CSRF check, (b) DNS-rebinding
  2026. pages that manage to forge an Origin header, and (c) accidental
  2027. discovery by scripts probing localhost ports.
  2028. Idempotent: returns the existing token if the file is already there
  2029. and non-empty. Only creates a new token on first run or if the file
  2030. is missing/empty.
  2031. """
  2032. # v5.11.1: read via _read_state_file (consistent OSError-safe read path).
  2033. _measure_facade = _compat.facade()
  2034. existing = _measure_facade._read_state_file(_measure_facade.DAEMON_TOKEN_PATH)
  2035. if len(existing) >= 16:
  2036. return existing
  2037. try:
  2038. _measure_facade.SNAPSHOT_DIR.mkdir(parents=True, exist_ok=True)
  2039. token = _measure_facade.secrets.token_urlsafe(32) # 43 chars, cryptographically random
  2040. # O_EXCL: atomic create — concurrent installers race on mkdir, not
  2041. # on truncating each other's token.
  2042. fd = _measure_facade.os.open(
  2043. str(_measure_facade.DAEMON_TOKEN_PATH),
  2044. _measure_facade.os.O_WRONLY | _measure_facade.os.O_CREAT | _measure_facade.os.O_EXCL,
  2045. 0o600,
  2046. )
  2047. with _measure_facade.os.fdopen(fd, "w", encoding="utf-8") as f:
  2048. f.write(token + "\n")
  2049. return token
  2050. except FileExistsError:
  2051. # Lost the race — another installer created it first. Read theirs.
  2052. return _measure_facade._read_state_file(_measure_facade.DAEMON_TOKEN_PATH)
  2053. except OSError:
  2054. return ""
  2055. def _read_state_file(path):
  2056. """Read a small single-value state file. Returns '' if absent/unreadable."""
  2057. _measure_facade = _compat.facade()
  2058. try:
  2059. if path.exists():
  2060. return path.read_text(encoding="utf-8").strip()
  2061. except OSError:
  2062. pass
  2063. return ""
  2064. def _read_daemon_token():
  2065. """Read the daemon token from disk. Returns empty string if missing."""
  2066. _measure_facade = _compat.facade()
  2067. return _measure_facade._read_state_file(_measure_facade.DAEMON_TOKEN_PATH)
  2068. def _read_dashboard_host_file():
  2069. """Return the persisted daemon bind-host, or '' if absent/unreadable."""
  2070. _measure_facade = _compat.facade()
  2071. return _measure_facade._read_state_file(_measure_facade.DAEMON_HOST_PATH)
  2072. def _persist_dashboard_host(persist=None):
  2073. """v5.11.1: persist the daemon bind-host so the service-managed daemon
  2074. (launchd/systemd/Task Scheduler) -- which starts with an empty environment --
  2075. binds the host the user asked for instead of always 127.0.0.1.
  2076. Reads TOKEN_OPTIMIZER_DASHBOARD_HOST (fallback TOKEN_OPTIMIZER_HOST):
  2077. - set + valid (in the allowlist): write it to DAEMON_HOST_PATH.
  2078. - set + invalid: warn naming the bad value and the allowed values; do NOT
  2079. persist (and do NOT clobber an existing valid file).
  2080. - unset: leave any existing file untouched. The setting is sticky so a
  2081. network setup survives re-running setup-daemon without the env var
  2082. (e.g. during an upgrade that re-invokes the installer).
  2083. persist=False resolves without writing (dry-run path), so the env > file >
  2084. default chain lives in exactly one place.
  2085. The setting is per-runtime: each daemon (Claude/Codex/Hermes/Copilot) has
  2086. its own DAEMON_HOST_PATH and persists independently, so run setup-daemon
  2087. under each runtime you want to expose on the network.
  2088. Returns the effective host for messaging (env var > file > '127.0.0.1').
  2089. """
  2090. _measure_facade = _compat.facade()
  2091. env_raw = _measure_facade.os.environ.get(
  2092. "TOKEN_OPTIMIZER_DASHBOARD_HOST",
  2093. _measure_facade.os.environ.get("TOKEN_OPTIMIZER_HOST", ""),
  2094. ).strip()
  2095. if env_raw:
  2096. if env_raw in _measure_facade._DAEMON_HOST_ALLOWLIST:
  2097. if persist:
  2098. try:
  2099. _measure_facade.SNAPSHOT_DIR.mkdir(parents=True, exist_ok=True)
  2100. # v5.11.1: O_CREAT|O_TRUNC with 0o600 + fdopen, matching
  2101. # the daemon-token write -- avoids a umask-widened world-readable
  2102. # file and the symlink-replacement write that plain write_text
  2103. # (which follows symlinks) would allow.
  2104. fd = _measure_facade.os.open(
  2105. str(_measure_facade.DAEMON_HOST_PATH),
  2106. _measure_facade.os.O_WRONLY | _measure_facade.os.O_CREAT | _measure_facade.os.O_TRUNC,
  2107. 0o600,
  2108. )
  2109. with _measure_facade.os.fdopen(fd, "w", encoding="utf-8") as f:
  2110. f.write(env_raw + "\n")
  2111. except OSError as e:
  2112. print(
  2113. f"[Token Optimizer] Warning: could not persist dashboard host "
  2114. f"to {_measure_facade.DAEMON_HOST_PATH}: {e}",
  2115. file=_measure_facade.sys.stderr,
  2116. )
  2117. return env_raw
  2118. # Invalid value: warn and fall through to the existing/default host.
  2119. print(
  2120. f"[Token Optimizer] Ignoring invalid dashboard host "
  2121. f"{env_raw!r}. Allowed: {', '.join(_measure_facade._DAEMON_HOST_ALLOWLIST)}.",
  2122. file=_measure_facade.sys.stderr,
  2123. )
  2124. # Env unset or invalid: honor any sticky persisted host, else default.
  2125. return _measure_facade._read_dashboard_host_file() or "127.0.0.1"
  2126. def _is_localhost_host_header(host_header):
  2127. """True iff the Host header is a recognised localhost form (DNS rebinding guard).
  2128. Accepts: 'localhost', 'localhost:<port>', '127.0.0.1', '127.0.0.1:<port>',
  2129. '[::1]', '[::1]:<port>'. Rejects anything else, closing the DNS-rebinding
  2130. leak path where an attacker-controlled hostname resolves to 127.0.0.1
  2131. and the browser sends same-origin requests whose Host is the attacker's
  2132. domain (H-1, 2026-04-16).
  2133. """
  2134. _measure_facade = _compat.facade()
  2135. if not host_header:
  2136. return False
  2137. h = host_header.strip().lower()
  2138. # Strip port if present (bracketed IPv6 is handled below)
  2139. if h.startswith("["):
  2140. # [::1] or [::1]:port
  2141. return h.startswith("[::1]") and (h == "[::1]" or h.startswith("[::1]:"))
  2142. # bare or ipv4-with-port
  2143. base = h.split(":", 1)[0]
  2144. return base in ("localhost", "127.0.0.1")
  2145. def _generate_daemon_script():
  2146. """Generate a minimal Python HTTP server script for the dashboard daemon.
  2147. v5.4.19 hardening (security):
  2148. - C-2: reject empty Origin and non-localhost Origin on POST.
  2149. - H-1: Host header allowlist (localhost / 127.0.0.1 / [::1]) — DNS rebind defense.
  2150. - H-2/M-4: per-install X-TO-Token header required on mutating endpoints.
  2151. - 3-strikes thrash breadcrumb when DASHBOARD is missing (avoid KeepAlive
  2152. hot-loop with launchd/systemd).
  2153. - honor breadcrumb tombstone to self-exit cleanly after uninstall.
  2154. - Serves /api/token to localhost Origin+Host only so dashboard JS can fetch it.
  2155. Paths are interpolated via repr() to produce properly escaped Python string
  2156. literals. Without this, Windows paths containing backslash escape sequences
  2157. (\\U, \\N, \\x, \\t) would raise SyntaxError when the generated daemon starts.
  2158. """
  2159. _measure_facade = _compat.facade()
  2160. dashboard_literal = repr(str(_measure_facade.DASHBOARD_PATH))
  2161. measure_path = _measure_facade.Path(_measure_facade.__file__).resolve()
  2162. measure_py_literal = repr(str(measure_path))
  2163. # A daemon can be generated by a runtime cache (for example .codex) while
  2164. # serving another runtime's plugin-data directory (for example .claude).
  2165. # Keep the marketplace identity so the generated resolver can select the
  2166. # unversioned source belonging to the served runtime before using this path.
  2167. measure_marketplace = "alexgreensh-token-optimizer"
  2168. try:
  2169. cache_index = measure_path.parts.index("cache")
  2170. if cache_index + 1 < len(measure_path.parts):
  2171. measure_marketplace = measure_path.parts[cache_index + 1]
  2172. except ValueError:
  2173. pass
  2174. measure_marketplace_literal = repr(measure_marketplace)
  2175. token_path_literal = repr(str(_measure_facade.DAEMON_TOKEN_PATH))
  2176. host_path_literal = repr(str(_measure_facade.DAEMON_HOST_PATH))
  2177. thrash_path_literal = repr(str(_measure_facade.DAEMON_THRASH_BREADCRUMB))
  2178. log_dir_literal = repr(str(_measure_facade.DAEMON_LOG_DIR))
  2179. tombstone_marker_literal = repr(_measure_facade._UNINSTALL_TOMBSTONE_MARKER)
  2180. magic_literal = repr(_measure_facade.DAEMON_IDENTITY_MAGIC)
  2181. return f'''#!/usr/bin/env python3
  2182. """Token Optimizer dashboard server daemon.
  2183. Auto-generated by measure.py v{_measure_facade.TOKEN_OPTIMIZER_VERSION}. Serves the dashboard HTML on localhost:{_measure_facade.DAEMON_PORT}.
  2184. The SessionEnd hook regenerates the HTML file; this daemon just serves what's on disk.
  2185. """
  2186. # Real module constant (the line above used to live INSIDE the
  2187. # docstring, so /api/health's reference to TOKEN_OPTIMIZER_DAEMON_VERSION raised
  2188. # NameError and the health route crashed on every call). Exposed as a cheap
  2189. # constant the non-blocking health route can answer from without doing work.
  2190. TOKEN_OPTIMIZER_DAEMON_VERSION = "{_measure_facade.TOKEN_OPTIMIZER_VERSION}"
  2191. import hmac
  2192. import http.server
  2193. import json
  2194. import os
  2195. import socketserver
  2196. import subprocess
  2197. import sys
  2198. import threading
  2199. import time
  2200. # The daemon now runs on a ThreadingTCPServer so a synchronous
  2201. # api/regenerate (10-17s) no longer blocks liveness probes (/api/health,
  2202. # /__to_ping) and other requests. Every module global mutated per-request is
  2203. # guarded by _STATE_LOCK so concurrent handler threads cannot race on:
  2204. # - _regen_inflight (overlap refusal)
  2205. # - _last_regen (freshness throttle)
  2206. # - _MEASURE_PY_CACHE (measure.py resolution cache)
  2207. # - _REJECT_LOG_LAST_TS (per-path reject-log throttle)
  2208. # The lock is held only for the brief read/modify/write of each flag, never
  2209. # across the long subprocess.run() of a regen (that would re-introduce the
  2210. # single-threaded block). Security checks (Host allowlist, Origin, hmac token)
  2211. # run unchanged in each thread; they read only constants / a per-request token
  2212. # file, so they need no lock.
  2213. _STATE_LOCK = threading.Lock()
  2214. # Unbounded-thread regression: ThreadingTCPServer spawns
  2215. # one worker thread per accepted connection with NO cap, and daemon_threads=True
  2216. # only affects EXIT, not the live count. A slowloris (many connections that never
  2217. # send a request line) or a flood of valid-token requests would otherwise spawn
  2218. # unbounded threads -- and, via the regen endpoints, unbounded child processes.
  2219. # Two conservative bounds close that (the daemon serves a single local user):
  2220. # - _WORKER_SEM caps concurrently-handled connections. The accept loop acquires
  2221. # it NON-BLOCKING before dispatching a worker, so it never stalls (health
  2222. # probes keep being accepted); on saturation it answers a bare 503 and closes
  2223. # instead of spawning another thread.
  2224. # - DAEMON_REQUEST_TIMEOUT is applied to each connection's socket (Handler.timeout
  2225. # -> settimeout), so a peer that connects but never finishes sending a request
  2226. # line is dropped after the timeout, freeing its worker. Without this a slow/
  2227. # idle connection could pin a bounded worker slot forever (slowloris).
  2228. DAEMON_MAX_WORKERS = 12
  2229. DAEMON_REQUEST_TIMEOUT = 10.0
  2230. _WORKER_SEM = threading.BoundedSemaphore(DAEMON_MAX_WORKERS)
  2231. # The daemon shells out to `python measure.py ...` for the v5-toggle,
  2232. # skill/MCP-manage, and manual-regenerate endpoints. python.exe is a console
  2233. # subsystem binary, so without CREATE_NO_WINDOW each of those spawns flashes a
  2234. # cmd window on Windows. getattr -> 0 on POSIX makes it a no-op there.
  2235. _NO_WINDOW = getattr(subprocess, "CREATE_NO_WINDOW", 0)
  2236. LOG_DIR = {log_dir_literal}
  2237. # The Scheduled Task now launches this script with pythonw.exe
  2238. # (GUI subsystem) instead of the .cmd launcher, because Task Scheduler paints a
  2239. # console window for any console-subsystem action and no creationflags of ours
  2240. # can stop it. pythonw gives the process NO std handles at all, so the
  2241. # stdout.log / stderr.log trail the .cmd used to provide would vanish -- and a
  2242. # silent daemon death is exactly the failure mode that once looked healthy for
  2243. # two days. Reopen the handles onto the same log files before anything runs.
  2244. LOG_CAP_BYTES = 1048576
  2245. def _cap_log(path, max_bytes=LOG_CAP_BYTES):
  2246. """Keep a log under max_bytes by dropping its older half. Truncates in place
  2247. so a handle another process holds (launchd, the shell redirect) stays valid.
  2248. Never raises."""
  2249. try:
  2250. if os.path.getsize(path) <= max_bytes:
  2251. return
  2252. with open(path, "rb") as fh:
  2253. fh.seek(-(max_bytes // 2), os.SEEK_END)
  2254. tail = fh.read()
  2255. nl = tail.find(b"\\n")
  2256. if 0 <= nl < len(tail) - 1:
  2257. tail = tail[nl + 1:]
  2258. with open(path, "r+b") as fh:
  2259. fh.write(tail)
  2260. fh.truncate()
  2261. except OSError:
  2262. pass
  2263. # Daemon logs used to grow forever (every traceback, with local paths).
  2264. for _log_name in ("stdout.log", "stderr.log"):
  2265. _cap_log(os.path.join(LOG_DIR, _log_name))
  2266. if sys.stdout is None or sys.stderr is None:
  2267. try:
  2268. os.makedirs(LOG_DIR, exist_ok=True)
  2269. if sys.stdout is None:
  2270. sys.stdout = open(os.path.join(LOG_DIR, "stdout.log"), "a",
  2271. encoding="utf-8", errors="replace", buffering=1)
  2272. if sys.stderr is None:
  2273. sys.stderr = open(os.path.join(LOG_DIR, "stderr.log"), "a",
  2274. encoding="utf-8", errors="replace", buffering=1)
  2275. except OSError:
  2276. pass
  2277. DASHBOARD = {dashboard_literal}
  2278. TOKEN_PATH = {token_path_literal}
  2279. HOST_PATH = {host_path_literal}
  2280. THRASH_PATH = {thrash_path_literal}
  2281. UNINSTALL_MARKER = {tombstone_marker_literal}
  2282. IDENTITY_MAGIC = {magic_literal}
  2283. PORT = {_measure_facade.DAEMON_PORT}
  2284. # thrash + tombstone guard. If dashboard has been missing for
  2285. # 3 consecutive starts (reset when found), write a tombstone and exit with
  2286. # code 0 so launchd/systemd consider this a "successful exit" and stop
  2287. # respawning us. Uninstall also writes this tombstone directly.
  2288. THRASH_LIMIT = 3
  2289. # Freshness-on-open: if the cached dashboard HTML is older than this when a GET
  2290. # arrives, kick off one background regen (throttled by the same interval) and
  2291. # serve the current file immediately, so the next open reflects recent activity.
  2292. DASHBOARD_FRESH_SECONDS = 120
  2293. _last_regen = 0.0
  2294. # The path this daemon was GENERATED against. Used only as a last-resort fallback:
  2295. # it points into a versioned plugin-cache directory that any later upgrade may prune,
  2296. # which silently broke every regeneration for upgrading users (the daemon kept
  2297. # listening, so health checks stayed green while refresh was dead).
  2298. MEASURE_PY_FALLBACK = {measure_py_literal}
  2299. MEASURE_PY_MARKETPLACE = {measure_marketplace_literal}
  2300. _MEASURE_PY_CACHE = ("", 0.0)
  2301. _regen_inflight = False
  2302. _regen_job = {{}}
  2303. REGEN_STEP_TIMEOUT = 45
  2304. # Regen race: the background stale-while-revalidate refresh
  2305. # is a fire-and-forget Popen, so nothing reaps its child inline. A daemon reaper
  2306. # thread waits for the child and clears _regen_inflight when it exits, bounded by
  2307. # this timeout so a hung child can never wedge the guard True forever (the manual
  2308. # regen path already caps each step at REGEN_STEP_TIMEOUT; this is one step + headroom).
  2309. REGEN_BG_TIMEOUT = 300
  2310. MEASURE_PY_RESOLVE_TTL = 300
  2311. REGEN_LOG = os.path.join(os.path.dirname(DASHBOARD), "daemon-regen.log")
  2312. def _log_regen(msg):
  2313. """Append a regeneration event. Never raises; the daemon must survive a dead log."""
  2314. _cap_log(REGEN_LOG)
  2315. try:
  2316. with open(REGEN_LOG, "a", encoding="utf-8") as f:
  2317. f.write("%s %s\\n" % (time.strftime("%Y-%m-%dT%H:%M:%S"), msg))
  2318. except OSError:
  2319. pass
  2320. def _reap_bg_regen(proc):
  2321. """Wait for a fire-and-forget background-regen child and clear _regen_inflight
  2322. when it exits, so the manual-regen path (which shares the flag under _STATE_LOCK)
  2323. can never overlap a background refresh. Bounded wait: if the child hangs past
  2324. REGEN_BG_TIMEOUT the guard is cleared anyway, so it can never stick True and
  2325. refuse regeneration forever. Runs on a daemon thread; never raises."""
  2326. global _regen_inflight
  2327. try:
  2328. proc.wait(timeout=REGEN_BG_TIMEOUT)
  2329. except Exception: # TimeoutExpired or any wait error: do not hold the guard forever
  2330. _log_regen("background regen reaper: child did not exit within %ss; clearing guard"
  2331. % REGEN_BG_TIMEOUT)
  2332. finally:
  2333. with _STATE_LOCK:
  2334. _regen_inflight = False
  2335. # Rate-limited trace for a rejected api/* POST. The M-4 token check runs BEFORE
  2336. # the api/regenerate handler, so a rejected regenerate click used to leave NO
  2337. # trace in daemon-regen.log -- it stayed empty for weeks while the user reported
  2338. # a dead button. This writes one line per _REJECT_LOG_MIN_GAP seconds PER PATH
  2339. # so a hammering/looping client cannot spam the log, but a real rejected click
  2340. # on a DIFFERENT endpoint is never invisible (was global, which dropped a
  2341. # rejected toggle trace after a recent rejected regenerate). The path is
  2342. # sanitized so a CR in the request path cannot inject/overwrite a log line
  2343. # via terminal carriage-return overprint.
  2344. _REJECT_LOG_LAST_TS: dict = {{}}
  2345. _REJECT_LOG_MIN_GAP = 30.0
  2346. # Cap the per-path throttle dict. It lives for the daemon's lifetime (launchd
  2347. # KeepAlive) and _log_reject_regen fires before token auth, so a client -- remote
  2348. # in opt-in network mode -- POSTing many distinct bad-token paths would otherwise
  2349. # grow it without bound (memory-leak / OOM-restart DoS). Evict oldest half at cap.
  2350. _REJECT_LOG_MAX_KEYS = 512
  2351. def _sanitize_log_path(path):
  2352. """Keep only printable ASCII in *path* so a request path containing control
  2353. chars cannot inject or overwrite a log line. Drops CR/LF AND ESC and every
  2354. other control byte (a bare ESC sequence like \\x1b[2J would clear the screen
  2355. when the log is cat'd). Returns the cleaned string."""
  2356. if not path:
  2357. return ""
  2358. return "".join(c for c in path if 32 <= ord(c) < 127)
  2359. def _log_reject_regen(path):
  2360. """One rate-limited line PER PATH when an api/* POST is rejected for a bad
  2361. token. Per-path throttle so a rejected toggle is still visible after a
  2362. recent rejected regenerate. Path is sanitized before writing."""
  2363. global _REJECT_LOG_LAST_TS
  2364. now = time.time()
  2365. clean = _sanitize_log_path(path)
  2366. # _REJECT_LOG_LAST_TS is a shared module global; under the
  2367. # threading server two POSTs can reject concurrently. Hold _STATE_LOCK only
  2368. # for the dict mutation, not for the file write below.
  2369. with _STATE_LOCK:
  2370. last = _REJECT_LOG_LAST_TS.get(clean, 0.0)
  2371. if now - last < _REJECT_LOG_MIN_GAP:
  2372. return
  2373. _REJECT_LOG_LAST_TS[clean] = now
  2374. if len(_REJECT_LOG_LAST_TS) > _REJECT_LOG_MAX_KEYS:
  2375. _stale = sorted(_REJECT_LOG_LAST_TS, key=_REJECT_LOG_LAST_TS.get)[:len(_REJECT_LOG_LAST_TS) // 2]
  2376. for _k in _stale:
  2377. _REJECT_LOG_LAST_TS.pop(_k, None)
  2378. _cap_log(REGEN_LOG)
  2379. try:
  2380. with open(REGEN_LOG, "a", encoding="utf-8") as f:
  2381. f.write("%s REJECT api/* POST token-mismatch path=%s\\n"
  2382. % (time.strftime("%Y-%m-%dT%H:%M:%S"), clean))
  2383. except OSError:
  2384. pass
  2385. def _resolve_measure_py():
  2386. """Find the NEWEST installed measure.py at call time, not at generation time.
  2387. Mirrors the discovery the token-dashboard skill already does: walk the known
  2388. install roots, read each copy's .claude-plugin/plugin.json version, keep the
  2389. highest. Resolving at call time is the whole point -- a baked absolute path
  2390. dies the moment an upgrade prunes the version that generated this file.
  2391. Cached for MEASURE_PY_RESOLVE_TTL so a busy daemon does not re-walk per request.
  2392. """
  2393. global _MEASURE_PY_CACHE
  2394. # Cache read under _STATE_LOCK so a concurrent freshness regen
  2395. # and a manual regen don't both walk the container and clobber the cache.
  2396. with _STATE_LOCK:
  2397. cached, when = _MEASURE_PY_CACHE
  2398. now = time.time()
  2399. if cached and now - when < MEASURE_PY_RESOLVE_TTL and os.path.isfile(cached):
  2400. return cached
  2401. # SECURITY: resolution stays inside the SAME plugin identity this daemon was
  2402. # generated from. An earlier version walked seven install roots with
  2403. # followlinks=True and picked whichever copy self-reported the highest version
  2404. # in its .claude-plugin/plugin.json, then executed it. That let any unrelated
  2405. # plugin shipping a nested token-optimizer/scripts/measure.py and a manifest
  2406. # claiming a huge version win the contest, and let a symlink under any walked
  2407. # root escape the roots entirely -- a code-execution path where there had
  2408. # previously been a static self-reference. Narrowed deliberately:
  2409. # - no directory walk and no symlink following
  2410. # - candidates must be siblings under the generating identity's version
  2411. # container, so nothing new is trusted
  2412. # - the version comes from the DIRECTORY NAME, not from file contents any
  2413. # other writer controls
  2414. # - the resolved path is containment-checked before it is ever executed
  2415. fallback = MEASURE_PY_FALLBACK
  2416. # Resolve against the install whose dashboard is actually being served.
  2417. # A .claude data dashboard may be generated by a .codex measure.py process;
  2418. # executing the fallback in that case writes a different snapshot and makes
  2419. # POST /api/regenerate return success without changing the visible file.
  2420. # The marketplace clone is version-independent and remains inside the same
  2421. # runtime's plugins root. Legacy/backups dashboards have no plugin-data
  2422. # shape, so they continue through the identity-scoped fallback below.
  2423. try:
  2424. dashboard_dir = os.path.realpath(os.path.dirname(DASHBOARD))
  2425. dashboard_parts = dashboard_dir.split(os.sep)
  2426. data_idx = next(
  2427. i for i in range(1, len(dashboard_parts))
  2428. if dashboard_parts[i - 1] == "plugins" and dashboard_parts[i] == "data")
  2429. if data_idx >= 1:
  2430. plugins_root = os.sep.join(dashboard_parts[:data_idx])
  2431. marketplaces_root = os.path.realpath(os.path.join(plugins_root, "marketplaces"))
  2432. # 5.13.3: a daemon generated by one runtime (.codex) can serve
  2433. # another runtime's (.claude) data dir. The baked MEASURE_PY_MARKETPLACE
  2434. # is the GENERATING runtime's identity, so under the SERVED runtime it
  2435. # may name a marketplace dir that does not exist -- the lookup missed
  2436. # and fell through to the stale cross-runtime fallback, so regenerate
  2437. # returned ok while the visible file never changed. Try the baked
  2438. # identity first, then discover the served runtime's OWN token-optimizer
  2439. # marketplace by scanning ITS marketplaces root. That root is the served
  2440. # runtime's own plugins tree (same trust boundary as this daemon);
  2441. # symlinks are rejected and every candidate is containment-checked.
  2442. served_identities = []
  2443. if MEASURE_PY_MARKETPLACE:
  2444. served_identities.append(MEASURE_PY_MARKETPLACE)
  2445. try:
  2446. for entry in sorted(os.listdir(marketplaces_root)):
  2447. if entry not in served_identities and "token-optimizer" in entry:
  2448. served_identities.append(entry)
  2449. except OSError:
  2450. pass
  2451. for served_ident in served_identities:
  2452. served_candidate = os.path.join(
  2453. marketplaces_root, served_ident,
  2454. "skills", "token-optimizer", "scripts", "measure.py")
  2455. served_real = os.path.realpath(served_candidate)
  2456. if (os.path.isfile(served_candidate) and not os.path.islink(served_candidate)
  2457. and (served_real == marketplaces_root
  2458. or served_real.startswith(marketplaces_root + os.sep))):
  2459. with _STATE_LOCK:
  2460. _MEASURE_PY_CACHE = (served_real, now)
  2461. return served_real
  2462. except (OSError, ValueError, StopIteration):
  2463. pass
  2464. # .../<identity>/token-optimizer/<version>/skills/token-optimizer/scripts/measure.py
  2465. version_dir = os.path.dirname(os.path.dirname(os.path.dirname(
  2466. os.path.dirname(os.path.abspath(fallback)))))
  2467. container = os.path.dirname(version_dir)
  2468. suffix = os.path.relpath(os.path.abspath(fallback), version_dir)
  2469. best_path, best_ver = "", None
  2470. try:
  2471. entries = os.listdir(container)
  2472. except OSError:
  2473. entries = []
  2474. container_real = os.path.realpath(container)
  2475. for name in entries:
  2476. cand_dir = os.path.join(container, name)
  2477. if os.path.islink(cand_dir) or not os.path.isdir(cand_dir):
  2478. continue
  2479. # Take the leading numeric run and stop at the first non-numeric
  2480. # component. A plain isdigit() filter silently DROPPED such parts, so
  2481. # "5.11.66-rc1" became (5, 11) and sorted BELOW (5, 11, 60) -- the
  2482. # daemon would pick an older stable over the newer build actually
  2483. # installed. Stopping instead of skipping keeps the tuple positional.
  2484. parts = []
  2485. for chunk in name.split("."):
  2486. digits = ""
  2487. for ch in chunk:
  2488. if not ch.isdigit():
  2489. break
  2490. digits += ch
  2491. if not digits:
  2492. break
  2493. parts.append(int(digits))
  2494. if not parts:
  2495. continue
  2496. cand = os.path.join(cand_dir, suffix)
  2497. if not os.path.isfile(cand) or os.path.islink(cand):
  2498. continue
  2499. # Containment: the real path must still sit under the real container.
  2500. cand_real = os.path.realpath(cand)
  2501. if not (cand_real == container_real or
  2502. cand_real.startswith(container_real + os.sep)):
  2503. continue
  2504. ver = tuple(parts)
  2505. if best_ver is None or ver > best_ver:
  2506. best_ver, best_path = ver, cand
  2507. if best_path:
  2508. with _STATE_LOCK:
  2509. _MEASURE_PY_CACHE = (best_path, now)
  2510. return best_path
  2511. if os.path.isfile(fallback):
  2512. return fallback
  2513. _log_regen("resolve failed: no installed measure.py under %s" % container)
  2514. return fallback
  2515. def _read_token():
  2516. """Read per-install CSRF token from disk. Empty string if missing/unreadable."""
  2517. try:
  2518. with open(TOKEN_PATH, "r", encoding="utf-8") as f:
  2519. return f.read().strip()
  2520. except OSError:
  2521. return ""
  2522. def _is_localhost_host(host_header):
  2523. """True iff Host header is a localhost form. Defeats DNS rebinding."""
  2524. if not host_header:
  2525. return False
  2526. h = host_header.strip().lower()
  2527. if h.startswith("["):
  2528. return h.startswith("[::1]") and (h == "[::1]" or h.startswith("[::1]:"))
  2529. base = h.split(":", 1)[0]
  2530. return base in ("localhost", "127.0.0.1")
  2531. def _is_localhost_origin(origin):
  2532. """True iff Origin header is http(s)://localhost|127.0.0.1|[::1] or file://.
  2533. v5.4.19 S-2 tightening: require the host to terminate (end-of-string,
  2534. ':', or '/') so a malicious Origin like 'http://127.0.0.1.evil.com'
  2535. cannot masquerade as localhost.
  2536. """
  2537. if not origin:
  2538. return False
  2539. for prefix in ("http://localhost", "http://127.0.0.1", "http://[::1]",
  2540. "https://localhost", "https://127.0.0.1", "https://[::1]"):
  2541. if origin.startswith(prefix):
  2542. tail = origin[len(prefix):]
  2543. if tail == "" or tail.startswith(":") or tail.startswith("/"):
  2544. return True
  2545. return origin.startswith("file://")
  2546. class Handler(http.server.SimpleHTTPRequestHandler):
  2547. # Slowloris defense: StreamRequestHandler.setup() applies this
  2548. # as self.connection.settimeout(...), so a peer that connects but never finishes
  2549. # sending a request line is dropped after DAEMON_REQUEST_TIMEOUT (raising
  2550. # socket.timeout, which BaseHTTPRequestHandler.handle_one_request catches and
  2551. # closes). Without it an idle connection would pin a bounded worker slot forever.
  2552. timeout = DAEMON_REQUEST_TIMEOUT
  2553. def __init__(self, *a, **kw):
  2554. d = os.path.dirname(DASHBOARD)
  2555. super().__init__(*a, directory=d, **kw)
  2556. def log_message(self, fmt, *a):
  2557. pass
  2558. def end_headers(self):
  2559. self.send_header("X-Content-Type-Options", "nosniff")
  2560. self.send_header("X-Frame-Options", "DENY")
  2561. self.send_header("Cache-Control", "no-cache")
  2562. self.send_header("Referrer-Policy", "no-referrer")
  2563. self.send_header(
  2564. "Content-Security-Policy",
  2565. "default-src 'none'; script-src 'unsafe-inline'; style-src 'unsafe-inline' https://fonts.googleapis.com; font-src https://fonts.gstatic.com; connect-src 'self'; img-src 'self' data:; base-uri 'none'; form-action 'none'; frame-ancestors 'none'",
  2566. )
  2567. super().end_headers()
  2568. def _is_dashboard_request(self):
  2569. f = os.path.basename(DASHBOARD)
  2570. clean = self.path.lstrip("/").split("?")[0]
  2571. return clean in ("", "token-optimizer", f)
  2572. def _is_identity_request(self):
  2573. return self.path.lstrip("/").split("?")[0] == "__to_ping"
  2574. def _respond_identity(self):
  2575. # Return the magic string so foreign listeners on PORT can't masquerade.
  2576. body = IDENTITY_MAGIC.encode("utf-8")
  2577. self.send_response(200)
  2578. self.send_header("Content-Type", "text/plain; charset=utf-8")
  2579. self.send_header("Content-Length", str(len(body)))
  2580. self.end_headers()
  2581. self.wfile.write(body)
  2582. def _json_response(self, code, data):
  2583. body = json.dumps(data).encode("utf-8")
  2584. self.send_response(code)
  2585. self.send_header("Content-Type", "application/json; charset=utf-8")
  2586. self.send_header("Content-Length", str(len(body)))
  2587. origin = self.headers.get("Origin", "")
  2588. if NETWORK_MODE:
  2589. allowed = origin or "*"
  2590. else:
  2591. allowed = origin if _is_localhost_origin(origin) else "http://localhost:{_measure_facade.DAEMON_PORT}"
  2592. self.send_header("Access-Control-Allow-Origin", allowed)
  2593. if allowed != "*":
  2594. self.send_header("Access-Control-Allow-Credentials", "true")
  2595. self.end_headers()
  2596. self.wfile.write(body)
  2597. def _require_localhost(self):
  2598. """Reject non-localhost Host headers unless NETWORK_MODE is active.
  2599. H-1 DNS rebinding defense — fires before we look at Origin/token.
  2600. In network mode, any Host is accepted (remote clients send their own).
  2601. """
  2602. if NETWORK_MODE:
  2603. return True
  2604. if not _is_localhost_host(self.headers.get("Host", "")):
  2605. self.send_error(421, "Misdirected Request")
  2606. return False
  2607. return True
  2608. def _serve_or_redirect(self, method):
  2609. if self._is_identity_request():
  2610. self._respond_identity()
  2611. return
  2612. clean = self.path.lstrip("/").split("?")[0]
  2613. if clean == "api/health":
  2614. self._json_response(200, {{"ok": True, "server": "token-optimizer-daemon", "version": TOKEN_OPTIMIZER_DAEMON_VERSION}})
  2615. return
  2616. if clean == "api/token":
  2617. # v5.11.1: the token endpoint is loopback-locked in BOTH modes.
  2618. # In network mode we serve the dashboard HTML to LAN visitors (view-
  2619. # only) but only hand out the CSRF token to clients connecting FROM
  2620. # the machine itself (client_address loopback), so a LAN visitor can
  2621. # never obtain the token and therefore can never toggle. Mutations
  2622. # are effectively localhost-only. In default mode we additionally
  2623. # require localhost Origin+Host (anti-exfiltration / DNS-rebind).
  2624. if NETWORK_MODE:
  2625. if self.client_address[0] not in ("127.0.0.1", "::1"):
  2626. self._json_response(403, {{"ok": False, "msg": "token endpoint is loopback-only"}})
  2627. return
  2628. else:
  2629. origin = self.headers.get("Origin", "")
  2630. origin_ok = (not origin) or _is_localhost_origin(origin)
  2631. if not origin_ok or not _is_localhost_host(self.headers.get("Host", "")):
  2632. self.send_error(403, "Forbidden")
  2633. return
  2634. self._json_response(200, {{"token": _read_token()}})
  2635. return
  2636. if not self._require_localhost():
  2637. return
  2638. if self._is_dashboard_request():
  2639. if method == "do_GET":
  2640. self._maybe_refresh_dashboard()
  2641. self.path = "/" + os.path.basename(DASHBOARD)
  2642. getattr(super(), method)()
  2643. else:
  2644. self.send_error(403, "Forbidden")
  2645. def _maybe_refresh_dashboard(self):
  2646. # Stale-while-revalidate: serve the current HTML now, and if it's stale
  2647. # kick off ONE throttled background regen so the next open is fresh. Never
  2648. # blocks the request; failures are silently ignored (best-effort).
  2649. global _last_regen, _regen_inflight
  2650. try:
  2651. mtime = os.path.getmtime(DASHBOARD)
  2652. except OSError:
  2653. return
  2654. now = time.time()
  2655. # _last_regen is a shared throttle stamp; under the threading
  2656. # server two GETs can race the stale check. Hold _STATE_LOCK only for the
  2657. # read+write of the stamp, not for the subprocess.Popen below.
  2658. #
  2659. # Regen race: the manual POST /api/regenerate path
  2660. # guards overlap with _regen_inflight; this background path used to touch
  2661. # only _last_regen, so under the threading server a background refresh and a
  2662. # manual regen (or two backgrounds) could run concurrently. Claim the SAME
  2663. # _regen_inflight flag here under the SAME lock: bail if a regen is already
  2664. # in flight, otherwise set it True and let _reap_bg_regen clear it when the
  2665. # fire-and-forget child exits (bounded, so it can never stick True).
  2666. with _STATE_LOCK:
  2667. if _regen_inflight:
  2668. return
  2669. if now - mtime <= DASHBOARD_FRESH_SECONDS or now - _last_regen <= DASHBOARD_FRESH_SECONDS:
  2670. return
  2671. _last_regen = now
  2672. _regen_inflight = True
  2673. # From here the flag is CLAIMED. It is handed to _reap_bg_regen only once the
  2674. # child is actually spawned; on every earlier exit path we must clear it
  2675. # ourselves or it would wedge True and refuse all future regeneration.
  2676. spawned = False
  2677. errf = None
  2678. try:
  2679. import subprocess
  2680. target = _resolve_measure_py()
  2681. if not os.path.isfile(target):
  2682. _log_regen("SKIP regen: measure.py not found at %s" % target)
  2683. return
  2684. # stderr is captured to a log rather than discarded. Swallowing it is what
  2685. # let a dead regeneration look identical to a healthy one for two days.
  2686. _cap_log(REGEN_LOG)
  2687. errf = open(REGEN_LOG, "a", encoding="utf-8")
  2688. # The regen child is a transient worker (writes the new HTML and
  2689. # exits); it does NOT need to survive the daemon, so DETACHED_PROCESS
  2690. # would be wrong. On Windows it only needs CREATE_NO_WINDOW to avoid
  2691. # a console flash; POSIX detaches via start_new_session. Inlined
  2692. # (not a shared helper) because this template generates a STANDALONE
  2693. # daemon script with no sibling helper module on its sys.path.
  2694. _regen_kwargs = dict(stdout=subprocess.DEVNULL, stderr=errf,
  2695. stdin=subprocess.DEVNULL, close_fds=True)
  2696. if os.name == "nt":
  2697. _flags = getattr(subprocess, "CREATE_NO_WINDOW", 0)
  2698. if _flags:
  2699. _regen_kwargs["creationflags"] = _flags
  2700. else:
  2701. _regen_kwargs["start_new_session"] = True
  2702. proc = subprocess.Popen(
  2703. [sys.executable, target, "dashboard", "--quiet"],
  2704. **_regen_kwargs,
  2705. )
  2706. # Hand the guard's lifetime to a daemon reaper: it clears _regen_inflight
  2707. # when the child exits (or after REGEN_BG_TIMEOUT if it hangs), so the
  2708. # manual regen path cannot overlap this one and the flag never sticks.
  2709. threading.Thread(target=_reap_bg_regen, args=(proc,), daemon=True).start()
  2710. spawned = True
  2711. except Exception as exc: # best-effort background path: never crash the serving thread
  2712. _log_regen("regen launch failed: %s" % exc)
  2713. finally:
  2714. if errf is not None:
  2715. # Popen duplicates the fd via start_new_session/creationflags;
  2716. # the parent must close its own copy so the file handle doesn't
  2717. # leak per stale-open (EMFILE after ~1024 regens).
  2718. errf.close()
  2719. if not spawned:
  2720. # Never reached the reaper -> release the guard we claimed so a
  2721. # failed/skipped background refresh can't block every future regen.
  2722. with _STATE_LOCK:
  2723. _regen_inflight = False
  2724. def do_OPTIONS(self):
  2725. self.send_response(200)
  2726. origin = self.headers.get("Origin", "")
  2727. if NETWORK_MODE:
  2728. allowed = origin or "*"
  2729. else:
  2730. allowed = origin if _is_localhost_origin(origin) else "http://localhost:{_measure_facade.DAEMON_PORT}"
  2731. self.send_header("Access-Control-Allow-Origin", allowed)
  2732. self.send_header("Access-Control-Allow-Methods", "GET, POST, OPTIONS")
  2733. self.send_header("Access-Control-Allow-Headers", "Content-Type, X-TO-Token")
  2734. if allowed != "*":
  2735. self.send_header("Access-Control-Allow-Credentials", "true")
  2736. self.end_headers()
  2737. def do_POST(self):
  2738. # Layered defense: Host allowlist (skipped in network mode), then Origin, then token.
  2739. if not NETWORK_MODE and not _is_localhost_host(self.headers.get("Host", "")):
  2740. self.send_error(421, "Misdirected Request")
  2741. return
  2742. origin = self.headers.get("Origin", "")
  2743. if not NETWORK_MODE and (not origin or not _is_localhost_origin(origin)):
  2744. self.send_error(403, "Forbidden: invalid origin")
  2745. return
  2746. expected_tok = _read_token()
  2747. got_tok = self.headers.get("X-TO-Token", "")
  2748. clean = self.path.lstrip("/").split("?")[0]
  2749. if not expected_tok or not hmac.compare_digest(expected_tok, got_tok):
  2750. # Leave a trace. This check fires BEFORE the api/regenerate
  2751. # handler, so without a log line a rejected regenerate click was
  2752. # indistinguishable from a dead button (daemon-regen.log stayed
  2753. # empty for weeks). Rate-limited so a looping client can't spam.
  2754. _log_reject_regen(clean)
  2755. self.send_error(403, "Forbidden: invalid token")
  2756. return
  2757. if clean == "api/v5/toggle":
  2758. length = int(self.headers.get("Content-Length", 0))
  2759. if length > 4096:
  2760. self._json_response(413, {{"ok": False, "msg": "payload too large"}})
  2761. return
  2762. try:
  2763. body = json.loads(self.rfile.read(length).decode("utf-8")) if length else {{}}
  2764. except (ValueError, UnicodeDecodeError):
  2765. self._json_response(400, {{"ok": False, "msg": "invalid json"}})
  2766. return
  2767. name = body.get("name", "")
  2768. enabled = bool(body.get("enabled", False))
  2769. # Validate name: alphanumeric + hyphens/underscores only.
  2770. # Prevents argument injection via subprocess argv.
  2771. import re as _re, subprocess
  2772. if not name or not _re.match(r'^[a-zA-Z0-9_-]{{1,64}}$', name):
  2773. self._json_response(400, {{"ok": False, "msg": "invalid feature name"}})
  2774. return
  2775. action = "enable" if enabled else "disable"
  2776. try:
  2777. result = subprocess.run(
  2778. [sys.executable, _resolve_measure_py(), "v5", action, name, "--json"],
  2779. capture_output=True, text=True, timeout=10,
  2780. creationflags=_NO_WINDOW
  2781. )
  2782. except (subprocess.TimeoutExpired, OSError) as e:
  2783. self._json_response(500, {{"ok": False, "msg": "toggle backend unavailable: " + str(e)}})
  2784. return
  2785. if result.returncode == 0:
  2786. v5_features = {{}}
  2787. try:
  2788. parsed = json.loads(result.stdout)
  2789. v5_features = parsed.get("v5_features", {{}})
  2790. except (ValueError, KeyError):
  2791. pass
  2792. state = "ENABLED" if enabled else "DISABLED"
  2793. self._json_response(200, {{"ok": True, "msg": state + ": " + name, "v5_features": v5_features}})
  2794. else:
  2795. self._json_response(500, {{"ok": False, "msg": result.stderr.strip()}})
  2796. return
  2797. if clean in ("api/skill/archive", "api/skill/restore", "api/mcp/disable", "api/mcp/enable"):
  2798. # Skill/MCP management. The dashboard POSTs here; the daemon proxies
  2799. # to `measure.py api-manage <endpoint> <name>` (single runtime-aware
  2800. # source of truth) and forwards its JSON. Previously these paths fell
  2801. # through to the 403 below, which the page surfaced as a misleading
  2802. # "Authentication failed" -- breaking skill/MCP toggles for every
  2803. # daemon user even though auth was fine.
  2804. length = int(self.headers.get("Content-Length", 0))
  2805. if length > 4096:
  2806. self._json_response(413, {{"ok": False, "msg": "payload too large"}})
  2807. return
  2808. try:
  2809. body = json.loads(self.rfile.read(length).decode("utf-8")) if length else {{}}
  2810. except (ValueError, UnicodeDecodeError):
  2811. self._json_response(400, {{"ok": False, "msg": "invalid json"}})
  2812. return
  2813. name = body.get("name", "")
  2814. # Permit real skill/MCP names (hyphens, dots, slashes, @, spaces);
  2815. # passed as a single argv element (no shell) so this is belt-and-
  2816. # suspenders against absurd input, not a shell-injection guard.
  2817. import re as _re, subprocess
  2818. if not name or not _re.match(r'^[A-Za-z0-9_.@:/ -]{{1,160}}$', name):
  2819. self._json_response(400, {{"ok": False, "msg": "invalid name"}})
  2820. return
  2821. endpoint = clean[len("api/"):] # e.g. "skill/archive"
  2822. try:
  2823. result = subprocess.run(
  2824. [sys.executable, _resolve_measure_py(), "api-manage", endpoint, name],
  2825. capture_output=True, text=True, timeout=15,
  2826. creationflags=_NO_WINDOW
  2827. )
  2828. except (subprocess.TimeoutExpired, OSError) as e:
  2829. self._json_response(500, {{"ok": False, "msg": "manage backend unavailable: " + str(e)}})
  2830. return
  2831. try:
  2832. payload = json.loads(result.stdout)
  2833. except (ValueError, KeyError):
  2834. self._json_response(500, {{"ok": False, "msg": (result.stderr or result.stdout or "manage failed").strip()[:200]}})
  2835. return
  2836. self._json_response(200 if payload.get("ok") else 500, payload)
  2837. return
  2838. if clean in ("api/regenerate", "api/regenerate/status"):
  2839. import subprocess
  2840. try:
  2841. length = int(self.headers.get("Content-Length", 0))
  2842. if length < 0 or length > 4096: raise ValueError()
  2843. body = json.loads(self.rfile.read(length).decode("utf-8")) if length else {{}}
  2844. if not isinstance(body, dict): raise ValueError()
  2845. days = body.get("days")
  2846. if days is not None and not (days == "all" or type(days) is int and 1 <= days <= 365000): raise ValueError()
  2847. except (ValueError, TypeError, AttributeError, UnicodeDecodeError):
  2848. self._json_response(400, {{"ok": False, "msg": "Invalid request."}})
  2849. return
  2850. global _regen_inflight, _regen_job
  2851. if clean.endswith("/status"):
  2852. with _STATE_LOCK:
  2853. result = dict(_regen_job) if body.get("job_id") == _regen_job.get("job_id") and _regen_job else None
  2854. self._json_response(200 if result else 404, result or {{"ok": False, "msg": "Regeneration job not found."}})
  2855. return
  2856. try:
  2857. job_id = os.urandom(16).hex()
  2858. except OSError:
  2859. self._json_response(500, {{"ok": False, "msg": "Could not start the update. Please try again."}})
  2860. return
  2861. with _STATE_LOCK:
  2862. if _regen_inflight:
  2863. self._json_response(409, {{"ok": False, "msg": "A regeneration is already running."}})
  2864. return
  2865. _regen_inflight = True
  2866. _regen_job = {{"ok": True, "job_id": job_id, "state": "running", "progress": "Collecting usage data."}}
  2867. def run_regen():
  2868. global _regen_inflight
  2869. try:
  2870. target = _resolve_measure_py()
  2871. if not os.path.isfile(target): raise OSError("measure.py missing")
  2872. for step in ("collect", "dashboard"):
  2873. with _STATE_LOCK:
  2874. _regen_job["progress"] = "Collecting usage data." if step == "collect" else "Updating the dashboard."
  2875. argv = [sys.executable, target, step, "--quiet"]
  2876. if step == "dashboard":
  2877. argv.append("--force")
  2878. if days is not None: argv.extend(["--days", str(days)])
  2879. result = subprocess.run(argv, capture_output=True, text=True,
  2880. timeout=180 if step == "dashboard" else REGEN_STEP_TIMEOUT,
  2881. creationflags=_NO_WINDOW, env={{**os.environ, "TOKEN_OPTIMIZER_INTERACTIVE": "1"}})
  2882. if result.returncode:
  2883. _log_regen("MANUAL regen failed at %s: %s" % (step, (result.stderr or result.stdout)[:300]))
  2884. raise RuntimeError("regeneration failed")
  2885. with _STATE_LOCK:
  2886. _regen_job.update(state="done", progress="Dashboard updated.")
  2887. except Exception as exc:
  2888. _log_regen("MANUAL regen failed: %s" % exc)
  2889. with _STATE_LOCK:
  2890. _regen_job.update(state="failed", progress="Could not update the dashboard. Please try again.")
  2891. finally:
  2892. with _STATE_LOCK:
  2893. _regen_inflight = False
  2894. try:
  2895. threading.Thread(target=run_regen, daemon=True).start()
  2896. except Exception:
  2897. with _STATE_LOCK:
  2898. _regen_inflight = False
  2899. _regen_job.update(state="failed", progress="Could not start the update. Please try again.")
  2900. self._json_response(202, {{"ok": True, "job_id": job_id}})
  2901. return
  2902. self.send_error(403, "Forbidden")
  2903. def do_GET(self):
  2904. self._serve_or_redirect("do_GET")
  2905. def do_HEAD(self):
  2906. self._serve_or_redirect("do_HEAD")
  2907. def _thrash_check_and_update():
  2908. """Return True to continue, False to tombstone-and-exit cleanly."""
  2909. # two breadcrumb kinds live at THRASH_PATH.
  2910. # - UNINSTALL tombstone: content == UNINSTALL_MARKER. Written by
  2911. # `measure.py` uninstall; it must OUTLIVE the uninstall and is cleared
  2912. # ONLY by a reinstall (setup_daemon unlinks it). Honor it FOREVER -- a
  2913. # surviving daemon script must never resurrect and mint a fresh CSRF
  2914. # token after a reported uninstall.
  2915. # - THRASH tombstone: EMPTY file, written by the 3-strikes path below when
  2916. # the dashboard has been missing. That is transient (e.g. an update
  2917. # window), so it self-heals after 60s once the dashboard is back.
  2918. if os.path.exists(THRASH_PATH):
  2919. try:
  2920. with open(THRASH_PATH, "r", encoding="utf-8") as f:
  2921. content = f.read().strip()
  2922. except OSError:
  2923. content = ""
  2924. if content == UNINSTALL_MARKER:
  2925. # Real uninstall: stay dead. Never self-clear, never serve.
  2926. return False
  2927. if content == "":
  2928. # Empty = 3-strikes thrash tombstone. Self-heal only when STALE: a
  2929. # stuck tombstone next to a healthy dashboard with this daemon still
  2930. # running is a thrash false-positive, not an uninstall.
  2931. try:
  2932. age = time.time() - os.path.getmtime(THRASH_PATH)
  2933. except OSError:
  2934. age = 0 # unreadable mtime: treat as fresh, preserve safety
  2935. if age < 60:
  2936. return False
  2937. try:
  2938. os.unlink(THRASH_PATH)
  2939. except OSError:
  2940. pass
  2941. # else: non-empty, non-marker -> a numeric thrash counter; fall through
  2942. # to the dashboard check / counter processing below.
  2943. if os.path.exists(DASHBOARD):
  2944. # Healthy start: clear any thrash counter.
  2945. try:
  2946. if os.path.exists(THRASH_PATH):
  2947. os.unlink(THRASH_PATH)
  2948. except OSError:
  2949. pass
  2950. return True
  2951. # dashboard is missing. Increment 3-strikes counter.
  2952. count = 0
  2953. try:
  2954. with open(THRASH_PATH, "r", encoding="utf-8") as f:
  2955. count = int((f.read() or "0").strip() or "0")
  2956. except (OSError, ValueError):
  2957. count = 0
  2958. count += 1
  2959. try:
  2960. os.makedirs(os.path.dirname(THRASH_PATH), exist_ok=True)
  2961. with open(THRASH_PATH, "w", encoding="utf-8") as f:
  2962. f.write(str(count) + "\\n")
  2963. except OSError:
  2964. pass
  2965. if count >= THRASH_LIMIT:
  2966. # Write tombstone (empty file) so future respawns exit immediately.
  2967. try:
  2968. with open(THRASH_PATH, "w", encoding="utf-8") as f:
  2969. f.write("")
  2970. except OSError:
  2971. pass
  2972. return False
  2973. # Under threshold: still exit 0 (don't fight KeepAlive) but without tombstone,
  2974. # so next SessionEnd regenerating dashboard.html will reset the counter.
  2975. return False
  2976. if not _thrash_check_and_update():
  2977. # Exit 0 so launchd's "KeepAlive on unsuccessful exit" stops respawning us.
  2978. sys.exit(0)
  2979. # Bind address. Default: localhost only. TOKEN_OPTIMIZER_DASHBOARD_HOST=0.0.0.0 enables
  2980. # network access (Tailscale Funnel, LAN). When network-bound, Host header checks are
  2981. # relaxed since remote clients send non-localhost Host headers. Token auth still applies.
  2982. # Resolution: env var -> persisted host file (HOST_PATH, written by
  2983. # setup-daemon) -> 127.0.0.1. The service manager starts us with an empty
  2984. # environment, so the env var is usually absent here; the host file carries the
  2985. # user's choice across to the daemon and survives version-bump auto-regen.
  2986. _LOCALHOST_ADDRS = ("127.0.0.1", "::1", "localhost")
  2987. _host_raw = os.environ.get("TOKEN_OPTIMIZER_DASHBOARD_HOST",
  2988. os.environ.get("TOKEN_OPTIMIZER_HOST", "")).strip()
  2989. if not _host_raw:
  2990. try:
  2991. with open(HOST_PATH, "r", encoding="utf-8") as _hf:
  2992. _host_raw = _hf.read().strip()
  2993. except FileNotFoundError:
  2994. # Normal: setup-daemon never persisted a host. Default to loopback.
  2995. _host_raw = ""
  2996. except OSError as _e:
  2997. # v5.11.1: a non-missing read failure (e.g. PermissionError) is
  2998. # worth a one-line note in the service-manager logs before we silently
  2999. # fall back to loopback -- otherwise a misconfigured host file looks
  3000. # like the daemon ignored the setting for no reason.
  3001. print(f"[Token Optimizer] Warning: could not read dashboard host file {{HOST_PATH}}: {{_e}}", file=sys.stderr)
  3002. _host_raw = ""
  3003. if not _host_raw:
  3004. _host_raw = "127.0.0.1"
  3005. HOST = _host_raw if _host_raw in _LOCALHOST_ADDRS + ("0.0.0.0",) else "127.0.0.1"
  3006. NETWORK_MODE = HOST == "0.0.0.0"
  3007. if NETWORK_MODE:
  3008. print(f"[Token Optimizer] Network mode: binding {{HOST}}:{{PORT}}", file=sys.stderr)
  3009. print(" Dashboard and toggle API accessible from LAN. Token auth required for mutations.", file=sys.stderr)
  3010. try:
  3011. # ThreadingTCPServer so a synchronous api/regenerate (10-17s)
  3012. # does NOT block liveness probes (/api/health, /__to_ping) or other
  3013. # requests. daemon_threads=True so worker threads die with the daemon and
  3014. # never block process exit; allow_reuse_address=True so a fast restart
  3015. # (KeepAlive respawn) does not hit TIME_WAIT on the port. Per-request shared
  3016. # state is guarded by _STATE_LOCK; the _regen_inflight flag still refuses
  3017. # overlapping regens. Security (Host allowlist, Origin, hmac token) runs
  3018. # unchanged in every thread.
  3019. class _DaemonServer(socketserver.ThreadingTCPServer):
  3020. daemon_threads = True
  3021. allow_reuse_address = True
  3022. # socketserver's default backlog is 5. A burst of connections (a dashboard
  3023. # page opening parallel requests, a probe storm) larger than the backlog
  3024. # is reset at connect before the accept loop can answer 200 or 503. Keep
  3025. # the backlog above the worker cap so saturation is always answered 503.
  3026. request_queue_size = 64
  3027. def process_request(self, request, client_address):
  3028. # Bound concurrent workers.
  3029. # Acquire NON-BLOCKING in the accept loop so it never stalls (health
  3030. # probes keep being accepted even at saturation). On saturation, answer
  3031. # a bare 503 and close instead of spawning another unbounded worker
  3032. # thread + child regen process. The permit is released in
  3033. # process_request_thread's finally, exactly once per successful acquire.
  3034. if not _WORKER_SEM.acquire(blocking=False):
  3035. try:
  3036. request.sendall(
  3037. b"HTTP/1.1 503 Service Unavailable\\r\\n"
  3038. b"Content-Length: 0\\r\\n"
  3039. b"Connection: close\\r\\n\\r\\n"
  3040. )
  3041. except OSError:
  3042. pass
  3043. self.shutdown_request(request)
  3044. return
  3045. super().process_request(request, client_address)
  3046. def process_request_thread(self, request, client_address):
  3047. # Release the worker permit on EVERY exit path (handler exception
  3048. # included) so the bounded pool can never leak permits and wedge shut.
  3049. try:
  3050. super().process_request_thread(request, client_address)
  3051. finally:
  3052. _WORKER_SEM.release()
  3053. with _DaemonServer((HOST, PORT), Handler) as httpd:
  3054. httpd.serve_forever()
  3055. except OSError:
  3056. # Port in use by another process (or our old instance still dying).
  3057. # Exit 0 cleanly so KeepAlive backs off via ThrottleInterval.
  3058. sys.exit(0)
  3059. '''
  3060. def _write_file_if_changed(path, content, mode=None):
  3061. """Write *content* to *path* only if it differs from what is already on disk.
  3062. Returns True if the file was created or its content changed, False if it
  3063. already matched (so nothing was written). macOS re-fires the "App Background
  3064. Activity" / background-item banner whenever a LaunchAgent plist is rewritten
  3065. -- even with byte-identical content -- so an unconditional rewrite on every
  3066. ensure/repair pass spams the user. Skipping the no-op write leaves the file
  3067. untouched and the banner silent.
  3068. """
  3069. _measure_facade = _compat.facade()
  3070. try:
  3071. if path.read_text(encoding="utf-8") == content:
  3072. if mode is not None:
  3073. try:
  3074. path.chmod(mode)
  3075. except OSError:
  3076. pass
  3077. return False
  3078. except (OSError, ValueError):
  3079. pass
  3080. path.write_text(content, encoding="utf-8")
  3081. if mode is not None:
  3082. try:
  3083. path.chmod(mode)
  3084. except OSError:
  3085. pass
  3086. return True
  3087. def _launchagent_loaded(label):
  3088. """True if a LaunchAgent with *label* is currently loaded in this GUI domain.
  3089. Used to decide whether a bootout+bootstrap is actually needed: if the plist
  3090. did not change AND the agent is already loaded, reloading it only re-fires the
  3091. macOS background-item banner for no benefit. Best-effort: any failure (no
  3092. launchctl, non-macOS, timeout) returns False so the caller falls back to the
  3093. reload it would have done anyway.
  3094. """
  3095. _measure_facade = _compat.facade()
  3096. getuid = getattr(_measure_facade.os, "getuid", None)
  3097. if getuid is None:
  3098. return False
  3099. try:
  3100. # 2s, not 10s: a loaded agent answers in milliseconds, and this runs on the
  3101. # SessionStart hot path. On timeout we return False and the caller falls
  3102. # back to the reload it would have done anyway.
  3103. r = _measure_facade.subprocess.run(
  3104. ["launchctl", "print", f"gui/{getuid()}/{label}"],
  3105. capture_output=True, text=True, timeout=2, creationflags=_measure_facade._NO_WINDOW,
  3106. )
  3107. return r.returncode == 0
  3108. except (OSError, _measure_facade.subprocess.TimeoutExpired):
  3109. return False
  3110. def _generate_plist():
  3111. """Generate the launchd plist XML for the dashboard daemon."""
  3112. _measure_facade = _compat.facade()
  3113. from xml.sax.saxutils import escape as _xml_escape
  3114. import shutil as _shutil
  3115. _which = _shutil.which("python3") or ""
  3116. python3_path = _which if (_which and "/shims/" not in _which) else "/usr/bin/python3"
  3117. daemon_script = _xml_escape(str(_measure_facade.SNAPSHOT_DIR / "dashboard-server.py"))
  3118. log_out = _xml_escape(str(_measure_facade.DAEMON_LOG_DIR / "stdout.log"))
  3119. log_err = _xml_escape(str(_measure_facade.DAEMON_LOG_DIR / "stderr.log"))
  3120. return f"""<?xml version="1.0" encoding="UTF-8"?>
  3121. <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
  3122. <plist version="1.0">
  3123. <dict>
  3124. <key>Label</key>
  3125. <string>{_measure_facade.DAEMON_LABEL}</string>
  3126. <key>ProgramArguments</key>
  3127. <array>
  3128. <string>{_xml_escape(python3_path)}</string>
  3129. <string>{daemon_script}</string>
  3130. </array>
  3131. <key>RunAtLoad</key>
  3132. <true/>
  3133. <key>KeepAlive</key>
  3134. <dict>
  3135. <key>SuccessfulExit</key>
  3136. <false/>
  3137. </dict>
  3138. <key>ThrottleInterval</key>
  3139. <integer>10</integer>
  3140. <key>StandardOutPath</key>
  3141. <string>{log_out}</string>
  3142. <key>StandardErrorPath</key>
  3143. <string>{log_err}</string>
  3144. </dict>
  3145. </plist>
  3146. """
  3147. def _ensure_dashboard_file(to_stderr=None):
  3148. """Shared: generate the initial dashboard HTML if missing.
  3149. All platform installers rely on the HTML file already existing on
  3150. disk so the daemon has something to serve. Idempotent.
  3151. ``to_stderr`` routes progress/error lines to stderr so they never
  3152. land in the SessionStart stdout capture.
  3153. """
  3154. _measure_facade = _compat.facade()
  3155. _out = _measure_facade.sys.stderr if to_stderr else _measure_facade.sys.stdout
  3156. if not _measure_facade.DASHBOARD_PATH.exists():
  3157. print(" Generating initial dashboard...", file=_out)
  3158. _measure_facade.generate_standalone_dashboard(quiet=True)
  3159. if not _measure_facade.DASHBOARD_PATH.exists():
  3160. print("[Error] Could not generate dashboard. Run 'measure.py dashboard' first.", file=_out)
  3161. return False
  3162. return True
  3163. def _verify_daemon_port(timeout_seconds=None, retries=None, retry_sleep=None):
  3164. """Probe 127.0.0.1:DAEMON_PORT to confirm OUR daemon is up (not a foreign listener).
  3165. retry_sleep (default 1): seconds to wait between retries. The SessionStart
  3166. ensure-health self-heal passes retry_sleep=0 with retries=1 so a single
  3167. quick probe of a dead port returns immediately instead of burning ~1-2s of
  3168. the hook budget on time.sleep -- the cheap-hot-path requirement.
  3169. v5.4.19: a TCP connect() alone can't tell us whether the
  3170. process on DAEMON_PORT is actually ours. A foreign app (or a stale orphan
  3171. under a different dashboard path) that happens to bind 24842 would pass
  3172. the connect check, and install code would assume success. We now follow
  3173. the connect with a GET /__to_ping and verify the body equals our magic
  3174. string. Only then do we claim the daemon is up.
  3175. Retry budget configurable via TOKEN_OPTIMIZER_DASHBOARD_TIMEOUT (total
  3176. seconds, default 30). Large trends DBs (~500+ sessions) can push Python
  3177. cold-start past 20s on first launch.
  3178. """
  3179. _measure_facade = _compat.facade()
  3180. if retries is None:
  3181. try:
  3182. total = max(0, int(_measure_facade.os.environ.get("TOKEN_OPTIMIZER_DASHBOARD_TIMEOUT", "30")))
  3183. except (ValueError, TypeError):
  3184. total = 30
  3185. retries = max(total // 2, 4)
  3186. import socket as _socket
  3187. import urllib.request
  3188. import urllib.error
  3189. url = f"http://127.0.0.1:{_measure_facade.DAEMON_PORT}/__to_ping"
  3190. for _attempt in range(retries):
  3191. try:
  3192. with _socket.socket(_socket.AF_INET, _socket.SOCK_STREAM) as s:
  3193. s.settimeout(timeout_seconds)
  3194. s.connect(("127.0.0.1", _measure_facade.DAEMON_PORT))
  3195. except (OSError, ConnectionRefusedError):
  3196. if retry_sleep:
  3197. _measure_facade.time.sleep(retry_sleep)
  3198. continue
  3199. # Port is open -- verify identity so we don't mistake a foreign listener for ours.
  3200. try:
  3201. req = urllib.request.Request(url, headers={"Host": f"127.0.0.1:{_measure_facade.DAEMON_PORT}"})
  3202. with urllib.request.urlopen(req, timeout=timeout_seconds) as resp:
  3203. body = resp.read(256).decode("utf-8", errors="replace").strip()
  3204. if body == _measure_facade.DAEMON_IDENTITY_MAGIC:
  3205. return True
  3206. # Wrong magic -- foreign listener or older build. Don't retry ident check.
  3207. return False
  3208. except (urllib.error.URLError, OSError, ValueError):
  3209. if retry_sleep:
  3210. _measure_facade.time.sleep(retry_sleep)
  3211. continue
  3212. return False
  3213. def _daemon_install_lock(soft_fail=None):
  3214. """Return a context manager that serialises concurrent setup-daemon runs.
  3215. v5.4.19: uses atomic mkdir as the mutex, but also reaps stale
  3216. locks whose owner died (SIGKILL, crash, etc.). A PID file inside the lock
  3217. directory records who took it; on contention, we check whether that PID is
  3218. still alive and the lock was taken recently (mtime < 10 minutes). If the
  3219. owner is gone OR the lock is older than 10 minutes, we reclaim it.
  3220. soft_fail=True: return an inert contextmanager on contention
  3221. rather than sys.exit(1). Used from hook paths where we must never kill
  3222. the calling Claude Code session.
  3223. """
  3224. _measure_facade = _compat.facade()
  3225. from contextlib import contextmanager
  3226. lock_dir = _measure_facade.SNAPSHOT_DIR / ".setup-daemon.lock"
  3227. pid_file = lock_dir / "pid"
  3228. MAX_LOCK_AGE = 10 * 60 # 10 minutes
  3229. def _pid_alive(pid):
  3230. _measure_facade = _compat.facade()
  3231. return _measure_facade._pid_is_running(pid)
  3232. def _try_reclaim_stale():
  3233. """Return True if we successfully reaped a stale lock, False otherwise."""
  3234. _measure_facade = _compat.facade()
  3235. try:
  3236. mtime = lock_dir.stat().st_mtime
  3237. except OSError:
  3238. return False
  3239. owner_pid = 0
  3240. try:
  3241. raw = pid_file.read_text(encoding="utf-8").strip()
  3242. owner_pid = int(raw)
  3243. except (OSError, ValueError):
  3244. owner_pid = 0
  3245. age = _measure_facade.time.time() - mtime
  3246. if not _pid_alive(owner_pid) or age > MAX_LOCK_AGE:
  3247. try:
  3248. if pid_file.exists():
  3249. pid_file.unlink()
  3250. lock_dir.rmdir()
  3251. return True
  3252. except OSError:
  3253. return False
  3254. return False
  3255. @contextmanager
  3256. def _inert():
  3257. _measure_facade = _compat.facade()
  3258. yield False # acquired=False: caller can detect "skipped"
  3259. @contextmanager
  3260. def _locked():
  3261. _measure_facade = _compat.facade()
  3262. acquired = False
  3263. try:
  3264. _measure_facade.SNAPSHOT_DIR.mkdir(parents=True, exist_ok=True)
  3265. try:
  3266. lock_dir.mkdir(exist_ok=False)
  3267. acquired = True
  3268. except FileExistsError:
  3269. # Try to reap a stale lock, then retry once.
  3270. if _try_reclaim_stale():
  3271. try:
  3272. lock_dir.mkdir(exist_ok=False)
  3273. acquired = True
  3274. except FileExistsError:
  3275. acquired = False
  3276. else:
  3277. acquired = False
  3278. if not acquired:
  3279. if soft_fail:
  3280. yield False
  3281. return
  3282. print("[Error] Another setup-daemon is already running.")
  3283. print(f" Lock file: {lock_dir}")
  3284. print(" If you're sure no other run is active, remove the lock and retry.")
  3285. _measure_facade.sys.exit(1)
  3286. # Record ownership inside the lock for reaping.
  3287. try:
  3288. pid_file.write_text(str(_measure_facade.os.getpid()), encoding="utf-8")
  3289. except OSError:
  3290. pass
  3291. yield True
  3292. finally:
  3293. if acquired:
  3294. try:
  3295. if pid_file.exists():
  3296. pid_file.unlink()
  3297. except OSError:
  3298. pass
  3299. try:
  3300. lock_dir.rmdir()
  3301. except OSError:
  3302. pass
  3303. return _locked()
  3304. def _reclaim_posix_daemon_port(port=None, script_name=None):
  3305. """SIGTERM any orphaned dashboard-server.py holding `port`.
  3306. Upgrading from a prior version (v5.2 -> v5.3) can leave the old
  3307. daemon alive after launchctl bootout, because bootout drops the
  3308. launchd job without sending SIGTERM to the child. A subsequent
  3309. bootstrap fails to bind. This helper identifies orphaned processes
  3310. that are running our OWN daemon script and terminates them cleanly.
  3311. Never kills a foreign process on the same port.
  3312. """
  3313. _measure_facade = _compat.facade()
  3314. try:
  3315. lsof = _measure_facade.subprocess.run(
  3316. ["lsof", "-nP", f"-iTCP:{port}", "-sTCP:LISTEN", "-t"],
  3317. capture_output=True, text=True, timeout=5, creationflags=_measure_facade._NO_WINDOW,
  3318. )
  3319. except (_measure_facade.subprocess.SubprocessError, OSError, FileNotFoundError):
  3320. return
  3321. if lsof.returncode != 0 or not lsof.stdout.strip():
  3322. return
  3323. for pid_str in lsof.stdout.strip().splitlines():
  3324. try:
  3325. pid = int(pid_str.strip())
  3326. except ValueError:
  3327. continue
  3328. try:
  3329. ps_out = _measure_facade.subprocess.run(
  3330. ["ps", "-o", "command=", "-p", str(pid)],
  3331. capture_output=True, text=True, timeout=5, creationflags=_measure_facade._NO_WINDOW,
  3332. )
  3333. except (_measure_facade.subprocess.SubprocessError, OSError, FileNotFoundError):
  3334. continue
  3335. if script_name in ps_out.stdout:
  3336. try:
  3337. _measure_facade.os.kill(pid, _measure_facade.signal.SIGTERM)
  3338. except (ProcessLookupError, PermissionError, OSError):
  3339. continue
  3340. # Give the old process a moment to release the port.
  3341. _measure_facade.time.sleep(0.5)
  3342. def _daemon_ports_to_reclaim(this_install_only: bool):
  3343. """Which daemon ports a POSIX uninstall must SIGTERM-reclaim.
  3344. A running-process kill was added on uninstall because
  3345. ``launchctl bootout`` / ``systemctl disable`` unregister the job without
  3346. stopping the child, leaving a daemon serving its port with the 0600 CSRF
  3347. token live in memory. But a sweep-all uninstall tears down EVERY runtime's
  3348. scheduler unit and EVERY identity's files, while the reclaim helper defaults
  3349. to ``DAEMON_PORT`` -- the RESOLVED runtime's port only. So a sibling
  3350. runtime's daemon (e.g. Codex on 24843 when cleanup runs from Claude) kept
  3351. running its token-authed mutating API after a reported uninstall.
  3352. Scoped (``this_install_only``): just the resolved runtime's port. Sweep-all:
  3353. every runtime's port (claude 24842 + codex/hermes/copilot/cursor).
  3354. ``script_name``
  3355. in the reclaim helper is runtime-generic, so this only ever kills our own
  3356. ``dashboard-server.py`` bound to one of these ports.
  3357. """
  3358. _measure_facade = _compat.facade()
  3359. if this_install_only:
  3360. return (_measure_facade.DAEMON_PORT,)
  3361. ports = {24842}
  3362. ports.update(_measure_facade._DAEMON_PORT_BY_RUNTIME.values())
  3363. return tuple(sorted(ports))
  3364. def _reclaim_daemon_ports(this_install_only: bool):
  3365. """Reclaim every port in scope; never crash the uninstall on cleanup."""
  3366. _measure_facade = _compat.facade()
  3367. for port in _measure_facade._daemon_ports_to_reclaim(this_install_only):
  3368. try:
  3369. _measure_facade._reclaim_posix_daemon_port(port=port)
  3370. except Exception: # noqa: BLE001 -- uninstall must never crash on cleanup
  3371. pass
  3372. def _daemon_access_lines(effective_host):
  3373. """v5.11.1: return the access-scope lines for install/dry-run output.
  3374. Localhost binds print the usual "only your machine" line. A 0.0.0.0 bind is
  3375. network mode -- the dashboard is reachable on the LAN, but token auth still
  3376. gates every mutation, so we say so explicitly instead of claiming localhost
  3377. only.
  3378. """
  3379. _measure_facade = _compat.facade()
  3380. if effective_host == "0.0.0.0":
  3381. return [
  3382. " - Network mode: reachable on your LAN (bound 0.0.0.0)",
  3383. " - Token auth still required for any change (toggle/disable)",
  3384. ]
  3385. return [" - Only accessible from your machine (localhost)"]
  3386. def _print_daemon_network_note(effective_host):
  3387. """Print a brief LAN note after a successful network-mode install."""
  3388. _measure_facade = _compat.facade()
  3389. if effective_host == "0.0.0.0":
  3390. print(" Network mode: dashboard reachable on your LAN (bound 0.0.0.0).")
  3391. print(" Token auth still required for any change.\n")
  3392. def _install_launchd_daemon(dry_run=None, soft_fail=None, effective_host=None):
  3393. """macOS: install the dashboard daemon via a LaunchAgent.
  3394. v5.4.19: added `soft_fail`. When True (hook paths),
  3395. errors return False instead of sys.exit(1) so we never kill the calling
  3396. Claude Code session. CLI calls keep the default False for backwards
  3397. compatibility -- users still see a hard failure + actionable hint.
  3398. Extracted verbatim from the pre-dispatcher setup_daemon. Re-running
  3399. overwrites the plist idempotently and bootouts any existing instance
  3400. first so we never fight a stale PID.
  3401. """
  3402. _measure_facade = _compat.facade()
  3403. if _measure_facade._scheduler_install_refused():
  3404. return False
  3405. def _fail(msg, hint=None):
  3406. # Under soft_fail we are inside a hook --
  3407. # stdout is session-visible context, so route errors to stderr instead
  3408. # of spamming "[Error] ..." into every SessionStart transcript.
  3409. _measure_facade = _compat.facade()
  3410. out = _measure_facade.sys.stderr if soft_fail else _measure_facade.sys.stdout
  3411. print(msg, file=out)
  3412. if hint:
  3413. print(hint, file=out)
  3414. if soft_fail:
  3415. return False
  3416. _measure_facade.sys.exit(1)
  3417. if dry_run:
  3418. print("[Token Optimizer] Dry run. Would install:\n")
  3419. print(" A tiny web server that makes your dashboard available at:")
  3420. print(f" http://localhost:{_measure_facade.DAEMON_PORT}/token-optimizer\n")
  3421. print(" What it does:")
  3422. print(" - Serves your dashboard file so you can bookmark the URL")
  3423. print(" - Starts automatically when you log into your Mac")
  3424. print(" - Restarts itself if it ever stops")
  3425. for line in _measure_facade._daemon_access_lines(effective_host):
  3426. print(line)
  3427. print(" - Uses ~2MB of memory\n")
  3428. print(" Files it creates:")
  3429. print(f" {_measure_facade.SNAPSHOT_DIR / 'dashboard-server.py'}")
  3430. print(f" {_measure_facade.PLIST_PATH}\n")
  3431. print(" No changes written.")
  3432. return True
  3433. if not _measure_facade._ensure_dashboard_file(to_stderr=soft_fail):
  3434. return _fail("[Token Optimizer] Dashboard file missing; cannot start daemon.")
  3435. # Clear any stale uninstall tombstone so a fresh install proceeds.
  3436. try:
  3437. if _measure_facade.DAEMON_THRASH_BREADCRUMB.exists():
  3438. _measure_facade.DAEMON_THRASH_BREADCRUMB.unlink()
  3439. except OSError:
  3440. pass
  3441. # Ensure per-install auth token exists before writing the daemon script.
  3442. _measure_facade._get_or_create_daemon_token()
  3443. with _measure_facade._daemon_install_lock(soft_fail=soft_fail) as acquired:
  3444. if soft_fail and not acquired:
  3445. # Another installer holds the lock; don't fight it from a hook path.
  3446. return False
  3447. # Write daemon script (catch OSError from permissions/quota issues).
  3448. try:
  3449. _measure_facade.SNAPSHOT_DIR.mkdir(parents=True, exist_ok=True)
  3450. _measure_facade.DAEMON_LOG_DIR.mkdir(parents=True, exist_ok=True)
  3451. daemon_script = _measure_facade.SNAPSHOT_DIR / "dashboard-server.py"
  3452. script_changed = _measure_facade._write_file_if_changed(daemon_script, _measure_facade._generate_daemon_script(), 0o755)
  3453. _measure_facade.LAUNCH_AGENTS_DIR.mkdir(parents=True, exist_ok=True)
  3454. plist_changed = _measure_facade._write_file_if_changed(_measure_facade.PLIST_PATH, _measure_facade._generate_plist())
  3455. except OSError as e:
  3456. return _fail(f"[Error] Could not write daemon files: {e}")
  3457. # Nothing changed and the daemon is already loaded and serving -> skip the
  3458. # bootout+bootstrap. Reloading a healthy, current daemon only re-fires the
  3459. # macOS "App Background Activity" banner for no benefit. (A reinstall after
  3460. # uninstall has an unloaded agent, so this guard is skipped and the reload
  3461. # below still runs.)
  3462. if (not script_changed and not plist_changed
  3463. and _measure_facade._launchagent_loaded(_measure_facade.DAEMON_LABEL)
  3464. and _measure_facade._verify_daemon_port(timeout_seconds=1, retries=1, retry_sleep=0)):
  3465. print("[Token Optimizer] Dashboard server already current and running.\n")
  3466. print(" Bookmark this URL:")
  3467. print(f" http://localhost:{_measure_facade.DAEMON_PORT}/token-optimizer")
  3468. return True
  3469. # Stop existing daemon if running (bootout is idempotent -- non-zero
  3470. # exit when nothing is loaded is expected and ignored).
  3471. try:
  3472. _measure_facade.subprocess.run(["launchctl", "bootout", f"gui/{_measure_facade.os.getuid()}", str(_measure_facade.PLIST_PATH)],
  3473. capture_output=True, timeout=10, creationflags=_measure_facade._NO_WINDOW)
  3474. except (OSError, _measure_facade.subprocess.TimeoutExpired):
  3475. pass
  3476. # bootout drops the launchd job but doesn't SIGTERM the Python
  3477. # child -- the port stays held until the orphan dies on its own.
  3478. # Reclaim it so bootstrap doesn't hit EADDRINUSE.
  3479. _measure_facade._reclaim_posix_daemon_port()
  3480. # Start daemon
  3481. try:
  3482. result = _measure_facade.subprocess.run(["launchctl", "bootstrap", f"gui/{_measure_facade.os.getuid()}", str(_measure_facade.PLIST_PATH)],
  3483. capture_output=True, text=True, timeout=20, creationflags=_measure_facade._NO_WINDOW)
  3484. except (OSError, _measure_facade.subprocess.TimeoutExpired) as e:
  3485. return _fail(f"[Error] launchctl bootstrap failed: {e}")
  3486. if result.returncode != 0:
  3487. return _fail(
  3488. f"[Error] Failed to start daemon: {result.stderr.strip()}",
  3489. hint=f" Plist written to: {_measure_facade.PLIST_PATH}\n Try manually: launchctl bootstrap gui/{_measure_facade.os.getuid()} {_measure_facade.PLIST_PATH}",
  3490. )
  3491. # Verify it's actually running. Budget ~16s covers slow cold
  3492. # starts. If we still can't reach the port, surface a clear
  3493. # do-NOT-uninstall instruction so users don't kill a booting
  3494. # daemon mid-init.
  3495. _measure_facade.time.sleep(1)
  3496. if _measure_facade._verify_daemon_port():
  3497. print("[Token Optimizer] Dashboard server installed and running.\n")
  3498. print(" Bookmark this URL:")
  3499. print(f" http://localhost:{_measure_facade.DAEMON_PORT}/token-optimizer\n")
  3500. _measure_facade._print_daemon_network_note(effective_host)
  3501. print(f" It updates automatically after every {_measure_facade.runtime_name_for_humans()} session.")
  3502. print(" Starts on login, so the URL always works.\n")
  3503. print(_measure_facade._hint(" To remove: python3 measure.py setup-daemon --uninstall"))
  3504. else:
  3505. print("[Token Optimizer] Server bootstrapped but port not yet reachable.")
  3506. print(f" Give it 30s, then open: http://localhost:{_measure_facade.DAEMON_PORT}/token-optimizer")
  3507. print(" Do NOT run --uninstall while it is still coming up -- that can")
  3508. print(f" orphan the process and block port {_measure_facade.DAEMON_PORT} for the next install.")
  3509. print(f" If still unreachable after 30s, check: {_measure_facade.DAEMON_LOG_DIR}/stderr.log")
  3510. return True
  3511. def _write_uninstall_tombstone(snapshot_dir=None):
  3512. """v5.4.19: write a marker breadcrumb file so that if the daemon
  3513. process somehow respawns (e.g., an orphaned LaunchAgent the user didn't clean
  3514. up), it exits cleanly on next start instead of resurrecting.
  3515. The breadcrumb content is the ``_UNINSTALL_TOMBSTONE_MARKER``
  3516. (not an empty file). The daemon's 3-strikes thrash path writes an EMPTY
  3517. breadcrumb that self-heals after 60s; an empty uninstall tombstone was
  3518. indistinguishable from it, so a surviving daemon self-cleared it after 60s
  3519. and minted a fresh CSRF token after a reported uninstall. The marker makes
  3520. the uninstall tombstone permanent (cleared only by a reinstall).
  3521. Surface failures to stderr so a silently
  3522. broken tombstone (e.g. SNAPSHOT_DIR unwritable) is visible to the user
  3523. rather than leaving them thinking uninstall succeeded while the daemon
  3524. keeps respawning.
  3525. The tombstone must OUTLIVE the uninstall. Each generated
  3526. dashboard-server.py bakes in its OWN identity's breadcrumb path (see the
  3527. `thrash_path_literal` in the daemon template) and checks it on start
  3528. ("noop-tombstoned"), so a per-identity tombstone is what keeps that
  3529. identity's orphaned daemon dead. `snapshot_dir` targets one identity; the
  3530. uninstall sweep calls this once per swept identity so no sibling is left
  3531. revivable. A legitimate reinstall clears it (setup_daemon unlinks the
  3532. tombstone before starting), which is the ONLY place it may be removed.
  3533. """
  3534. _measure_facade = _compat.facade()
  3535. snap = _measure_facade.SNAPSHOT_DIR if snapshot_dir is None else _measure_facade.Path(snapshot_dir)
  3536. breadcrumb = snap / ".daemon-thrash"
  3537. try:
  3538. snap.mkdir(parents=True, exist_ok=True)
  3539. # Write a distinctive marker (NOT an empty file) so a
  3540. # surviving daemon script can tell an uninstall tombstone (permanent
  3541. # until reinstall) from a transient 3-strikes thrash tombstone (which
  3542. # self-heals after 60s). See _UNINSTALL_TOMBSTONE_MARKER.
  3543. with open(breadcrumb, "w", encoding="utf-8") as f:
  3544. f.write(_measure_facade._UNINSTALL_TOMBSTONE_MARKER)
  3545. except OSError as e:
  3546. _measure_facade.sys.stderr.write(
  3547. f"[Token Optimizer] Warning: could not write uninstall tombstone "
  3548. f"at {breadcrumb}: {e}\n"
  3549. f" If the daemon is still running after uninstall, remove the "
  3550. f"LaunchAgent/task/unit manually.\n"
  3551. )
  3552. def _daemon_install_failed_marker_state():
  3553. """Tri-state stat of the sticky marker: 'present' | 'absent' | 'unknown'.
  3554. The old implementation used ``os.path.exists``, which
  3555. maps EVERY OSError to False -- so the written ``except OSError`` policy was
  3556. dead code, and (worse) an AV/permission denial made a PRESENT marker read
  3557. as ABSENT, silently reopening the retry loop the marker exists to stop --
  3558. intermittently, which is the worst kind of flap to debug. ``os.stat`` with
  3559. explicit errno handling makes present-but-unreadable ('unknown')
  3560. distinguishable from a definitive ENOENT ('absent'). Never raises.
  3561. """
  3562. _measure_facade = _compat.facade()
  3563. try:
  3564. _measure_facade.os.stat(str(_measure_facade.DAEMON_INSTALL_FAILED_BREADCRUMB))
  3565. return "present"
  3566. except (FileNotFoundError, NotADirectoryError):
  3567. # The only DEFINITIVE evidence of absence: the path (or a parent
  3568. # component) does not exist.
  3569. return "absent"
  3570. except OSError:
  3571. # EACCES/EIO/...: the marker may well exist; we just cannot see it.
  3572. return "unknown"
  3573. def _daemon_install_failed_marker_present():
  3574. """True when a prior daemon install/revive/self-heal failed and has NOT been
  3575. cleared (by an explicit successful ``setup-daemon``, or by a verified-live
  3576. daemon disproving the record).
  3577. Checked FIRST by every path that would otherwise (re)spawn an installer:
  3578. ``_ensure_dashboard_daemon`` (SessionStart self-heal AND the ``daemon-revive``
  3579. subcommand, including ``force=True``) and ``_daemon_midsession_pulse``
  3580. (per-turn revive). A filesystem breadcrumb, not a config flag, for the same
  3581. reason the uninstall tombstone is: a corrupt/unreadable config.json must not
  3582. silently re-arm the retry loop.
  3583. Policy (now explicit instead of an accident of exists()):
  3584. 'unknown' counts as PRESENT. This gate only ever suppresses a costly
  3585. retry, so failing CLOSED on a transient stat error keeps the gate stable
  3586. -- a momentary AV denial must not flap it open and resume the per-prompt
  3587. retry loop. Only a definitive ENOENT reads as absent, and the
  3588. clear-on-verified-healthy path in ``_ensure_dashboard_daemon`` still
  3589. un-wedges a stale marker the moment the daemon is observably alive.
  3590. """
  3591. _measure_facade = _compat.facade()
  3592. return _measure_facade._daemon_install_failed_marker_state() != "absent"
  3593. def _daemon_install_failed_reason():
  3594. """First line of the sticky marker (``<iso8601> <reason>``), or None.
  3595. Purely for user-facing surfaces (ensure-health's one-liner, doctor) --
  3596. presence, not content, is what gates behaviour. Never raises.
  3597. """
  3598. _measure_facade = _compat.facade()
  3599. try:
  3600. line = _measure_facade.DAEMON_INSTALL_FAILED_BREADCRUMB.read_text(
  3601. encoding="utf-8", errors="replace").strip().splitlines()
  3602. return line[0] if line else None
  3603. except OSError:
  3604. return None
  3605. def _write_daemon_install_failed_marker(reason):
  3606. """Persist the sticky install-failed marker. Best effort, never raises.
  3607. Callers must only arm this for DEFINITIVE,
  3608. permanent failure classes (MS-Store alias, schtasks missing, task creation
  3609. denied by policy) -- see the DAEMON_INSTALL_FAILED_BREADCRUMB comment.
  3610. Transient failures rely on the existing throttles instead.
  3611. Content is ``<iso8601> <reason>`` for the user/support (surfaced by
  3612. ensure-health and doctor): only the file's PRESENCE is load-bearing.
  3613. """
  3614. _measure_facade = _compat.facade()
  3615. try:
  3616. if _measure_facade._daemon_install_failed_marker_present():
  3617. return
  3618. _measure_facade.SNAPSHOT_DIR.mkdir(parents=True, exist_ok=True)
  3619. try:
  3620. # Match take_snapshot's 0o700 -- when this mkdir is the
  3621. # first creator, the state dir must not land world-readable.
  3622. _measure_facade.os.chmod(str(_measure_facade.SNAPSHOT_DIR), 0o700)
  3623. except OSError:
  3624. pass
  3625. stamp = _measure_facade.datetime.now(_measure_facade.timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ")
  3626. with open(_measure_facade.DAEMON_INSTALL_FAILED_BREADCRUMB, "w", encoding="utf-8") as f:
  3627. f.write(f"{stamp} {reason}\n")
  3628. except OSError as _e:
  3629. # An unwritable state dir means we cannot make the refusal sticky. The
  3630. # throttles still bound the retry rate; do not crash a hook over it --
  3631. # but SAY so: a silently-degraded stickiness would leave the
  3632. # per-window retry (and its Windows flashes) with zero explanation.
  3633. try:
  3634. print(
  3635. "[Token Optimizer] Warning: could not persist the daemon "
  3636. f"install-failed marker ({_measure_facade.DAEMON_INSTALL_FAILED_BREADCRUMB}): "
  3637. f"{_e}. Failed installs will keep retrying on the normal "
  3638. "throttle instead of stopping permanently.",
  3639. file=_measure_facade.sys.stderr,
  3640. )
  3641. except Exception:
  3642. pass
  3643. def _clear_daemon_install_failed_marker():
  3644. """Remove the sticky install-failed marker.
  3645. Legitimate callers, and NOTHING else (a time- or session-based clear would
  3646. restore the per-prompt retry loop the marker exists to stop):
  3647. * ``setup_daemon``'s explicit install branch, after the platform
  3648. installer reports success -- the user proved the problem is fixed;
  3649. * ``_ensure_dashboard_daemon`` on VERIFIED success -- a live,
  3650. identity-checked daemon (or a completed install/restart) disproves the
  3651. "structurally broken" record, e.g. after the install-lock race where
  3652. the losing session armed the marker while the winner installed fine.
  3653. Returns True only when the marker is VERIFIABLY gone
  3654. (read back after the unlink, never assumed). A clear that did not clear
  3655. warns on stderr -- the pre-fix shape swallowed the OSError, so
  3656. ``setup-daemon`` printed its full "installed and running" success story
  3657. while every self-heal path stayed permanently disabled, undiagnosably.
  3658. Never raises.
  3659. """
  3660. _measure_facade = _compat.facade()
  3661. try:
  3662. _measure_facade.DAEMON_INSTALL_FAILED_BREADCRUMB.unlink()
  3663. except FileNotFoundError:
  3664. return True
  3665. except OSError:
  3666. pass
  3667. # Read back: only a definitive ENOENT counts as cleared. 'present' AND
  3668. # 'unknown' both mean the wedge may still stand -- say so.
  3669. if _measure_facade._daemon_install_failed_marker_state() == "absent":
  3670. return True
  3671. try:
  3672. print(
  3673. "[Token Optimizer] Warning: could not remove "
  3674. f"{_measure_facade.DAEMON_INSTALL_FAILED_BREADCRUMB} -- daemon self-heal stays "
  3675. "DISABLED until this file is removed (delete it manually, then "
  3676. f"re-run: {_measure_facade._measure_cli()} setup-daemon).",
  3677. file=_measure_facade.sys.stderr,
  3678. )
  3679. except Exception:
  3680. pass
  3681. return False
  3682. def _uninstall_launchd_daemon(this_install_only=None, dry_run=None):
  3683. """macOS: stop and remove the LaunchAgent + daemon script(s).
  3684. Unified output: track what is actually
  3685. deleted so we don't print a contradictory "Nothing to remove" header
  3686. followed by a "Deleted: script.py" line when the plist is gone but
  3687. the script file remains from a half-uninstall.
  3688. v5.11.68: identity-sweeping by default. The LaunchAgent plist
  3689. is per-RUNTIME (shared across ``token-optimizer-*`` identities of the same
  3690. runtime), so it is removed once. The per-identity files
  3691. (``dashboard-server.py``, ``daemon-token``, ``dashboard-host``,
  3692. ``.daemon-thrash``) live in EACH identity's snapshot dir; a sibling
  3693. identity's daemon script + 0600 CSRF token would otherwise outlive the
  3694. uninstall. Sweep every identity unless ``this_install_only`` opts out.
  3695. Also bootout every runtime's LaunchAgent by name so a registration whose
  3696. data dir already vanished still gets unregistered.
  3697. v5.11.68 (cleanup): ``dry_run=True`` makes the uninstaller
  3698. side-effect-free (no bootout, no file deletion, no tombstone write) so the
  3699. cleanup command's ``--dry-run`` is a true preview.
  3700. """
  3701. _measure_facade = _compat.facade()
  3702. if _measure_facade._daemon_snapshot_sandboxed():
  3703. return False
  3704. if dry_run:
  3705. # Dry-run: report what WOULD be removed, touch nothing.
  3706. would_remove = []
  3707. for label in _measure_facade._scheduler_names_to_sweep(
  3708. this_install_only, _measure_facade._ALL_LAUNCH_AGENT_LABELS, _measure_facade.DAEMON_LABEL):
  3709. plist = _measure_facade.LAUNCH_AGENTS_DIR / f"{label}.plist"
  3710. if plist.exists():
  3711. would_remove.append(str(plist))
  3712. per_identity: list[tuple[Path, list[str]]] = []
  3713. for snap_dir in _measure_facade._daemon_identity_snapshot_dirs(this_install_only):
  3714. files = _measure_facade._daemon_per_identity_files(snap_dir)
  3715. identity_would = [str(files[k]) for k in ("daemon_script", "daemon_token", "daemon_host") if files[k].exists()]
  3716. if identity_would:
  3717. per_identity.append((snap_dir, identity_would))
  3718. would_remove.extend(identity_would)
  3719. if would_remove:
  3720. print("[Token Optimizer] Dashboard daemon would be removed.")
  3721. for path in would_remove:
  3722. print(f" Would delete: {path}")
  3723. if per_identity and not this_install_only:
  3724. print(" Would sweep all token-optimizer-* identities:")
  3725. for snap_dir, items in per_identity:
  3726. print(f" {snap_dir}: {len(items)} file(s)")
  3727. else:
  3728. print("[Token Optimizer] No daemon artifacts found. Nothing to remove.")
  3729. return
  3730. # Tombstone FIRST so any racing respawn exits cleanly.
  3731. _measure_facade._write_uninstall_tombstone()
  3732. removed = []
  3733. # Sweep the OS scheduler by name across every runtime variant. The plist
  3734. # file lives at a fixed per-runtime path under ~/Library/LaunchAgents.
  3735. for label in _measure_facade._scheduler_names_to_sweep(
  3736. this_install_only, _measure_facade._ALL_LAUNCH_AGENT_LABELS, _measure_facade.DAEMON_LABEL):
  3737. plist = _measure_facade.LAUNCH_AGENTS_DIR / f"{label}.plist"
  3738. # Bootout by LABEL, unconditionally. The old code only
  3739. # booted out when the plist FILE existed, so a job still loaded in
  3740. # launchd whose plist had already been deleted (half-uninstall, manual
  3741. # rm, platform GC) was never unregistered and kept respawning.
  3742. try:
  3743. _measure_facade.subprocess.run(["launchctl", "bootout", f"gui/{_measure_facade.os.getuid()}/{label}"],
  3744. capture_output=True, timeout=10, creationflags=_measure_facade._NO_WINDOW)
  3745. except (OSError, _measure_facade.subprocess.TimeoutExpired):
  3746. pass
  3747. if plist.exists():
  3748. try:
  3749. _measure_facade.subprocess.run(["launchctl", "bootout", f"gui/{_measure_facade.os.getuid()}", str(plist)],
  3750. capture_output=True, timeout=10, creationflags=_measure_facade._NO_WINDOW)
  3751. except (OSError, _measure_facade.subprocess.TimeoutExpired):
  3752. pass
  3753. if _measure_facade._unlink_if_exists(plist):
  3754. removed.append(str(plist))
  3755. # Unregistering the job does NOT stop the process --
  3756. # `launchctl bootout` / `systemctl disable` drop the registration without
  3757. # SIGTERMing the running child, so the daemon kept serving its port with
  3758. # the 0600 CSRF token live in memory until logout. Reclaim the port(s) now
  3759. # that the tombstone is down (so nothing races a respawn) and the job is
  3760. # unregistered. On a sweep-all uninstall this covers EVERY runtime's port
  3761. # (a sibling runtime's daemon + live CSRF token otherwise
  3762. # survived), scoped back to the resolved runtime under --this-install-only.
  3763. # Never kills a foreign process on a port -- the helper verifies the command
  3764. # line is our own dashboard-server.py.
  3765. _measure_facade._reclaim_daemon_ports(this_install_only)
  3766. # Sweep per-identity daemon files across every token-optimizer-* identity.
  3767. per_identity_removed: list[tuple[Path, list[str]]] = []
  3768. sweep_failed: list[str] = []
  3769. for snap_dir in _measure_facade._daemon_identity_snapshot_dirs(this_install_only):
  3770. identity_removed, identity_failed = _measure_facade._sweep_identity_daemon_files(
  3771. snap_dir, ("daemon_script", "daemon_token", "daemon_host"))
  3772. sweep_failed.extend(identity_failed)
  3773. # The .daemon-thrash tombstone must PERSIST for every
  3774. # swept identity. Each identity's generated dashboard-server.py checks
  3775. # its OWN breadcrumb path on start and exits "noop-tombstoned"; the
  3776. # previous unlink here re-armed self-revive, so an orphaned LaunchAgent
  3777. # /task/unit could resurrect the daemon and mint a fresh 0600 token
  3778. # after a "successful" uninstall. Only a legitimate reinstall clears it
  3779. # (setup_daemon unlinks before start). NOT counted toward `removed` so
  3780. # the "Nothing to remove" honesty invariant holds.
  3781. _measure_facade._write_uninstall_tombstone(snap_dir)
  3782. if identity_removed:
  3783. per_identity_removed.append((snap_dir, identity_removed))
  3784. removed.extend(identity_removed)
  3785. _measure_facade._print_identity_sweep_report(
  3786. removed, per_identity_removed, sweep_failed, this_install_only)
  3787. def _is_ms_store_python_alias(exe_path):
  3788. """True if sys.executable looks like a Microsoft Store App Execution Alias.
  3789. App Execution Aliases (path contains WindowsApps) only launch from
  3790. interactive user shell tokens. Task Scheduler's launch context
  3791. cannot resolve them, so pythonw would exit silently and leave the
  3792. user stuck in the 'wait 30s' message forever.
  3793. """
  3794. _measure_facade = _compat.facade()
  3795. if not exe_path:
  3796. return False
  3797. normalized = str(exe_path).replace("/", "\\").lower()
  3798. return "\\windowsapps\\" in normalized
  3799. def _resolve_windows_pythonw():
  3800. """Best-effort path to pythonw.exe on Windows. Returns None if only
  3801. an MS Store alias is available -- the caller must refuse install.
  3802. """
  3803. _measure_facade = _compat.facade()
  3804. try:
  3805. exe = _measure_facade.Path(_measure_facade.sys.executable)
  3806. except (TypeError, ValueError):
  3807. return None
  3808. if _measure_facade._is_ms_store_python_alias(exe):
  3809. return None
  3810. candidate = exe.with_name("pythonw.exe")
  3811. if candidate.exists() and not _measure_facade._is_ms_store_python_alias(candidate):
  3812. return str(candidate)
  3813. return None
  3814. def _windows_gui_python():
  3815. """pythonw.exe (GUI subsystem) next to ``sys.executable``, or None.
  3816. The ONE interpreter that never allocates a console. Every
  3817. console-subsystem child (python.exe, py.exe, cmd.exe) gets a window from
  3818. Windows itself unless the PARENT passes CREATE_NO_WINDOW -- and when the
  3819. parent is Task Scheduler, we do not get to pass anything. Launching
  3820. pythonw.exe moves the guarantee into the binary, where no parent can undo it.
  3821. Thin alias over ``_resolve_windows_pythonw`` (which already refuses MS Store
  3822. App Execution Aliases -- those do not resolve under Task Scheduler's launch
  3823. context) so the no-flash call sites read by intent rather than by platform trivia.
  3824. Returns None off Windows and whenever no usable twin exists; every caller
  3825. must fall back to its pre-fix behaviour.
  3826. """
  3827. _measure_facade = _compat.facade()
  3828. if _measure_facade.os.name != "nt":
  3829. return None
  3830. try:
  3831. return _measure_facade._resolve_windows_pythonw()
  3832. except Exception:
  3833. # Interpreter SELECTION must never be able to break a spawn. The
  3834. # resolver builds a WindowsPath, which raises on a host whose pathlib
  3835. # flavour disagrees with os.name (real case: a POSIX box with os.name
  3836. # monkeypatched; plausible case: an embedded/frozen interpreter with a
  3837. # non-path sys.executable). Degrade to "no twin" -> caller keeps
  3838. # sys.executable, which is exactly the pre-fix behaviour.
  3839. return None
  3840. def _windows_task_exec_action(daemon_script_path, launcher_path):
  3841. """Return ``(command, arguments)`` for the Scheduled Task's ``<Exec>``.
  3842. The idle-pop: the task action used to be the generated
  3843. ``dashboard-launcher.cmd``. Task Scheduler runs a .cmd through cmd.exe, a
  3844. CONSOLE-subsystem process, so EVERY fire of the task painted a console
  3845. window on the user's desktop -- at logon, at boot, and on each
  3846. ``schtasks /Run`` from the daemon revive / restart self-heal (which is what
  3847. made windows pop while the machine looked idle). ``<Hidden>true</Hidden>``
    • Note: Usual in code (tag names in paths and patterns): Contains a <hidden> block, a common wrapper for instructions hidden in a tool description
    • Line 4,245 on GitHub
  3848. does NOT suppress this; it only hides the task from the Task Scheduler UI's
  3849. default filter. CREATE_NO_WINDOW cannot help either -- our process is not
  3850. the parent, Task Scheduler is.
  3851. Preferred action: ``pythonw.exe <daemon_script>``. pythonw is GUI-subsystem,
  3852. so Windows never allocates a console for it no matter who launches it.
  3853. Fallback (no usable pythonw -- MS Store alias, or no twin next to
  3854. sys.executable): the old .cmd launcher, which still WORKS and still captures
  3855. stdout/stderr, it just keeps the console window. Better a visible window than
  3856. a daemon that will not start.
  3857. The .cmd is written either way: it stays the fallback, it is what the
  3858. uninstall sweep already knows to delete (``_daemon_per_identity_files``), and
  3859. it is a usable manual repro for a user debugging a dead daemon.
  3860. """
  3861. _measure_facade = _compat.facade()
  3862. pyw = _measure_facade._windows_gui_python()
  3863. if pyw:
  3864. return pyw, f'"{daemon_script_path}"'
  3865. return str(launcher_path), ""
  3866. def _compose_windows_user_id():
  3867. """Return DOMAIN\\user when domain-joined, bare username in workgroup.
  3868. Bare %USERNAME% in <LogonTrigger> fails to match
  3869. Windows' logon event on domain-joined machines because the event
  3870. carries the fully-qualified account name.
  3871. """
  3872. _measure_facade = _compat.facade()
  3873. username = _measure_facade.os.environ.get("USERNAME", "").strip()
  3874. if not username:
  3875. # Last-ditch fallback: parse the final path element of USERPROFILE.
  3876. username = _measure_facade.os.environ.get("USERPROFILE", "").split("\\")[-1].strip()
  3877. if not username:
  3878. return None
  3879. domain = _measure_facade.os.environ.get("USERDOMAIN", "").strip()
  3880. computer = _measure_facade.os.environ.get("COMPUTERNAME", "").strip()
  3881. # Workgroup machines report USERDOMAIN == COMPUTERNAME -- strip it
  3882. # so the UserId stays bare and matches the local logon event.
  3883. if domain and domain.upper() != computer.upper():
  3884. return f"{domain}\\{username}"
  3885. return username
  3886. def _generate_windows_launcher_cmd(daemon_script_path, log_dir):
  3887. """Generate a .cmd shim that resolves Python at runtime and redirects
  3888. daemon stdout/stderr so future failures leave a trail.
  3889. Rationale: pointing Task
  3890. Scheduler directly at a versioned pythonw.exe path breaks on Python
  3891. upgrades and loses stdout/stderr. A .cmd wrapper dispatches via
  3892. a runtime interpreter ladder and captures daemon output into
  3893. DAEMON_LOG_DIR so port conflicts and import errors no longer
  3894. silently vanish.
  3895. READ THIS BEFORE REORDERING ANYTHING:
  3896. * This shim is now the FALLBACK path only. The Scheduled Task action
  3897. normally points straight at pythonw.exe (see
  3898. ``_windows_task_exec_action``), because Task Scheduler hosts a .cmd
  3899. inside console-subsystem cmd.exe and paints a window on EVERY fire --
  3900. logon, boot, and each ``schtasks /Run`` from the revive/restart
  3901. self-heal. Nothing inside a .cmd can prevent its own cmd.exe host.
  3902. **HONEST LIMIT: when this shim is the task action, the no-flash bug is NOT fixed,
  3903. only REDUCED -- from several console processes per fire (the cmd.exe
  3904. host plus up to three where.exe probes plus a console-subsystem
  3905. interpreter) down to the one unavoidable cmd.exe host window. One flash
  3906. per fire remains for this population, and no rewrite of the shim's
  3907. CONTENTS can ever remove it.** That is why the shim is used only when no
  3908. usable pythonw twin exists.
  3909. * The interpreter ladder was ``py.exe -3`` FIRST, which is exactly
  3910. backwards: py.exe is console-subsystem and wins on any host with the
  3911. Python Launcher installed (the common case), so the LONG-LIVED daemon
  3912. itself ran as a console process. GUI-subsystem interpreters now come
  3913. first: pythonw.exe -> pyw.exe -3 -> py.exe -3 -> python.exe.
  3914. * The three ``where <exe>`` probes are gone. where.exe is itself a console
  3915. binary, so the old shim spawned up to three extra console processes per
  3916. fire. Detection is now cmd's own "command not found" code (9009) from a
  3917. direct invocation, which costs no extra process.
  3918. The ladder is written FLAT (no parenthesised blocks) on purpose:
  3919. ``%ERRORLEVEL%`` inside a ``( ... )`` block expands once at block-parse
  3920. time, so a block form would test a stale value.
  3921. # The bare-PATH ``pythonw.exe`` rung needs a
  3922. WindowsApps guard: a Microsoft Store App Execution Alias exits 0 SILENTLY
  3923. under Task Scheduler's non-interactive token, so an unguarded first rung
  3924. would "succeed", ``exit /b 0``, and no daemon would ever start -- on hosts
  3925. where the pre-fix ``py -3``-first ladder worked. The rung therefore
  3926. resolves the PATH hit with ``%%~$PATH:I`` (pure cmd, no extra process --
  3927. where.exe is banned here) and skips itself when the hit lives under
  3928. WindowsApps. pyw.exe/py.exe are real launchers in C:\\Windows, never Store
  3929. aliases, so they keep plain rungs.
  3930. ``%`` is legal in Windows paths but expands at cmd
  3931. parse time (``%VAR%``), silently mis-targeting DAEMON_SCRIPT and both log
  3932. redirects; ``%%`` is cmd's literal-percent escape inside a batch file, so
  3933. interpolated paths are escaped below.
  3934. """
  3935. _measure_facade = _compat.facade()
  3936. def _rung(invocation):
  3937. _measure_facade = _compat.facade()
  3938. return (
  3939. f'{invocation} "%DAEMON_SCRIPT%" 1>>"%STDOUT_LOG%" 2>>"%STDERR_LOG%"\r\n'
  3940. 'if not "%ERRORLEVEL%"=="9009" exit /b %ERRORLEVEL%\r\n'
  3941. )
  3942. daemon_script_path = str(daemon_script_path).replace("%", "%%")
  3943. log_dir = str(log_dir).replace("%", "%%")
  3944. return (
  3945. "@echo off\r\n"
  3946. # cmd.exe reads a batch file in the console (OEM) code page, but this
  3947. # file is written as UTF-8: a profile path like C:\Users\Jos\u00e9 was
  3948. # garbled and the daemon never started. Switch to UTF-8 first; this
  3949. # line and everything above it is plain ASCII.
  3950. "chcp 65001\r\n"
  3951. "REM Token Optimizer dashboard daemon launcher (v5.3.2+).\r\n"
  3952. "REM Auto-generated. Resolves Python at runtime so interpreter upgrades\r\n"
  3953. "REM do not break the scheduled task.\r\n"
  3954. "REM GUI-subsystem interpreters first (pythonw/pyw) so the daemon\r\n"
  3955. "REM itself never owns a console. 9009 = cmd's 'command not found'.\r\n"
  3956. "REM The pythonw.exe rung skips Microsoft Store aliases (WindowsApps):\r\n"
  3957. "REM they exit 0 silently under Task Scheduler and would end the ladder\r\n"
  3958. "REM with no daemon started.\r\n"
  3959. "setlocal\r\n"
  3960. f'set "DAEMON_SCRIPT={daemon_script_path}"\r\n'
  3961. f'set "STDOUT_LOG={log_dir}\\stdout.log"\r\n'
  3962. f'set "STDERR_LOG={log_dir}\\stderr.log"\r\n'
  3963. 'if not exist "%DAEMON_SCRIPT%" (\r\n'
  3964. ' echo [%DATE% %TIME%] daemon script missing: "%DAEMON_SCRIPT%" >> "%STDERR_LOG%"\r\n'
  3965. " exit /b 1\r\n"
  3966. ")\r\n"
  3967. 'set "PYW_EXE="\r\n'
  3968. 'for %%I in (pythonw.exe) do set "PYW_EXE=%%~$PATH:I"\r\n'
  3969. 'if defined PYW_EXE if /i not "%PYW_EXE:windowsapps=%"=="%PYW_EXE%" set "PYW_EXE="\r\n'
  3970. 'if defined PYW_EXE "%PYW_EXE%" "%DAEMON_SCRIPT%" 1>>"%STDOUT_LOG%" 2>>"%STDERR_LOG%"\r\n'
  3971. 'if defined PYW_EXE if not "%ERRORLEVEL%"=="9009" exit /b %ERRORLEVEL%\r\n'
  3972. + _rung("pyw.exe -3")
  3973. + _rung("py.exe -3")
  3974. + _rung("python.exe")
  3975. + 'echo [%DATE% %TIME%] No Python found on PATH -- install python.org or winget install Python.Python.3 >> "%STDERR_LOG%"\r\n'
  3976. "exit /b 2\r\n"
  3977. )
  3978. def _probe_windows_port_owner(port):
  3979. """Return a human-readable string identifying the PID bound to port,
  3980. or None if the port is free or we can't tell.
  3981. A foreign service on 24842 would leave the user in
  3982. an indefinite 'wait 30s' state. Pre-install probe surfaces the
  3983. offending PID so the user can decide whether to reclaim the port.
  3984. """
  3985. _measure_facade = _compat.facade()
  3986. try:
  3987. result = _measure_facade.subprocess.run(
  3988. ["netstat", "-ano"],
  3989. capture_output=True, text=True, timeout=10, errors="replace", creationflags=_measure_facade._NO_WINDOW,
  3990. )
  3991. except (_measure_facade.subprocess.SubprocessError, OSError, FileNotFoundError):
  3992. return None
  3993. if result.returncode != 0:
  3994. return None
  3995. needle = f":{port}"
  3996. for line in result.stdout.splitlines():
  3997. if needle in line and "LISTENING" in line.upper():
  3998. parts = line.split()
  3999. pid = parts[-1] if parts else "?"
  4000. return f"pid={pid} (netstat: {line.strip()})"
  4001. return None
  4002. def _generate_schtasks_xml(task_name, user_id, command, arguments=None):
  4003. """Build a Task Scheduler XML payload for the dashboard daemon.
  4004. Uses the documented Task Scheduler 1.2 schema (MS-LEARN:
  4005. task-scheduler-schema-reference). Key choices:
  4006. - LogonTrigger so the daemon starts whenever the user logs in
  4007. (equivalent to launchd's RunAtLoad + KeepAlive on Mac).
  4008. - Hidden=true keeps it out of the default Task Scheduler UI
  4009. filter; advanced users can still show hidden tasks.
  4010. - StartWhenAvailable=true so a missed logon trigger (e.g., after
  4011. a reboot at a login prompt timeout) still fires.
  4012. - ExecutionTimeLimit=PT0S = no time limit (daemon runs forever).
  4013. - DisallowStartIfOnBatteries=false so laptop users get their
  4014. bookmarkable URL on battery power too.
  4015. ``command``/``arguments`` come from
  4016. ``_windows_task_exec_action`` and are normally pythonw.exe + the daemon
  4017. script, NOT the .cmd launcher. See that helper for why <Hidden> was never
  4018. enough. ``<Arguments>`` is emitted only when non-empty so the .cmd fallback
  4019. still produces the exact pre-fix XML shape.
  4020. """
  4021. _measure_facade = _compat.facade()
  4022. from xml.sax.saxutils import escape as _xml_escape
  4023. # Escape &, <, > plus ' and " so domain\user strings like
  4024. # "O'Brien & Co\\bob" survive the XML round-trip cleanly.
  4025. def xml_escape(s):
  4026. _measure_facade = _compat.facade()
  4027. return _xml_escape(s, {"'": "&apos;", '"': "&quot;"})
  4028. _ = task_name # kept for API symmetry; no longer embedded via <URI>
  4029. # No <URI> element: it is optional per the Task Scheduler 1.2 schema
  4030. # and creates a mismatch class when enterprise GPO relocates tasks
  4031. # into subfolders. /TN in the schtasks /Create call is sufficient.
  4032. # A BootTrigger requires elevated registration even with LeastPrivilege.
  4033. # This is a per-user service: logon is the correct boundary, including
  4034. # logins following Fast Startup. Keep installation non-elevated.
  4035. return (
  4036. '<?xml version="1.0" encoding="UTF-16"?>\n'
  4037. '<Task version="1.2" xmlns="http://schemas.microsoft.com/windows/2004/02/mit/task">\n'
  4038. " <RegistrationInfo>\n"
  4039. f" <Description>Serves the Token Optimizer dashboard on http://localhost:{_measure_facade.DAEMON_PORT}/token-optimizer</Description>\n"
  4040. " </RegistrationInfo>\n"
  4041. " <Triggers>\n"
  4042. " <LogonTrigger>\n"
  4043. " <Enabled>true</Enabled>\n"
  4044. f" <UserId>{xml_escape(user_id)}</UserId>\n"
  4045. " </LogonTrigger>\n"
  4046. " </Triggers>\n"
  4047. " <Principals>\n"
  4048. ' <Principal id="Author">\n'
  4049. f" <UserId>{xml_escape(user_id)}</UserId>\n"
  4050. " <LogonType>InteractiveToken</LogonType>\n"
  4051. " <RunLevel>LeastPrivilege</RunLevel>\n"
  4052. " </Principal>\n"
  4053. " </Principals>\n"
  4054. " <Settings>\n"
  4055. " <MultipleInstancesPolicy>IgnoreNew</MultipleInstancesPolicy>\n"
  4056. " <DisallowStartIfOnBatteries>false</DisallowStartIfOnBatteries>\n"
  4057. " <StopIfGoingOnBatteries>false</StopIfGoingOnBatteries>\n"
  4058. " <AllowHardTerminate>true</AllowHardTerminate>\n"
  4059. " <StartWhenAvailable>true</StartWhenAvailable>\n"
  4060. " <RunOnlyIfNetworkAvailable>false</RunOnlyIfNetworkAvailable>\n"
  4061. " <IdleSettings>\n"
  4062. " <StopOnIdleEnd>false</StopOnIdleEnd>\n"
  4063. " <RestartOnIdle>false</RestartOnIdle>\n"
  4064. " </IdleSettings>\n"
  4065. " <AllowStartOnDemand>true</AllowStartOnDemand>\n"
  4066. " <Enabled>true</Enabled>\n"
  4067. " <Hidden>true</Hidden>\n"
  4068. " <RunOnlyIfIdle>false</RunOnlyIfIdle>\n"
  4069. " <WakeToRun>false</WakeToRun>\n"
  4070. " <ExecutionTimeLimit>PT0S</ExecutionTimeLimit>\n"
  4071. " <Priority>7</Priority>\n"
  4072. " </Settings>\n"
  4073. ' <Actions Context="Author">\n'
  4074. " <Exec>\n"
  4075. f" <Command>{xml_escape(str(command))}</Command>\n"
  4076. + (f" <Arguments>{xml_escape(str(arguments))}</Arguments>\n"
  4077. if arguments else "")
  4078. + " </Exec>\n"
  4079. " </Actions>\n"
  4080. "</Task>\n"
  4081. )
  4082. def _install_task_scheduler_daemon(dry_run=None, soft_fail=None, effective_host=None):
  4083. """Windows: register a per-user Scheduled Task that runs the dashboard
  4084. daemon at logon, survives reboot, and stays out of the foreground.
  4085. Idempotent: schtasks /Create /F overwrites an existing task. The
  4086. bootout-then-bootstrap pattern from macOS maps to /End then /Create:
  4087. we stop any running instance before re-registering so we never
  4088. fight a stale process on port 24842.
  4089. soft_fail=True (hook paths, e.g. ensure-health self-heal): every error
  4090. path returns False instead of sys.exit(1) so SystemExit can never escape
  4091. and kill the calling Claude Code session. Returns True on success.
  4092. Not tested on a real Windows box from the development Mac. First
  4093. Windows user to run this is the de facto smoke test. Full rollback
  4094. is one command: `measure.py setup-daemon --uninstall`.
  4095. """
  4096. _measure_facade = _compat.facade()
  4097. if _measure_facade._scheduler_install_refused():
  4098. return False
  4099. def _fail(msg, *extra, permanent_reason=None):
  4100. # Only DEFINITIVE, permanent failure classes
  4101. # arm the sticky no-retry marker (MS-Store alias, schtasks missing,
  4102. # policy-denied task creation). Transient failures stay retryable and
  4103. # are bounded by the existing ensure/revive/heal throttles.
  4104. _measure_facade = _compat.facade()
  4105. if permanent_reason:
  4106. _measure_facade._write_daemon_install_failed_marker(permanent_reason)
  4107. # Under soft_fail we are inside a hook --
  4108. # stdout is session-visible context, so route errors to stderr instead
  4109. # of spamming "[Error] ..." into every SessionStart transcript.
  4110. out = _measure_facade.sys.stderr if soft_fail else _measure_facade.sys.stdout
  4111. print(msg, file=out)
  4112. for line in extra:
  4113. print(line, file=out)
  4114. if soft_fail:
  4115. return False
  4116. _measure_facade.sys.exit(1)
  4117. if dry_run:
  4118. print("[Token Optimizer] Dry run (Windows). Would install:\n")
  4119. print(" A tiny web server that makes your dashboard available at:")
  4120. print(f" http://localhost:{_measure_facade.DAEMON_PORT}/token-optimizer\n")
  4121. print(" What it does:")
  4122. print(" - Registers a per-user Scheduled Task (no UAC / admin rights needed)")
  4123. print(" - Starts automatically when you log into Windows")
  4124. print(" - Launched hidden by the Scheduled Task (via 'py -3', a stable")
  4125. print(" interpreter path across upgrades; falls back to pythonw.exe)")
  4126. for line in _measure_facade._daemon_access_lines(effective_host):
  4127. print(line)
  4128. print(" - Uses ~2MB of memory\n")
  4129. print(" Files it creates:")
  4130. print(f" {_measure_facade.SNAPSHOT_DIR / 'dashboard-server.py'}")
  4131. print(f" Scheduled Task: {_measure_facade.WINDOWS_TASK_NAME}\n")
  4132. print(" No changes written.")
  4133. return
  4134. if not _measure_facade._ensure_dashboard_file(to_stderr=soft_fail):
  4135. return _fail("[Error] Dashboard file missing; cannot start daemon.")
  4136. # Microsoft Store Python's App Execution Alias cannot be
  4137. # launched from Task Scheduler's service context. Refuse install
  4138. # with a clear remediation path rather than registering a task
  4139. # that will silently fail every logon.
  4140. if _measure_facade._is_ms_store_python_alias(_measure_facade.sys.executable):
  4141. return _fail(
  4142. "[Error] Microsoft Store Python detected (App Execution Alias).",
  4143. " Task Scheduler cannot launch Store aliases as background services,",
  4144. " so the daemon would never start. Install a real Python build instead:",
  4145. " winget install Python.Python.3.12",
  4146. " or download from python.org (make sure 'Add to PATH' is checked).",
  4147. " Then re-run: python -m measure setup-daemon",
  4148. # Structurally impossible until the user installs a real Python --
  4149. # exactly the class the sticky marker exists for.
  4150. permanent_reason="ms-store-python-alias",
  4151. )
  4152. # Compose DOMAIN\user when domain-joined so LogonTrigger
  4153. # actually fires on corporate machines.
  4154. user_id = _measure_facade._compose_windows_user_id()
  4155. if not user_id:
  4156. return _fail(
  4157. "[Error] Could not determine Windows username from %USERNAME%.",
  4158. " Set the USERNAME environment variable and retry.",
  4159. )
  4160. # Surface the port owner before we try to install so the
  4161. # user isn't stuck in an indefinite 'wait 30s' loop if 24842 is
  4162. # already bound by something else.
  4163. owner = _measure_facade._probe_windows_port_owner(_measure_facade.DAEMON_PORT)
  4164. if owner:
  4165. return _fail(
  4166. f"[Error] Port {_measure_facade.DAEMON_PORT} is already bound: {owner}",
  4167. " Release the port (or run --uninstall if it is our own stale",
  4168. " pythonw), then re-run setup. Reclaim: taskkill /PID <pid> /F",
  4169. )
  4170. _measure_facade._get_or_create_daemon_token()
  4171. with _measure_facade._daemon_install_lock(soft_fail=soft_fail) as acquired:
  4172. if soft_fail and not acquired:
  4173. # Another installer holds the lock; don't fight it from a hook path.
  4174. return False
  4175. _measure_facade.SNAPSHOT_DIR.mkdir(parents=True, exist_ok=True)
  4176. _measure_facade.DAEMON_LOG_DIR.mkdir(parents=True, exist_ok=True)
  4177. daemon_script = _measure_facade.SNAPSHOT_DIR / "dashboard-server.py"
  4178. daemon_script.write_text(_measure_facade._generate_daemon_script(), encoding="utf-8")
  4179. # .cmd launcher resolves Python at runtime (py -> pythonw ->
  4180. # python) and redirects daemon stdout/stderr into logs so future
  4181. # failures leave a trail instead of vanishing.
  4182. # write_BYTES, not write_text: the shim is explicitly CRLF, and
  4183. # text-mode writing translates "\n" to os.linesep, which on Windows
  4184. # turns every "\r\n" into "\r\r\n". Bytes also makes the file
  4185. # byte-comparable, which is what lets _heal_windows_launcher_shim be
  4186. # idempotent instead of rewriting it on every ensure-health.
  4187. launcher_path = _measure_facade.SNAPSHOT_DIR / _measure_facade.WINDOWS_LAUNCHER_NAME
  4188. launcher_path.write_bytes(
  4189. _measure_facade._generate_windows_launcher_cmd(
  4190. str(daemon_script), str(_measure_facade.DAEMON_LOG_DIR)).encode("utf-8"),
  4191. )
  4192. # Prefer a GUI-subsystem pythonw.exe action over the .cmd
  4193. # launcher so Task Scheduler cannot paint a console window on every
  4194. # fire (logon, boot, and every `schtasks /Run` from the revive path).
  4195. _task_command, _task_arguments = _measure_facade._windows_task_exec_action(
  4196. str(daemon_script), str(launcher_path))
  4197. if not _task_arguments:
  4198. print("[Token Optimizer] Note: pythonw.exe not found next to this "
  4199. "interpreter; falling back to the .cmd launcher, which shows "
  4200. "a console window when the task runs.")
  4201. xml_payload = _measure_facade._generate_schtasks_xml(
  4202. task_name=_measure_facade.WINDOWS_TASK_NAME,
  4203. user_id=user_id,
  4204. command=_task_command,
  4205. arguments=_task_arguments,
  4206. )
  4207. # UTF-16 LE with BOM: matches schtasks /Query native output and
  4208. # pins the byte order so future Python encoding defaults can't
  4209. # drift into BE and break schtasks XML parsing.
  4210. xml_path = _measure_facade.SNAPSHOT_DIR / ".schtasks-daemon.xml"
  4211. xml_path.write_bytes(b"\xff\xfe" + xml_payload.encode("utf-16-le"))
  4212. # Stop any prior instance -- safe to fail if task doesn't exist.
  4213. # FileNotFoundError here means schtasks.exe itself is missing from the
  4214. # machine: a STRUCTURAL condition (no scheduler to register with), the
  4215. # one exception class worth the sticky marker.
  4216. try:
  4217. _measure_facade.subprocess.run(
  4218. ["schtasks", "/End", "/TN", _measure_facade.WINDOWS_TASK_NAME],
  4219. capture_output=True, text=True, errors="replace", creationflags=_measure_facade._NO_WINDOW,
  4220. )
  4221. # Register (or overwrite) the task.
  4222. create = _measure_facade.subprocess.run(
  4223. ["schtasks", "/Create", "/XML", str(xml_path),
  4224. "/TN", _measure_facade.WINDOWS_TASK_NAME, "/F"],
  4225. capture_output=True, text=True, errors="replace", creationflags=_measure_facade._NO_WINDOW,
  4226. )
  4227. except FileNotFoundError:
  4228. return _fail(
  4229. "[Error] schtasks.exe not found on this system.",
  4230. " The dashboard daemon needs the Windows Task Scheduler CLI.",
  4231. permanent_reason="schtasks-missing",
  4232. )
  4233. if create.returncode != 0:
  4234. _stderr_lower = (create.stderr or "").lower()
  4235. # "Access is denied" / GPO-blocked task creation is a policy
  4236. # decision, not a blip -- it will fail identically next prompt.
  4237. # Everything else (transient service hiccup, odd one-off exit)
  4238. # stays retryable under the throttles.
  4239. _denied = ("access is denied" in _stderr_lower
  4240. or "denied" in _stderr_lower)
  4241. return _fail(
  4242. f"[Error] schtasks /Create failed: {create.stderr.strip()}",
  4243. " Common causes: locked-down enterprise policy blocking task",
  4244. " creation, or schtasks.exe missing. Try manually with:",
  4245. f" schtasks /Create /XML \"{xml_path}\" /TN {_measure_facade.WINDOWS_TASK_NAME} /F",
  4246. permanent_reason=(
  4247. "schtasks-create-access-denied" if _denied else None),
  4248. )
  4249. # Fire the task immediately so the user's first URL click works.
  4250. _measure_facade.subprocess.run(
  4251. ["schtasks", "/Run", "/TN", _measure_facade.WINDOWS_TASK_NAME],
  4252. capture_output=True, text=True, errors="replace", creationflags=_measure_facade._NO_WINDOW,
  4253. )
  4254. _measure_facade.time.sleep(1)
  4255. if _measure_facade._verify_daemon_port():
  4256. print("[Token Optimizer] Dashboard server installed and running.\n")
  4257. print(" Bookmark this URL:")
  4258. print(f" http://localhost:{_measure_facade.DAEMON_PORT}/token-optimizer\n")
  4259. _measure_facade._print_daemon_network_note(effective_host)
  4260. print(f" It updates automatically after every {_measure_facade.runtime_name_for_humans()} session.")
  4261. print(" Starts at logon, so the URL always works.\n")
  4262. print(" To remove: python -m measure setup-daemon --uninstall")
  4263. else:
  4264. print("[Token Optimizer] Task registered but port not yet reachable.")
  4265. print(f" Give it 30s, then open: http://localhost:{_measure_facade.DAEMON_PORT}/token-optimizer")
  4266. print(" Do NOT run --uninstall while it is still coming up.")
  4267. print(f" If still unreachable, check daemon logs: {_measure_facade.DAEMON_LOG_DIR}\\stderr.log")
  4268. print(f" and task status: schtasks /Query /TN {_measure_facade.WINDOWS_TASK_NAME} /V /FO LIST")
  4269. # Install succeeded (task registered); port may still be warming up.
  4270. return True
  4271. def _uninstall_task_scheduler_daemon(this_install_only=None, dry_run=None):
  4272. """Windows: stop and remove the dashboard daemon scheduled task.
  4273. Cleans orphan XML files from any prior naming convention via glob so
  4274. version drift doesn't leave artifacts behind.
  4275. v5.11.68: identity-sweeping by default. The scheduled task is
  4276. per-RUNTIME (shared across ``token-optimizer-*`` identities of the same
  4277. runtime), so it is removed once, and we query+delete EVERY runtime's task
  4278. name so a task whose data dir already vanished still gets unregistered.
  4279. The per-identity files (``dashboard-server.py``, the .cmd launcher, the
  4280. schtasks XML, ``daemon-token``, ``dashboard-host``, ``.daemon-thrash``)
  4281. live in EACH identity's snapshot dir; a sibling's daemon script + 0600
  4282. CSRF token would otherwise outlive the uninstall. Sweep every identity
  4283. unless ``this_install_only`` opts out.
  4284. v5.11.68 (cleanup): ``dry_run=True`` makes the uninstaller
  4285. side-effect-free (no schtasks calls, no file deletion, no tombstone write)
  4286. so the cleanup command's ``--dry-run`` is a true preview.
  4287. """
  4288. _measure_facade = _compat.facade()
  4289. if _measure_facade._daemon_snapshot_sandboxed():
  4290. return False
  4291. if dry_run:
  4292. would_remove = []
  4293. per_identity: list[tuple[Path, list[str]]] = []
  4294. for snap_dir in _measure_facade._daemon_identity_snapshot_dirs(this_install_only):
  4295. files = _measure_facade._daemon_per_identity_files(snap_dir)
  4296. identity_would = [str(files[k]) for k in ("daemon_script", "windows_launcher", "daemon_token", "daemon_host") if files[k].exists()]
  4297. try:
  4298. for xml_path in snap_dir.glob("*schtasks-daemon*.xml"):
  4299. identity_would.append(str(xml_path))
  4300. except OSError:
  4301. pass
  4302. if identity_would:
  4303. per_identity.append((snap_dir, identity_would))
  4304. would_remove.extend(identity_would)
  4305. if would_remove:
  4306. print("[Token Optimizer] Dashboard daemon would be removed.")
  4307. for path in would_remove:
  4308. print(f" Would delete: {path}")
  4309. if per_identity and not this_install_only:
  4310. print(" Would sweep all token-optimizer-* identities:")
  4311. for snap_dir, items in per_identity:
  4312. print(f" {snap_dir}: {len(items)} file(s)")
  4313. else:
  4314. print("[Token Optimizer] No daemon artifacts found. Nothing to remove.")
  4315. return
  4316. # Tombstone FIRST so any racing respawn exits cleanly.
  4317. _measure_facade._write_uninstall_tombstone()
  4318. removed = []
  4319. # Sweep the OS scheduler by name across every runtime variant so a task
  4320. # whose data dir already vanished still gets unregistered.
  4321. for task_name in _measure_facade._scheduler_names_to_sweep(
  4322. this_install_only, _measure_facade._ALL_WINDOWS_TASK_NAMES, _measure_facade.WINDOWS_TASK_NAME):
  4323. try:
  4324. _measure_facade.subprocess.run(
  4325. ["schtasks", "/End", "/TN", task_name],
  4326. capture_output=True, text=True, errors="replace", timeout=10, creationflags=_measure_facade._NO_WINDOW,
  4327. )
  4328. except (OSError, _measure_facade.subprocess.TimeoutExpired):
  4329. pass
  4330. try:
  4331. delete = _measure_facade.subprocess.run(
  4332. ["schtasks", "/Delete", "/TN", task_name, "/F"],
  4333. capture_output=True, text=True, errors="replace", timeout=10, creationflags=_measure_facade._NO_WINDOW,
  4334. )
  4335. except (OSError, _measure_facade.subprocess.TimeoutExpired):
  4336. delete = _measure_facade.subprocess.CompletedProcess(args=[], returncode=1, stdout="", stderr="timeout")
  4337. if delete.returncode == 0:
  4338. removed.append(f"Scheduled Task: {task_name}")
  4339. # Sweep per-identity daemon files across every token-optimizer-* identity.
  4340. per_identity_removed: list[tuple[Path, list[str]]] = []
  4341. sweep_failed: list[str] = []
  4342. for snap_dir in _measure_facade._daemon_identity_snapshot_dirs(this_install_only):
  4343. identity_removed, identity_failed = _measure_facade._sweep_identity_daemon_files(
  4344. snap_dir, ("daemon_script", "windows_launcher", "daemon_token", "daemon_host"))
  4345. sweep_failed.extend(identity_failed)
  4346. # The .daemon-thrash tombstone must PERSIST for every
  4347. # swept identity. Each identity's generated dashboard-server.py checks
  4348. # its OWN breadcrumb path on start and exits "noop-tombstoned"; the
  4349. # previous unlink here re-armed self-revive, so an orphaned LaunchAgent
  4350. # /task/unit could resurrect the daemon and mint a fresh 0600 token
  4351. # after a "successful" uninstall. Only a legitimate reinstall clears it
  4352. # (setup_daemon unlinks before start). NOT counted toward `removed` so
  4353. # the "Nothing to remove" honesty invariant holds.
  4354. _measure_facade._write_uninstall_tombstone(snap_dir)
  4355. # Glob-clean any schtasks XML regardless of version-specific naming
  4356. # (".schtasks-daemon.xml", "schtasks-daemon.xml", etc.).
  4357. try:
  4358. for xml_path in snap_dir.glob("*schtasks-daemon*.xml"):
  4359. try:
  4360. xml_path.unlink()
  4361. identity_removed.append(str(xml_path))
  4362. except OSError:
  4363. pass
  4364. except OSError:
  4365. pass
  4366. if identity_removed:
  4367. per_identity_removed.append((snap_dir, identity_removed))
  4368. removed.extend(identity_removed)
  4369. _measure_facade._print_identity_sweep_report(
  4370. removed, per_identity_removed, sweep_failed, this_install_only)
  4371. def _generate_linux_launcher_sh(daemon_script_path, log_dir):
  4372. """Generate a POSIX shell launcher that resolves python at runtime.
  4373. Parallels the Windows .cmd launcher. ExecStart pointing directly at
  4374. sys.executable would break when the user's python install moves
  4375. (pipx venv removed, distro upgrade, brew switch), leaving a
  4376. stranded unit that journald spams forever. The launcher re-resolves
  4377. every invocation (python3 -> python), so Python changes no longer
  4378. invalidate the installed unit.
  4379. """
  4380. _measure_facade = _compat.facade()
  4381. return (
  4382. "#!/bin/sh\n"
  4383. "# Token Optimizer dashboard daemon launcher (v5.3.4+).\n"
  4384. "# Auto-generated by measure.py. Resolves Python at runtime so\n"
  4385. "# interpreter moves/upgrades don't invalidate the systemd unit.\n"
  4386. "set -u\n"
  4387. f'DAEMON_SCRIPT="{daemon_script_path}"\n'
  4388. f'STDOUT_LOG="{log_dir}/stdout.log"\n'
  4389. f'STDERR_LOG="{log_dir}/stderr.log"\n'
  4390. 'if [ ! -f "$DAEMON_SCRIPT" ]; then\n'
  4391. ' echo "daemon script missing: $DAEMON_SCRIPT" >>"$STDERR_LOG" 2>&1 || true\n'
  4392. " exit 1\n"
  4393. "fi\n"
  4394. "for PY in python3 python; do\n"
  4395. ' if command -v "$PY" >/dev/null 2>&1; then\n'
  4396. ' exec "$PY" "$DAEMON_SCRIPT"\n'
  4397. " fi\n"
  4398. "done\n"
  4399. 'echo "no python interpreter found on PATH" >>"$STDERR_LOG" 2>&1 || true\n'
  4400. "exit 2\n"
  4401. )
  4402. def _probe_systemd_user_bus():
  4403. """Return True if `systemctl --user` can actually reach the user bus.
  4404. `systemctl --user --version` succeeds even when the user-bus is
  4405. dead (SSH session without linger, cron context, WSL2 w/o systemd).
  4406. A real reachability check runs `list-units` which requires the
  4407. dbus-session. Distinguishes "systemd present but bus dead" from
  4408. "systemd absent" so the installer can emit a targeted error rather
  4409. than planting a unit file that will never start.
  4410. """
  4411. _measure_facade = _compat.facade()
  4412. try:
  4413. result = _measure_facade.subprocess.run(
  4414. ["systemctl", "--user", "list-units", "--no-legend"],
  4415. capture_output=True, text=True, timeout=5, creationflags=_measure_facade._NO_WINDOW,
  4416. )
  4417. except (_measure_facade.subprocess.SubprocessError, OSError, FileNotFoundError):
  4418. return False
  4419. return result.returncode == 0
  4420. def _systemd_user_unit_path():
  4421. """Resolve the target path for the user systemd unit.
  4422. Respects $XDG_CONFIG_HOME when set (the freedesktop spec), falling
  4423. back to ~/.config. Creating the parent directory is the installer's
  4424. responsibility.
  4425. """
  4426. _measure_facade = _compat.facade()
  4427. xdg = _measure_facade.os.environ.get("XDG_CONFIG_HOME", "").strip()
  4428. base = _measure_facade.Path(xdg) if xdg else _measure_facade.Path.home() / ".config"
  4429. return base / "systemd" / "user" / _measure_facade.SYSTEMD_UNIT_NAME
  4430. def _systemd_user_unit_path_for(unit_name: str) -> Path:
  4431. """Resolve the unit path for an arbitrary unit name.
  4432. Same base resolution as ``_systemd_user_unit_path`` but for any runtime
  4433. variant's unit name, so the sweeping uninstall can remove a sibling
  4434. runtime's registration whose data dir already vanished.
  4435. """
  4436. _measure_facade = _compat.facade()
  4437. xdg = _measure_facade.os.environ.get("XDG_CONFIG_HOME", "").strip()
  4438. base = _measure_facade.Path(xdg) if xdg else _measure_facade.Path.home() / ".config"
  4439. return base / "systemd" / "user" / unit_name
  4440. def _generate_systemd_user_unit(launcher_path, log_dir):
  4441. """Build a systemd user unit for the dashboard daemon.
  4442. ExecStart points at a shell launcher (_generate_linux_launcher_sh)
  4443. rather than a hardcoded sys.executable. The launcher resolves
  4444. Python at runtime so pipx venv removals, distro upgrades, or brew
  4445. swaps don't strand the unit. The ExecStart path
  4446. is double-quoted per systemd.service(5) so paths with spaces in
  4447. HOME survive tokenization.
  4448. Design choices:
  4449. - Restart=on-failure + RestartSec=10 mirrors macOS launchd
  4450. KeepAlive on-non-zero-exit behavior.
  4451. - WantedBy=default.target so enable links the unit into the
  4452. user's default target (graphical-session.target isn't
  4453. universal across distros).
  4454. - StandardOutput/StandardError piped to append-mode log files
  4455. under DAEMON_LOG_DIR so port conflicts and import errors
  4456. leave a trail, matching the Windows .cmd launcher.
  4457. - Type=simple (default) -- the daemon runs in the foreground of
  4458. its own process group, which is what we want.
  4459. """
  4460. _measure_facade = _compat.facade()
  4461. return (
  4462. "[Unit]\n"
  4463. "Description=Token Optimizer Dashboard\n"
  4464. "Documentation=https://github.com/alexgreensh/token-optimizer\n"
  4465. "After=default.target\n"
  4466. "\n"
  4467. "[Service]\n"
  4468. f'ExecStart="{launcher_path}"\n'
  4469. "Restart=on-failure\n"
  4470. "RestartSec=10\n"
  4471. f"StandardOutput=append:{log_dir}/stdout.log\n"
  4472. f"StandardError=append:{log_dir}/stderr.log\n"
  4473. "\n"
  4474. "[Install]\n"
  4475. "WantedBy=default.target\n"
  4476. )
  4477. def _install_systemd_user_daemon(dry_run=None, soft_fail=None, effective_host=None):
  4478. """Linux: install the dashboard daemon via systemd --user.
  4479. User-scoped unit, no root required. Survives reboot when the user
  4480. has lingering enabled (loginctl enable-linger $USER -- we surface
  4481. the hint but don't auto-enable because some distros require
  4482. sudo/pkexec for that operation).
  4483. soft_fail=True (hook paths, e.g. ensure-health self-heal): every error
  4484. path returns False instead of sys.exit(1) so SystemExit can never escape
  4485. and kill the calling Claude Code session. Returns True on success. CLI
  4486. callers keep the default False for the hard-failure + actionable-hint UX.
  4487. """
  4488. _measure_facade = _compat.facade()
  4489. if _measure_facade._scheduler_install_refused():
  4490. return False
  4491. def _fail(msg, *extra):
  4492. # Under soft_fail we are inside a hook --
  4493. # stdout is session-visible context, so route errors to stderr instead
  4494. # of spamming "[Error] ..." into every SessionStart transcript.
  4495. _measure_facade = _compat.facade()
  4496. out = _measure_facade.sys.stderr if soft_fail else _measure_facade.sys.stdout
  4497. print(msg, file=out)
  4498. for line in extra:
  4499. print(line, file=out)
  4500. if soft_fail:
  4501. return False
  4502. _measure_facade.sys.exit(1)
  4503. unit_path = _measure_facade._systemd_user_unit_path()
  4504. daemon_script = _measure_facade.SNAPSHOT_DIR / "dashboard-server.py"
  4505. if dry_run:
  4506. print("[Token Optimizer] Dry run (Linux). Would install:\n")
  4507. print(" A tiny web server that makes your dashboard available at:")
  4508. print(f" http://localhost:{_measure_facade.DAEMON_PORT}/token-optimizer\n")
  4509. print(" What it does:")
  4510. print(" - Registers a systemd --user unit (no root needed)")
  4511. print(" - Starts automatically at login via default.target")
  4512. print(" - Restarts itself on failure (Restart=on-failure)")
  4513. for line in _measure_facade._daemon_access_lines(effective_host):
  4514. print(line)
  4515. print(" - Uses ~2MB of memory\n")
  4516. print(" Files it creates:")
  4517. print(f" {daemon_script}")
  4518. print(f" {unit_path}\n")
  4519. print(" Tip: `loginctl enable-linger $USER` makes the daemon survive")
  4520. print(" logout without requiring an active session. (Needs sudo on some distros.)\n")
  4521. print(" No changes written.")
  4522. return
  4523. # Reachability check: `systemctl --user --version`
  4524. # succeeds even when the user-bus is dead (SSH w/o linger, cron,
  4525. # WSL2 without systemd=true). Probing list-units requires the bus,
  4526. # so a positive result confirms we can actually enable a unit.
  4527. if not _measure_facade._probe_systemd_user_bus():
  4528. return _fail(
  4529. "[Error] systemctl --user is not reachable on this system.",
  4530. " Likely causes:",
  4531. " - systemd is not installed (minimal container, WSL2 w/o systemd=true)",
  4532. " - this is a headless SSH session without lingering enabled",
  4533. " - the user dbus is not running in this shell",
  4534. " Try: loginctl enable-linger $USER (may need sudo on some distros)",
  4535. _measure_facade._hint(" Then re-run: python3 measure.py setup-daemon"),
  4536. f" Meanwhile, the dashboard file still works: {_measure_facade.DASHBOARD_PATH.as_uri()}",
  4537. )
  4538. if not _measure_facade._ensure_dashboard_file(to_stderr=soft_fail):
  4539. return _fail("[Error] Dashboard file missing; cannot start daemon.")
  4540. _measure_facade._get_or_create_daemon_token()
  4541. with _measure_facade._daemon_install_lock(soft_fail=soft_fail) as acquired:
  4542. if soft_fail and not acquired:
  4543. # Another installer holds the lock; don't fight it from a hook path.
  4544. return False
  4545. # Ordering: stop + reclaim FIRST, BEFORE touching
  4546. # unit/daemon files, so a failed stop never races a half-written
  4547. # new unit. On failure after write, rollback removes artifacts
  4548. # so re-running doesn't inherit a broken half-install.
  4549. _measure_facade.subprocess.run(
  4550. ["systemctl", "--user", "disable", "--now", _measure_facade.SYSTEMD_UNIT_NAME],
  4551. capture_output=True, text=True, errors="replace", creationflags=_measure_facade._NO_WINDOW,
  4552. )
  4553. _measure_facade._reclaim_posix_daemon_port()
  4554. _measure_facade.SNAPSHOT_DIR.mkdir(parents=True, exist_ok=True)
  4555. _measure_facade.DAEMON_LOG_DIR.mkdir(parents=True, exist_ok=True)
  4556. daemon_script.write_text(_measure_facade._generate_daemon_script(), encoding="utf-8")
  4557. try:
  4558. daemon_script.chmod(0o755)
  4559. except OSError:
  4560. pass
  4561. launcher_path = _measure_facade.SNAPSHOT_DIR / _measure_facade.LINUX_LAUNCHER_NAME
  4562. launcher_path.write_text(
  4563. _measure_facade._generate_linux_launcher_sh(str(daemon_script), str(_measure_facade.DAEMON_LOG_DIR)),
  4564. encoding="utf-8",
  4565. )
  4566. try:
  4567. launcher_path.chmod(0o755)
  4568. except OSError:
  4569. pass
  4570. unit_path.parent.mkdir(parents=True, exist_ok=True)
  4571. unit_path.write_text(
  4572. _measure_facade._generate_systemd_user_unit(
  4573. launcher_path=str(launcher_path),
  4574. log_dir=str(_measure_facade.DAEMON_LOG_DIR),
  4575. ),
  4576. encoding="utf-8",
  4577. )
  4578. def _rollback(reason):
  4579. """Remove everything this install just wrote so a retry
  4580. isn't blocked by half-installed artifacts."""
  4581. _measure_facade = _compat.facade()
  4582. print(f" Rolling back: {reason}")
  4583. for p in (unit_path, launcher_path):
  4584. try:
  4585. if p.exists():
  4586. p.unlink()
  4587. except OSError:
  4588. pass
  4589. _measure_facade.subprocess.run(
  4590. ["systemctl", "--user", "daemon-reload"],
  4591. capture_output=True, text=True, errors="replace", creationflags=_measure_facade._NO_WINDOW,
  4592. )
  4593. reload_result = _measure_facade.subprocess.run(
  4594. ["systemctl", "--user", "daemon-reload"],
  4595. capture_output=True, text=True, errors="replace", creationflags=_measure_facade._NO_WINDOW,
  4596. )
  4597. if reload_result.returncode != 0:
  4598. _rollback("daemon-reload failed")
  4599. return _fail(f"[Error] systemctl daemon-reload failed: {reload_result.stderr.strip()}")
  4600. enable_result = _measure_facade.subprocess.run(
  4601. ["systemctl", "--user", "enable", "--now", _measure_facade.SYSTEMD_UNIT_NAME],
  4602. capture_output=True, text=True, errors="replace", creationflags=_measure_facade._NO_WINDOW,
  4603. )
  4604. if enable_result.returncode != 0:
  4605. _rollback("enable failed")
  4606. return _fail(
  4607. f"[Error] systemctl --user enable failed: {enable_result.stderr.strip()}",
  4608. " Common causes: XDG_RUNTIME_DIR missing (SSH w/o linger), user bus",
  4609. " refusing a root-owned unit file (bind-mounted home), or a polkit",
  4610. " prompt that couldn't reach a TTY.",
  4611. )
  4612. _measure_facade.time.sleep(1)
  4613. if _measure_facade._verify_daemon_port():
  4614. print("[Token Optimizer] Dashboard server installed and running.\n")
  4615. print(" Bookmark this URL:")
  4616. print(f" http://localhost:{_measure_facade.DAEMON_PORT}/token-optimizer\n")
  4617. _measure_facade._print_daemon_network_note(effective_host)
  4618. print(f" It updates automatically after every {_measure_facade.runtime_name_for_humans()} session.")
  4619. print(" Starts at login via default.target.\n")
  4620. print(" Survive logout: loginctl enable-linger $USER (may need sudo)")
  4621. print(_measure_facade._hint(" To remove: python3 measure.py setup-daemon --uninstall"))
  4622. else:
  4623. print("[Token Optimizer] Unit enabled but port not yet reachable.")
  4624. print(f" Give it 30s, then open: http://localhost:{_measure_facade.DAEMON_PORT}/token-optimizer")
  4625. print(" Do NOT run --uninstall while it is still coming up.")
  4626. print(f" Diagnose: systemctl --user status {_measure_facade.SYSTEMD_UNIT_NAME}")
  4627. print(f" Logs: {_measure_facade.DAEMON_LOG_DIR}/stderr.log")
  4628. # Install succeeded (unit enabled); port may still be warming up.
  4629. return True
  4630. def _uninstall_systemd_user_daemon(this_install_only=None, dry_run=None):
  4631. """Linux: stop and remove the systemd --user dashboard unit.
  4632. v5.11.68: identity-sweeping by default. The systemd unit is
  4633. per-RUNTIME (shared across ``token-optimizer-*`` identities of the same
  4634. runtime), so it is removed once, and we disable+remove EVERY runtime's
  4635. unit so a unit whose data dir already vanished still gets unregistered.
  4636. The per-identity files (``dashboard-server.py``, the launcher script,
  4637. ``daemon-token``, ``dashboard-host``, ``.daemon-thrash``) live in EACH
  4638. identity's snapshot dir; a sibling's daemon script + 0600 CSRF token
  4639. would otherwise outlive the uninstall. Sweep every identity unless
  4640. ``this_install_only`` opts out.
  4641. v5.11.68 (cleanup): ``dry_run=True`` makes the uninstaller
  4642. side-effect-free (no systemctl calls, no file deletion, no tombstone write)
  4643. so the cleanup command's ``--dry-run`` is a true preview.
  4644. """
  4645. _measure_facade = _compat.facade()
  4646. if _measure_facade._daemon_snapshot_sandboxed():
  4647. return False
  4648. if dry_run:
  4649. would_remove = []
  4650. per_identity: list[tuple[Path, list[str]]] = []
  4651. for unit_name in _measure_facade._scheduler_names_to_sweep(
  4652. this_install_only, _measure_facade._ALL_SYSTEMD_UNIT_NAMES, _measure_facade.SYSTEMD_UNIT_NAME):
  4653. unit_path = _measure_facade._systemd_user_unit_path_for(unit_name)
  4654. if unit_path.exists():
  4655. would_remove.append(str(unit_path))
  4656. for snap_dir in _measure_facade._daemon_identity_snapshot_dirs(this_install_only):
  4657. files = _measure_facade._daemon_per_identity_files(snap_dir)
  4658. identity_would = [str(files[k]) for k in ("daemon_script", "linux_launcher", "daemon_token", "daemon_host") if files[k].exists()]
  4659. if identity_would:
  4660. per_identity.append((snap_dir, identity_would))
  4661. would_remove.extend(identity_would)
  4662. if would_remove:
  4663. print("[Token Optimizer] Dashboard daemon would be removed.")
  4664. for path in would_remove:
  4665. print(f" Would delete: {path}")
  4666. if per_identity and not this_install_only:
  4667. print(" Would sweep all token-optimizer-* identities:")
  4668. for snap_dir, items in per_identity:
  4669. print(f" {snap_dir}: {len(items)} file(s)")
  4670. else:
  4671. print("[Token Optimizer] No daemon artifacts found. Nothing to remove.")
  4672. return
  4673. # Tombstone FIRST so any racing respawn exits cleanly.
  4674. _measure_facade._write_uninstall_tombstone()
  4675. removed = []
  4676. # Sweep the OS scheduler by name across every runtime variant.
  4677. for unit_name in _measure_facade._scheduler_names_to_sweep(
  4678. this_install_only, _measure_facade._ALL_SYSTEMD_UNIT_NAMES, _measure_facade.SYSTEMD_UNIT_NAME):
  4679. try:
  4680. _measure_facade.subprocess.run(
  4681. ["systemctl", "--user", "disable", "--now", unit_name],
  4682. capture_output=True, text=True, errors="replace", timeout=10, creationflags=_measure_facade._NO_WINDOW,
  4683. )
  4684. except (OSError, _measure_facade.subprocess.TimeoutExpired):
  4685. pass
  4686. unit_path = _measure_facade._systemd_user_unit_path_for(unit_name)
  4687. if _measure_facade._unlink_if_exists(unit_path):
  4688. removed.append(str(unit_path))
  4689. # daemon-reload AFTER unit file removal so systemd drops the units
  4690. # from its in-memory catalog.
  4691. try:
  4692. _measure_facade.subprocess.run(
  4693. ["systemctl", "--user", "daemon-reload"],
  4694. capture_output=True, text=True, errors="replace", timeout=10, creationflags=_measure_facade._NO_WINDOW,
  4695. )
  4696. except (OSError, _measure_facade.subprocess.TimeoutExpired):
  4697. pass
  4698. # Unregistering the job does NOT stop the process --
  4699. # `launchctl bootout` / `systemctl disable` drop the registration without
  4700. # SIGTERMing the running child, so the daemon kept serving its port with
  4701. # the 0600 CSRF token live in memory until logout. Reclaim the port(s) now
  4702. # that the tombstone is down (so nothing races a respawn) and the job is
  4703. # unregistered. On a sweep-all uninstall this covers EVERY runtime's port
  4704. # (a sibling runtime's daemon + live CSRF token otherwise
  4705. # survived), scoped back to the resolved runtime under --this-install-only.
  4706. # Never kills a foreign process on a port -- the helper verifies the command
  4707. # line is our own dashboard-server.py.
  4708. _measure_facade._reclaim_daemon_ports(this_install_only)
  4709. # Sweep per-identity daemon files across every token-optimizer-* identity.
  4710. per_identity_removed: list[tuple[Path, list[str]]] = []
  4711. sweep_failed: list[str] = []
  4712. for snap_dir in _measure_facade._daemon_identity_snapshot_dirs(this_install_only):
  4713. identity_removed, identity_failed = _measure_facade._sweep_identity_daemon_files(
  4714. snap_dir, ("daemon_script", "linux_launcher", "daemon_token", "daemon_host"))
  4715. sweep_failed.extend(identity_failed)
  4716. # The .daemon-thrash tombstone must PERSIST for every
  4717. # swept identity. Each identity's generated dashboard-server.py checks
  4718. # its OWN breadcrumb path on start and exits "noop-tombstoned"; the
  4719. # previous unlink here re-armed self-revive, so an orphaned LaunchAgent
  4720. # /task/unit could resurrect the daemon and mint a fresh 0600 token
  4721. # after a "successful" uninstall. Only a legitimate reinstall clears it
  4722. # (setup_daemon unlinks before start). NOT counted toward `removed` so
  4723. # the "Nothing to remove" honesty invariant holds.
  4724. _measure_facade._write_uninstall_tombstone(snap_dir)
  4725. if identity_removed:
  4726. per_identity_removed.append((snap_dir, identity_removed))
  4727. removed.extend(identity_removed)
  4728. _measure_facade._print_identity_sweep_report(
  4729. removed, per_identity_removed, sweep_failed, this_install_only)
  4730. def _normalized_platform():
  4731. """Return a normalized platform label.
  4732. Hedges against frozen builds or test harnesses that
  4733. monkey-patch platform.system() can return non-canonical casing
  4734. ("darwin", "Macintosh", "MacOSX"). We normalize a small allowlist
  4735. of aliases to the canonical CPython labels.
  4736. """
  4737. _measure_facade = _compat.facade()
  4738. raw = (_measure_facade.platform.system() or "").strip()
  4739. lower = raw.lower()
  4740. if lower in ("darwin", "macintosh", "macosx", "mac os x", "mac"):
  4741. return "Darwin"
  4742. if lower == "windows":
  4743. return "Windows"
  4744. if lower == "linux":
  4745. return "Linux"
  4746. return raw
  4747. def _setup_daemon_cli(args):
  4748. """`setup-daemon` CLI entry. A refusal used to return "noop-sandbox" to a
  4749. caller that ignored it, so the command printed nothing and the user had
  4750. no idea why the dashboard never came back. Prints the reason, exit 1."""
  4751. _measure_facade = _compat.facade()
  4752. dry = "--dry-run" in args
  4753. uninstall = "--uninstall" in args
  4754. this_install_only = "--this-install-only" in args
  4755. result = _measure_facade.setup_daemon(dry_run=dry, uninstall=uninstall, this_install_only=this_install_only)
  4756. if result == "noop-sandbox":
  4757. detail = "" if (uninstall or _measure_facade._daemon_snapshot_sandboxed()) else _measure_facade._scheduler_job_points_elsewhere()
  4758. print("[Token Optimizer] setup-daemon " + _measure_facade._SCHEDULER_REFUSAL
  4759. + (" (existing job: %s)" % detail if detail else ""))
  4760. return 1
  4761. return 0
  4762. def setup_daemon(dry_run=None, uninstall=None, this_install_only=None, latch_optout=None):
  4763. """Install or remove the persistent dashboard HTTP server daemon.
  4764. ``latch_optout`` (default True) controls whether an uninstall persists the
  4765. sticky ``daemon_disabled`` opt-out that stops the SessionStart self-heal from
  4766. ever resurrecting the daemon. The dedicated ``setup-daemon --uninstall``
  4767. command -- whose whole purpose is "turn my dashboard off" -- passes True.
  4768. ``cleanup`` (the full-uninstall tidy) passes False: it removes the hooks too,
  4769. so the self-heal simply stops running after a genuine uninstall, and if the
  4770. user only tidied up while keeping Token Optimizer active, the self-heal must
  4771. be free to bring the dashboard back on its own. A tidy is not a deliberate
  4772. opt-out, and must never silently kill the dashboard until a manual command.
  4773. Dispatches to a platform-specific installer/uninstaller. macOS uses
  4774. launchd (LaunchAgent); Windows and Linux installers ship in future
  4775. releases. All platforms share the daemon script and port
  4776. (DAEMON_PORT = 24842) so the bookmarkable URL is identical
  4777. everywhere.
  4778. v5.11.68: ``--uninstall`` is identity-sweeping by default
  4779. (removes the daemon script + 0600 CSRF token from EVERY
  4780. ``token-optimizer-*`` identity, and unregisters every runtime's scheduler
  4781. artifact). Pass ``this_install_only=True`` (``--this-install-only``) to
  4782. clean only the resolved identity, for a user intentionally running
  4783. side-by-side installs who only wants this one gone.
  4784. """
  4785. _measure_facade = _compat.facade()
  4786. if (_measure_facade._daemon_snapshot_sandboxed() if uninstall else _measure_facade._scheduler_install_refused()):
  4787. return "noop-sandbox"
  4788. system = _measure_facade._normalized_platform()
  4789. if uninstall:
  4790. if system == "Darwin":
  4791. _measure_facade._uninstall_launchd_daemon(this_install_only=this_install_only, dry_run=dry_run)
  4792. elif system == "Windows":
  4793. _measure_facade._uninstall_task_scheduler_daemon(this_install_only=this_install_only, dry_run=dry_run)
  4794. elif system == "Linux":
  4795. _measure_facade._uninstall_systemd_user_daemon(this_install_only=this_install_only, dry_run=dry_run)
  4796. else:
  4797. print(f"[Token Optimizer] Unsupported platform for daemon uninstall: {system}")
  4798. # Sticky opt-out: persist the user's DELIBERATE intent so the SessionStart
  4799. # ensure-health self-install (`_ensure_dashboard_daemon`) never resurrects
  4800. # a daemon the user explicitly turned off. Mirrors the quality-bar opt-out.
  4801. # Gated on `latch_optout`: only the dedicated `setup-daemon --uninstall`
  4802. # command sets it. `cleanup` (full-uninstall tidy) passes latch_optout=
  4803. # False -- it removes the hooks too, so a genuine uninstall stops the
  4804. # self-heal anyway, and a user who merely tidied while keeping Token
  4805. # Optimizer active must get the dashboard self-healed back automatically,
  4806. # never left silently dead pending a manual `setup-daemon`.
  4807. if not dry_run and latch_optout:
  4808. _measure_facade._set_daemon_disabled(True)
  4809. elif dry_run:
  4810. # A dry-run does NOT write the sticky opt-out, so the SessionStart
  4811. # self-heal would reinstall the daemon. Say so, or the user thinks a
  4812. # preview opted them out for good.
  4813. print(" [Token Optimizer] (dry run: sticky opt-out NOT written -- "
  4814. "the daemon will be reinstalled at next SessionStart. Run "
  4815. "`setup-daemon --uninstall` without --dry-run to opt out for good.)")
  4816. return
  4817. # v5.11.1: resolve the bind-host once, before branching, so every OS
  4818. # installer (and dry-run messaging) shares the same effective host. The
  4819. # daemon runs under launchd/systemd/Task Scheduler with an empty env, so it
  4820. # reads the persisted value instead of the env var. Dry-run resolves the
  4821. # same env > file > default chain but skips the write, staying
  4822. # side-effect-free.
  4823. effective_host = _measure_facade._persist_dashboard_host(persist=not dry_run)
  4824. # Explicit install = explicit opt-in: clear any sticky opt-out so the
  4825. # ensure-health self-heal resumes maintaining this daemon. Symmetric with
  4826. # the quality-bar install path. Skipped on dry-run (side-effect-free).
  4827. if not dry_run:
  4828. _measure_facade._set_daemon_disabled(False)
  4829. # Every platform's uninstall leaves a permanent tombstone the daemon
  4830. # obeys forever; an explicit install is what clears it. Done here, not
  4831. # per installer: only the launchd installer cleared it, so on Windows
  4832. # and Linux `setup-daemon` after `--uninstall` left the daemon dead.
  4833. try:
  4834. if _measure_facade.DAEMON_THRASH_BREADCRUMB.exists():
  4835. _measure_facade.DAEMON_THRASH_BREADCRUMB.unlink()
  4836. except OSError:
  4837. pass
  4838. if system == "Darwin":
  4839. _installed_ok = _measure_facade._install_launchd_daemon(dry_run=dry_run, effective_host=effective_host)
  4840. elif system == "Windows":
  4841. _installed_ok = _measure_facade._install_task_scheduler_daemon(dry_run=dry_run, effective_host=effective_host)
  4842. elif system == "Linux":
  4843. _installed_ok = _measure_facade._install_systemd_user_daemon(dry_run=dry_run, effective_host=effective_host)
  4844. else:
  4845. print(f"[Error] Dashboard daemon not supported on {system}.")
  4846. print(f" Open the dashboard file directly: {_measure_facade.DASHBOARD_PATH.as_uri()}")
  4847. _measure_facade.sys.exit(1)
  4848. # THE ONLY place the sticky install-failed marker is cleared.
  4849. # An explicit `setup-daemon` that actually succeeded is the user telling us
  4850. # the structural problem is fixed. Not on dry-run (side-effect-free), and
  4851. # not on a failed install -- the installers return False under soft_fail and
  4852. # sys.exit(1) otherwise, so neither reaches here with a truthy result.
  4853. if not dry_run and _installed_ok:
  4854. _measure_facade._clear_daemon_install_failed_marker()
  4855. def _retained_paths() -> list[tuple[str, Path]]:
  4856. """User-data paths preserved by design across an uninstall.
  4857. Session snapshots, compaction checkpoints, trend aggregates, and the
  4858. quality-bar cache are NOT removed by the cleanup command. They are
  4859. disclosed in --dry-run so the user knows what stays.
  4860. """
  4861. _measure_facade = _compat.facade()
  4862. paths: list[tuple[str, Path]] = []
  4863. try:
  4864. for ident in _measure_facade._all_plugin_data_dirs():
  4865. paths.append((f"session data ({ident.name})", ident / "data"))
  4866. paths.append((f"checkpoints ({ident.name})", ident / "data" / "checkpoints"))
  4867. paths.append((f"trends ({ident.name})", ident / "data" / "trends"))
  4868. except Exception: # noqa: BLE001
  4869. pass
  4870. # Always include the resolved snapshot dir even under a sandbox override.
  4871. paths.append(("session data (this install)", _measure_facade.SNAPSHOT_DIR))
  4872. # Compaction checkpoints (conversation-context captures) live under the
  4873. # per-session state base, which on a script install is NOT inside
  4874. # SNAPSHOT_DIR (#106 named them as undisclosed). Disclose them too.
  4875. try:
  4876. listed = {p for _, p in paths}
  4877. if _measure_facade.CHECKPOINT_DIR not in listed:
  4878. paths.append(("checkpoints (this install)", _measure_facade.CHECKPOINT_DIR))
  4879. except NameError:
  4880. pass
  4881. return paths
  4882. def _backup_settings_file(dest_dir: Path) -> Path | None:
  4883. """Copy settings.json into ``dest_dir``. Returns the backup path or None."""
  4884. _measure_facade = _compat.facade()
  4885. try:
  4886. if not _measure_facade.SETTINGS_PATH.is_file():
  4887. return None
  4888. dest_dir.mkdir(parents=True, exist_ok=True)
  4889. dest = dest_dir / "settings.json"
  4890. i = 1
  4891. while dest.exists():
  4892. dest = dest_dir / f"settings.json-{i}"
  4893. i += 1
  4894. _measure_facade.shutil.copy2(_measure_facade.SETTINGS_PATH, dest)
  4895. return dest
  4896. except OSError:
  4897. return None
  4898. def _is_our_hook_entry(entry, event="", group=None) -> bool:
  4899. """True when a settings.json hook entry is one Token Optimizer wrote (cleanup).
  4900. Proof only (owned_hooks.cleanup_owner): recorded in the provenance ledger,
  4901. a shipped hooks.json entry for an install root, or byte-identical to a
  4902. command setup_hook / setup_smart_compact / setup_quality_bar generated for
  4903. an install's measure.py. A command that merely names our subcommands or
  4904. scripts proves nothing and stays: a dangling entry the user can delete
  4905. beats silently deleting their automation.
  4906. """
  4907. _measure_facade = _compat.facade()
  4908. import owned_hooks
  4909. return owned_hooks.cleanup_owner(_measure_facade)(event, {"hooks": [entry]} if group is None else group, entry)
  4910. def _remove_our_settings_entries(settings: dict) -> list[str]:
  4911. """Strip Token Optimizer's OWN entries from ``settings`` in place.
  4912. Removes only entries we own:
  4913. - ``statusLine`` when its command references our ``statusline.js`` +
  4914. ``token-optimizer`` (the same matcher _is_quality_bar_installed uses).
  4915. - ``UserPromptSubmit`` hook groups whose command references
  4916. ``quality-cache`` (our cache hook).
  4917. - ``SessionEnd`` hook entries that are ours (reuses
  4918. _remove_token_optimizer_session_end_hooks).
  4919. - ``compactInstructions`` when it is the block setup-smart-compact
  4920. generated (removed) or holds our appended section (only that section
  4921. is removed; the user's own text stays).
  4922. Foreign statusLines, foreign hooks, and every other settings key are left
  4923. byte-identical. Returns a list of human-readable descriptions of what was
  4924. removed. Never raises.
  4925. """
  4926. _measure_facade = _compat.facade()
  4927. removed: list[str] = []
  4928. if not isinstance(settings, dict):
  4929. return removed
  4930. # statusLine
  4931. sl = settings.get("statusLine")
  4932. if isinstance(sl, dict):
  4933. cmd = sl.get("command", "")
  4934. if _measure_facade._statusline_cmd_is_ours(cmd):
  4935. del settings["statusLine"]
  4936. removed.append("statusLine (token-optimizer)")
  4937. # Hook entries we own, across every event.
  4938. #
  4939. # The old loop dropped ANY group whose filtered hook list
  4940. # came out empty -- including a user-authored group that never held a hook
  4941. # of ours (an empty `hooks: []`, or a group of foreign hooks in an event we
  4942. # also use) -- and miscounted each one as "UserPromptSubmit quality-cache
  4943. # hook". A group is now removed ONLY when it actually contained one of our
  4944. # hooks and filtering emptied it; user groups pass through untouched.
  4945. #
  4946. # The sweep covered UserPromptSubmit + SessionEnd only,
  4947. # leaving our smart-compact family (PreCompact / SessionStart / Stop /
  4948. # SessionEnd) dangling in settings.json after cleanup -- commands pointing
  4949. # into a plugin tree that is about to be deleted. Every event is now
  4950. # filtered with the same ownership predicate.
  4951. hooks = settings.get("hooks")
  4952. if isinstance(hooks, dict):
  4953. import owned_hooks
  4954. owns = owned_hooks.cleanup_owner(_measure_facade) # proof only, as _is_our_hook_entry
  4955. for event in list(hooks.keys()):
  4956. groups = hooks.get(event)
  4957. if not isinstance(groups, list):
  4958. continue
  4959. new_groups = []
  4960. n_removed = 0
  4961. for group in groups:
  4962. if not isinstance(group, dict):
  4963. new_groups.append(group)
  4964. continue
  4965. entries = group.get("hooks")
  4966. if not isinstance(entries, list):
  4967. new_groups.append(group)
  4968. continue
  4969. kept = [h for h in entries if not owns(event, group, h)]
  4970. n_ours = len(entries) - len(kept)
  4971. if n_ours == 0:
  4972. # Nothing of ours here -- leave the group exactly as-is,
  4973. # even when it is empty (it is the user's to keep).
  4974. new_groups.append(group)
  4975. continue
  4976. n_removed += n_ours
  4977. if kept:
  4978. group["hooks"] = kept
  4979. new_groups.append(group)
  4980. # else: the group held only our hooks -> drop the whole group.
  4981. if n_removed:
  4982. removed.append(f"{n_removed} {event} hook(s)")
  4983. if new_groups:
  4984. hooks[event] = new_groups
  4985. else:
  4986. del hooks[event]
  4987. # SessionEnd hooks (reuses the proven filter)
  4988. n_session_end = _measure_facade._remove_token_optimizer_session_end_hooks(settings)
  4989. if n_session_end:
  4990. removed.append(f"{n_session_end} SessionEnd hook(s)")
  4991. # compactInstructions written by setup-smart-compact (#106: it outlived
  4992. # cleanup). Only our generated block / appended section is removed.
  4993. if _measure_facade._remove_our_compact_instructions(settings):
  4994. removed.append("compactInstructions (token-optimizer)")
  4995. return removed
  4996. def cleanup(dry_run=None, this_install_only=None):
  4997. """One-command uninstall cleanup for Token Optimizer.
  4998. Orchestrates the three uninstall surfaces:
  4999. 1. Daemon: stop + remove the dashboard daemon across all
  5000. ``token-optimizer-*`` identities (or just this install under
  5001. ``this_install_only``), via ``setup_daemon(uninstall=True, ...)``.
  5002. 2. Host config (``settings.json``): back up first, then remove ONLY our
  5003. own entries (statusLine, quality-cache hook, SessionEnd hooks), leaving
  5004. every other key byte-identical.
  5005. 3. Manifests (``installed_plugins.json`` / ``known_marketplaces.json``):
  5006. back up first, then remove our STALE entries via
  5007. ``install_reconcile.reconcile_uninstall(remove=True)``.
  5008. Session data, checkpoints, and trends are PRESERVED by design and
  5009. disclosed in the report.
  5010. ``--dry-run`` is side-effect-free and doubles as retained-paths
  5011. DISCLOSURE: it prints exactly what WOULD be removed and what stays, so
  5012. the user can review before agreeing to a real cleanup.
  5013. """
  5014. _measure_facade = _compat.facade()
  5015. print("[Token Optimizer] Cleanup", "(dry run)" if dry_run else "")
  5016. print(f" {'=' * 60}")
  5017. if this_install_only:
  5018. print(" Scope: this install only (--this-install-only)")
  5019. else:
  5020. print(" Scope: all token-optimizer-* identities")
  5021. print()
  5022. # 1. Daemon
  5023. print(" [1/3] Dashboard daemon")
  5024. # latch_optout=False: a cleanup is a tidy, not a deliberate "turn my dashboard
  5025. # off forever". This removes the daemon now, but does NOT set the sticky
  5026. # daemon_disabled flag -- so if Token Optimizer stays active, the SessionStart
  5027. # self-heal brings the dashboard back automatically, with no manual command.
  5028. # A genuine full uninstall removes the hooks here too, so the self-heal simply
  5029. # stops running; the sticky flag was never what kept a real uninstall gone.
  5030. _measure_facade.setup_daemon(dry_run=dry_run, uninstall=True, this_install_only=this_install_only, latch_optout=False)
  5031. # The launchd LaunchAgent plist is removed by the macOS uninstaller only.
  5032. # Defensively unlink it here too so a stray plist (a HOME migrated or
  5033. # restored from a Mac, a copied backup) is always cleared, regardless of
  5034. # the platform running cleanup -- it is just a file. Skipped on dry-run so
  5035. # the preview stays side-effect-free.
  5036. if not dry_run:
  5037. _measure_facade._unlink_if_exists(_measure_facade.PLIST_PATH)
  5038. # Same reasoning for the other platforms' per-identity daemon files:
  5039. # each platform's uninstaller sweeps only its own launcher, so a HOME
  5040. # synced or migrated from another OS kept dashboard-launcher.cmd /
  5041. # .schtasks-daemon.xml (Windows) or the Linux launcher after cleanup
  5042. # (#106). They are plain files once the scheduler job is gone.
  5043. if not _measure_facade._daemon_snapshot_sandboxed():
  5044. for _snap in _measure_facade._daemon_identity_snapshot_dirs(this_install_only):
  5045. _gone, _ = _measure_facade._sweep_identity_daemon_files(
  5046. _snap, ("daemon_script", "daemon_token", "daemon_host",
  5047. "windows_launcher", "linux_launcher"))
  5048. try:
  5049. for _xml in _snap.glob("*schtasks-daemon*.xml"):
  5050. if _measure_facade._unlink_if_exists(_xml):
  5051. _gone.append(str(_xml))
  5052. except OSError:
  5053. pass
  5054. for _path in _gone:
  5055. print(f" Deleted: {_path}")
  5056. print()
  5057. # 2. settings.json
  5058. print(" [2/3] Host config (settings.json)")
  5059. settings, _ = _measure_facade._read_settings_json()
  5060. our_entries = _measure_facade._remove_our_settings_entries(settings) # mutate a copy
  5061. if our_entries:
  5062. if dry_run:
  5063. for entry in our_entries:
  5064. print(f" Would remove: {entry}")
  5065. else:
  5066. # Do NOT claim "Removed" before the write lands. The
  5067. # entries are echoed after a confirmed successful write below; a
  5068. # refused read or a denied lease prints a WARNING instead, so the
  5069. # report can never assert a removal that did not happen.
  5070. pass
  5071. if not dry_run:
  5072. backup_root = _measure_facade.CLAUDE_DIR / "_backups" / "token-optimizer"
  5073. stamp = _measure_facade.datetime.now().strftime("%Y%m%d-%H%M%S")
  5074. backup_dest = backup_root / stamp
  5075. backed = _measure_facade._backup_settings_file(backup_dest)
  5076. if backed is None:
  5077. print(f" WARNING: could not back up settings.json; entries left in place.")
  5078. else:
  5079. # Re-read under the backup, re-remove, write atomically.
  5080. #
  5081. # A failed re-read yields {} ("unknown"), and
  5082. # writing that would erase every key the user owns. Refuse to
  5083. # write on a bad read and say so -- their entries stay in
  5084. # place, which is recoverable; an emptied settings.json is not.
  5085. fresh, _, read_ok = _measure_facade._read_settings_json_checked()
  5086. if not read_ok:
  5087. print(
  5088. " WARNING: settings.json became unreadable (malformed "
  5089. "or permission denied) after the backup; refusing to "
  5090. "write.\n"
  5091. f" Entries left in place. Backup: {backed}"
  5092. )
  5093. else:
  5094. _measure_facade._remove_our_settings_entries(fresh)
  5095. # `_remove_our_settings_entries` deletes exactly one
  5096. # TOP-LEVEL key when it is ours: `statusLine`. Everything
  5097. # else it touches is nested inside `hooks`. Declare that one
  5098. # key so the choke-point guard permits this deliberate
  5099. # uninstall removal and still blocks accidental key loss.
  5100. # A denied advisory lease makes the write a
  5101. # silent no-op. Reporting "Removed" for a write that never
  5102. # landed leaves the user with the dangling statusLine this
  5103. # command exists to clear, so surface it instead.
  5104. # user_initiated: cleanup is an explicit user command. Under
  5105. # the hook cohort throttle any Token Optimizer settings
  5106. # write in the previous ~10s (a SessionStart heal, a setup
  5107. # command) made this write lose the lease, and the whole
  5108. # removal was skipped (#106 leftovers).
  5109. if _measure_facade._write_settings_atomic(
  5110. fresh, allow_removing_keys={"statusLine", "compactInstructions"},
  5111. user_initiated=True):
  5112. for entry in our_entries:
  5113. print(f" Removed: {entry}")
  5114. print(f" Backup: {backed}")
  5115. else:
  5116. print(
  5117. " WARNING: settings.json is locked by another "
  5118. "Token Optimizer process; NOTHING was removed.\n"
  5119. " Re-run cleanup in a few seconds (no Claude Code "
  5120. f"session should be starting). Backup: {backed}"
  5121. )
  5122. else:
  5123. print(" No Token Optimizer entries found in settings.json.")
  5124. print()
  5125. # 2b. Subagent prompt-cache TTL: the uninstall path calls the SAME undo
  5126. # as `subagent-cache disable` -- remove the key only when the marker says
  5127. # Token Optimizer set it and the value is still "1h". Dry-run previews.
  5128. print(" [2b/3] Subagent cache (subagentPromptCacheTtl)")
  5129. try:
  5130. if dry_run:
  5131. _sc_marker = _measure_facade._subagent_cache_read_marker()
  5132. if _sc_marker and _sc_marker.get("state") == "set":
  5133. print(" Would remove: subagentPromptCacheTtl (set by Token Optimizer)")
  5134. else:
  5135. print(" Nothing to remove (no marker says Token Optimizer set it)")
  5136. else:
  5137. _sc_undo = _measure_facade.subagent_cache_disable()
  5138. if _sc_undo.get("changed"):
  5139. print(" Removed: subagentPromptCacheTtl (was set by Token Optimizer)")
  5140. elif _sc_undo.get("state") == "platform-gap":
  5141. print(" Skipped: not Claude Code")
  5142. else:
  5143. print(f" Nothing to remove ({_sc_undo.get('state')})")
  5144. except Exception as _sc_exc:
  5145. print(f" WARNING: subagent cache undo failed: {_sc_exc}")
  5146. print()
  5147. # 3. Manifests
  5148. print(" [3/3] Plugin manifests (installed_plugins / known_marketplaces)")
  5149. import install_reconcile
  5150. reconcile = install_reconcile.reconcile_uninstall(
  5151. _measure_facade.CLAUDE_DIR, dry_run=dry_run, remove=True,
  5152. daemon_already_stopped=True,
  5153. )
  5154. for entry in reconcile.get("reported", []):
  5155. stale_tag = " (stale)" if entry.get("stale") else " (active, kept)"
  5156. print(f" {entry['file']}::{entry['key']}{stale_tag}")
  5157. for warning in reconcile.get("warnings", []):
  5158. print(f" {warning}")
  5159. if reconcile.get("backup"):
  5160. print(f" Backup: {reconcile['backup']}")
  5161. for removed in reconcile.get("removed", []):
  5162. print(f" {'Would remove' if dry_run else 'Removed'}: {removed}")
  5163. if not reconcile.get("reported"):
  5164. print(" No Token Optimizer manifest entries found.")
  5165. print()
  5166. # Retained-paths disclosure (always shown, dry-run or not).
  5167. print(" Retained by design (session data, checkpoints, trends):")
  5168. for label, path in _measure_facade._retained_paths():
  5169. exists = path.exists() if hasattr(path, "exists") else False
  5170. tag = " (present)" if exists else " (absent)"
  5171. print(f" {label}: {path}{tag}")
  5172. print()
  5173. if dry_run:
  5174. print(" Dry run: nothing was modified. Re-run without --dry-run to apply.")
  5175. else:
  5176. print(" Cleanup complete. Re-enable later with: setup-quality-bar / setup-daemon")
  5177. print()
  5178. def _daemon_service_installed(system=None):
  5179. """True when the OS service manager has the dashboard daemon REGISTERED.
  5180. This is the "is it installed" half of the ensure-health health check; the
  5181. "is it alive" half is `_verify_daemon_port`. Kept cheap: macOS/Linux are
  5182. stat-only (no subprocess); only Windows shells out (one locale-independent
  5183. `schtasks /Query`, matching the rest of the Windows daemon code which avoids
  5184. parsing localized text output). Never raises.
  5185. Returns False on unsupported platforms so the ensure path no-ops there.
  5186. """
  5187. _measure_facade = _compat.facade()
  5188. system = system or _measure_facade._normalized_platform()
  5189. try:
  5190. if system == "Darwin":
  5191. return _measure_facade.PLIST_PATH.exists()
  5192. if system == "Linux":
  5193. return _measure_facade._systemd_user_unit_path().exists()
  5194. if system == "Windows":
  5195. try:
  5196. rc = _measure_facade.subprocess.run(
  5197. ["schtasks", "/Query", "/TN", _measure_facade.WINDOWS_TASK_NAME],
  5198. capture_output=True, timeout=5, creationflags=_measure_facade._NO_WINDOW,
  5199. ).returncode
  5200. return rc == 0
  5201. except (OSError, _measure_facade.subprocess.TimeoutExpired):
  5202. return False
  5203. except OSError:
  5204. return False
  5205. return False
  5206. def _daemon_ensure_throttle_seconds():
  5207. """Throttle window (seconds) for the COSTLY half of the ensure path
  5208. (install/restart). The cheap presence + port probe runs every SessionStart;
  5209. an actual install attempt is retried at most once per this interval so a
  5210. persistently-failing install (e.g. unwritable LaunchAgents dir) doesn't shell
  5211. out on every prompt. Mirrors the 24h hook-heal throttle. Read lazily (not a
  5212. module-level constant) because `_int_env` is defined further down the file.
  5213. Overridable via TOKEN_OPTIMIZER_DAEMON_ENSURE_THROTTLE for tests.
  5214. """
  5215. _measure_facade = _compat.facade()
  5216. return _measure_facade._int_env("TOKEN_OPTIMIZER_DAEMON_ENSURE_THROTTLE", 24 * 3600)
  5217. def _daemon_pulse_probe_seconds():
  5218. """Cadence (seconds) of the mid-session liveness PROBE. Within this window the
  5219. per-turn pulse is a single config read -- no stat, no socket. Overridable via
  5220. TOKEN_OPTIMIZER_DAEMON_PULSE_PROBE (tests)."""
  5221. _measure_facade = _compat.facade()
  5222. return _measure_facade._int_env("TOKEN_OPTIMIZER_DAEMON_PULSE_PROBE", 60)
  5223. def _daemon_pulse_revive_seconds():
  5224. """Cadence (seconds) of the mid-session REVIVE. A persistently-dead daemon
  5225. (e.g. its port stolen by a foreign process) is retried at most this often so
  5226. the pulse never spawns a revive every minute. Independent of the 24h
  5227. SessionStart ensure throttle. Overridable via TOKEN_OPTIMIZER_DAEMON_PULSE_REVIVE."""
  5228. _measure_facade = _compat.facade()
  5229. return _measure_facade._int_env("TOKEN_OPTIMIZER_DAEMON_PULSE_REVIVE", 300)
  5230. def _ensure_dashboard_daemon(force=None):
  5231. """SessionStart self-heal: INSTALL the dashboard daemon if missing, RESTART
  5232. it if installed-but-dead, NO-OP when healthy. Default-on for every Claude
  5233. user (loopback-only, token-authed, ~0 cost), with a sticky `daemon_disabled`
  5234. opt-out -- mirroring the quality-bar self-install, NOT the marker-gated
  5235. keep-warm scheduler (that gate exists because keep-warm spends tokens; this
  5236. daemon does not).
  5237. Cheap-gate-first so the healthy hot path costs only a config read + a stat +
  5238. a fast (sleepless) port probe -- no install lock, no install subprocess,
  5239. well inside the ensure-health budget. The one costly branch (install/restart)
  5240. is throttled to once per `_daemon_ensure_throttle_seconds()` via a config
  5241. timestamp; `force=True` bypasses the throttle (tests / explicit repair).
  5242. Concurrency: this does NOT take `_daemon_install_lock` itself -- the platform
  5243. installers self-serialise on that lock (soft_fail=True), so two parallel
  5244. sessions can't double-install; the loser soft-skips and we resolve the final
  5245. status from on-disk reality. Restart is lock-free (kickstart/restart is
  5246. idempotent), matching the auto-update block.
  5247. Security: a silent auto-install always binds LOOPBACK (127.0.0.1) regardless
  5248. of any persisted dashboard host -- a default-on install must never surface a
  5249. network-exposed daemon. An explicit `setup-daemon` still honours a chosen host.
  5250. Distinct from the daemon AUTO-UPDATE block in run_ensure_health: that block
  5251. refreshes a STALE script for an already-installed daemon; this function
  5252. handles the ABSENT / dead-service case. Run this AFTER the update block so
  5253. update owns has-daemon and ensure owns no-daemon.
  5254. Returns one of: 'noop-foreign', 'noop-disabled', 'noop-install-failed',
  5255. 'noop-unsupported', 'noop-healthy', 'noop-throttled', 'installed',
  5256. 'install-failed', 'restarted', 'restart-stale', 'restart-failed'.
  5257. ('restart-stale' propagates up from _restart_dashboard_daemon's
  5258. landing-verification.) Never raises.
  5259. """
  5260. # An explicit snapshot override is an isolation boundary, including when a
  5261. # detached daemon-revive child calls this with force=True. Frozen flag
  5262. # only up here: this is the SessionStart hot path, and the full guard's
  5263. # job check runs `schtasks /Query` on Windows. It runs below, once the
  5264. # cheap gates and the health probe say a repair is actually due.
  5265. _measure_facade = _compat.facade()
  5266. if _measure_facade._daemon_snapshot_sandboxed():
  5267. return "noop-sandbox"
  5268. # Claude Code cloud VM: no browser to open the URL in, and no systemd user
  5269. # bus to install a unit on (the install path fails in containers).
  5270. if _measure_facade._remote_lean_profile():
  5271. return "noop-remote"
  5272. # Cheapest gates first -- all pure/stat, no subprocess.
  5273. if _measure_facade._is_foreign_runtime() or _measure_facade.detect_runtime() != "claude":
  5274. return "noop-foreign"
  5275. if _measure_facade._read_config_flag("daemon_disabled", False):
  5276. return "noop-disabled"
  5277. # A prior install/self-heal failed for a DEFINITIVE reason.
  5278. # Refuse. BEFORE the `force` check on purpose -- `daemon-revive`
  5279. # (force=True) is the per-turn retry path this marker exists to stop, and
  5280. # on Windows each retry costs the user a cmd-window flash.
  5281. # A live, identity-verified daemon DISPROVES the
  5282. # "structurally broken" record (the classic case: an install-lock race
  5283. # where the losing session armed the marker while the winner installed
  5284. # fine). One cheap sleepless socket probe -- no subprocess, cannot flash --
  5285. # buys the un-wedge; otherwise the refusal stands until `setup-daemon`.
  5286. if _measure_facade._daemon_install_failed_marker_present():
  5287. if _measure_facade._verify_daemon_port(timeout_seconds=1, retries=1, retry_sleep=0):
  5288. _measure_facade._clear_daemon_install_failed_marker()
  5289. return "noop-healthy"
  5290. return "noop-install-failed"
  5291. system = _measure_facade._normalized_platform()
  5292. if system not in ("Darwin", "Linux", "Windows"):
  5293. return "noop-unsupported"
  5294. installed = _measure_facade._daemon_service_installed(system)
  5295. # Fast, sleepless probe: this runs on the SessionStart budget, so a single
  5296. # quick attempt with NO inter-retry sleep is all we can afford.
  5297. alive = _measure_facade._verify_daemon_port(timeout_seconds=1, retries=1, retry_sleep=0)
  5298. if installed and alive:
  5299. return "noop-healthy"
  5300. # A repair is due. Refuse it (before stamping the throttle) when the
  5301. # account's job belongs to another install: writing ours would re-point it.
  5302. if _measure_facade._scheduler_install_refused():
  5303. return "noop-sandbox"
  5304. # We need the costly path (install or restart). Throttle the ATTEMPT so a
  5305. # persistently-failing install doesn't shell out every prompt. The cheap
  5306. # probe above still runs every time, so a daemon that comes back healthy is
  5307. # detected instantly; only the repair attempt is rate-limited.
  5308. if not force:
  5309. last = _measure_facade._read_config_flag("last_daemon_ensure_attempt", 0)
  5310. try:
  5311. if _measure_facade.time.time() - float(last or 0) < _measure_facade._daemon_ensure_throttle_seconds():
  5312. return "noop-throttled"
  5313. except (TypeError, ValueError):
  5314. pass
  5315. # Stamp BEFORE the attempt so a failure (or a hard exit upstream) can't
  5316. # loop the costly path next session.
  5317. try:
  5318. _measure_facade._write_config_flag("last_daemon_ensure_attempt", int(_measure_facade.time.time()))
  5319. except Exception:
  5320. pass
  5321. if installed:
  5322. # Installed but dead -> restart (lock-free, idempotent).
  5323. # 'restart-failed' is NOT
  5324. # marker-worthy. It is reached through a blanket `except Exception`
  5325. # whose common inhabitants are TRANSIENT -- a 5s launchctl kickstart
  5326. # TimeoutExpired on a busy Mac, a slow systemd, an AV-locked schtasks.
  5327. # An INSTALLED daemon proves the structure works; the 24h ensure
  5328. # throttle already bounds retry cost. A verified restart clears any
  5329. # stale marker a concurrent session may have raced in.
  5330. _status = _measure_facade._restart_dashboard_daemon(system)
  5331. if _status == "restarted":
  5332. _measure_facade._clear_daemon_install_failed_marker()
  5333. return _status
  5334. # Not installed -> run the platform installer. Each installer self-locks with
  5335. # soft_fail so a concurrent session can't deadlock or double-install. Force
  5336. # loopback for this silent install. `except (Exception, SystemExit)` is
  5337. # belt-and-suspenders: installers return False under soft_fail, but a future
  5338. # edit that reintroduces sys.exit must NEVER kill the SessionStart hook.
  5339. try:
  5340. if not _measure_facade._ensure_dashboard_file(to_stderr=True):
  5341. # Transient class (disk full once, a regen hiccup): the 24h
  5342. # throttle bounds the retry. Never marker-worthy.
  5343. return "install-failed"
  5344. _measure_facade._get_or_create_daemon_token()
  5345. if system == "Darwin":
  5346. ok = _measure_facade._install_launchd_daemon(soft_fail=True, effective_host="127.0.0.1")
  5347. elif system == "Linux":
  5348. ok = _measure_facade._install_systemd_user_daemon(soft_fail=True, effective_host="127.0.0.1")
  5349. else: # Windows
  5350. ok = _measure_facade._install_task_scheduler_daemon(soft_fail=True, effective_host="127.0.0.1")
  5351. if ok is False:
  5352. # Real failure OR a concurrent installer won the lock and soft-skipped
  5353. # us. Resolve from reality so a race doesn't log a false failure.
  5354. # The loser's instantaneous recheck races the
  5355. # winner's multi-second install window -- an ambiguous result must
  5356. # NOT arm the sticky marker (the installer itself arms it at the
  5357. # exact sites that can classify a failure as permanent).
  5358. if _measure_facade._daemon_service_installed(system):
  5359. _measure_facade._clear_daemon_install_failed_marker()
  5360. return "installed"
  5361. return "install-failed"
  5362. # Verified success supersedes any failure record.
  5363. _measure_facade._clear_daemon_install_failed_marker()
  5364. return "installed"
  5365. except (Exception, SystemExit) as _e:
  5366. try:
  5367. print(
  5368. "[Token Optimizer] dashboard daemon self-install failed: "
  5369. f"{str(_e).splitlines()[0] if str(_e) else type(_e).__name__}",
  5370. file=_measure_facade.sys.stderr,
  5371. )
  5372. except Exception:
  5373. pass
  5374. # A raised exception is UNCLASSIFIED -- OSError from a full disk, an
  5375. # AV-locked file, a gui-domain bootstrap over SSH. All recoverable by
  5376. # tomorrow; the throttle bounds them. Definitive classes are armed
  5377. # inside the installer where they can actually be identified.
  5378. return "install-failed"
  5379. def _daemon_served_version(port=None, total_timeout=None):
  5380. """Return the version string OUR live daemon reports on /api/health, or None
  5381. if it can't be determined (no daemon, no HTTP, a foreign listener, an old
  5382. daemon without the version field, or timeout). Bounded total wait; never
  5383. raises.
  5384. Used to verify a restart actually LANDED rather than trusting the service
  5385. manager's exit code -- an orphan we couldn't reap, or a job that failed to
  5386. bind, leaves the OLD version serving even though kickstart returned 0. A
  5387. None result means "unknown" and callers MUST treat it as a safe degrade
  5388. (assume restarted), never as proof of staleness: a pre-fix orphan has no
  5389. version field, and the reaper already kills those regardless.
  5390. IDENTITY: only trusts the response when `server == "token-optimizer-daemon"`,
  5391. mirroring _verify_daemon_port's magic-string check, so a foreign listener on
  5392. the port that happens to return JSON with a "version" key is ignored.
  5393. BUDGET: default wait is short (TOKEN_OPTIMIZER_DAEMON_VERIFY_TIMEOUT, 2.0s)
  5394. because this runs inside the SessionStart hook. A slow-cold-start install
  5395. (large trends DB) can exceed it, in which case the probe returns None and
  5396. the caller safe-degrades -- the reap already did the real work; the probe is
  5397. belt-and-suspenders. Operators who want a stronger landing check can raise
  5398. the env knob.
  5399. """
  5400. _measure_facade = _compat.facade()
  5401. if port is None:
  5402. port = _measure_facade.DAEMON_PORT
  5403. if total_timeout is None:
  5404. total_timeout = _measure_facade._float_env("TOKEN_OPTIMIZER_DAEMON_VERIFY_TIMEOUT", 2.0)
  5405. import urllib.request
  5406. deadline = _measure_facade.time.time() + total_timeout
  5407. while _measure_facade.time.time() < deadline:
  5408. try:
  5409. with urllib.request.urlopen(
  5410. f"http://127.0.0.1:{port}/api/health", timeout=1
  5411. ) as resp:
  5412. body = resp.read(2048).decode("utf-8", "replace")
  5413. data = _measure_facade.json.loads(body)
  5414. if data.get("server") != "token-optimizer-daemon":
  5415. # A foreign listener on our port -> not ours; unknown, degrade.
  5416. return None
  5417. ver = data.get("version")
  5418. if ver:
  5419. return str(ver)
  5420. # Reachable OUR daemon but no version field -> old (pre-fix) code.
  5421. return None
  5422. except Exception:
  5423. _measure_facade.time.sleep(0.2)
  5424. continue
  5425. return None
  5426. def _daemon_midsession_pulse():
  5427. """Per-turn liveness pulse (UserPromptSubmit) that revives a dashboard daemon
  5428. which died MID-session.
  5429. `_ensure_dashboard_daemon` self-heals only at SessionStart, so a daemon that
  5430. dies during a long session stays dead until the next session begins. This runs
  5431. once per user turn with its OWN short cadence, reuses the same tombstone-safe /
  5432. opt-out-respecting primitive, and NEVER blocks the turn:
  5433. * Probe throttle (60s): within the window the whole pulse is one config read.
  5434. No stat, no socket -- the per-turn hot-path cost is ~a small JSON read.
  5435. * Once per window: a sleepless 1s port probe. Alive -> done.
  5436. * Dead AND revive throttle (300s) open: spawn a DETACHED `daemon-revive`
  5437. subprocess (fire-and-forget) so kickstart + landing-verification run OFF
  5438. the hot path. The next probe (<=60s later) confirms it came back.
  5439. FIX-SPEC-DAEMON req 2: the 300s revive throttle is SKIPPED when the
  5440. scheduler registration (plist/unit/task) is ABSENT -- a missing plist is
  5441. the "something removed it" state, not the transient "port not up yet"
  5442. case the throttle bounds, so a swept daemon self-revives promptly. The
  5443. throttle still applies to the ordinary dead-but-INSTALLED case.
  5444. Concurrency: the throttle timestamps live in the shared config.json (machine-
  5445. wide, not per-session), so cadence is bounded across parallel sessions. A rare
  5446. two-session race just spawns two detached revives, which are idempotent
  5447. (`kickstart -k`) and self-serialise on `_daemon_install_lock` inside the
  5448. installer. Returns a short status string. Never raises, never blocks.
  5449. """
  5450. _measure_facade = _compat.facade()
  5451. try:
  5452. if _measure_facade._remote_lean_profile():
  5453. return "noop-remote"
  5454. # Frozen sandbox flag only: this pulse runs on every UserPromptSubmit.
  5455. # The "job owned by another install" check reads the scheduler (on
  5456. # Windows a `schtasks /Query`, up to 10 s when Task Scheduler is
  5457. # wedged), so it lives in the writers that would actually re-point a
  5458. # job (_install_*, _restart_dashboard_daemon, the heals, setup_daemon),
  5459. # never on a per-prompt path.
  5460. if _measure_facade._daemon_snapshot_sandboxed():
  5461. return "noop-sandbox"
  5462. # SAFETY gates run EVERY turn, BEFORE the probe throttle: a
  5463. # disabled/uninstalled/thrashing daemon must NEVER be revived, not even on
  5464. # the 59/60 throttled turns. All below are cheap (a stat + a small config
  5465. # read + platform.system(); no scheduler query); detect_runtime is paid
  5466. # by quality-cache regardless.
  5467. if _measure_facade._is_foreign_runtime() or _measure_facade.detect_runtime() != "claude":
  5468. return "noop-foreign"
  5469. # Filesystem tombstone is AUTHORITATIVE and independent of config.json: a
  5470. # corrupt config would make `daemon_disabled` read False and defeat the
  5471. # opt-out (CORR-1). The breadcrumb is present after uninstall (size 0) or
  5472. # during a thrash back-off (>0); either way, do NOT revive.
  5473. try:
  5474. if _measure_facade.os.path.exists(str(_measure_facade.DAEMON_THRASH_BREADCRUMB)):
  5475. return "noop-tombstoned"
  5476. except OSError:
  5477. pass
  5478. if _measure_facade._read_config_flag("daemon_disabled", False):
  5479. return "noop-disabled"
  5480. # Sticky install-failed marker. Checked here, alongside the
  5481. # tombstone and BEFORE the probe throttle, so a permanently-broken
  5482. # install stops costing a revive spawn (and on Windows a cmd-window
  5483. # flash) every ~5min for the rest of time. Cleared only by an explicit
  5484. # successful `setup-daemon`.
  5485. if _measure_facade._daemon_install_failed_marker_present():
  5486. return "noop-install-failed"
  5487. if _measure_facade._normalized_platform() not in ("Darwin", "Linux", "Windows"):
  5488. return "noop-unsupported"
  5489. # Probe throttle: within the window the remaining cost is one config read.
  5490. now = _measure_facade.time.time()
  5491. last_probe = _measure_facade._read_config_flag("last_daemon_midsession_probe", 0)
  5492. try:
  5493. if now - float(last_probe or 0) < _measure_facade._daemon_pulse_probe_seconds():
  5494. return "pulse-throttled"
  5495. except (TypeError, ValueError):
  5496. pass
  5497. # Stamp BEFORE the socket work so a slow/hung probe can't let the next turn
  5498. # through the throttle and pile up socket checks.
  5499. _measure_facade._write_config_flag("last_daemon_midsession_probe", int(now))
  5500. # Cheap sleepless liveness probe (identity-checked inside _verify_daemon_port).
  5501. if _measure_facade._verify_daemon_port(timeout_seconds=1, retries=1, retry_sleep=0):
  5502. return "noop-healthy"
  5503. # Dead. A missing scheduler registration (launchd plist / systemd unit /
  5504. # scheduled task absent) is a clear "something removed it" state -- distinct
  5505. # from the transient "port not up yet" case the 300s revive throttle is meant
  5506. # to bound (FIX-SPEC-DAEMON req 2). Bypass the revive throttle for the
  5507. # plist-absent case so a swept daemon self-revives promptly mid-session; keep
  5508. # the throttle for the ordinary dead-but-installed case. Fail-open: a probe
  5509. # error degrades to "installed" so the conservative throttle still applies
  5510. # (never bypass on uncertainty, never raise into the hot path).
  5511. try:
  5512. _service_installed = _measure_facade._daemon_service_installed(_measure_facade._normalized_platform())
  5513. except Exception: # noqa: BLE001 -- protection must never raise
  5514. _service_installed = True
  5515. if _service_installed:
  5516. last_revive = _measure_facade._read_config_flag("last_daemon_midsession_revive", 0)
  5517. try:
  5518. if now - float(last_revive or 0) < _measure_facade._daemon_pulse_revive_seconds():
  5519. return "revive-throttled"
  5520. except (TypeError, ValueError):
  5521. pass
  5522. _measure_facade._write_config_flag("last_daemon_midsession_revive", int(now))
  5523. # Detached fire-and-forget revive (or first-time install) -- the user's turn
  5524. # must NEVER block on kickstart + landing verification. POSIX detaches via
  5525. # start_new_session; Windows ignores that, so it needs creationflags
  5526. # (DETACHED_PROCESS + CREATE_NEW_PROCESS_GROUP + CREATE_BREAKAWAY_FROM_JOB)
  5527. # or the child dies with the hook's job object (CXP-1). The child runs
  5528. # `daemon-revive`, which calls _ensure_dashboard_daemon(force=True).
  5529. # Routed through spawn_detached (single source of truth) so the
  5530. # CREATE_BREAKAWAY_FROM_JOB retry path is included at the exact CXP-1
  5531. # site it was built for.
  5532. # pythonw.exe on Windows (GUI subsystem) in addition to the
  5533. # detach flags -- the revive child is stdio-DEVNULL fire-and-forget.
  5534. _proc = _measure_facade.spawn_detached(
  5535. [_measure_facade._detached_python_exe(), str(_measure_facade.MEASURE_PY_PATH), "daemon-revive"],
  5536. stdout=_measure_facade.subprocess.DEVNULL, stderr=_measure_facade.subprocess.DEVNULL,
  5537. stdin=_measure_facade.subprocess.DEVNULL)
  5538. if _proc is None:
  5539. _measure_facade._log_spawn_failure("daemon-revive spawn failed")
  5540. # A failed spawn is the
  5541. # LEAST structural failure in the set -- fork EAGAIN under load,
  5542. # AV transiently locking the exe, a corrupt-but-replaceable
  5543. # pythonw twin. The 300s revive throttle already bounds the retry
  5544. # cost, and a failed CreateProcess never painted a window anyway.
  5545. # Never arm the sticky marker from here.
  5546. return "revive-spawn-failed"
  5547. return "revive-spawned"
  5548. except Exception:
  5549. return "pulse-error"
  5550. def _windows_action_is_console_flasher(command):
  5551. """True when a Scheduled Task ``<Command>`` is OUR console-subsystem
  5552. action shape (and therefore safe to rewrite).
  5553. Conservative on BOTH sides on purpose. An unrecognised command (a venv
  5554. shim, a user-edited action, a wrapper we do not ship) returns False: we only
  5555. rewrite actions we can positively identify as ours-and-broken, because the
  5556. repair re-registers the whole task. "Positively
  5557. identify" now means it -- a ``.cmd``/``.bat`` matches ONLY when its
  5558. basename is the one launcher we actually generate
  5559. (``WINDOWS_LAUNCHER_NAME``), not any batch file that happens to sit in our
  5560. task slot. And ``pythonw.exe``/``pyw.exe`` return False so the heal is
  5561. idempotent by construction -- once migrated, the same check reports "not
  5562. broken" and nothing runs.
  5563. The value arrives from ``schtasks /Query /XML``, so
  5564. entities are unescaped first (``&quot;C:\\...\\dashboard-launcher.cmd&quot;``
  5565. must classify the same as its unquoted form) -- consistent with
  5566. ``_windows_action_is_dead_path``.
  5567. """
  5568. _measure_facade = _compat.facade()
  5569. if not command:
  5570. return False
  5571. try:
  5572. from xml.sax.saxutils import unescape as _xml_unescape
  5573. # Default unescape covers only &amp;/&lt;/&gt; -- the quote entities
  5574. # schtasks emits for a quoted <Command> need the extra dict.
  5575. cleaned = _xml_unescape(
  5576. str(command), {"&quot;": '"', "&apos;": "'"}
  5577. ).strip().strip('"').strip()
  5578. except Exception:
  5579. cleaned = str(command).strip().strip('"').strip()
  5580. name = cleaned.replace("/", "\\").rsplit("\\", 1)[-1].strip().strip('"').lower()
  5581. if name.endswith(".cmd") or name.endswith(".bat"):
  5582. return name == _measure_facade.WINDOWS_LAUNCHER_NAME.lower()
  5583. return name in _measure_facade._WINDOWS_CONSOLE_ACTION_EXES
  5584. def _windows_task_action_info(task_name=None):
  5585. """Return ``(command, enabled)`` for the registered task, or ``(None, None)``
  5586. if the task does not exist / cannot be read.
  5587. ``command`` is the task's ``<Command>``. A None command means "no task" and
  5588. every caller MUST treat it as "do nothing" -- never as "install one".
  5589. Healing an absent task would resurrect a daemon the user declined or
  5590. uninstalled.
  5591. ``enabled`` is the SETTINGS-level ``<Enabled>`` from the same XML (the
  5592. trigger-level ``<Enabled>`` elements live outside ``<Settings>`` and are
  5593. ignored): False when the user disabled the task (Task Scheduler UI
  5594. right-click Disable, or ``schtasks /Change /DISABLE``), True when
  5595. explicitly enabled, None when the XML carries no Settings-level flag.
  5596. A disabled task must be treated exactly like a
  5597. tombstone -- pre-fix that opt-out was durable, and the heal must not
  5598. re-register it ``<Enabled>true</Enabled>``.
  5599. ``schtasks /Query /XML`` emits UTF-16, which text-mode decoding turns into
  5600. ASCII interleaved with NULs; stripping NULs recovers the payload under both
  5601. that and a plain UTF-8 console.
  5602. """
  5603. _measure_facade = _compat.facade()
  5604. try:
  5605. query = _measure_facade.subprocess.run(
  5606. ["schtasks", "/Query", "/TN", task_name or _measure_facade.WINDOWS_TASK_NAME,
  5607. "/XML", "ONE"],
  5608. capture_output=True, text=True, errors="replace", timeout=10,
  5609. creationflags=_measure_facade._NO_WINDOW,
  5610. )
  5611. except (OSError, _measure_facade.subprocess.SubprocessError):
  5612. return None, None
  5613. if query.returncode != 0:
  5614. return None, None
  5615. raw = (query.stdout or "").replace("\x00", "")
  5616. match = _measure_facade.re.search(r"<Command>(.*?)</Command>", raw, _measure_facade.re.S | _measure_facade.re.I)
  5617. if not match:
  5618. return None, None
  5619. command = match.group(1).strip()
  5620. enabled = None
  5621. settings_match = _measure_facade.re.search(r"<Settings>(.*?)</Settings>", raw, _measure_facade.re.S | _measure_facade.re.I)
  5622. if settings_match:
  5623. en = _measure_facade.re.search(r"<Enabled>\s*(true|false)\s*</Enabled>",
  5624. settings_match.group(1), _measure_facade.re.I)
  5625. if en:
  5626. enabled = en.group(1).lower() == "true"
  5627. return command, enabled
  5628. def _windows_task_action_command(task_name=None):
  5629. """Back-compat shim over ``_windows_task_action_info`` (command only)."""
  5630. _measure_facade = _compat.facade()
  5631. return _measure_facade._windows_task_action_info(task_name)[0]
  5632. def _windows_action_path_exists(path):
  5633. """``os.path.exists`` behind a seam so tests can stub filesystem reality
  5634. for Windows-shaped paths that cannot exist on the test host. Never raises."""
  5635. _measure_facade = _compat.facade()
  5636. try:
  5637. return _measure_facade.os.path.exists(path)
  5638. except (OSError, ValueError):
  5639. return False
  5640. def _windows_action_is_dead_path(command):
  5641. """True when the task ``<Command>`` is an absolute Windows path that no
  5642. longer exists on disk.
  5643. The preferred task action bakes the ABSOLUTE path of
  5644. the pythonw twin that ran the install. Versioned python.org installs and
  5645. deletable venvs make that path go dead on upgrade/uninstall -- and a dead
  5646. pythonw path is invisible to ``_windows_action_is_console_flasher`` (it
  5647. classifies the NAME, not liveness), so without this check no heal layer
  5648. ever repairs it and the daemon is permanently dead behind green
  5649. diagnostics.
  5650. Only absolute (drive-letter or UNC) paths are judged; a bare PATH-resolved
  5651. name ("pythonw.exe") cannot be classified dead from here. The XML value is
  5652. unescaped and unquoted first so ``&amp;`` / quoted paths round-trip.
  5653. Never raises.
  5654. """
  5655. _measure_facade = _compat.facade()
  5656. if not command:
  5657. return False
  5658. try:
  5659. from xml.sax.saxutils import unescape as _xml_unescape
  5660. cmd = _xml_unescape(
  5661. str(command), {"&quot;": '"', "&apos;": "'"}
  5662. ).strip().strip('"').strip()
  5663. if not cmd:
  5664. return False
  5665. if not (_measure_facade.re.match(r"^[A-Za-z]:[\\/]", cmd) or cmd.startswith("\\\\")):
  5666. return False
  5667. return not _measure_facade._windows_action_path_exists(cmd)
  5668. except Exception:
  5669. return False
  5670. def _daemon_resurrection_blocked():
  5671. """THE shared refusal gate for every daemon heal/revive path.
  5672. The individual heal entry points each grew their own
  5673. subset of these checks, and the ones that forgot (the second
  5674. ``_heal_windows_task_action`` call site inside ``_restart_dashboard_daemon``,
  5675. the ensure-health auto-update block) became resurrection doors for daemons
  5676. the user turned off. One helper, checked by every path that could
  5677. (re)start a daemon, so no future call site can bypass a gate by accident.
  5678. Returns the blocking reason (``"sandbox"`` | ``"tombstoned"`` |
  5679. ``"disabled"`` | ``"install-failed"``) or None when the action may proceed. The tombstone
  5680. stat fails open (matching ``_daemon_install_failed_marker_present``: an
  5681. unreadable state dir is not evidence of intent). Never raises.
  5682. Reads only the frozen sandbox flag, never the scheduler: this gate sits on
  5683. hot hook paths. A job owned by another install is refused by the writers
  5684. themselves (_scheduler_install_refused), and doctor reports it on its own.
  5685. """
  5686. _measure_facade = _compat.facade()
  5687. if _measure_facade._daemon_snapshot_sandboxed():
  5688. return "sandbox"
  5689. try:
  5690. if _measure_facade.os.path.exists(str(_measure_facade.DAEMON_THRASH_BREADCRUMB)):
  5691. return "tombstoned"
  5692. except OSError:
  5693. pass
  5694. if _measure_facade._read_config_flag("daemon_disabled", False):
  5695. return "disabled"
  5696. if _measure_facade._daemon_install_failed_marker_present():
  5697. return "install-failed"
  5698. return None
  5699. def _heal_windows_task_action(stop_first=None, throttle=None):
  5700. """One-shot migration: lift an already-registered Scheduled Task
  5701. off a console action (the .cmd launcher, or a bare py.exe/python.exe) onto
  5702. the GUI-subsystem pythonw.exe action.
  5703. Without this, only NEW installs get the fix. Every existing Windows user
  5704. keeps the console action forever, because the auto-update path refreshes
  5705. ``dashboard-server.py`` and then merely ``/End`` + ``/Run``s the task that is
  5706. already registered -- it never re-writes the task definition.
  5707. Deliberately narrow, because re-registering a scheduled task is not free:
  5708. * Windows only, and only when a usable pythonw twin actually exists --
  5709. there is nothing better to migrate TO otherwise.
  5710. * Only when the task EXISTS and its action is positively identified as a
  5711. console flasher. An absent task is left absent.
  5712. * Re-registers through ``_install_task_scheduler_daemon``, the same code
  5713. path a fresh install uses, so heal and install can never drift.
  5714. * Preserves the user's persisted bind host (``persist=False`` reads the
  5715. env > file > default chain WITHOUT writing), so a migration cannot
  5716. silently move a deliberately network-exposed daemon back to loopback.
  5717. ``stop_first``: ``_install_task_scheduler_daemon`` refuses to install while
  5718. something holds DAEMON_PORT, and a running daemon is exactly that. Callers
  5719. that have already stopped the task (``_restart_dashboard_daemon``, between
  5720. its /End and /Run) pass False. The ensure-health entry point passes True.
  5721. On failure the restoring ``/Run`` fires from a ``finally`` (an
  5722. installer that RAISES must not strand the daemon) -- but ONLY when the
  5723. still-registered action is neither a console flasher nor a dead path
  5724. (/Run-ing a .cmd action IS one extra console flash per session,
  5725. the exact console flash the fix set out to remove; the LogonTrigger revives the daemon
  5726. at next logon instead).
  5727. ``throttle``: the ensure-health path passes True so a persistently-failing
  5728. heal attempts at most once per ensure-throttle window (24h) instead of
  5729. /End-ing the live daemon + sleeping 2s on every SessionStart. The
  5730. restart-path caller passes False -- it is already bounded by its own
  5731. throttles and runs with the port freed.
  5732. Returns True only when the re-registration succeeded. Never raises.
  5733. """
  5734. # Per-account scheduler job: never touched from a redirected HOME
  5735. # (a test sandbox or scratch run would re-point the user's live job).
  5736. # Frozen flag here; the job check (a `schtasks /Query`) waits until the
  5737. # cheap gates below say a heal is actually due.
  5738. _measure_facade = _compat.facade()
  5739. if _measure_facade._daemon_snapshot_sandboxed():
  5740. return False
  5741. try:
  5742. if _measure_facade.os.name != "nt":
  5743. return False
  5744. # UNIFIED GATE: the refusal gates live HERE,
  5745. # inside the primitive, so BOTH call sites (ensure-health heal and
  5746. # _restart_dashboard_daemon's /End->/Run window) and any future one
  5747. # are gated by construction.
  5748. if _measure_facade._daemon_resurrection_blocked() is not None:
  5749. return False
  5750. command, task_enabled = _measure_facade._windows_task_action_info()
  5751. if command is None:
  5752. return False # no task -> nothing to heal, and nothing to create
  5753. if task_enabled is False:
  5754. # The user DISABLED our task -- the natural Windows
  5755. # "off without uninstalling". Same posture as the tombstone:
  5756. # leave it alone; never re-register it <Enabled>true</Enabled>.
  5757. return False
  5758. flasher = _measure_facade._windows_action_is_console_flasher(command)
  5759. dead = _measure_facade._windows_action_is_dead_path(command)
  5760. if not flasher and not dead:
  5761. return False # already hidden and alive; idempotent no-op
  5762. if flasher and not dead and not _measure_facade._windows_gui_python():
  5763. return False # nothing better to migrate TO (a DEAD action is
  5764. # always worth re-registering: the .cmd fallback beats a daemon
  5765. # that can never start again)
  5766. # A heal is due. Refuse it when the registered task belongs to
  5767. # another install (re-registering would re-point it at this one).
  5768. if _measure_facade._scheduler_install_refused():
  5769. return False
  5770. if throttle:
  5771. # A heal that keeps failing (policy-blocked /Create,
  5772. # foreign process on the port) must not bounce the daemon every
  5773. # SessionStart. Stamp BEFORE the attempt, mirroring the ensure
  5774. # throttle; a SUCCESSFUL heal makes the next classify a no-op
  5775. # above, so the stamp never delays healthy behaviour.
  5776. try:
  5777. last = _measure_facade._read_config_flag("last_daemon_flash_heal_attempt", 0)
  5778. if (_measure_facade.time.time() - float(last or 0)
  5779. < _measure_facade._daemon_ensure_throttle_seconds()):
  5780. return False
  5781. except (TypeError, ValueError):
  5782. pass
  5783. try:
  5784. _measure_facade._write_config_flag(
  5785. "last_daemon_flash_heal_attempt", int(_measure_facade.time.time()))
  5786. except Exception:
  5787. pass
  5788. if stop_first:
  5789. try:
  5790. _measure_facade.subprocess.run(
  5791. ["schtasks", "/End", "/TN", _measure_facade.WINDOWS_TASK_NAME],
  5792. capture_output=True, timeout=5, creationflags=_measure_facade._NO_WINDOW,
  5793. )
  5794. _measure_facade.time.sleep(2) # let the daemon release DAEMON_PORT
  5795. except (OSError, _measure_facade.subprocess.SubprocessError):
  5796. pass
  5797. ok = False
  5798. try:
  5799. host = _measure_facade._persist_dashboard_host(persist=False)
  5800. ok = bool(_measure_facade._install_task_scheduler_daemon(
  5801. soft_fail=True, effective_host=host))
  5802. finally:
  5803. if not ok and stop_first:
  5804. # We stopped a working daemon and failed to re-register it
  5805. # (returned False OR raised -- hence the finally).
  5806. # Put it back ONLY if firing it cannot flash: re-read the
  5807. # action and refuse to /Run a still-flasher or dead command.
  5808. try:
  5809. _current, _en = _measure_facade._windows_task_action_info()
  5810. if (_current is not None
  5811. and not _measure_facade._windows_action_is_console_flasher(_current)
  5812. and not _measure_facade._windows_action_is_dead_path(_current)):
  5813. _measure_facade.subprocess.run(
  5814. ["schtasks", "/Run", "/TN", _measure_facade.WINDOWS_TASK_NAME],
  5815. capture_output=True, timeout=5,
  5816. creationflags=_measure_facade._NO_WINDOW,
  5817. )
  5818. except (OSError, _measure_facade.subprocess.SubprocessError):
  5819. pass
  5820. return ok
  5821. except Exception:
  5822. return False
  5823. def _legacy_daemon_dir():
  5824. """Pre-plugin-data-migration daemon dir (the auto-update block writes the
  5825. refreshed ``dashboard-server.py`` here too). Behind a helper so tests can
  5826. pin it into a sandbox -- RUNTIME_DIR is NOT covered by the
  5827. TOKEN_OPTIMIZER_SNAPSHOT_DIR override."""
  5828. _measure_facade = _compat.facade()
  5829. return _measure_facade.RUNTIME_DIR / "_backups" / "token-optimizer"
  5830. def _heal_one_launcher_shim(launcher, daemon_script):
  5831. """Rewrite ONE existing launcher shim to today's generated content.
  5832. Shared mechanics for both shim locations: byte-compare, link-safety
  5833. refusal, atomic mkstemp+os.replace. Returns True when the file was
  5834. rewritten. Never raises past its caller's guard.
  5835. """
  5836. _measure_facade = _compat.facade()
  5837. if not launcher.is_file():
  5838. return False
  5839. # BYTES on both sides. The shim is explicitly CRLF, and text-mode I/O
  5840. # rewrites line endings in both directions (universal-newlines on read,
  5841. # os.linesep on write), so a text comparison would never match and this
  5842. # would rewrite the file on every single ensure-health.
  5843. desired = _measure_facade._generate_windows_launcher_cmd(
  5844. str(daemon_script), str(_measure_facade.DAEMON_LOG_DIR)).encode("utf-8")
  5845. if launcher.read_bytes() == desired:
  5846. return False
  5847. # Link-safety: refuse to write through a symlink or a
  5848. # hardlinked name -- a planted link would turn this rewrite into an
  5849. # arbitrary-file overwrite with known content (Windows hardlinks need
  5850. # no privilege at all).
  5851. if launcher.is_symlink():
  5852. return False
  5853. if _measure_facade.os.lstat(str(launcher)).st_nlink > 1:
  5854. return False
  5855. # The regenerated ladder redirects into DAEMON_LOG_DIR; a legacy-era shim
  5856. # may predate that dir, and a missing redirect target fails the rung.
  5857. try:
  5858. _measure_facade.DAEMON_LOG_DIR.mkdir(parents=True, exist_ok=True)
  5859. except OSError:
  5860. pass
  5861. # Atomic replace (the _heal_keepwarm_plist_path pattern): a crash
  5862. # mid-write must not leave a truncated .cmd that Task Scheduler would
  5863. # still execute, and cmd.exe must never resume reading a half-written
  5864. # batch file at its saved byte offset.
  5865. fd, tmp = _measure_facade.tempfile.mkstemp(prefix=".dashboard-launcher.",
  5866. suffix=".tmp", dir=str(launcher.parent))
  5867. try:
  5868. with _measure_facade.os.fdopen(fd, "wb") as f:
  5869. f.write(desired)
  5870. _measure_facade.os.replace(tmp, str(launcher))
  5871. tmp = None
  5872. finally:
  5873. if tmp is not None:
  5874. try:
  5875. _measure_facade.os.unlink(tmp)
  5876. except OSError:
  5877. pass
  5878. return True
  5879. def _heal_windows_launcher_shim():
  5880. """Rewrite an on-disk ``dashboard-launcher.cmd`` left by an older build.
  5881. The shim generated before the no-flash fix tried console ``py.exe -3`` FIRST and probed
  5882. with three ``where.exe`` calls, so on any host with the Python Launcher
  5883. installed the long-lived daemon itself ran as a console process. Even where
  5884. the shim is only the fallback action, an installed copy keeps that shape
  5885. until something rewrites it.
  5886. BOTH shim locations are healed -- the plugin-data
  5887. SNAPSHOT_DIR and the legacy pre-migration dir
  5888. (``~/.claude/_backups/token-optimizer``). The auto-update block already
  5889. knows about both paths for ``dashboard-server.py``; users whose Scheduled
  5890. Task action still points at the LEGACY .cmd would otherwise keep the
  5891. console-first ladder forever. Each shim is regenerated against its own
  5892. directory's daemon script so the legacy action keeps launching the legacy
  5893. (auto-update-refreshed) script.
  5894. Only touches files that ALREADY EXIST (never creates one -- that would be
  5895. half-installing a daemon the user may have declined), and only when the
  5896. content differs from what we would generate today, which makes it a no-op
  5897. on the second run. Returns True when anything was rewritten. Never raises.
  5898. """
  5899. _measure_facade = _compat.facade()
  5900. healed_any = False
  5901. if _measure_facade.os.name != "nt":
  5902. return False
  5903. for base_dir in (_measure_facade.SNAPSHOT_DIR, _measure_facade._legacy_daemon_dir()):
  5904. try:
  5905. launcher = base_dir / _measure_facade.WINDOWS_LAUNCHER_NAME
  5906. daemon_script = base_dir / "dashboard-server.py"
  5907. if _measure_facade._heal_one_launcher_shim(launcher, daemon_script):
  5908. healed_any = True
  5909. except OSError:
  5910. continue
  5911. except Exception:
  5912. continue
  5913. return healed_any
  5914. def _heal_windows_console_flash():
  5915. """Runtime self-heal: repair already-installed Windows artifacts
  5916. that flash a console window.
  5917. A generator fix only helps the NEXT install. The reporter already has the
  5918. flashing task and shim on disk, and nothing in the update path rewrites
  5919. either -- the auto-update block refreshes ``dashboard-server.py`` and
  5920. restarts the task it finds. This is the migration that reaches the installed
  5921. base, and it is modelled on ``_heal_keepwarm_plist_path`` /
  5922. ``_migrate_statusline_to_stable_path``: best-effort, fail-open, called once
  5923. per ensure-health.
  5924. Repairs, in order:
  5925. (a) the on-disk ``dashboard-launcher.cmd`` shim (console-first interpreter
  5926. ladder + where.exe probes), and
  5927. (b) the Scheduled Task action, onto hidden pythonw.exe -- via the same
  5928. generation code a fresh install uses, so heal and install cannot drift.
  5929. Refusal gates, checked FIRST, so a heal can never resurrect something the
  5930. user turned off:
  5931. * not Windows -> strict no-op (macOS/Linux never reach schtasks);
  5932. * ``_daemon_resurrection_blocked()`` -- THE shared refusal gate
  5933. (uninstall/thrash tombstone, sticky ``daemon_disabled`` opt-out,
  5934. sticky ``.daemon-install-failed`` marker). The task heal re-checks the
  5935. same gate internally (plus the task's own user-set ``<Enabled>``
  5936. state), so its other call sites are equally safe.
  5937. IDEMPOTENT: both steps compare against the value they would write and return
  5938. False when it already matches, so a second run does nothing and the printed
  5939. line appears exactly once. The task heal is additionally THROTTLED
  5940. (``throttle=True``): a persistently-failing re-registration attempts at
  5941. most once per ensure-throttle window instead of bouncing the live daemon
  5942. on every SessionStart.
  5943. Returns True when anything was actually repaired. Never raises.
  5944. """
  5945. _measure_facade = _compat.facade()
  5946. if _measure_facade.os.name != "nt":
  5947. return False
  5948. if _measure_facade._daemon_resurrection_blocked() is not None:
  5949. return False
  5950. healed_shim = _measure_facade._heal_windows_launcher_shim()
  5951. healed_task = False
  5952. # Reading the task's action costs a `schtasks /Query`. A healthy install
  5953. # needs no heal, so look at most once per ensure-throttle window instead of
  5954. # on every SessionStart. Stamped before the read, like the heal throttle.
  5955. try:
  5956. last = float(_measure_facade._read_config_flag("last_windows_task_action_check", 0) or 0)
  5957. except (TypeError, ValueError):
  5958. last = 0.0
  5959. if _measure_facade.time.time() - last >= _measure_facade._daemon_ensure_throttle_seconds():
  5960. try:
  5961. _measure_facade._write_config_flag("last_windows_task_action_check", int(_measure_facade.time.time()))
  5962. except Exception:
  5963. pass
  5964. healed_task = _measure_facade._heal_windows_task_action(stop_first=True, throttle=True)
  5965. return bool(healed_shim or healed_task)
  5966. def _restart_dashboard_daemon(system):
  5967. """Restart a dashboard daemon via its service manager, reaping any orphaned
  5968. port-holder first and verifying the restart actually landed.
  5969. Factored out of run_ensure_health's auto-update block so the ensure path can
  5970. reuse the exact same reap/kickstart/restart/Run calls. Returns 'restarted',
  5971. 'restart-stale' (service restart ran but the OLD version is still serving),
  5972. or 'restart-failed'. Never raises.
  5973. WHY REAP FIRST: `launchctl kickstart -k` only restarts the launchd job's OWN
  5974. child. An orphaned dashboard-server.py -- from a prior label, a manual
  5975. launch, or ANOTHER runtime (Codex/OpenCode) on the same port -- survives the
  5976. restart and keeps binding DAEMON_PORT, serving the dashboard from its old
  5977. hardcoded measure.py path (the silent version-skew drift). Reaping the port
  5978. holder BEFORE restart frees the port so the service manager's fresh child
  5979. can bind. The reaper only SIGTERMs our own dashboard-server.py and never a
  5980. foreign process, so this is safe.
  5981. IDEMPOTENCY / RACE GUARD: if the port is ALREADY serving the current version
  5982. -- e.g. a sibling SessionStart just restarted it -- do nothing and report
  5983. success. Without this, two near-simultaneous sessions race: session B's reap
  5984. would SIGTERM session A's freshly-bound correct daemon, causing a restart
  5985. flap. Checking the served version first makes the whole restart idempotent.
  5986. """
  5987. _measure_facade = _compat.facade()
  5988. if _measure_facade._daemon_snapshot_sandboxed():
  5989. return "noop-sandbox"
  5990. try:
  5991. # Already current (a sibling session fixed it)? Do not reap/restart,
  5992. # and do not pay for the job check below (`schtasks /Query` on
  5993. # Windows): nothing is about to be written.
  5994. if _measure_facade._daemon_served_version() == _measure_facade.TOKEN_OPTIMIZER_VERSION:
  5995. return "restarted"
  5996. # About to reap and restart: refuse when the job is another install's.
  5997. if _measure_facade._scheduler_install_refused():
  5998. return "noop-sandbox"
  5999. # Reap our own orphaned port-holder first (never a foreign process).
  6000. try:
  6001. _measure_facade._reclaim_posix_daemon_port()
  6002. except Exception:
  6003. pass
  6004. _run_result = None # Windows-only landing signal (schtasks /Run rc)
  6005. if system == "Darwin":
  6006. uid = _measure_facade.subprocess.run(
  6007. ["id", "-u"], capture_output=True, text=True, creationflags=_measure_facade._NO_WINDOW
  6008. ).stdout.strip()
  6009. _measure_facade.subprocess.run(
  6010. ["launchctl", "kickstart", "-k", f"gui/{uid}/{_measure_facade.DAEMON_LABEL}"],
  6011. capture_output=True, timeout=5, creationflags=_measure_facade._NO_WINDOW,
  6012. )
  6013. elif system == "Linux":
  6014. _measure_facade.subprocess.run(
  6015. ["systemctl", "--user", "restart", _measure_facade.SYSTEMD_UNIT_NAME],
  6016. capture_output=True, timeout=10, creationflags=_measure_facade._NO_WINDOW,
  6017. )
  6018. elif system == "Windows":
  6019. _measure_facade.subprocess.run(
  6020. ["schtasks", "/End", "/TN", _measure_facade.WINDOWS_TASK_NAME],
  6021. capture_output=True, timeout=5, creationflags=_measure_facade._NO_WINDOW,
  6022. )
  6023. _measure_facade.time.sleep(2)
  6024. # An install registered by an older build still has the
  6025. # .cmd launcher as its task ACTION, and End/Run just re-runs whatever
  6026. # is registered -- so the console flash would survive every update
  6027. # for the entire installed base. Migrate the action here, between
  6028. # /End and /Run: the task is stopped, so the port is free and the
  6029. # re-registration's port-owner pre-check can pass.
  6030. _measure_facade._heal_windows_task_action()
  6031. _run_result = _measure_facade.subprocess.run(
  6032. ["schtasks", "/Run", "/TN", _measure_facade.WINDOWS_TASK_NAME],
  6033. capture_output=True, timeout=5, creationflags=_measure_facade._NO_WINDOW,
  6034. )
  6035. else:
  6036. return "restart-failed"
  6037. # Verify the restart LANDED: the process now serving DAEMON_PORT must
  6038. # report the CURRENT version. None (unknown -- no HTTP, or a pre-fix
  6039. # orphan without the version field) is a SAFE DEGRADE to 'restarted',
  6040. # never a false 'restart-stale'; the reap above already handles those.
  6041. served = _measure_facade._daemon_served_version()
  6042. if served is not None and served != _measure_facade.TOKEN_OPTIMIZER_VERSION:
  6043. return "restart-stale"
  6044. # A nonzero `schtasks /Run` with nothing
  6045. # serving the port is a DEMONSTRABLY failed restart (e.g. a disabled
  6046. # task, or an action whose baked interpreter path went dead). The
  6047. # None-version safe-degrade above must not convert that into a false
  6048. # "restarted" -- that is how a permanently-dead daemon stayed green.
  6049. if (_run_result is not None
  6050. and getattr(_run_result, "returncode", 0) != 0
  6051. and served is None
  6052. and not _measure_facade._verify_daemon_port(
  6053. timeout_seconds=1, retries=1, retry_sleep=0)):
  6054. return "restart-failed"
  6055. return "restarted"
  6056. except Exception:
  6057. return "restart-failed"
  6058. def _apply_daemon_restart_outcome(restart_status):
  6059. """Apply escalation + resolve the honest log message for a restart result.
  6060. Extracted from run_ensure_health's auto-update block so the escalation
  6061. decision is unit-testable. Returns a (level, message) tuple; the caller logs
  6062. `message` to stdout for the success levels ('ok', 'ok-reinstall') and to
  6063. stderr otherwise ('stale', 'failed'). Never raises.
  6064. On 'restart-stale' the service restart ran but the OLD version is still
  6065. serving the port -- an orphan the service manager couldn't reap. Escalate by
  6066. killing the port holder DIRECTLY, THEN forcing a clean reinstall. Reaping
  6067. first is essential: an alive-but-wrong-version orphan makes the port read as
  6068. healthy, so _ensure_dashboard_daemon(force=True) would no-op ('noop-healthy')
  6069. without the reap.
  6070. DO-NO-HARM GATE: only escalate when the forced reinstall can ACTUALLY run.
  6071. _ensure_dashboard_daemon early-returns a noop under a non-Claude runtime
  6072. (e.g. Codex, which has its own daemon), a `daemon_disabled` opt-out, or an
  6073. unsupported platform -- and `force` bypasses only the throttle, not those
  6074. gates. Reaping and then no-op'ing the reinstall would leave the user with NO
  6075. daemon (worse than a stale-but-alive one). In those cases, leave the running
  6076. daemon in place and log honestly.
  6077. """
  6078. _measure_facade = _compat.facade()
  6079. if restart_status == "restarted":
  6080. return ("ok", f"Auto-updated daemon to v{_measure_facade.TOKEN_OPTIMIZER_VERSION}")
  6081. if restart_status == "restart-stale":
  6082. if (_measure_facade._is_foreign_runtime()
  6083. or _measure_facade.detect_runtime() != "claude"
  6084. or _measure_facade._read_config_flag("daemon_disabled", False)
  6085. or _measure_facade._normalized_platform() not in ("Darwin", "Linux", "Windows")):
  6086. # A forced reinstall would no-op here -- do not reap a daemon we
  6087. # cannot replace. Leave the stale-but-alive daemon running.
  6088. return ("stale",
  6089. "daemon serving a stale version after auto-update; "
  6090. "run: measure.py setup-daemon")
  6091. try:
  6092. _measure_facade._reclaim_posix_daemon_port()
  6093. except Exception:
  6094. pass
  6095. escalated = _measure_facade._ensure_dashboard_daemon(force=True)
  6096. if escalated in ("installed", "restarted"):
  6097. return ("ok-reinstall",
  6098. f"Auto-updated daemon to v{_measure_facade.TOKEN_OPTIMIZER_VERSION} (via reinstall)")
  6099. return ("stale",
  6100. f"daemon still serving a stale version after auto-update "
  6101. f"({escalated}); run: measure.py setup-daemon")
  6102. return ("failed", "daemon restart failed after auto-update")
cowork/token-optimizer/skills/token-optimizer/scripts/bash_compress.py · Bundled script · 2,429 lines · 1 finding marked
  1. #!/usr/bin/env python3
  2. """Token Optimizer v5: Bash Output Compression Wrapper.
  3. Shared PostToolUse and host-bridge compression; optional direct wrapper:
  4. bash_compress.py git status
  5. bash_compress.py pytest tests/
  6. The legacy bash_hook.py no longer rewrites PreToolUse commands. Direct calls
  7. capture and compress output; errors retain raw output unchanged (fail-open).
  8. Security:
  9. - shell=True is NEVER used
  10. - Token preservation scan runs on PRE-compression output
  11. - Output buffered completely before writing to stdout
  12. - Partial output on timeout is NEVER compressed
  13. """
  14. from __future__ import annotations
  15. import hashlib
  16. import json
  17. import os
  18. import re
  19. import shlex
  20. import subprocess
  21. import sys
  22. import time
  23. import unicodedata
  24. from pathlib import Path
  25. # Windows: spawning a console exe (git, where, tasklist, ...) from a console-less
  26. # hook flashes a cmd window on every Bash call. CREATE_NO_WINDOW suppresses
  27. # it. getattr-guarded so it is 0 -- a no-op -- on POSIX and on builds without it.
  28. _NO_WINDOW = getattr(subprocess, "CREATE_NO_WINDOW", 0)
  29. # ---------------------------------------------------------------------------
  30. # Claude Code tool-output truncation baseline (never show more than baseline)
  31. # ---------------------------------------------------------------------------
  32. # Claude Code 2.1.247 (2026-08) replaces any tool result LARGER than
  33. # CC_PERSISTED_OUTPUT_THRESHOLD_CHARS with a ~2.2KB "<persisted-output>"
  34. # stub (measured: a 41.1KB `ls -la /usr/bin` result -> a 2,206-char stub).
  35. # Token Optimizer must NEVER make the model see more chars than that
  36. # baseline would, so these two numbers are the single tuning point: if a
  37. # future Claude Code version changes its truncation behavior, adjust them
  38. # together here (see _enforce_baseline_invariant).
  39. CC_PERSISTED_OUTPUT_THRESHOLD_CHARS = 30_000 # raw tool result size that triggers CC's stub
  40. CC_PERSISTED_OUTPUT_STUB_CHARS = 2_206 # size of the stub CC shows for such results
  41. # ---------------------------------------------------------------------------
  42. # Token/credential preservation patterns (scanned PRE-compression)
  43. # ---------------------------------------------------------------------------
  44. try:
  45. from credential_patterns import PATTERNS_ONLY as _TOKEN_PATTERNS
  46. except ImportError:
  47. # Standalone execution or import failure: inline minimal set
  48. _TOKEN_PATTERNS = [
  49. re.compile(r"AKIA[0-9A-Z]{16}"),
  50. re.compile(r"sk-[a-zA-Z0-9]{20,}"),
  51. re.compile(r"sk-ant-[a-zA-Z0-9\-]{20,}"),
  52. re.compile(r"ghp_[a-zA-Z0-9]{36}"),
  53. re.compile(r"gho_[a-zA-Z0-9]{36}"),
  54. re.compile(r"ghs_[a-zA-Z0-9]{36}"),
  55. re.compile(r"ghr_[a-zA-Z0-9]{36}"),
  56. re.compile(r"github_pat_[a-zA-Z0-9_]{80,}"),
  57. re.compile(r"npm_[a-zA-Z0-9]{36}"),
  58. re.compile(r"xoxb-[0-9]+-[a-zA-Z0-9]+"),
  59. re.compile(r"xoxp-[0-9]+-[a-zA-Z0-9]+"),
  60. re.compile(r"xoxa-[0-9]+-[a-zA-Z0-9]+"),
  61. re.compile(r"sk_live_[a-zA-Z0-9]{24,}"),
  62. re.compile(r"rk_live_[a-zA-Z0-9]{24,}"),
  63. re.compile(r"hf_[a-zA-Z0-9]{34}"),
  64. re.compile(r"gl(?:pat|dt|rt|cbt|ptt|ft|imt|agent|soat)-[a-zA-Z0-9_.\-]{20,}"),
  65. re.compile(r"Bearer\s+[a-zA-Z0-9\-._~+/]+=*", re.I),
  66. re.compile(r"AIza[0-9A-Za-z_\-]{35}"),
  67. re.compile(r"ya29\.[0-9A-Za-z_\-]{20,}"),
  68. re.compile(r"eyJ[A-Za-z0-9_\-]{10,}\.[A-Za-z0-9_\-]{10,}\.[A-Za-z0-9_\-]{10,}"),
  69. re.compile(r"-----BEGIN [A-Z ]*PRIVATE KEY-----"),
  70. re.compile(r"(?:postgres|postgresql|mysql|mongodb|mongodb\+srv|redis)://[^:\s/]+:[^@\s]+@", re.I),
  71. re.compile(r"https?://[^:\s/@]+:[^@\s]+@", re.I),
  72. ]
  73. # ANSI escape code patterns.
  74. #
  75. # _ANSI_CSI_RE drops colour/style/cursor escape sequences.
  76. #
  77. # _ANSI_OSC8_RE matches a full OSC 8 hyperlink -- opener, label text, closer --
  78. # and is used with a ``\1`` backreference so the visible label text survives
  79. # (captured as group 1). The previous combined regex swallowed the label along
  80. # with the opener/closer, which meant any credential embedded in the visible
  81. # hyperlink text vanished before the preservation scan ran.
  82. _ANSI_CSI_RE = re.compile(r"\x1b\[[0-9;]*[a-zA-Z]")
  83. _ANSI_OSC8_RE = re.compile(r"\x1b\]8;[^\x07]*\x07([^\x1b]*)\x1b\]8;;\x07")
  84. # Stderr patterns that indicate failure even with exit code 0 (linters often
  85. # emit errors on stderr but exit 0 because they only reported warnings).
  86. # Localised error markers are included so non-English toolchains still trip
  87. # the tee. The patterns are case-insensitive for Latin scripts and literal
  88. # for CJK scripts (case folding is a no-op there).
  89. _ERROR_STDERR_PATTERNS = [
  90. re.compile(r"\berror\s*:", re.I),
  91. re.compile(r"\bfatal\s*:", re.I),
  92. re.compile(r"\bpanic\s*:", re.I),
  93. re.compile(r"\bFAILED\b"),
  94. re.compile(r"\bTraceback\b"),
  95. # Localised variants:
  96. re.compile(r"\bfehler\s*:", re.I), # German
  97. re.compile(r"\berreur\s*:", re.I), # French
  98. re.compile(r"\berrore\s*:", re.I), # Italian
  99. re.compile(r"\bошибка\s*:", re.I), # Russian
  100. re.compile(r"错误\s*[::]"), # Chinese (simplified)
  101. re.compile(r"錯誤\s*[::]"), # Chinese (traditional)
  102. re.compile(r"エラー\s*[::]"), # Japanese
  103. re.compile(r"오류\s*[::]"), # Korean
  104. ]
  105. def _strip_ansi(text):
  106. """Remove ANSI escape codes. Preserve visible text from OSC 8 hyperlinks.
  107. Two-step replace: first collapse OSC 8 hyperlinks to their visible label
  108. (via the captured group 1), then drop CSI escape sequences. This keeps
  109. any credential that happens to appear inside the visible label text
  110. available to the pre-compression preservation scan, instead of dropping
  111. it with the surrounding hyperlink structure.
  112. """
  113. text = _ANSI_OSC8_RE.sub(r"\1", text)
  114. return _ANSI_CSI_RE.sub("", text)
  115. def _looks_like_failure(returncode, stderr):
  116. """Return True when the command should not have its output compressed.
  117. Triggers on non-zero exit, OR on exit code 0 with an error pattern on
  118. stderr (common for linters that print "error:" on stderr while exiting 0).
  119. Fail-open: on any surprise, treat as non-failure so the normal compression
  120. path still runs.
  121. """
  122. try:
  123. if returncode not in (0, None):
  124. return True
  125. if not stderr:
  126. return False
  127. for pat in _ERROR_STDERR_PATTERNS:
  128. if pat.search(stderr):
  129. return True
  130. except Exception:
  131. return False
  132. return False
  133. # Error markers in scripts/languages the English-keyword handlers would
  134. # otherwise drop. Any line matching one of these is re-injected through the
  135. # same preservation path credentials use, so a foreign-language error
  136. # surfaces even when the compression handler speaks only English.
  137. #
  138. # Latin-script entries require a trailing colon so short Latin tokens that
  139. # also appear in English tech vocabulary do not false-positive. FOUT
  140. # (Flash Of Unstyled Text) in web-perf output, FEL in Free Electron Laser
  141. # toolchains, and Hata propagation model references are common enough
  142. # that a bare word boundary would preserve unrelated lines and inflate
  143. # the compressed output.
  144. #
  145. # CJK-script entries stay as literal character sequences since those
  146. # characters do not collide with English tech tokens.
  147. _FOREIGN_ERROR_PATTERNS = [
  148. re.compile(r"\bfout\s*[::]", re.I), # Dutch
  149. re.compile(r"\bfel\s*[::]", re.I), # Swedish
  150. re.compile(r"\bvirhe\s*[::]", re.I), # Finnish
  151. re.compile(r"\bhiba\s*[::]", re.I), # Hungarian
  152. re.compile(r"\bhata\s*[::]", re.I), # Turkish
  153. re.compile(r"\berro\s*[::]", re.I), # Portuguese / Spanish
  154. re.compile(r"\bblad\s*[::]", re.I), # Polish
  155. re.compile(r"l\u1ed7i\s*[::]", re.I), # Vietnamese
  156. re.compile(r"\u0e02\u0e49\u0e2d\u0e1c\u0e34\u0e14\u0e1e\u0e25\u0e32\u0e14"), # Thai
  157. re.compile(r"\u05e9\u05d2\u05d9\u05d0\u05d4"), # Hebrew
  158. re.compile(r"\u062e\u0637\u0623"), # Arabic
  159. re.compile(r"\u062e\u0637\u0627"), # Persian
  160. ]
  161. def _find_preserved_lines(text):
  162. """Find line indices that must survive compression (credentials + errors).
  163. Two scans run here:
  164. 1. Token patterns (credentials) — never drop a line carrying a secret.
  165. 2. Foreign-language error markers — the English-keyword handlers below
  166. will happily drop a Dutch "fout:" line because it does not contain
  167. "error"/"failed"/etc. We flag it here so the re-injection path
  168. brings it back after compression runs.
  169. """
  170. preserved = set()
  171. for i, line in enumerate(text.splitlines()):
  172. for pat in _TOKEN_PATTERNS:
  173. if pat.search(line):
  174. preserved.add(i)
  175. break
  176. else:
  177. # Only scan error patterns when the line did not already match
  178. # a credential pattern (small optimization).
  179. for pat in _ERROR_STDERR_PATTERNS:
  180. if pat.search(line):
  181. preserved.add(i)
  182. break
  183. else:
  184. for pat in _FOREIGN_ERROR_PATTERNS:
  185. if pat.search(line):
  186. preserved.add(i)
  187. break
  188. return preserved
  189. # Identifier-shaped token: >=6 chars, contains a letter, not a bare
  190. # number/timestamp. Captures UUIDs, URLs, hashes, emails, dotted names, IDs --
  191. # the kinds of things a user greps a log FOR -- while ignoring counters,
  192. # timestamps, and short plain words.
  193. _DISTINCTIVE_TOKEN_RE = re.compile(r"[A-Za-z0-9][A-Za-z0-9._:/@\-]{5,}")
  194. _BARE_NUMERIC_RE = re.compile(r"^[0-9][0-9:.\-T/]*$")
  195. def _find_distinctive_lines(text, rare_max=2, cap_frac=0.30, min_lines=8):
  196. """Find lines carrying a token that is RARE within this output (the needle).
  197. Aggressive summarizers (search/grep -> top-N + count, generic log -> head/
  198. tail) drop the one line the user actually wants when it is not near the top.
  199. The model then re-runs a narrower query to recover it, spending the tokens
  200. the compression just saved. This scan flags lines whose distinctive token
  201. (a UUID, URL, hash, email, id -- something with identifier shape) appears on
  202. only a couple of lines, so the shared re-injection path keeps that needle.
  203. Rarity, not type, is the signal. A token on EVERY line (a per-line request
  204. id, a ubiquitous host) is boilerplate and is NOT preserved -- otherwise a
  205. log where every line carries an id would preserve every line and defeat
  206. compression. If more than ``cap_frac`` of lines look distinctive the output
  207. is too varied to have a single needle, so we bail (return nothing) and let
  208. normal compression run unhindered. Never raises: on any surprise the caller
  209. still has the credential/error scan and progressive-disclosure archive.
  210. """
  211. try:
  212. lines = text.splitlines()
  213. n = len(lines)
  214. if n < min_lines:
  215. return set() # too small to bother; full output likely survives
  216. from collections import Counter
  217. toksets = []
  218. df = Counter()
  219. for ln in lines:
  220. toks = set()
  221. for m in _DISTINCTIVE_TOKEN_RE.finditer(ln):
  222. t = m.group(0)
  223. if _BARE_NUMERIC_RE.match(t):
  224. continue # counter / timestamp, not a needle
  225. if not any(c.isalpha() for c in t):
  226. continue
  227. toks.add(t)
  228. toksets.append(toks)
  229. for t in toks:
  230. df[t] += 1
  231. distinctive = {
  232. i for i, toks in enumerate(toksets)
  233. if any(df[t] <= rare_max for t in toks)
  234. }
  235. if len(distinctive) > n * cap_frac:
  236. return set() # too varied to needle-preserve; do not defeat compression
  237. return distinctive
  238. except Exception:
  239. return set() # fail open -- credential scan + archive still protect data
  240. # ---------------------------------------------------------------------------
  241. # Compression patterns (one per command family)
  242. # ---------------------------------------------------------------------------
  243. def _compress_git_status(output):
  244. """Compress git status to one-line summary."""
  245. lines = output.strip().splitlines()
  246. branch = "?"
  247. ahead_behind = ""
  248. staged_files = []
  249. unstaged_files = []
  250. untracked_files = []
  251. unmerged_files = []
  252. section = None
  253. for line in lines:
  254. if line.startswith("On branch "):
  255. branch = line.replace("On branch ", "").strip()
  256. elif "ahead" in line or "behind" in line:
  257. ahead_behind = line.strip().lstrip("(").rstrip(")")
  258. elif line.strip() in (
  259. "nothing to commit, working tree clean",
  260. "nothing to commit (working directory clean)",
  261. "nothing to commit, working directory clean",
  262. ):
  263. # Anchored equality check: the string "nothing to commit" can
  264. # legitimately appear inside a filename under "Untracked files:"
  265. # (e.g. `nothing to commit.txt`). A substring match there would
  266. # falsely report the tree clean and hide real untracked work.
  267. return f"branch: {branch}, clean{f' ({ahead_behind})' if ahead_behind else ''}"
  268. elif "Changes to be committed:" in line:
  269. section = "staged"
  270. elif "Changes not staged" in line:
  271. section = "unstaged"
  272. elif "Untracked files:" in line:
  273. section = "untracked"
  274. elif "Unmerged paths:" in line:
  275. section = "unmerged"
  276. elif (line.startswith("\t") or line.startswith(" ")) and section:
  277. fname = line.strip()
  278. # Strip prefixes: staged/unstaged AND merge-conflict prefixes, so
  279. # conflicted files are captured (they were silently dropped before).
  280. for prefix in ("new file:", "modified:", "deleted:", "renamed:", "copied:",
  281. "both modified:", "both added:", "both deleted:",
  282. "added by us:", "added by them:",
  283. "deleted by us:", "deleted by them:"):
  284. if fname.startswith(prefix):
  285. fname = fname[len(prefix):].strip()
  286. break
  287. if section == "staged":
  288. staged_files.append(fname)
  289. elif section == "unstaged":
  290. unstaged_files.append(fname)
  291. elif section == "untracked":
  292. untracked_files.append(fname)
  293. elif section == "unmerged":
  294. unmerged_files.append(fname)
  295. if (branch == "?" and not ahead_behind and not staged_files
  296. and not unstaged_files and not untracked_files and not unmerged_files):
  297. # Nothing recognised: the input is not git status output (for example
  298. # a "git status"-hinted capture of a grep or git log). Hand the raw
  299. # output back rather than emit "branch: ?" and silently drop the
  300. # whole capture; the caller's generic fallback then applies.
  301. return output
  302. parts = [f"branch: {branch}"]
  303. if ahead_behind:
  304. parts.append(ahead_behind)
  305. if staged_files:
  306. parts.append(f"{len(staged_files)} staged: {', '.join(staged_files)}")
  307. if unstaged_files:
  308. parts.append(f"{len(unstaged_files)} unstaged: {', '.join(unstaged_files)}")
  309. if untracked_files:
  310. parts.append(f"{len(untracked_files)} untracked: {', '.join(untracked_files)}")
  311. if unmerged_files:
  312. # Merge conflicts must NEVER be silently dropped -- surface prominently.
  313. parts.append(f"{len(unmerged_files)} UNMERGED (merge conflicts): {', '.join(unmerged_files)}")
  314. return "\n".join(parts) if len(parts) > 2 else ", ".join(parts)
  315. def _compress_git_log_basic(output):
  316. """Identity: shapes the summarizer must not touch (custom formats, patches,
  317. signatures, stat listings) are delivered exactly as git printed them."""
  318. return output
  319. _GIT_LOG_COMMIT_RE = re.compile(r"^commit ([0-9a-f]{40}|[0-9a-f]{64})(?: \((.*)\))?$")
  320. _GIT_LOG_HEADER_RE = re.compile(r"^(Author|Date|Merge|Commit|CommitDate|AuthorDate):\s*(.*)$")
  321. _GIT_LOG_TRAILER_RE = re.compile(
  322. r"^(Signed-off-by|Co-authored-by|Reviewed-by|Acked-by|Tested-by|Reported-by|"
  323. r"Suggested-by|Helped-by|Cc|Change-Id|Fixes|Closes|Refs?)\s*:", re.IGNORECASE)
  324. _GIT_LOG_ALL_HASHES_RE = re.compile(r"(?m)^commit ([0-9a-f]{40}|[0-9a-f]{64})\b")
  325. _GIT_LOG_OUT_HASH_RE = re.compile(r"(?m)^([0-9a-f]{10}) ")
  326. # Separators str.splitlines() honours beyond "\n" (and NUL): a message holding
  327. # one would be cut mid-line, so such output is delivered untouched.
  328. _AMBIGUOUS_SEPARATORS_RE = re.compile("[\x00\r\x0b\x0c\x1c-\x1e\x85\u2028\u2029]")
  329. _GIT_LOG_MIN_COMMITS = 5
  330. _GIT_LOG_MAX_COMMITS = 50000
  331. _GIT_LOG_SHORT = 10
  332. def _plural(n, one, many):
  333. return f"{n} {one if n == 1 else many}"
  334. def _git_log_date(raw):
  335. """'Thu Oct 15 12:39:24 2026 +0300' -> '2026-10-15 12:39 +0300' (raw on any doubt)."""
  336. try:
  337. import datetime as _dt
  338. parts = raw.split()
  339. stamp = _dt.datetime.strptime(" ".join(parts[:5]), "%a %b %d %H:%M:%S %Y")
  340. tz = parts[5] if len(parts) > 5 and re.fullmatch(r"[+-]\d{4}", parts[5]) else ""
  341. return (stamp.strftime("%Y-%m-%d %H:%M") + (" " + tz if tz else "")).strip()
  342. except Exception:
  343. return raw.strip()
  344. def _compress_git_log(output):
  345. """Summarize default-format ``git log``: one line per commit.
  346. Keeps, for EVERY commit: 10-char hash, date, author name, subject (plus
  347. decorations and a merge marker). Bodies, trailers (Signed-off-by,
  348. Co-authored-by ...) and author emails are condensed to counts. Anything
  349. that is not cleanly parseable default-format output (patches, --stat,
  350. signatures, graph, custom formats, ambiguous separators) is returned
  351. unchanged, as is any result whose hash recount disagrees with the input.
  352. A last commit cut off by ``| head`` is listed as truncated, never guessed.
  353. """
  354. if _AMBIGUOUS_SEPARATORS_RE.search(output):
  355. return output
  356. input_hashes = _GIT_LOG_ALL_HASHES_RE.findall(output)
  357. if not _GIT_LOG_MIN_COMMITS <= len(input_hashes) <= _GIT_LOG_MAX_COMMITS:
  358. return output
  359. commits = []
  360. cur = None
  361. stream = output.split("\n")
  362. if stream and stream[-1] == "":
  363. stream.pop() # the newline that ends the stream is not a blank line
  364. for line in stream:
  365. m = _GIT_LOG_COMMIT_RE.match(line)
  366. if m:
  367. cur = {"hash": m.group(1), "deco": m.group(2), "hdr": {}, "msg": [], "in_msg": False}
  368. commits.append(cur)
  369. continue
  370. if cur is None:
  371. if line.strip():
  372. return output # preamble: not default format
  373. continue
  374. if not cur["in_msg"]:
  375. hm = _GIT_LOG_HEADER_RE.match(line)
  376. if hm:
  377. cur["hdr"][hm.group(1)] = hm.group(2)
  378. continue
  379. if not line.strip():
  380. cur["in_msg"] = True
  381. continue
  382. return output # unknown header shape (gpg lines, reflog ...)
  383. # message region: default format indents every message line by 4 spaces
  384. if line.strip() and not line.startswith(" "):
  385. return output # patch / stat / graph content
  386. cur["msg"].append(line[4:] if line.startswith(" ") else line.strip())
  387. if len(commits) != len(input_hashes) or [c["hash"] for c in commits] != input_hashes:
  388. return output
  389. def _incomplete(c):
  390. # Cut off only when the headers really are incomplete: no Date line, or
  391. # the stream ends before the blank line that closes the headers. A
  392. # commit with an empty message is complete: "(no message)".
  393. return ("Date" not in c["hdr"] and "AuthorDate" not in c["hdr"]) or not c["in_msg"]
  394. truncated = None
  395. if _incomplete(commits[-1]):
  396. truncated = commits[-1]
  397. for c in commits[:-1]:
  398. if "Date" not in c["hdr"] and "AuthorDate" not in c["hdr"]:
  399. return output
  400. out = []
  401. for c in commits:
  402. if c is truncated:
  403. out.append(f"{c['hash'][:_GIT_LOG_SHORT]} [truncated: the output ends inside this "
  404. f"commit; its date, author and subject are not in the original stream]")
  405. continue
  406. hdr = c["hdr"]
  407. author = re.sub(r"\s*<[^>]*>\s*$", "", hdr.get("Author", "?")).strip() or "?"
  408. date = _git_log_date(hdr.get("Date") or hdr.get("AuthorDate", ""))
  409. msg = list(c["msg"])
  410. while msg and not msg[0].strip():
  411. msg.pop(0)
  412. subject_lines = []
  413. while msg and msg[0].strip():
  414. subject_lines.append(msg.pop(0).strip()) # git's subject is the first paragraph
  415. subject = " ".join(subject_lines) or "(no message)"
  416. rest = [m for m in msg if m.strip()]
  417. trailers = [m for m in rest if _GIT_LOG_TRAILER_RE.match(m.strip())]
  418. body = len(rest) - len(trailers)
  419. flags = []
  420. if "Merge" in hdr:
  421. flags.append("merge")
  422. if body:
  423. flags.append("+" + _plural(body, "body line", "body lines"))
  424. if trailers:
  425. flags.append("+" + _plural(len(trailers), "trailer", "trailers"))
  426. coauth = [re.sub(r"\s*<[^>]*>\s*$", "", t.split(":", 1)[1]).strip()
  427. for t in trailers if t.strip().lower().startswith("co-authored-by")]
  428. if coauth:
  429. flags.append("co-authors: " + ", ".join(coauth[:3]))
  430. line = f"{c['hash'][:_GIT_LOG_SHORT]} {date} {author} | {subject}"
  431. if c["deco"]:
  432. line += f" ({c['deco']})"
  433. if flags:
  434. line += " [" + "; ".join(flags) + "]"
  435. out.append(line)
  436. body_text = "\n".join(out)
  437. # Recount from the text that will actually be delivered.
  438. if _GIT_LOG_OUT_HASH_RE.findall(body_text) != [h[:_GIT_LOG_SHORT] for h in input_hashes]:
  439. return output
  440. header = (f"[git log: {_plural(len(commits), 'commit', 'commits')}, one line each: "
  441. f"{_GIT_LOG_SHORT}-char hash, date, author, subject. "
  442. f"Text in [ ] counts omitted body lines and trailers and names co-authors; "
  443. f"text in ( ) is branch/tag decoration. Author emails and message bodies are "
  444. f"omitted; the full original output is recoverable.]")
  445. return header + "\n" + body_text
  446. _FIND_MIN_PATHS = 40 # fewer paths pass through unchanged
  447. _FIND_MAX_PATHS = 50000 # larger lists are delivered untouched (time bound)
  448. _FIND_DIR_KEEP = 8 # a directory with <= this many entries lists all names
  449. _FIND_DIR_KEEP_SEARCH = 40 # ... in a name/path search, where the names are the answer
  450. _FIND_EXAMPLES = 3
  451. _FIND_NOISE_DIRS = frozenset({
  452. ".git", "node_modules", "__pycache__", ".venv", "venv", ".tox", ".pytest_cache",
  453. ".mypy_cache", ".ruff_cache", ".next", ".gradle", "target", ".idea",
  454. })
  455. _FIND_SEARCH_PREDICATES = frozenset({
  456. "-name", "-iname", "-path", "-ipath", "-wholename", "-iwholename", "-regex", "-iregex",
  457. })
  458. _FIND_GLOBAL_OPTIONS = frozenset({"-H", "-L", "-P", "-E", "-X", "-x", "-s", "-d"})
  459. def _find_ext(name):
  460. base = name.rsplit("/", 1)[-1]
  461. if "." in base.lstrip("."):
  462. return "*." + base.rsplit(".", 1)[-1]
  463. return "(no ext)"
  464. def _find_name(name):
  465. return shlex.quote(name) if re.search(r"[\s'\"\\$`*?\[\](){}<>|&;#~!]", name) else name
  466. def _find_summary(label, names, with_examples=True, unit="entry", unit_many="entries",
  467. count=None):
  468. count = len(names) if count is None else count
  469. exts = {}
  470. for n in names:
  471. exts[_find_ext(n)] = exts.get(_find_ext(n), 0) + 1
  472. ranked = sorted(exts.items(), key=lambda kv: (-kv[1], kv[0]))
  473. shown = ", ".join(f"{e} x{k}" for e, k in ranked[:4])
  474. if len(ranked) > 4:
  475. shown += f", {_plural(len(ranked) - 4, 'other type', 'other types')}"
  476. text = f"* {_find_name(label)} ({_plural(count, unit, unit_many)}" + (f": {shown})" if shown else ")")
  477. if with_examples and names:
  478. text += " e.g. " + " ".join(_find_name(n) for n in names[:_FIND_EXAMPLES])
  479. return text
  480. def _find_command_scope(command_str):
  481. """(roots, search_mode) read from the find invocation, or (None, False)."""
  482. if not command_str:
  483. return None, False
  484. try:
  485. tokens = shlex.split(command_str)
  486. except ValueError:
  487. return None, False
  488. if "|" in tokens:
  489. tokens = tokens[:tokens.index("|")]
  490. idx = next((i for i, t in enumerate(tokens) if t == "find"), None)
  491. if idx is None:
  492. return None, False
  493. rest = tokens[idx + 1:]
  494. roots = []
  495. i = 0
  496. while i < len(rest):
  497. t = rest[i]
  498. if t in _FIND_GLOBAL_OPTIONS or re.fullmatch(r"-O\d", t):
  499. i += 1
  500. elif t == "-D":
  501. i += 2
  502. else:
  503. break
  504. while i < len(rest) and not (rest[i].startswith("-") or rest[i] in ("(", "!", ")")):
  505. roots.append(rest[i])
  506. i += 1
  507. search = any(t in _FIND_SEARCH_PREDICATES for t in rest[i:])
  508. return (roots or ["."]), search
  509. def _norm_components(path):
  510. parts = [p for p in path.split("/") if p not in ("", ".")]
  511. return (path.startswith("/"), tuple(parts))
  512. def _under(root, path):
  513. r_abs, r = _norm_components(root)
  514. p_abs, p = _norm_components(path)
  515. return r_abs == p_abs and p[:len(r)] == r
  516. def _find_recount(text, paths):
  517. """Recount, from the delivered text, how many paths it accounts for."""
  518. expected = set(paths)
  519. seen = set()
  520. total = 0
  521. try:
  522. for ln in text.split("\n")[1:]:
  523. if ln.startswith("* "):
  524. t = shlex.split(ln[2:])
  525. m = re.fullmatch(r"\((\d+)", t[1])
  526. total += int(m.group(1))
  527. elif ln.startswith("= "):
  528. for name in shlex.split(ln[2:]):
  529. if name not in expected or name in seen:
  530. return False
  531. seen.add(name)
  532. total += 1
  533. else:
  534. t = shlex.split(ln)
  535. label = t[0]
  536. for name in t[1:]:
  537. full = label + (name[:-1] if name.endswith("/") else name)
  538. if full not in expected or full in seen:
  539. return False
  540. seen.add(full)
  541. total += 1
  542. except (ValueError, IndexError, AttributeError):
  543. return False
  544. return total == len(paths)
  545. def _compress_find(output, command_str=None):
  546. """Summarize a plain ``find`` path list, grouped by directory.
  547. Small lists pass through. Otherwise every directory with few entries lists
  548. every name once (its prefix stated once, not per path); larger directories
  549. collapse to a count, a type breakdown and example names; with a plain
  550. full-tree listing, noise trees (.git, node_modules, __pycache__ ...)
  551. collapse whole. Noise is never collapsed when the find root is inside it or
  552. when the command is a name/path search. The total path count is always
  553. stated. Anything that is not plainly a path list is returned unchanged.
  554. """
  555. if _AMBIGUOUS_SEPARATORS_RE.search(output):
  556. return output
  557. paths = [ln for ln in output.split("\n") if ln]
  558. if not _FIND_MIN_PATHS <= len(paths) <= _FIND_MAX_PATHS:
  559. return output
  560. if len(set(paths)) != len(paths):
  561. return output
  562. for p in paths:
  563. # tab/control chars, leading blanks (uniq -c, columns), error lines,
  564. # trailing or doubled slashes: not plainly a list of find paths.
  565. if (len(p) > 1024 or p != p.strip() or p.startswith("find:")
  566. or re.search(r"[\x00-\x1f\x7f]", p) or "//" in p
  567. or (len(p) > 1 and p.endswith("/")) or re.match(r"^\S+\s{2,}\S", p)):
  568. return output
  569. roots, search_mode = _find_command_scope(command_str)
  570. if roots is not None:
  571. prefixes = [r.rstrip("/") or "/" for r in roots]
  572. if not all(any(p == r or p.startswith(r if r.endswith("/") else r + "/")
  573. for r in prefixes) for p in paths):
  574. return output
  575. else:
  576. forms = {("abs" if p.startswith("/") else "dot" if p.split("/")[0] == "." else "rel")
  577. for p in paths}
  578. if len(forms) != 1:
  579. return output
  580. keep = _FIND_DIR_KEEP_SEARCH if search_mode else _FIND_DIR_KEEP
  581. noise_roots = {}
  582. groups = {}
  583. special = []
  584. dir_set = set()
  585. for p in paths:
  586. d = p
  587. while "/" in d: # every ancestor directory, even of a collapsed tree
  588. d = d[:d.rfind("/")]
  589. if not d:
  590. break
  591. dir_set.add(d)
  592. if p in (".", "/") or (not p.startswith("/") and "/" not in p):
  593. special.append(p)
  594. continue
  595. parts = p.split("/")
  596. cut = None
  597. if not search_mode:
  598. for i, comp in enumerate(parts):
  599. if comp in _FIND_NOISE_DIRS:
  600. cut = i
  601. break
  602. if cut is not None:
  603. root = "/".join(parts[:cut + 1])
  604. if roots is not None and any(_under(root, r) for r in roots):
  605. cut = None # the find root is inside this noise tree
  606. if cut is not None:
  607. noise_roots.setdefault(root, []).append("/".join(parts[cut + 1:]))
  608. continue
  609. parent = p[:p.rfind("/")] or "/"
  610. groups.setdefault(parent, []).append(parts[-1])
  611. out = []
  612. collapsed = False
  613. if special:
  614. out.append("= " + " ".join(_find_name(s) for s in special))
  615. for parent in sorted(groups):
  616. names = groups[parent]
  617. label = parent if parent.endswith("/") else parent + "/"
  618. disp = []
  619. for n in names:
  620. full = (parent if parent != "/" else "") + "/" + n
  621. disp.append(n + "/" if full in dir_set else n)
  622. if len(names) <= keep:
  623. out.append(_find_name(label) + " " + " ".join(_find_name(n) for n in disp))
  624. else:
  625. collapsed = True
  626. out.append(_find_summary(label, names))
  627. for root in sorted(noise_roots):
  628. collapsed = True
  629. members = noise_roots[root]
  630. named = [m for m in members if m]
  631. out.append(_find_summary(root + "/", named, unit="path collapsed",
  632. unit_many="paths collapsed", count=len(members)))
  633. head = (f"[find: {_plural(len(paths), 'path', 'paths')} total, grouped by directory; "
  634. f"a name ending in / has its own children listed in that directory's group.")
  635. if special:
  636. head += " A line starting with = lists top-level paths such as the search root itself."
  637. if collapsed:
  638. head += (" Lines starting with * are collapsed groups: they show a count, a type "
  639. "breakdown and a few example names only, not every entry.")
  640. head += " The full original output is recoverable.]"
  641. result = head + "\n" + "\n".join(out)
  642. if not _find_recount(result, paths):
  643. return output
  644. return result
  645. def _compress_git_diff(output):
  646. """Compress git diff: keep file names and stats, truncate large diffs."""
  647. lines = output.strip().splitlines()
  648. if len(lines) <= 50:
  649. return output # small diff, keep full
  650. # Extract summary stats
  651. additions = 0
  652. deletions = 0
  653. for line in lines:
  654. if line.startswith("diff --git"):
  655. pass
  656. elif line.startswith("+++"):
  657. pass
  658. elif line.startswith("+") and not line.startswith("+++"):
  659. additions += 1
  660. elif line.startswith("-") and not line.startswith("---"):
  661. deletions += 1
  662. # Keep first 30 lines of actual diff content
  663. changed = [ln for ln in lines if ln.startswith("diff --git ")]
  664. result_lines = lines[:30]
  665. if len(lines) > 30:
  666. result_lines.append(f"\n... ({len(lines) - 30} more lines, +{additions}/-{deletions} total)")
  667. if changed:
  668. # Keep every changed FILENAME so "did file X change?" is answerable.
  669. names = [c.split(" b/")[-1] for c in changed]
  670. result_lines.append(f"All {len(names)} changed files: " + ", ".join(names))
  671. return "\n".join(result_lines)
  672. _PYTEST_SUMMARY_TAIL_LINES = 40
  673. def _compress_pytest(output):
  674. """Compress pytest/cypress/playwright/mocha/karma/rspec/unittest/gradle/tox/deno/bun/ava test output.
  675. Extracts trailing summary lines (anything containing passed, passing,
  676. failed, failing, error, pending, or skipped) plus the explicit
  677. FAILURES/ERRORS section when pytest emits one. Cypress and playwright
  678. speak in "passing" / "failing" terms, so the reverse-scan accepts both
  679. vocabularies. The v5.8 extensions add "ran " (unittest), "build
  680. successful"/"build failed" (gradle), "congratulations" (tox), and
  681. "ok |" (deno) to the summary keyword set. The scan window is bounded
  682. by ``_PYTEST_SUMMARY_TAIL_LINES`` to avoid walking the whole output on
  683. pathological inputs, but is wide enough to survive 15-20 trailing
  684. warning lines emitted after the summary.
  685. """
  686. lines = output.strip().splitlines()
  687. if not lines:
  688. return output
  689. if len(lines) < 15:
  690. return output # too short to meaningfully compress
  691. summary_kw = ("passed", "passing", "failed", "failing", "error",
  692. "pending", "skipped",
  693. # v5.8 test-runner extension summary markers
  694. "ran ", "build successful", "build failed",
  695. "congratulations", "ok |",
  696. # v5.8 bun test runner (uses "pass"/"fail" without -ed/-ing)
  697. "pass,", "fail,")
  698. # Reverse-scan the tail for a contiguous block of summary lines.
  699. # Accept up to 5 non-matching lines inside the block (trailing warnings)
  700. # before declaring the summary region over.
  701. summary_block = []
  702. non_match_run = 0
  703. MAX_NON_MATCH = 5
  704. for line in reversed(lines[-_PYTEST_SUMMARY_TAIL_LINES:]):
  705. stripped = line.strip().lstrip("=").strip()
  706. if not stripped:
  707. if summary_block:
  708. non_match_run += 1
  709. if non_match_run > MAX_NON_MATCH:
  710. break
  711. continue
  712. if any(kw in stripped.lower() for kw in summary_kw):
  713. summary_block.append(stripped)
  714. non_match_run = 0
  715. elif summary_block:
  716. non_match_run += 1
  717. if non_match_run > MAX_NON_MATCH:
  718. break
  719. summary_block.reverse()
  720. summary_line = "\n".join(summary_block)
  721. # Find failure section (pytest-native)
  722. failure_lines = []
  723. in_failures = False
  724. for line in lines:
  725. if "FAILURES" in line or "ERRORS" in line:
  726. in_failures = True
  727. continue
  728. if in_failures:
  729. if line.startswith("=" * 10):
  730. break # end of failures section
  731. failure_lines.append(line)
  732. if failure_lines:
  733. failure_text = "\n".join(failure_lines[:30])
  734. if len(failure_lines) > 30:
  735. failure_text += f"\n... ({len(failure_lines) - 30} more failure lines)"
  736. return f"{summary_line}\n\n{failure_text}"
  737. return summary_line if summary_line else output
  738. def _compress_jest(output):
  739. """Compress jest/vitest: keep summary + failure details."""
  740. lines = output.strip().splitlines()
  741. summary_lines = []
  742. failure_lines = []
  743. for line in lines:
  744. if "Tests:" in line or "Test Suites:" in line or "Time:" in line:
  745. summary_lines.append(line.strip())
  746. elif "FAIL" in line and ("::" in line or ">" in line):
  747. failure_lines.append(line.strip())
  748. elif line.strip().startswith("Expected:") or line.strip().startswith("Received:"):
  749. failure_lines.append(line.strip())
  750. result = "\n".join(summary_lines)
  751. if failure_lines:
  752. result += "\n\nFailures:\n" + "\n".join(failure_lines[:20])
  753. return result if result.strip() else output
  754. def _compress_npm_install(output):
  755. """Compress npm/pip install: summary only."""
  756. lines = output.strip().splitlines()
  757. result = []
  758. for line in lines:
  759. stripped = line.strip()
  760. if not stripped:
  761. continue
  762. # Keep: added/removed/audited summary, vulnerability count, warnings
  763. if any(kw in stripped.lower() for kw in [
  764. "added", "removed", "audited", "packages",
  765. "vulnerabilit", "up to date", "successfully installed",
  766. "warn", "error", "fatal",
  767. ]):
  768. result.append(stripped)
  769. return "\n".join(result) if result else output
  770. def _ls_entry_name(line):
  771. """Extract the entry NAME from one ls line, spaces and symlinks intact.
  772. ``split()[-1]`` is wrong for two real cases: a filename with spaces ("My
  773. Important Doc.txt" -> "Doc.txt") and a symlink (``link -> target`` -> the
  774. target, hiding the link's own name). Both make the agent search for a name
  775. that "isn't there" and infer false absence. For long format (``ls -l``) the
  776. name is everything after the 8 metadata columns; for a plain listing the
  777. whole line is the name.
  778. """
  779. toks = line.split()
  780. if not toks:
  781. return line
  782. first = toks[0]
  783. is_long = (
  784. len(toks) >= 9
  785. and len(first) in (10, 11)
  786. and first[0] in "bcdlps-"
  787. and set(first[1:10]) <= set("rwxXsStT-")
  788. )
  789. if is_long:
  790. name = line.split(None, 8)[-1]
  791. # symlink "link -> target": keep the link's own name, not the target.
  792. return name.split(" -> ", 1)[0].rstrip()
  793. # plain listing (ls -1 / find output): the whole line is the path/name.
  794. return line.rstrip()
  795. def _compress_ls(output):
  796. """Compress directory listing: truncate at 50 entries."""
  797. lines = output.strip().splitlines()
  798. if len(lines) <= 50:
  799. return output
  800. result = lines[:50]
  801. # Keep every entry NAME (cheap) so "is file X here?" stays answerable; only
  802. # the metadata columns of entries past 50 are dropped, never the names.
  803. rest = lines[50:]
  804. names = [_ls_entry_name(ln) for ln in rest]
  805. result.append(f"... plus {len(rest)} more (names only, none dropped): "
  806. f"{', '.join(names)}")
  807. return "\n".join(result)
  808. # Lint rule-code patterns. Ordered by specificity so the first match wins.
  809. # The CAPS+digits pattern catches ruff/flake8/pylint/shellcheck/pydocstyle.
  810. # The trailing kebab pattern catches eslint rule names. The CamelCase/slash
  811. # pattern catches rubocop cops. The parenthesised trailing word catches
  812. # golangci-lint linter tags like `(typecheck)` or `(govet)`.
  813. _LINT_CODE_PATTERNS = [
  814. re.compile(r"\b([A-Z]{1,4}\d{2,5})\b"), # F401, E501, SC2086, W0611
  815. re.compile(r"\s([a-z][a-z0-9]*(?:-[a-z0-9]+)+)\s*$"), # eslint trailing kebab
  816. re.compile(r"\b([A-Z][A-Za-z]+/[A-Z][A-Za-z]+)\b"), # rubocop Style/Foo
  817. re.compile(r"\(([a-z][a-z0-9]+)\)\s*$"), # golangci (typecheck)
  818. ]
  819. # Prefixes that look like lint rule codes but are not. A line like
  820. # "vulnerable to CVE2024" or "HTTP404 on /api" would otherwise get grouped
  821. # as if CVE2024 / HTTP404 were linter rules. The handler discards matches
  822. # whose first two letters are any of these prefixes.
  823. _LINT_CODE_BLOCKLIST_PREFIXES = (
  824. "CVE", "RFC", "ISO", "HTTP", "ISBN", "USD", "EUR", "GBP",
  825. "JPY", "SHA", "MD5", "PGP", "TCP", "UDP", "DNS",
  826. )
  827. def _compress_build(output):
  828. """Compress JS/TS/Go build output (tsc/webpack/esbuild/vite/next/go build).
  829. Keeps lines that look like errors, warnings, or final summary markers.
  830. Drops the bulk of bundler chatter (asset lists, file size tables,
  831. incremental compile stats). When there is no recognisable error/warning
  832. or summary signal at all, returns the raw output unchanged — the 10%
  833. ratio gate in ``compress()`` then throws the compression away on its own.
  834. The handler never fabricates a positive "build clean" claim.
  835. """
  836. lines = output.splitlines()
  837. if len(lines) < 20:
  838. return output
  839. error_kw = ("error", "warning", "failed", "fatal")
  840. summary_kw = (
  841. "compiled successfully", "built in", "build finished",
  842. "build completed", "done in", "errors", "warnings",
  843. "bundled ", "chunk ", "emitted ", "hash:", "version:",
  844. )
  845. # "Found 0 errors" / "0 warnings" / "no errors" are CLEAN signals that
  846. # happen to contain an error_kw substring. Treat them as summary,
  847. # not as error/warning, so a clean build is not reported with a
  848. # misleading "1 error/warning lines" header. The match is
  849. # left-anchored so a real error line like
  850. # `error TS2322: Expected 'no errors' but found 'undefined'` is not
  851. # mis-routed to the summary bucket, and so a line like
  852. # `10 errors in 5 files` cannot match the `0 errors` prefix.
  853. clean_summary_patterns = (
  854. "found 0 error", "0 errors", "0 warnings", "no errors",
  855. "no warnings", "0 problems",
  856. )
  857. # "Found N errors" (N > 0) is an aggregate summary, not an individual
  858. # error line. Counting it as an error inflates the handler's "N
  859. # error/warning lines" header. We detect it via a regex and route to
  860. # the summary bucket.
  861. found_n_errors_re = re.compile(r"^found\s+\d+\s+(error|warning|problem)s?", re.I)
  862. # "Errors Files" style tsc trailing column headers are also summary.
  863. error_header_res = re.compile(r"^(errors?\s+files?|errors?\s+warnings?)$", re.I)
  864. errors = []
  865. summaries = []
  866. total_kept = 0
  867. for raw in lines:
  868. stripped = raw.strip()
  869. if not stripped:
  870. continue
  871. low = stripped.lower()
  872. if any(low.startswith(pat) for pat in clean_summary_patterns):
  873. summaries.append(stripped)
  874. continue
  875. if found_n_errors_re.match(low) or error_header_res.match(low):
  876. summaries.append(stripped)
  877. continue
  878. if any(kw in low for kw in error_kw):
  879. errors.append(stripped)
  880. total_kept += 1
  881. if total_kept > 80: # bound even on pathological error dumps
  882. break
  883. elif any(kw in low for kw in summary_kw):
  884. summaries.append(stripped)
  885. if not errors and not summaries:
  886. # No recognisable signal — hand the raw output back unchanged. The
  887. # 10% ratio gate in compress() will see zero compression and also
  888. # return raw. Do NOT fabricate a "no errors detected" marker: that
  889. # would be a positive claim about build state we cannot verify.
  890. return output
  891. parts = []
  892. if errors:
  893. parts.append(f"{len(errors)} error/warning lines:")
  894. parts.extend(errors[:40])
  895. if len(errors) > 40:
  896. parts.append(f"... ({len(errors) - 40} more errors/warnings)")
  897. if summaries:
  898. parts.append("")
  899. parts.extend(summaries[-5:])
  900. return "\n".join(parts)
  901. def _compress_list(output):
  902. """Compress long listing commands (pip list / npm ls / docker ps / brew list).
  903. Shows the first ~10 entries verbatim, then a "... N more" placeholder,
  904. and preserves any trailing summary line. Header lines (first non-empty
  905. line plus common "Name Version" style headers) survive to keep the
  906. output interpretable. Short lists (<20 entries) pass through raw.
  907. """
  908. lines = output.splitlines()
  909. non_empty = [ln for ln in lines if ln.strip()]
  910. if len(non_empty) < 20:
  911. return output
  912. header_lines = []
  913. data_lines = []
  914. seen_data = False
  915. for line in lines:
  916. stripped = line.strip()
  917. if not stripped:
  918. continue
  919. # Detect header separators (all dashes, "Package Version", etc.)
  920. if not seen_data and (
  921. set(stripped) <= set("-= ")
  922. or any(stripped.lower().startswith(prefix) for prefix in (
  923. "package", "name ", "container", "image", "repository",
  924. ))
  925. ):
  926. header_lines.append(line)
  927. continue
  928. seen_data = True
  929. data_lines.append(line)
  930. if len(data_lines) < 20:
  931. return output
  932. keep_count = 10
  933. result = list(header_lines)
  934. result.extend(data_lines[:keep_count])
  935. rest = data_lines[keep_count:]
  936. # Which column holds the identifying NAME depends on the command:
  937. # - docker ps: NAMES is the LAST column (first col is CONTAINER ID) -> split()[-1]
  938. # - npm/pnpm ls: lines are a glyph tree (+--, |--, `--) -> strip glyph, take pkg
  939. # - pip list / brew / apt: NAME is the FIRST column -> split()[0]
  940. header_blob = " ".join(header_lines).upper()
  941. tree_glyphs = ("+--", "|--", "`--", "\\--", "├", "└", "│", "|__")
  942. def _name_of(ln):
  943. toks = ln.split()
  944. if not toks:
  945. return ""
  946. # npm-style tree row: drop leading glyph tokens, keep the package spec.
  947. if any(g in ln for g in tree_glyphs):
  948. cleaned = ln
  949. for g in ("+--", "|--", "`--", "\\--", "├──",
  950. "└──", "│", "|__"):
  951. cleaned = cleaned.replace(g, " ")
  952. ct = cleaned.split()
  953. return ct[0] if ct else ""
  954. # docker ps: NAMES is the last whitespace-column.
  955. if "CONTAINER ID" in header_blob and "NAMES" in header_blob:
  956. return toks[-1]
  957. return toks[0]
  958. names = [n for n in (_name_of(ln) for ln in rest) if n]
  959. result.append(
  960. f"... plus {len(rest)} more entries (names only; a name not listed is "
  961. f"not present): {', '.join(names)}")
  962. return "\n".join(result)
  963. def _compress_progress(output):
  964. """Compress build progress output (docker build, cargo build verbose, etc.).
  965. Drops noisy progress lines (layer staging, "Downloading", "Extracting",
  966. cache hit indicators) and keeps lines that signal status, warnings,
  967. errors, or final outcomes. Safe for pip install / cargo build / docker
  968. build style output. Unknown lines fall through as summary context when
  969. there are few enough of them to matter.
  970. """
  971. lines = output.splitlines()
  972. if len(lines) < 20:
  973. return output
  974. noisy_prefixes = (
  975. "Downloading ", "Collecting ", "Resolving ", "Building wheels ",
  976. "Created wheel", "Using cached ", "Requirement already ",
  977. "Extracting", "Transferring",
  978. )
  979. # docker-buildx progress markers: `#12 DONE 0.1s`, `#12 ...`, etc.
  980. docker_progress_re = re.compile(r"^#\d+\s+")
  981. # docker pull layer fetch progress (shows up as `abc123456789: Pulling...`)
  982. pull_progress_re = re.compile(
  983. r"^[a-f0-9]{12}:\s+(Pulling|Waiting|Already|Extracting|Download|Verifying)"
  984. )
  985. keep = []
  986. dropped = 0
  987. for raw in lines:
  988. stripped = raw.strip()
  989. if not stripped:
  990. continue
  991. low = stripped.lower()
  992. # Always keep errors, warnings, failures, and final success markers.
  993. if any(kw in low for kw in ("error", "warning", "fail", "fatal",
  994. "successfully", "built ",
  995. "installed", "complete", "naming to",
  996. "exporting")):
  997. keep.append(stripped)
  998. continue
  999. # Drop noisy progress chatter.
  1000. if stripped.startswith(noisy_prefixes):
  1001. dropped += 1
  1002. continue
  1003. if docker_progress_re.match(stripped) and "DONE" not in stripped:
  1004. dropped += 1
  1005. continue
  1006. if pull_progress_re.match(stripped):
  1007. dropped += 1
  1008. continue
  1009. # Keep everything else as context.
  1010. keep.append(stripped)
  1011. if dropped < max(10, len(lines) // 4):
  1012. return output # not noisy enough to justify compression
  1013. return "\n".join(keep)
  1014. def _tree_line_depth(line):
  1015. """Return the logical depth of a `tree` output line.
  1016. Walks the 4-character prefix chunks `tree` emits: vertical pipes ("│ "),
  1017. blanks (" "), and branch markers ("├── "/"└── "). Each pipe or blank
  1018. chunk is one level; a branch marker contributes the final +1 for the leaf.
  1019. Root-level names with no prefix return 0.
  1020. """
  1021. i = 0
  1022. depth = 0
  1023. while i + 4 <= len(line):
  1024. chunk = line[i:i + 4]
  1025. if chunk == "\u2502 " or chunk == " ": # "│ " or four spaces
  1026. depth += 1
  1027. i += 4
  1028. elif chunk == "\u251c\u2500\u2500 " or chunk == "\u2514\u2500\u2500 ": # ├── / └──
  1029. return depth + 1
  1030. else:
  1031. return depth
  1032. return depth
  1033. def _compress_tree(output):
  1034. """Compress the `tree` command: keep entries up to depth 2 plus the summary.
  1035. Depth is inferred from `tree`'s box-drawing prefix. Root (depth 0), direct
  1036. children (depth 1), and grandchildren (depth 2) are kept; anything deeper
  1037. is dropped and replaced with a single "(N entries at depth > 2 truncated)"
  1038. marker. The trailing "N directories, M files" summary line survives
  1039. because summary lines have depth 0. Credentials appearing in deep file
  1040. names are preserved by the pre-compression token scan.
  1041. """
  1042. lines = output.splitlines()
  1043. if len(lines) < 20:
  1044. return output # not enough to bother
  1045. result = []
  1046. truncated = 0
  1047. for line in lines:
  1048. if not line.strip():
  1049. result.append(line)
  1050. continue
  1051. depth = _tree_line_depth(line)
  1052. if depth <= 2:
  1053. result.append(line)
  1054. else:
  1055. truncated += 1
  1056. if truncated == 0:
  1057. return output
  1058. result.append(f"... ({truncated} entries at depth > 2 truncated)")
  1059. return "\n".join(result)
  1060. def _compress_logs(output):
  1061. """Compress log-like output (tail/journalctl).
  1062. Detects runs of adjacent duplicate lines and collapses them to a single
  1063. line plus a "(xN)" marker. Only activates when the duplicate rate is at
  1064. least 30% of the input, so normal mixed logs pass through raw.
  1065. Credential safety: the collapsed line is ``<original> (xN)``, so the
  1066. original line text is kept verbatim as a prefix. Any credential on the
  1067. line is therefore preserved by the line itself — the pre-compression
  1068. token scan still runs as a backstop, and its re-injection check (``if
  1069. preserved_line not in compressed_text``) correctly finds the credential
  1070. as a substring and skips adding a duplicate copy.
  1071. """
  1072. lines = output.splitlines()
  1073. if len(lines) < 20:
  1074. return output # short logs: not worth the compression risk
  1075. collapsed = []
  1076. dup_removed = 0
  1077. i = 0
  1078. while i < len(lines):
  1079. current = lines[i]
  1080. run = 1
  1081. while i + run < len(lines) and lines[i + run] == current:
  1082. run += 1
  1083. if run > 1:
  1084. collapsed.append(f"{current} (x{run})")
  1085. dup_removed += run - 1
  1086. else:
  1087. collapsed.append(current)
  1088. i += run
  1089. # Require meaningful dup density before accepting the compression.
  1090. if dup_removed < max(10, len(lines) // 3):
  1091. return output
  1092. return "\n".join(collapsed)
  1093. def _compress_lint(output):
  1094. """Compress lint output (eslint/ruff/flake8/rubocop/shellcheck/biome/pylint/golangci-lint).
  1095. Group findings by rule code, show top-5 codes with a single sample line,
  1096. preserve trailing summary lines (e.g. "Found 6 errors") verbatim. Unknown
  1097. lines fall through uncounted so the handler stays robust against format
  1098. drift — credentials are preserved via the PRE-compression token scan.
  1099. """
  1100. lines = output.splitlines()
  1101. if len(lines) < 10:
  1102. return output
  1103. counts = {}
  1104. samples = {}
  1105. summary_lines = []
  1106. total_findings = 0
  1107. files_with_findings = set()
  1108. for line in lines:
  1109. stripped = line.strip()
  1110. if not stripped:
  1111. continue
  1112. matched_code = None
  1113. for pat in _LINT_CODE_PATTERNS:
  1114. # Walk every match of this pattern on the line so a blocked
  1115. # candidate (e.g. CVE2024) does not shadow a real lint code
  1116. # (e.g. E501) that appears later in the same line.
  1117. for m in pat.finditer(stripped):
  1118. candidate = m.group(1)
  1119. if candidate.startswith(_LINT_CODE_BLOCKLIST_PREFIXES):
  1120. continue
  1121. matched_code = candidate
  1122. break
  1123. if matched_code:
  1124. break
  1125. if matched_code:
  1126. counts[matched_code] = counts.get(matched_code, 0) + 1
  1127. # Capture the file this finding is in (path:line:col: CODE format) so
  1128. # "does file X have lint errors?" is answerable, not just top-5 codes.
  1129. # Grab everything before the first :LINE[:COL] group -- this keeps
  1130. # Windows drive paths (C:\...) and paths containing spaces, which a
  1131. # plain split(":")[0] would truncate to the drive letter or a prose
  1132. # word, silently dropping the file from a list that reads as complete.
  1133. fm = re.match(r"^(.*?):\d+(?::\d+)?[:\s]", stripped)
  1134. head = fm.group(1).strip() if fm else ""
  1135. if head and ("/" in head or "\\" in head or "." in head):
  1136. files_with_findings.add(head)
  1137. if matched_code not in samples:
  1138. sample = stripped
  1139. if len(sample) > 140:
  1140. sample = sample[:137] + "..."
  1141. samples[matched_code] = sample
  1142. total_findings += 1
  1143. else:
  1144. # Trailing summary context: "Found 6 errors.", "✖ 12 problems", etc.
  1145. low = stripped.lower()
  1146. if any(tok in low for tok in ("found", "error", "warning", "problem",
  1147. "clean", "passed", "failed", "checked")):
  1148. summary_lines.append(stripped)
  1149. if total_findings < 5:
  1150. return output # not worth the complexity for short lint runs
  1151. parts = [f"{total_findings} findings across {len(counts)} rule codes"]
  1152. ranked = sorted(counts.items(), key=lambda kv: (-kv[1], kv[0]))
  1153. for code, cnt in ranked[:5]:
  1154. sample = samples.get(code, "")
  1155. parts.append(f" {code} x{cnt}: {sample}")
  1156. if len(ranked) > 5:
  1157. tail_count = sum(c for _, c in ranked[5:])
  1158. parts.append(f" ... {len(ranked) - 5} other codes ({tail_count} findings)")
  1159. if files_with_findings:
  1160. flist = sorted(files_with_findings)
  1161. # Deliberately NOT an "all files listed" claim: eslint's grouped format
  1162. # carries no path on finding lines, so this list can under-count. Never
  1163. # let the agent infer a file is clean just because it is absent here.
  1164. parts.append(f"Files with findings (>= {len(flist)}, parsed from finding "
  1165. f"lines, may under-count): {', '.join(flist)}")
  1166. if summary_lines:
  1167. parts.append("")
  1168. for line in summary_lines[-3:]:
  1169. parts.append(line)
  1170. return "\n".join(parts)
  1171. # ---------------------------------------------------------------------------
  1172. # Pattern dispatch
  1173. # ---------------------------------------------------------------------------
  1174. _GIT_LOG_SHAPE_FLAGS = ("--oneline", "--format", "--pretty", "--porcelain", "--null",
  1175. "--name-status", "--name-only", "--numstat", "--stat", "--shortstat",
  1176. "--dirstat", "--summary", "--patch", "--graph", "--raw", "--cc",
  1177. "--diff", "--show-signature", "--abbrev-commit", "--walk-reflogs",
  1178. "--left-right", "--cherry", "-L")
  1179. _CONTENT_SHAPED_PATTERNS = frozenset({"find", "git_log"})
  1180. _PIPE_TRIMMERS = frozenset({"head", "tail", "sort"})
  1181. _GIT_LOG_PATH_SEP = "--"
  1182. _GIT_LOG_SHAPE_LETTERS = frozenset("zpuc")
  1183. _GIT_LOG_SHAPE_EXACT = frozenset({"-p", "-u", "-z", "-c", "-m", "-s"})
  1184. _FIND_FPRINT_PREFIX = "-fprint"
  1185. _FIND_RAW_ARGS = frozenset({"-print0", "-printf", "-exec", "-execdir", "-ok", "-okdir",
  1186. "-delete", "-ls", "-fls"})
  1187. def _detect_pattern(command_str):
  1188. """Detect which compression pattern to use based on command."""
  1189. # U9: user exclude runs BEFORE built-in detection so a user can remove a
  1190. # built-in command from compression (returns None = raw passthrough).
  1191. # User add runs AFTER built-in detection (at the bottom) so it only
  1192. # extends, never replaces a built-in handler.
  1193. try:
  1194. from command_filters import get_effective_filters
  1195. eff = get_effective_filters()
  1196. if eff.is_user_excluded(command_str):
  1197. return None
  1198. except Exception:
  1199. pass
  1200. try:
  1201. tokens = shlex.split(command_str)
  1202. except ValueError:
  1203. return None
  1204. if not tokens:
  1205. return None
  1206. # Pipelines: dispatch on the stage that DEFINES the output shape. The last
  1207. # stage usually does -- `cat f | grep X` produces grep output and must get
  1208. # the aggressive search summarizer, exactly like a bare `grep X f`, not the
  1209. # timid generic path. But a terminal TRIMMER (`head`/`tail`/`sort`) has no
  1210. # handler of its own; there we fall back to the FIRST stage, which still
  1211. # defines the shape (`du | sort | head` -> du). So: try the last stage, then
  1212. # the first. shlex keeps a quoted pipe (`grep 'a|b'`) inside its token, so
  1213. # only a real stage-separator `|` splits here. Only reached for read-only
  1214. # pipelines (the PostToolUse hook gates on is_read_only_pipeline first).
  1215. if "|&" in tokens or ("|" not in tokens and any("|" in t for t in tokens)):
  1216. # `|&` is a pipe too, and `f|sed` is one when written without spaces.
  1217. # Re-tokenize with punctuation split off; quoted `|` stays inside its token.
  1218. try:
  1219. lexer = shlex.shlex(command_str, posix=True, punctuation_chars=True)
  1220. lexer.whitespace_split = True
  1221. punct = list(lexer)
  1222. if "|" in punct or "|&" in punct:
  1223. tokens = ["|" if t == "|&" else t for t in punct]
  1224. except ValueError:
  1225. return None
  1226. if "|" in tokens:
  1227. pipes = [i for i, t in enumerate(tokens) if t == "|"]
  1228. last_stage = tokens[pipes[-1] + 1:]
  1229. first_stage = tokens[:pipes[0]]
  1230. later_stages, cur_stage = [], []
  1231. for t in tokens:
  1232. if t == "|":
  1233. later_stages.append(cur_stage)
  1234. cur_stage = []
  1235. else:
  1236. cur_stage.append(t)
  1237. later_stages.append(cur_stage)
  1238. first_p = _detect_pattern(" ".join(shlex.quote(t) for t in first_stage)) if first_stage else None
  1239. if first_p in _CONTENT_SHAPED_PATTERNS and all(
  1240. next((t for t in st if "=" not in t), "") in _PIPE_TRIMMERS
  1241. for st in later_stages[1:]):
  1242. return first_p # `| head`, `| tail`, `| sort` keep the list shape
  1243. for stage in (last_stage, first_stage):
  1244. if stage:
  1245. p = _detect_pattern(" ".join(shlex.quote(t) for t in stage))
  1246. if p is not None:
  1247. if stage is first_stage and p in _CONTENT_SHAPED_PATTERNS:
  1248. # find / git log output is summarized only when every
  1249. # later stage merely trims or orders it. cut, uniq,
  1250. # xargs, awk, sed, wc, stat ... change the content, so
  1251. # the result is no longer a path list / commit log.
  1252. stages, cur_stage = [], []
  1253. for t in tokens:
  1254. if t == "|":
  1255. stages.append(cur_stage)
  1256. cur_stage = []
  1257. else:
  1258. cur_stage.append(t)
  1259. stages.append(cur_stage)
  1260. for later in stages[1:]:
  1261. name = next((t for t in later if "=" not in t), "")
  1262. if name not in _PIPE_TRIMMERS:
  1263. return "find_raw" if p.startswith("find") else "git_log_basic"
  1264. return p
  1265. return None
  1266. # Strip leading env vars
  1267. cmd_start = 0
  1268. while cmd_start < len(tokens) and "=" in tokens[cmd_start]:
  1269. cmd_start += 1
  1270. if cmd_start >= len(tokens):
  1271. return None
  1272. cmd = tokens[cmd_start]
  1273. subcmd = tokens[cmd_start + 1] if cmd_start + 1 < len(tokens) else ""
  1274. if cmd == "git":
  1275. if subcmd in ("status",):
  1276. # Only the default human-readable format is parseable. Any
  1277. # flag that switches to machine-readable output (--porcelain
  1278. # v1/v2, --short/-s, -z) routes to raw so the handler does not
  1279. # swallow file lists it cannot recognise.
  1280. git_args = tokens[cmd_start + 2:]
  1281. for arg in git_args:
  1282. if arg.startswith("--porcelain") or arg.startswith("--short"):
  1283. return None
  1284. # Short-flag clusters (-s, -sb, -bs, -z, -sz): any
  1285. # cluster containing s or z changes the output format.
  1286. if arg.startswith("-") and not arg.startswith("--"):
  1287. if "s" in arg[1:] or "z" in arg[1:]:
  1288. return None
  1289. return "git_status"
  1290. elif subcmd in ("log",):
  1291. # Only the default human-readable format is summarized. Anything
  1292. # that switches the output shape (custom/one-line formats, NUL
  1293. # separation, porcelain, name/stat listings, patches, graph) keeps
  1294. # the light cleanup so no machine-readable line is rewritten.
  1295. for arg in tokens[cmd_start + 2:]:
  1296. if arg == _GIT_LOG_PATH_SEP:
  1297. break
  1298. if arg.startswith(_GIT_LOG_SHAPE_FLAGS) or arg in _GIT_LOG_SHAPE_EXACT:
  1299. return "git_log_basic"
  1300. if arg.startswith("-") and not arg.startswith("--") and (
  1301. set(arg[1:]) & _GIT_LOG_SHAPE_LETTERS):
  1302. return "git_log_basic"
  1303. return "git_log"
  1304. elif subcmd in ("diff", "show"):
  1305. return "git_diff"
  1306. elif subcmd in ("branch",):
  1307. # `git branch` lists branches — route through the list handler
  1308. # so long branch lists truncate with a summary.
  1309. return "list"
  1310. elif cmd in ("pytest", "py.test") or (cmd in ("python", "python3") and subcmd == "-m" and
  1311. cmd_start + 2 < len(tokens) and tokens[cmd_start + 2] == "pytest"):
  1312. return "pytest"
  1313. elif cmd in ("jest", "vitest") or (cmd == "npx" and subcmd in ("jest", "vitest")):
  1314. return "jest"
  1315. elif cmd == "rspec":
  1316. return "pytest" # similar enough format
  1317. # v5.1 test-runner extensions (dispatch-only: cypress/playwright/mocha/
  1318. # karma all report in "passing/failing" terminology; the extended pytest
  1319. # handler now reads all five formats.)
  1320. elif cmd == "mocha" or (cmd == "npx" and subcmd == "mocha"):
  1321. return "pytest"
  1322. elif cmd == "karma" or (cmd == "npx" and subcmd == "karma"):
  1323. return "pytest"
  1324. elif cmd == "cypress" and subcmd == "run":
  1325. return "pytest"
  1326. elif cmd == "npx" and subcmd == "cypress":
  1327. return "pytest"
  1328. elif cmd == "playwright" and subcmd == "test":
  1329. return "pytest"
  1330. elif cmd == "npx" and subcmd == "playwright":
  1331. return "pytest"
  1332. elif cmd in ("go", "cargo") and subcmd == "test":
  1333. return "pytest"
  1334. # v5.8 test-runner extensions: unittest, tox, nox, gradle, mvn, deno, bun, ava
  1335. elif cmd in ("python", "python3") and subcmd == "-m" and cmd_start + 2 < len(tokens) and tokens[cmd_start + 2] == "unittest":
  1336. return "pytest"
  1337. elif cmd in ("tox", "nox", "nox-sessions"):
  1338. return "pytest"
  1339. elif cmd in ("gradle", "gradlew", "./gradlew") and subcmd == "test":
  1340. return "pytest"
  1341. elif cmd == "mvn" and subcmd == "test":
  1342. return "pytest"
  1343. elif cmd == "deno" and subcmd == "test":
  1344. return "pytest"
  1345. elif cmd == "bun" and subcmd == "test":
  1346. return "pytest"
  1347. elif cmd == "ava" or (cmd == "npx" and subcmd == "ava"):
  1348. return "pytest"
  1349. elif cmd == "npm" and subcmd in ("install", "ci"):
  1350. return "npm_install"
  1351. elif cmd == "npm" and subcmd == "test":
  1352. # `npm test` dispatches to whatever the project wires up, usually
  1353. # jest/vitest/mocha. Route through the pytest handler, which
  1354. # understands all of them.
  1355. return "pytest"
  1356. elif cmd in ("pip", "pip3") and subcmd == "install":
  1357. return "npm_install"
  1358. elif cmd == "cargo" and subcmd == "build":
  1359. return "npm_install"
  1360. elif cmd == "find":
  1361. # Machine-oriented or acting find (NUL/printf/exec/delete/ls) is never
  1362. # rewritten; a plain path list gets the directory-grouping summarizer.
  1363. if any(a in _FIND_RAW_ARGS or a.startswith(_FIND_FPRINT_PREFIX) for a in tokens[cmd_start + 1:]):
  1364. return "find_raw"
  1365. return "find"
  1366. elif cmd == "ls":
  1367. return "ls"
  1368. # v5.1 lint handlers
  1369. elif cmd in ("eslint", "flake8", "pylint", "shellcheck", "rubocop"):
  1370. return "lint"
  1371. elif cmd == "ruff" and subcmd == "check":
  1372. return "lint"
  1373. elif cmd == "biome" and subcmd == "lint":
  1374. return "lint"
  1375. elif cmd == "golangci-lint" and subcmd == "run":
  1376. return "lint"
  1377. # v5.1 logs handler (read-only log inspection). `cat` is intentionally
  1378. # NOT whitelisted — only tail and journalctl, which are log-specific.
  1379. elif cmd in ("tail", "journalctl"):
  1380. return "logs"
  1381. # v5.1 tree handler
  1382. elif cmd == "tree":
  1383. return "tree"
  1384. # v5.1 progress handler (docker build, docker pull)
  1385. elif cmd == "docker" and subcmd in ("build", "pull"):
  1386. return "progress"
  1387. # v5.1 build handler
  1388. elif cmd in ("tsc", "webpack", "esbuild"):
  1389. return "build"
  1390. elif cmd in ("vite", "next") and subcmd == "build":
  1391. return "build"
  1392. elif cmd == "go" and subcmd == "build":
  1393. return "build"
  1394. # v5.1 list handler
  1395. elif cmd in ("pip", "pip3") and subcmd == "list":
  1396. return "list"
  1397. elif cmd == "npm" and subcmd == "ls":
  1398. return "list"
  1399. elif cmd == "pnpm" and subcmd == "list":
  1400. return "list"
  1401. elif cmd == "docker" and subcmd == "ps":
  1402. return "list"
  1403. elif cmd == "brew" and subcmd == "list":
  1404. return "list"
  1405. elif cmd == "sqlite3":
  1406. return "sqlite3"
  1407. elif cmd in ("wc", "du", "df"):
  1408. return "disk_stats"
  1409. elif cmd == "printenv":
  1410. return "list"
  1411. elif cmd == "docker" and subcmd in ("exec", "logs", "inspect"):
  1412. if subcmd == "logs":
  1413. return "logs"
  1414. return "docker_output"
  1415. elif cmd == "kubectl" and subcmd in ("get", "describe", "logs"):
  1416. if subcmd == "logs":
  1417. return "logs"
  1418. return "k8s"
  1419. elif cmd == "kubectl" and subcmd in ("top", "events"):
  1420. return "k8s"
  1421. elif cmd in ("gcloud", "aws", "az"):
  1422. return "cloud_cli"
  1423. elif cmd in ("jq", "yq"):
  1424. return "json"
  1425. elif cmd in ("python", "python3") and subcmd == "-m" and cmd_start + 2 < len(tokens) and tokens[cmd_start + 2] == "json.tool":
  1426. return "json"
  1427. elif cmd in ("csvtool", "mlr", "csvcut"):
  1428. return "csv"
  1429. elif cmd in ("node", "deno", "bun") and any(".json" in arg for arg in tokens[cmd_start + 1:]):
  1430. return "json"
  1431. # v5.9 search results handler (grep/ripgrep output)
  1432. elif cmd in ("grep", "rg", "ag", "ack"):
  1433. return "search_results"
  1434. # U9: user TOML command filters (add only — exclude already ran at the
  1435. # top). Built-in detection ran first, so user add only extends, never
  1436. # replaces a built-in handler for the same command. The _is_safe_add gate
  1437. # already ran in the loader, so add entries are read-only and
  1438. # handler-validated.
  1439. try:
  1440. add_entry = eff.find_user_add(command_str)
  1441. if add_entry is not None:
  1442. return add_entry.handler
  1443. except Exception:
  1444. pass
  1445. return None
  1446. def _compress_sqlite3(output):
  1447. """Compress sqlite3 query output: truncate large result sets."""
  1448. lines = output.strip().splitlines()
  1449. if len(lines) < 30:
  1450. return output
  1451. header = lines[:2]
  1452. data = lines[2:22]
  1453. result = header + data
  1454. result.append(f"... ({len(lines) - 22} more rows, {len(lines)} total)")
  1455. return "\n".join(result)
  1456. def _compress_disk_stats(output):
  1457. """Compress du/df/wc output: keep header + totals."""
  1458. lines = output.strip().splitlines()
  1459. if len(lines) < 20:
  1460. return output
  1461. head = lines[:3]
  1462. tail = lines[-5:]
  1463. kept = len(head) + len(tail)
  1464. result = head
  1465. result.append(f"... ({len(lines) - kept} entries omitted)")
  1466. result.extend(tail)
  1467. return "\n".join(result)
  1468. def _compress_docker_output(output):
  1469. """Compress docker exec/logs/inspect output."""
  1470. stripped = output.strip()
  1471. if stripped.startswith("[") or stripped.startswith("{"):
  1472. try:
  1473. data = json.loads(stripped[:200_000])
  1474. if isinstance(data, list) and len(data) > 0:
  1475. first_preview = json.dumps(data[0], indent=2)[:500]
  1476. return f"[{len(data)} items, first:\n{first_preview}\n...]"
  1477. elif isinstance(data, dict):
  1478. keys = list(data.keys())[:10]
  1479. return f"Object with {len(data)} keys: {', '.join(keys)}"
  1480. except (json.JSONDecodeError, RecursionError, TypeError):
  1481. pass
  1482. return _compress_logs(output)
  1483. # ---------------------------------------------------------------------------
  1484. # JSON / CSV / stack-trace / k8s / cloud-CLI handlers
  1485. # ---------------------------------------------------------------------------
  1486. _JSON_MIN_CHARS = 500
  1487. _JSON_MAX_PARSE = 500_000
  1488. _JSON_CELL_CAP = 100
  1489. _JSON_MAX_COLS = 40
  1490. _JSON_MIN_ROWS = 3
  1491. _JSON_TOTAL_CAP = 8192
  1492. _JSON_MIN_REDUCTION = 0.40
  1493. def _compress_json(output):
  1494. """Compress JSON output: columnar for arrays-of-dicts, key-summary for objects."""
  1495. stripped = output.strip()
  1496. if len(stripped) < _JSON_MIN_CHARS:
  1497. return output
  1498. if not (stripped.startswith("[") or stripped.startswith("{")):
  1499. return output
  1500. try:
  1501. data = json.loads(stripped[:_JSON_MAX_PARSE])
  1502. except (json.JSONDecodeError, RecursionError, TypeError):
  1503. return output
  1504. if isinstance(data, list):
  1505. dict_rows = [r for r in data if isinstance(r, dict)]
  1506. if len(dict_rows) >= max(_JSON_MIN_ROWS, int(len(data) * 0.8)):
  1507. cols: list[str] = []
  1508. seen: set[str] = set()
  1509. for row in dict_rows:
  1510. for k in row.keys():
  1511. if k not in seen:
  1512. seen.add(k)
  1513. cols.append(str(k))
  1514. if cols and len(cols) <= _JSON_MAX_COLS:
  1515. def _cell(v) -> str:
  1516. s = v if isinstance(v, str) else json.dumps(v, ensure_ascii=False)
  1517. s = s.replace("\n", " ").replace("\t", " ")
  1518. return s[:_JSON_CELL_CAP] + ("…" if len(s) > _JSON_CELL_CAP else "")
  1519. lines = [
  1520. f"JSON array ({len(data)} items), columnar (schema-stripped, values preserved):",
  1521. "cols: " + ", ".join(cols),
  1522. ]
  1523. for row in data:
  1524. if isinstance(row, dict):
  1525. lines.append("- " + " | ".join(_cell(row.get(c, "")) for c in cols))
  1526. else:
  1527. lines.append("- " + _cell(row))
  1528. result = "\n".join(lines)
  1529. if len(result) <= _JSON_TOTAL_CAP and len(result) < len(stripped) * (1.0 - _JSON_MIN_REDUCTION):
  1530. return result
  1531. if len(data) > 20:
  1532. first = json.dumps(data[0], indent=2, ensure_ascii=False)[:500]
  1533. return f"JSON array ({len(data)} items), first:\n{first}\n... ({len(data) - 1} more items)"
  1534. return output
  1535. if isinstance(data, dict):
  1536. keys = list(data.keys())[:50]
  1537. result = f"JSON object ({len(data)} keys): {', '.join(keys)}"
  1538. if len(result) < len(stripped) * 0.9:
  1539. return result
  1540. return output
  1541. return output
  1542. _CSV_MIN_LINES = 20
  1543. _CSV_MAX_COLS = 40
  1544. _CSV_CELL_CAP = 80
  1545. _CSV_TOTAL_CAP = 8192
  1546. _CSV_PREVIEW_ROWS = 15
  1547. def _compress_csv(output):
  1548. """Compress CSV/TSV output: keep header + first rows, summarize the rest."""
  1549. lines = output.strip().splitlines()
  1550. if len(lines) < _CSV_MIN_LINES:
  1551. return output
  1552. delimiter = "\t" if "\t" in lines[0] else ","
  1553. try:
  1554. import csv as _csv
  1555. reader = _csv.reader(lines, delimiter=delimiter)
  1556. header = next(reader, None)
  1557. if not header:
  1558. return output
  1559. cols = header[:_CSV_MAX_COLS]
  1560. data_rows = list(reader)
  1561. total = len(data_rows)
  1562. if total < _CSV_MIN_LINES - 1:
  1563. return output
  1564. preview = data_rows[:_CSV_PREVIEW_ROWS]
  1565. lines_out = [delimiter.join(header)]
  1566. for row in preview:
  1567. row = row[:_CSV_MAX_COLS]
  1568. lines_out.append(delimiter.join(
  1569. (c[:_CSV_CELL_CAP] + "…" if len(c) > _CSV_CELL_CAP else c) for c in row
  1570. ))
  1571. lines_out.append(f"... ({total - len(preview)} more rows, {total} total)")
  1572. result = "\n".join(lines_out)
  1573. if len(result) > _CSV_TOTAL_CAP:
  1574. lines_out = lines_out[:2] + [lines_out[-1]]
  1575. result = "\n".join(lines_out)
  1576. return result
  1577. except Exception:
  1578. return output
  1579. _STACK_TRACE_FRAME_RE = re.compile(r'^\s+(?:at\s+)?(.+?)\s+')
  1580. _STACK_TRACE_FILE_RE = re.compile(r'\((.+?):(\d+)(?::(\d+))?\)')
  1581. _STACK_TRACE_MIN_LINES = 15
  1582. _STACK_TRACE_MAX_FRAMES = 20
  1583. _STACK_TRACE_MARKERS = re.compile(
  1584. r'(?:Traceback \(most recent call last\)|'
  1585. r'^\s+at\s+[\w.$]+\(|'
  1586. r'Caused by:|'
  1587. r'Exception in thread|'
  1588. r'^\s+at\s+[\w.]+\([^)]+:\d+\))',
  1589. re.MULTILINE
  1590. )
  1591. def _looks_like_stack_trace(text: str) -> bool:
  1592. """Check if output contains a stack trace worth compressing."""
  1593. lines = text.splitlines()
  1594. if len(lines) < _STACK_TRACE_MIN_LINES:
  1595. return False
  1596. return bool(_STACK_TRACE_MARKERS.search(text))
  1597. def _compress_stack_trace(output):
  1598. """Compress stack traces: keep exception message + top N frames, summarize the rest."""
  1599. lines = output.splitlines()
  1600. if len(lines) < _STACK_TRACE_MIN_LINES:
  1601. return output
  1602. exception_lines: list[str] = []
  1603. frames: list[str] = []
  1604. in_traceback = False
  1605. for line in lines:
  1606. stripped = line.strip()
  1607. # Detect start of traceback section
  1608. if stripped.startswith("Traceback") or stripped.startswith("Exception in thread"):
  1609. in_traceback = True
  1610. exception_lines.append(line)
  1611. continue
  1612. # Lines after the traceback section that look like the final exception
  1613. # (e.g. "ValueError: ...", "java.lang.NPE")
  1614. if in_traceback and not stripped.startswith("File ") and not stripped.startswith("at ") and not _STACK_TRACE_FRAME_RE.match(line):
  1615. # This is the exception message line (after all frames)
  1616. if stripped and not stripped.startswith("..."):
  1617. exception_lines.append(line)
  1618. continue
  1619. if in_traceback and (stripped.startswith("File ") or stripped.startswith("at ") or _STACK_TRACE_FRAME_RE.match(line)):
  1620. frames.append(line)
  1621. elif not in_traceback:
  1622. exception_lines.append(line)
  1623. else:
  1624. frames.append(line)
  1625. if not frames:
  1626. return output
  1627. kept_frames = frames[:_STACK_TRACE_MAX_FRAMES]
  1628. omitted = len(frames) - len(kept_frames)
  1629. result_lines = list(exception_lines)
  1630. result_lines.extend(kept_frames)
  1631. if omitted > 0:
  1632. result_lines.append(f"... ({omitted} stack frames omitted)")
  1633. result = "\n".join(result_lines)
  1634. if len(result) >= len(output) * 0.9:
  1635. return output
  1636. return result
  1637. _K8S_MIN_LINES = 20
  1638. _K8S_MAX_COLS = 40
  1639. _K8S_CELL_CAP = 60
  1640. _K8S_PREVIEW_ROWS = 15
  1641. def _compress_k8s(output):
  1642. """Compress kubectl get/describe/top/events output: keep header + preview rows."""
  1643. lines = output.strip().splitlines()
  1644. if len(lines) < _K8S_MIN_LINES:
  1645. return output
  1646. if lines[0].startswith("NAME") or lines[0].startswith("NAMESPACE"):
  1647. header = lines[0].split()
  1648. data_lines = lines[1:]
  1649. total = len(data_lines)
  1650. if total < _K8S_MIN_LINES - 1:
  1651. return output
  1652. preview = data_lines[:_K8S_PREVIEW_ROWS]
  1653. result_lines = [lines[0]]
  1654. for row in preview:
  1655. result_lines.append(row)
  1656. rest = data_lines[_K8S_PREVIEW_ROWS:]
  1657. # Keep the NAME (first column) of every dropped row so "is pod X running?"
  1658. # is answerable; a name not listed here genuinely does not exist. Without
  1659. # this the agent infers false absence from the truncated preview.
  1660. rest_names = [r.split()[0] for r in rest if r.split()]
  1661. if rest_names:
  1662. result_lines.append(
  1663. f"... plus {len(rest)} more rows ({total} total; names only, "
  1664. f"a name not listed has no matching resource): {', '.join(rest_names)}")
  1665. result = "\n".join(result_lines)
  1666. return result
  1667. return _compress_logs(output)
  1668. _CLOUD_CLI_MIN_CHARS = 1000
  1669. _CLOUD_CLI_HEAD = 15
  1670. _CLOUD_CLI_TAIL = 10
  1671. def _compress_cloud_cli(output):
  1672. """Compress gcloud/aws/az output: keep head + tail, preserve important lines."""
  1673. lines = output.strip().splitlines()
  1674. if len(lines) < _CLOUD_CLI_MIN_CHARS // 40:
  1675. return output
  1676. collapsed: list[str] = []
  1677. run_val = None
  1678. run_count = 0
  1679. for ln in lines:
  1680. if ln == run_val:
  1681. run_count += 1
  1682. else:
  1683. if run_val is not None:
  1684. collapsed.append(f"{run_val} (x{run_count})" if run_count > 1 else run_val)
  1685. run_val = ln
  1686. run_count = 1
  1687. if run_val is not None:
  1688. collapsed.append(f"{run_val} (x{run_count})" if run_count > 1 else run_val)
  1689. if len(collapsed) <= _CLOUD_CLI_HEAD + _CLOUD_CLI_TAIL:
  1690. return "\n".join(collapsed)
  1691. head = collapsed[:_CLOUD_CLI_HEAD]
  1692. tail = collapsed[len(collapsed) - _CLOUD_CLI_TAIL:]
  1693. middle = collapsed[_CLOUD_CLI_HEAD:len(collapsed) - _CLOUD_CLI_TAIL]
  1694. important = [ln for ln in middle if _GENERIC_IMPORTANT.search(ln)][:_CLOUD_CLI_HEAD * 5]
  1695. omitted = len(middle) - len(important)
  1696. out = list(head)
  1697. if important:
  1698. out.append(f"... {omitted} lines omitted (kept {len(important)} error/warning lines) ...")
  1699. out.extend(important)
  1700. else:
  1701. out.append(f"... {omitted} lines omitted ...")
  1702. out.extend(tail)
  1703. return "\n".join(out)
  1704. _SEARCH_FILE_LINE_RE = re.compile(r'^(.+?):(\d+)[:](.*)$')
  1705. def _compress_search_results(output):
  1706. """Compress grep/ripgrep/ag/ack output: group by file, keep top matches per file.
  1707. Search output is typically `file:line:content` (or `file:line:content` with
  1708. color codes). For large result sets, we group by file, keep the first N
  1709. matches per file, and summarize the rest. Files with only 1-2 matches are
  1710. kept verbatim. The goal is the same as a skilled human skimming search
  1711. results: see which files have hits, see a few representative lines, skip
  1712. the rest.
  1713. """
  1714. if not output:
  1715. return output
  1716. lines = output.strip().splitlines()
  1717. if len(lines) < 30:
  1718. return output
  1719. # Strip ANSI color codes from grep/rg output
  1720. cleaned = _strip_ansi(output)
  1721. lines = cleaned.strip().splitlines()
  1722. if len(lines) < 30:
  1723. return output
  1724. # Group lines by file (non-greedy up to :digits: to handle colons in filenames)
  1725. files: dict[str, list[str]] = {}
  1726. no_file: list[str] = []
  1727. for ln in lines:
  1728. m = _SEARCH_FILE_LINE_RE.match(ln)
  1729. if m:
  1730. fname = m.group(1)
  1731. files.setdefault(fname, []).append(ln)
  1732. else:
  1733. no_file.append(ln)
  1734. if not files:
  1735. return output # not file:line format, don't compress
  1736. from bash_search_results import render_search_results
  1737. return render_search_results(files, no_file)
  1738. _PATTERN_HANDLERS = {
  1739. "git_status": _compress_git_status,
  1740. "git_log": _compress_git_log,
  1741. "git_log_basic": _compress_git_log_basic,
  1742. "find": _compress_find,
  1743. "find_raw": lambda output: output,
  1744. "git_diff": _compress_git_diff,
  1745. "pytest": _compress_pytest,
  1746. "jest": _compress_jest,
  1747. "npm_install": _compress_npm_install,
  1748. "ls": _compress_ls,
  1749. "lint": _compress_lint,
  1750. "logs": _compress_logs,
  1751. "tree": _compress_tree,
  1752. "progress": _compress_progress,
  1753. "list": _compress_list,
  1754. "build": _compress_build,
  1755. "sqlite3": _compress_sqlite3,
  1756. "disk_stats": _compress_disk_stats,
  1757. "docker_output": _compress_docker_output,
  1758. "json": _compress_json,
  1759. "csv": _compress_csv,
  1760. "stack_trace": _compress_stack_trace,
  1761. "k8s": _compress_k8s,
  1762. "cloud_cli": _compress_cloud_cli,
  1763. "search_results": _compress_search_results,
  1764. }
  1765. # Maps _detect_pattern() output to the feature name stored in compression_events.
  1766. _COMPRESS_FEATURE_MAP = {
  1767. "git_status": "bash_compress_git",
  1768. "git_log": "bash_compress_git",
  1769. "git_log_basic": "bash_compress_git",
  1770. "find": "bash_compress_ls",
  1771. "find_raw": "bash_compress_ls",
  1772. "git_diff": "bash_compress_git",
  1773. "pytest": "bash_compress_pytest",
  1774. "jest": "bash_compress_jest",
  1775. "npm_install": "bash_compress_npm",
  1776. "ls": "bash_compress_ls",
  1777. "lint": "bash_compress_lint",
  1778. "logs": "bash_compress_logs",
  1779. "tree": "bash_compress_tree",
  1780. "progress": "bash_compress_progress",
  1781. "list": "bash_compress_list",
  1782. "build": "bash_compress_build",
  1783. "sqlite3": "bash_compress_build",
  1784. "disk_stats": "bash_compress_list",
  1785. "docker_output": "bash_compress_build",
  1786. "json": "bash_compress_json",
  1787. "csv": "bash_compress_csv",
  1788. "stack_trace": "bash_compress_stack",
  1789. "k8s": "bash_compress_k8s",
  1790. "cloud_cli": "bash_compress_cloud",
  1791. "search_results": "bash_compress_search",
  1792. }
  1793. # ---------------------------------------------------------------------------
  1794. # Main entry point
  1795. # ---------------------------------------------------------------------------
  1796. _GENERIC_MIN_CHARS = 2000 # only compress reasonably large unmatched output
  1797. _GENERIC_HEAD = 20 # lines kept from the top
  1798. _GENERIC_TAIL = 20 # lines kept from the bottom
  1799. _GENERIC_IMPORTANT = re.compile(
  1800. r"\b(error|errors|warn|warning|fail|failed|failure|exception|traceback|"
  1801. r"fatal|panic|denied|refused|timeout|timed out)\b",
  1802. re.IGNORECASE,
  1803. )
  1804. def _compress_generic(text):
  1805. """Generic structural/log compression for output matching no known pattern.
  1806. Collapses consecutive duplicate lines, keeps head + tail, and preserves
  1807. every error/warning line anywhere in the body. Deterministic, fail-open
  1808. (caller wraps in try/except), and credential lines are independently
  1809. re-injected by the PRE-compression scan in compress().
  1810. """
  1811. lines = text.splitlines()
  1812. # Collapse runs of identical consecutive lines into "<line> (xN)".
  1813. collapsed: list[str] = []
  1814. run_val = None
  1815. run_count = 0
  1816. def _flush_run():
  1817. if run_val is None:
  1818. return
  1819. if run_count > 1:
  1820. collapsed.append(f"{run_val} (x{run_count})")
  1821. else:
  1822. collapsed.append(run_val)
  1823. for ln in lines:
  1824. if ln == run_val:
  1825. run_count += 1
  1826. else:
  1827. _flush_run()
  1828. run_val = ln
  1829. run_count = 1
  1830. _flush_run()
  1831. if len(collapsed) <= _GENERIC_HEAD + _GENERIC_TAIL:
  1832. return "\n".join(collapsed)
  1833. head = collapsed[:_GENERIC_HEAD]
  1834. tail = collapsed[len(collapsed) - _GENERIC_TAIL:]
  1835. middle = collapsed[_GENERIC_HEAD:len(collapsed) - _GENERIC_TAIL]
  1836. _IMPORTANT_CAP = _GENERIC_HEAD * 10
  1837. important = [ln for ln in middle if _GENERIC_IMPORTANT.search(ln)][:_IMPORTANT_CAP]
  1838. omitted = len(middle) - len(important)
  1839. out: list[str] = list(head)
  1840. if important:
  1841. out.append(f"... {omitted} lines omitted (kept {len(important)} error/warning lines) ...")
  1842. out.extend(important)
  1843. else:
  1844. out.append(f"... {omitted} lines omitted ...")
  1845. out.extend(tail)
  1846. return "\n".join(out)
  1847. def _baseline_visible_chars(raw_chars):
  1848. """Chars Claude Code would show for a raw tool result of ``raw_chars`` chars.
  1849. UNIT C baseline: results at or under CC's persisted-output threshold are
  1850. shown in full; larger results are collapsed to the ~2.2KB stub (CC 2.1.247
  1851. behavior -- see the CC_PERSISTED_OUTPUT_* constants above).
  1852. """
  1853. if raw_chars <= CC_PERSISTED_OUTPUT_THRESHOLD_CHARS:
  1854. return raw_chars
  1855. return CC_PERSISTED_OUTPUT_STUB_CHARS
  1856. def _unicode_safe_head(text, limit):
  1857. """Keep the cap without splitting marks, format controls or ZWJ runs.
  1858. Walk only the bounded visible prefix, never an unbounded combining tail.
  1859. Dropping the whole boundary cluster preserves its meaning in the archive
  1860. and prevents an over-cap preview from hiding the recovery pointer.
  1861. This is a conservative boundary guard, not a full Unicode segmenter.
  1862. """
  1863. cut = min(max(0, limit), len(text))
  1864. if cut == len(text):
  1865. return text
  1866. safe = 0
  1867. for boundary in range(1, cut + 1):
  1868. following = text[boundary]
  1869. code = ord(following)
  1870. attaches = (unicodedata.category(following) in ('Mn', 'Mc', 'Me', 'Cf')
  1871. or 0x1F3FB <= code <= 0x1F3FF)
  1872. if not attaches and text[boundary - 1] != '\u200d':
  1873. safe = boundary
  1874. return text[:safe]
  1875. def _enforce_baseline_invariant(text, raw_output, archive_key):
  1876. """Never show the model MORE chars than the CC baseline would.
  1877. Claude Code 2.1.247 already truncates any tool result larger than
  1878. ~30KB down to a ~2.2KB "<persisted-output>" stub on its own, so the
  1879. baseline the model sees for such outputs is only the stub -- NOT the
  1880. raw result. When our compressed preview + archive pointer would exceed
  1881. that baseline, shrink the preview to fit UNDER the stub while keeping
  1882. the pointer intact, so the full original stays retrievable via
  1883. `expand <key>` and CC does not re-stub (and thereby swallow) our output.
  1884. Three branches:
  1885. * text already <= baseline -> unchanged (the common case).
  1886. * text IS the raw output -> left untouched: CC truncates raw output
  1887. at its own layer to the same stub it would show without Token
  1888. Optimizer, i.e. exactly the baseline (defer, fail-open, no data
  1889. loss beyond what CC itself does).
  1890. * lossy preview (pointer failed to build) -> keep the most useful
  1891. head of the preview, capped at the baseline.
  1892. """
  1893. cap = _baseline_visible_chars(len(raw_output))
  1894. if len(text) <= cap:
  1895. return text
  1896. if text == raw_output:
  1897. return text # CC stubs raw output itself; that IS the baseline
  1898. if archive_key:
  1899. try:
  1900. from archive_result import build_archive_pointer
  1901. pointer = build_archive_pointer("", len(raw_output), archive_key)
  1902. if pointer and len(pointer) <= cap:
  1903. # text is preview + "\n\n" + pointer; text > cap guarantees
  1904. # text[:cap - len(pointer)] cuts inside the preview, never
  1905. # into the pointer region, so no duplicated pointer bytes.
  1906. return _unicode_safe_head(text, cap - len(pointer)) + pointer
  1907. except Exception:
  1908. pass # fall through to plain head-cap
  1909. return _unicode_safe_head(text, cap)
  1910. def compress(command_str, raw_output, returncode=0, stderr=""):
  1911. """Compress CLI output based on command pattern.
  1912. Returns compressed output. On any issue, returns raw output.
  1913. Token preservation scan runs FIRST on raw output.
  1914. Tee-on-failure: if the command failed (non-zero exit) or printed error
  1915. patterns on stderr even with exit 0, return raw output verbatim. Never
  1916. compress failure output — the user needs the full signal to debug.
  1917. """
  1918. from post_run_classify import unsafe_compound_output, echo_boundary_preserved
  1919. if unsafe_compound_output(command_str):
  1920. return raw_output
  1921. if _looks_like_failure(returncode, stderr):
  1922. return raw_output # fail-open tee: full output, unchanged
  1923. if not raw_output or len(raw_output) < 100:
  1924. return raw_output # too small to bother
  1925. # Strip ANSI codes (always safe)
  1926. cleaned = _strip_ansi(raw_output)
  1927. # Preserve credentials, errors, and distinctive tokens before summarizing.
  1928. # Git log and find keep their own hashes and paths.
  1929. _summarizer_pattern = False
  1930. try:
  1931. _summarizer_pattern = _detect_pattern(command_str) in ("git_log", "find")
  1932. except Exception:
  1933. pass
  1934. preserved_lines = _find_preserved_lines(cleaned)
  1935. if not _summarizer_pattern:
  1936. preserved_lines = preserved_lines | _find_distinctive_lines(cleaned)
  1937. from preserved_reinject import find_priority_lines, reinject
  1938. priority_lines = find_priority_lines(cleaned, _ERROR_STDERR_PATTERNS + _FOREIGN_ERROR_PATTERNS)
  1939. preserved_lines = preserved_lines | priority_lines
  1940. # Detect pattern
  1941. pattern = _detect_pattern(command_str)
  1942. if pattern is None:
  1943. # Output-based detection: stack traces can appear from any command.
  1944. # Check before generic compression so tracebacks get frame-limited.
  1945. if _looks_like_stack_trace(cleaned):
  1946. pattern = "stack_trace"
  1947. if pattern is None:
  1948. # U4: generic structural/log compression for unmatched large output.
  1949. # Small outputs pass through; large ones get deduped + head/tail with
  1950. # all error/warning lines kept. The shared 10% gate below still applies.
  1951. if len(cleaned) < _GENERIC_MIN_CHARS:
  1952. return cleaned
  1953. try:
  1954. compressed = _compress_generic(cleaned)
  1955. except Exception:
  1956. return cleaned # fail open
  1957. else:
  1958. handler = _PATTERN_HANDLERS.get(pattern)
  1959. if handler is None:
  1960. return cleaned
  1961. try:
  1962. if pattern == "find":
  1963. compressed = handler(cleaned, command_str)
  1964. else:
  1965. compressed = handler(cleaned)
  1966. except Exception:
  1967. return cleaned # fail open
  1968. # Cap credential reinjection; the full original remains archived.
  1969. if preserved_lines:
  1970. compressed = reinject(compressed, cleaned, preserved_lines, priority_lines, _summarizer_pattern)
  1971. if not echo_boundary_preserved(command_str, cleaned, compressed):
  1972. return raw_output
  1973. try:
  1974. from token_estimate import estimate_tokens as _est
  1975. except Exception:
  1976. _est = lambda t: len(t.encode("utf-8", errors="replace")) // 4
  1977. original_tokens = _est(cleaned)
  1978. compressed_tokens = _est(compressed)
  1979. if original_tokens > 0 and (1.0 - compressed_tokens / original_tokens) < 0.10:
  1980. return cleaned # not worth the risk
  1981. return compressed
  1982. def _run_original(command_args):
  1983. """Execute the command uncompressed and relay its output and exit code."""
  1984. try:
  1985. result = subprocess.run(
  1986. command_args,
  1987. shell=False,
  1988. capture_output=True,
  1989. text=True,
  1990. encoding="utf-8",
  1991. errors="replace",
  1992. timeout=60,
  1993. creationflags=_NO_WINDOW,
  1994. )
  1995. except subprocess.TimeoutExpired:
  1996. sys.exit(124)
  1997. except OSError as exc:
  1998. print(f"bash_compress.py: {exc}", file=sys.stderr)
  1999. sys.exit(127)
  2000. sys.stdout.write(result.stdout or "")
  2001. sys.stdout.flush()
  2002. sys.stderr.write(result.stderr or "")
  2003. sys.stderr.flush()
  2004. sys.exit(result.returncode)
  2005. def main():
  2006. """Run a command through compression wrapper."""
  2007. if len(sys.argv) < 2:
  2008. print("Usage: bash_compress.py <command...>", file=sys.stderr)
  2009. sys.exit(1)
  2010. command_args = sys.argv[1:]
  2011. command_str = shlex.join(command_args)
  2012. # R13a: independently re-validate the argv before running anything, so a
  2013. # cached "always allow" on this wrapper's command prefix can never turn
  2014. # bash_compress.py into an unguarded command runner. Exact same whitelist +
  2015. # dangerous-character gate the Antigravity / Copilot / Claude bridges apply
  2016. # before rewriting. A direct `bash_compress.py <anything-not-whitelisted>`
  2017. # (or a hostile `rm -rf /`, `curl | sh`, etc.) is refused and spawns
    • Note: Quoted as data in code: Runs `rm -rf` against a root or home-directory path
    • Line 2,269 on GitHub
  2018. # nothing.
  2019. #
  2020. # The gate module is dependency-free, but if it cannot be imported (stale
  2021. # or partial payload copy) the wrapper must fail OPEN: the command runs
  2022. # uncompressed exactly as the user typed it. Refusing here would silently
  2023. # drop whitelisted commands on every platform that shares this wrapper.
  2024. try:
  2025. import bash_whitelist as _self_check
  2026. except Exception:
  2027. _self_check = None
  2028. if _self_check is None:
  2029. _run_original(command_args)
  2030. sys.exit(0)
  2031. if _self_check.has_dangerous_chars(command_str) or not _self_check.is_whitelisted(command_str):
  2032. print("bash_compress.py: command is not eligible for compression", file=sys.stderr)
  2033. sys.exit(1)
  2034. try:
  2035. result = subprocess.run(
  2036. command_args,
  2037. shell=False,
  2038. capture_output=True,
  2039. text=True,
  2040. encoding="utf-8",
  2041. errors="replace",
  2042. timeout=60,
  2043. creationflags=_NO_WINDOW,
  2044. )
  2045. stdout = result.stdout or ""
  2046. stderr = result.stderr or ""
  2047. raw_output = stdout + stderr
  2048. # Memory safety: commands like `find /`, `tree /`, or `journalctl`
  2049. # under a large time window can emit hundreds of MB through
  2050. # capture_output. Compression would duplicate that payload multiple
  2051. # times across regex scans and rebuilt strings, so above 5MB we
  2052. # emit raw and skip the compression path entirely.
  2053. MAX_COMPRESS_BYTES = 5 * 1024 * 1024
  2054. if len(raw_output.encode("utf-8", errors="replace")) > MAX_COMPRESS_BYTES:
  2055. sys.stdout.write(raw_output)
  2056. sys.stdout.flush()
  2057. sys.exit(result.returncode)
  2058. compressed = compress(
  2059. command_str,
  2060. raw_output,
  2061. returncode=result.returncode,
  2062. stderr=stderr,
  2063. )
  2064. # Progressive disclosure: archive the raw original BEFORE compression
  2065. # so the model can retrieve it via `expand <key>` if needed. Fail-open:
  2066. # if archiving fails, serve the original instead of a lossy preview.
  2067. _archive_key = None
  2068. if compressed != raw_output and len(raw_output) > 500:
  2069. try:
  2070. from archive_result import archive_entry_exists, archive_original, build_archive_pointer
  2071. _session_id = os.environ.get("CLAUDE_SESSION_ID", "")
  2072. _archive_key = hashlib.sha256(
  2073. f"{_session_id}|{command_str}|{time.time()}|{os.urandom(4).hex()}".encode("utf-8", errors="replace")
  2074. ).hexdigest()[:16]
  2075. if archive_original(raw_output, _session_id, _archive_key, "Bash") is not None:
  2076. # Re-check the entry survived any concurrent retention prune
  2077. # before we emit a pointer that would strand the model.
  2078. if archive_entry_exists(_session_id, _archive_key):
  2079. compressed = build_archive_pointer(compressed, len(raw_output), _archive_key)
  2080. else:
  2081. # Entry was pruned before we could point to it. The lossy
  2082. # `compressed` preview has no recovery path once the archive
  2083. # copy is gone, so serving it here would be silent, permanent
  2084. # data loss. raw_output is still in scope untouched -- serve
  2085. # that instead, matching the guarantee archive_result.py /
  2086. # read_cache.py already give on the same post-prune re-check
  2087. # (they fall back to the full original rather than a preview).
  2088. compressed = raw_output
  2089. _archive_key = None
  2090. else:
  2091. _archive_key = None
  2092. except Exception:
  2093. _archive_key = None
  2094. # Search matches must remain recoverable even below the archive floor
  2095. # or when the archive write fails. Other handlers keep their contracts.
  2096. if _archive_key is None and _detect_pattern(command_str) == "search_results":
  2097. compressed = raw_output
  2098. # Record savings to trends.db if compression was meaningful
  2099. try:
  2100. orig_bytes = len(raw_output.encode("utf-8", errors="replace"))
  2101. comp_bytes = len(compressed.encode("utf-8", errors="replace"))
  2102. if comp_bytes < orig_bytes * 0.9:
  2103. _pattern = _detect_pattern(command_str)
  2104. _feature = _COMPRESS_FEATURE_MAP.get(_pattern or "")
  2105. if _feature is None and _pattern is None:
  2106. _feature = "bash_generic" # U4: unmatched-output coverage
  2107. if _feature:
  2108. from measure import _log_compression_event
  2109. # C-2: command_pattern is persisted to trends.db's
  2110. # compression_events table. Redact BEFORE truncating so an
  2111. # inline secret (Bearer token, mysql -pPASSWORD,
  2112. # PGPASSWORD=... psql) never reaches disk in cleartext.
  2113. # Mirrors the fix in bash_compress_hook._log_event. When the
  2114. # redactor is unavailable or refuses (a broken custom
  2115. # pattern file fails closed), skip the event entirely —
  2116. # persisting the raw command would store what the configured
  2117. # rules exist to keep off disk.
  2118. try:
  2119. from credential_patterns import redact_credentials as _redact
  2120. _safe_pattern = _redact(command_str)[:100]
  2121. except Exception:
  2122. _safe_pattern = None
  2123. if _safe_pattern is not None:
  2124. _log_compression_event(
  2125. feature=_feature,
  2126. original_text=raw_output,
  2127. compressed_text=compressed,
  2128. session_id=os.environ.get("CLAUDE_SESSION_ID", ""),
  2129. command_pattern=_safe_pattern,
  2130. quality_preserved=True,
  2131. verified=True,
  2132. tier="measured",
  2133. )
  2134. except Exception:
  2135. pass
  2136. # UNIT C guard: the final gate on what the model sees. If our
  2137. # preview + archive pointer would exceed what Claude Code itself
  2138. # would show for this output (full raw below ~30KB, ~2.2KB stub
  2139. # above it), shrink it so Token Optimizer never loses to baseline.
  2140. # Fail-open: a guard failure must never break the pipeline.
  2141. try:
  2142. compressed = _enforce_baseline_invariant(compressed, raw_output, _archive_key)
  2143. except Exception:
  2144. pass
  2145. # Buffer completely, then write
  2146. sys.stdout.write(compressed)
  2147. sys.stdout.flush()
  2148. sys.exit(result.returncode)
  2149. except subprocess.TimeoutExpired as e:
  2150. # NEVER compress partial output on timeout
  2151. partial = ""
  2152. if e.stdout:
  2153. partial += e.stdout if isinstance(e.stdout, str) else e.stdout.decode("utf-8", errors="replace")
  2154. if e.stderr:
  2155. partial += e.stderr if isinstance(e.stderr, str) else e.stderr.decode("utf-8", errors="replace")
  2156. sys.stdout.write(partial)
  2157. sys.stdout.write("\n[TIMEOUT after 60s - output may be incomplete]\n")
  2158. sys.stdout.flush()
  2159. sys.exit(124)
  2160. except Exception as e:
  2161. # Fail open: emit the error so Claude sees it instead of empty output
  2162. sys.stderr.write(f"[bash_compress: wrapper error: {type(e).__name__}: {e}]\n")
  2163. sys.exit(1)
  2164. if __name__ == "__main__":
  2165. main()

Ratings and comments

Sign in to rate this skill.

Sign in

Free to use. If it’s helped, pass it on.