Skip to content
Back to search

dsh-tool-normalizer

> An auto-healing layer for model tool calls: it silently fixes the failures that used to cost you a full retry round-trip, and shows you the receipts.

Safety scan

A static text scan, not a guarantee. Select the badge to see what was scanned and every finding.

Checked today
Badge for your README
findsafeskills safety rating

Markdown

[![findsafeskills safety rating](/api/listings/merenguesl--dsh-tool-normalizer/badge.svg)](/listings/merenguesl--dsh-tool-normalizer)

HTML

<a href="/listings/merenguesl--dsh-tool-normalizer"><img src="/api/listings/merenguesl--dsh-tool-normalizer/badge.svg" alt="findsafeskills safety rating" /></a>

Install

i

Install instructions from the README

As written by the repo's author. findsafeskills has not verified these steps.

In DeepSeek Harness, plugins are managed per composition profile (`web`, `headless`, `tui`, etc.).

Step 1: Install Plugin into your Target Profile

Using the `dsh` CLI (or `pnpm dsh` from monorepo root):

# 1. Install into Web UI profile (Includes Settings Dashboard)
dsh plugin --profile web add dsh-tool-normalizer
# (or if running from source repository)
pnpm dsh plugin --profile web add dsh-tool-normalizer

# 2. Install into Headless automation profile
dsh plugin --profile headless add dsh-tool-normalizer

# 3. Install into TUI terminal profile
dsh plugin --profile tui add dsh-tool-normalizer

Local Development Link (Optional)

If you are developing or testing local changes:

pnpm dsh plugin --profile web add ./plugins/dsh-tool-normalizer

Step 2: Launch and Verify

# Boot Web UI mode
dsh web
# (or from source)
pnpm dsh web

Open your browser, navigate to **Settings (⚙️)** ➔ **Tool Normalizer**, and observe real-time tool execution metrics and auto-healing in action!

---

What was scanned

The files the safety scan read, as written by the repository's author and shown as plain text. Lines with a finding are marked.

README.md · README · 269 lines
  1. # dsh-tool-normalizer
  2. [![dsh-plugin](https://img.shields.io/badge/dsh--plugin-tools-blue)](https://github.com/awesome-dsh-plugin/awesome-dsh-plugin)
  3. [![License: MIT](https://img.shields.io/badge/License-MIT-green.svg)](LICENSE)
  4. [![npm version](https://img.shields.io/npm/v/dsh-tool-normalizer.svg)](https://www.npmjs.com/package/dsh-tool-normalizer)
  5. > An auto-healing layer for model tool calls: it silently fixes the failures that used to cost you a full retry round-trip, and shows you the receipts.
  6. [中文文档 (README.zh.md)](./README.zh.md)
  7. ![Tool Normalizer dashboard: 97.7% healing rate, 837 healed calls, 153.3M tokens saved](assets/dashboard-hero.png)
  8. ## Why this exists
  9. Every failed tool call costs a full model round-trip: the error comes back, the model re-reads the entire conversation context, and tries again. In a large workspace that retry resubmits ~180k input tokens — so a ~5% tool failure rate quietly inflates your token bill by more than 2× on the affected turns, and your agent visibly stumbles every few minutes.
  10. This plugin sits on the `tools/execute` pipeline and repairs the failure before it ever reaches the model: a missing `description` is filled in, a forgotten file read is performed and the edit retried, a relative path is resolved, an unrecoverable error gets an actionable hint appended. Measured across 192 real sessions (15,460 tool calls):
  11. - Surfaced error rate fell from **7.95% → 2.20%** (about **72% fewer** visible failures).
  12. - About **82%** of would-be errors were healed automatically (837 healed vs 182 residual).
  13. - Each heal avoids one full-context retransmission (median ~160k input tokens), totaling **~153M tokens** — roughly **115%** of what the models actually consumed in the same window, i.e. without healing the token spend would have been ~2.2× (estimated: token-meter pressure × skipped round-trips).
  14. What you get is an agent that stops tripping over its own tool calls — plus a dashboard that proves it:
  15. ![Before/after diff of a healed file edit](assets/dashboard-trace.png)
  16. ## What it does for you
  17. 1. **Fixes calls before they fail** — schema mismatches (`command` → `code`, missing `description`, Markdown fences), Code-Mode inner-call descriptions, relative paths and view ranges.
  18. 2. **Retries what is safe to retry** — one scoped read-then-retry for guarded file mutations, one bounded range retry; anchor failures are never retried blindly.
  19. 3. **Shows everything** — KPI cards, per-tool and per-category rankings, and a filterable before/after trace, one click away in Settings:
  20. | Rankings & root causes | Healing rules |
  21. | --- | --- |
  22. | ![Per-tool and per-category rankings](assets/dashboard-analytics.png) | ![Six healing rules, all active](assets/dashboard-rules.png) |
  23. ---
  24. ## 📖 Background & Empirical Motivation
  25. In DeepSeek Harness, tool execution reliability is critical for autonomous agent loops. An empirical analysis across **111 persisted sessions** (containing **11,176 total tool invocations**) revealed **543 tool call errors** (a **4.86% error rate** across 53.2% of sessions).
  26. Detailed root-cause analysis identified four primary structural error drivers:
  27. 1. **`INVALID_ARGS` Schema Incompatibilities (13.6%, 74 cases)**:
  28. - The model frequently treats `run_code` as `bash`, supplying `{"command": "..."}` instead of `{"description": "...", "code": "..."}`.
  29. - The model frequently omits the required `description` field in `run_code`.
  30. 2. **`UNKNOWN_TOOL` Code-Mode Cognitive Inertia (12.3%, 67 cases)**:
  31. - When Code-Mode is active, only `run_code` is exposed directly to the model. However, the model regularly hallucinates direct tool calls like `read`, `bash`, `write`, or `grep`, which fail immediately with `UNKNOWN_TOOL`.
  32. 3. **`CODE_RUN_FAILED` In-Sandbox Failures (46.8%, 254 cases)**:
  33. - JavaScript syntax errors caused by multiline shell or Python scripts nested inside JS template strings with unescaped backticks or newlines.
  34. 4. **File System Safety Policy Violations (5.5%, 30 cases)**:
  35. - Violating DSH's read-before-edit invariant (`FS_NOT_OBSERVED`), editor `view_range` line count out-of-bounds, or using relative paths instead of absolute paths.
  36. ### Production rollout effects (v0.4.0 · 192 sessions / 15,460 calls)
  37. Sessions before the plugin's first activation (7,182 calls, **7.95%** error rate) versus after (8,289 calls, **2.20%**):
  38. - `INVALID_ARGS` missing-description failures fell from 74 to 2 (outer `RUN_CODE_DESC` plus preemptive `INNER_DESC` heals).
  39. - Inner `description` omissions surfacing as `CODE_RUN_FAILED` fell from 45 to 0 (598 preemptive `INNER_DESC` successes in the plugin log).
  40. - `FS_NOT_OBSERVED` fell from 10 to 0 (169 observe-then-retry successes).
  41. - `UNKNOWN_TOOL` halved (71 → 35) but persists: PTC-collapsed calls are denied before the waterfall and stay unobservable to any plugin, so v0.4.0 appends a reissue hint to those errors instead of silently dropping them.
  42. - Residual `CODE_RUN_FAILED` syntax failures are semantic breakage no safe rewrite can guess (Python pasted as JS, wrong APIs); v0.4.0 appends a parse-failure hint for those.
  43. Counterfactual upper bound: without the 837 healed successes, the post window would have shown ≈12.3% instead of 2.20%. Token savings sum measured retransmission avoided (token-meter pressure × skipped round-trips), never a hardcoded constant. Since v0.4.0 the plugin's own nested recoveries are excluded from interception counts, so the denominator is user-facing calls only.
  44. ---
  45. ## 🎯 What Problems `dsh-tool-normalizer` Solves
  46. `dsh-tool-normalizer` acts as a low-overhead, deterministic safety middleware: normalization and healing on the `tools/execute` waterfall, statistics in the `tools/result` observer off the dispatch hot path, paired with an integrated Web UI diagnostics dashboard.
  47. ```
  48. Model Tool Call
  49. │
  50. ▼
  51. ┌────────────────────────────────────────────────────────┐
  52. │ dsh-tool-normalizer (Plugin) │
  53. │ │
  54. │ 1. run_code Normalizer (command ➔ code, description) │
  55. │ 2. Safe Direct-Call Recovery (context-preserving nested dispatch) │
  56. │ 3. Range & Path Normalizer (relative paths, real bounds) │
  57. │ 4. Dynamic Prompt Guidance (minimal token footprint) │
  58. │ 5. Real-Time Telemetry & Statistics Tracker │
  59. └────────────────────────────────────────────────────────┘
  60. │
  61. ▼
  62. Best-effort Recovery of Repairable Errors
  63. │
  64. ▼
  65. [Web UI] Settings ➔ Tool Normalizer & Diagnostics Page
  66. ```
  67. ### Key Features
  68. - 🛠️ **`run_code` Schema Auto-Healing**:
  69. - Automatically wraps `{"command": "git status"}` or `{"cmd": "..."}` into valid `run_code` JavaScript dispatches. Empty or non-string commands are left for the host to reject loudly instead of healing into a silent no-op.
  70. - Fills in missing `description` fields with sensible contextual defaults.
  71. - Strips accidental Markdown code block fences (e.g. ````typescript ...````).
  72. - **Program syntax self-healing**: when the emitted `code` does not parse, repairs the three mechanical breakage classes the host's async-function executor rejects — truncated tails (code ending inside an unclosed string or call), Python-style triple-quoted strings (`'''`/`"""` spans containing a newline) rewritten as escaped template literals, and stray unescaped backticks inside template literals. Every repair is re-verified with the same `new AsyncFunction` parse the host uses; valid programs are never touched.
  73. - 🌉 **Code-Mode Direct Tool Bridging**:
  74. - When an `UNKNOWN_TOOL` result reaches `tools/execute` and the target is visible in the active agent scope, the plugin re-dispatches it through the host's `tools.execute()` API as a nested call, preserving agent/session ownership, cancellation, contexts, and terminal state. Bridgeable names cover `bash/read/write/grep/edit/glob/str_replace_editor/job_output/job_kill` plus `web_fetch/web_search/todo_write/skill/ask_user_question`.
  75. - Scope note: under the PTC (`code`) presentation collapse, the host rejects a direct call before any listener runs; a plugin cannot intercept that path. For those errors v0.4.0 appends a ready-to-paste `run_code` reissue hint to the original error text instead. The plugin never invokes a tool definition's `execute()` method directly.
  76. - 💡 **Unrecoverable-Error Hints** (`errorHints`, default on):
  77. - PTC-collapsed direct calls and unrepairable `run_code` parse failures keep their original error text with one appended actionable hint, so the model can correct itself in the same round-trip. Set `errorHints: false` to preserve byte-identical host errors.
  78. - 🩹 **Inner-Call Description Injection**:
  79. - Before a `run_code` program executes, inserts a generated description only into a `tools.*()` call whose active tool schema marks `description` as required. Open schemas such as `read`, `glob`, and `grep` are left unchanged.
  80. - 📐 **Editor Parameter & Bounds Normalization**:
  81. - Corrects structural and inverted `view_range` values in `str_replace_editor` and `edit`; when the real error reports a line count, it retries with that bound and preserves the `-1` end-of-file sentinel.
  82. - Resolves relative file paths to absolute paths against the session working directory for `str_replace_editor` only, which rejects relative paths outright. The `edit`/`read`/`write` family resolves them against the session workspace itself, so the plugin leaves those paths untouched (see the `read`/`write` note below).
  83. - 🩹 **Observe-then-Retry Recovery**:
  84. - After `FS_NOT_OBSERVED` or `FS_STALE_VERSION`, the plugin reads the target and retries the mutation once through the host dispatcher; anchor failures (`FS_EDIT_NOT_FOUND`, `FS_AMBIGUOUS_EDIT`) are never retried blindly — a best-effort refresh updates the observed version so the next model retry is not additionally blocked. Normal calls do not pay for a speculative read.
  85. - 📈 **Projected Token Savings**:
  86. - Measures the input tokens each successful healing avoids from the host's token-meter: the session's one-request context pressure multiplied by the skipped model round-trips, shown in the dashboard. A composition without `@deepseek-ai/dsh-token-meter` reports zero instead of guessing.
  87. - Live observability: every settled call updates aggregate counters in the `tools/result` observer, off the dispatch hot path — including pre-execute/guard denials the `tools/execute` wrapper never sees, counted honestly as untouched failures. Healing attempts and failures append detailed JSONL events to `~/.dsh/tool-normalizer-events.jsonl`; successful untouched pass-through calls are aggregated in `tool-normalizer-summary.json` by default instead of expanding the detail log. The detail log rotates above 2 MB (newest 1 MB retained); the summary coalesces to at most one write per second and debug lines are emitted only for failures and heals.
  88. - Diagnostic previews keep both the beginning and end of long arguments and include a bounded summary of the fields or dispatch path that changed.
  89. - The dashboard reads live data from the same-origin feed `GET /plugin-api/tool-normalizer/stats`, registered by the node half when a webserver is present.
  90. - 📊 **Web UI Execution & Diagnostics Dashboard**:
  91. - Embedded directly into DSH's **Settings (`settings.section`)** panel.
  92. - Displays real-time KPI metrics (Total Interceptions, Auto-Healed Count, Healing Success Rate %, Unrecovered Errors).
  93. - Visual breakdown by tool and category with progress meters.
  94. - Filterable live table of execution logs showing original input vs. normalized payload.
  95. - v0.4.1 UI fixes: active filter pills and tabs no longer render unreadable filled labels under dark themes (tinted ring + brand text instead of filled background); untouched pass-through rows use a neutral tone instead of success green; a sixth rule card documents error hints. Screenshots above were captured from a live deployment (dark hero, light detail views).
  96. ---
  97. ## 🧭 UI Location & Design Rationale
  98. **Placement**: DeepSeek Harness Settings panel (`settings.section` with ID `tool-normalizer`, order `25`).
  99. ### Rationale
  100. 1. **Consistency with DSH Architecture**: In DSH Web UI, developer diagnostics and usage metrics (like `dsh-usage-atlas`, Model configuration, and Plugin inventory) are hosted as first-class sections inside the Settings panel.
  101. 2. **Zero Conversation Clutter**: Placing diagnostics in Settings keeps the primary agent chat canvas distraction-free while remaining just one click away via the gear icon in the sidebar rail.
  102. 3. **Unified Management**: Allows administrators and developers to observe runtime error rates and clear logs in the same panel where they configure models and plugins.
  103. ---
  104. ## 🚀 Installation & Quick Start
  105. In DeepSeek Harness, plugins are managed per composition profile (`web`, `headless`, `tui`, etc.).
  106. ### Step 1: Install Plugin into your Target Profile
  107. Using the `dsh` CLI (or `pnpm dsh` from monorepo root):
  108. ```sh
  109. # 1. Install into Web UI profile (Includes Settings Dashboard)
  110. dsh plugin --profile web add dsh-tool-normalizer
  111. # (or if running from source repository)
  112. pnpm dsh plugin --profile web add dsh-tool-normalizer
  113. # 2. Install into Headless automation profile
  114. dsh plugin --profile headless add dsh-tool-normalizer
  115. # 3. Install into TUI terminal profile
  116. dsh plugin --profile tui add dsh-tool-normalizer
  117. ```
  118. #### Local Development Link (Optional)
  119. If you are developing or testing local changes:
  120. ```sh
  121. pnpm dsh plugin --profile web add ./plugins/dsh-tool-normalizer
  122. ```
  123. ### Step 2: Launch and Verify
  124. ```sh
  125. # Boot Web UI mode
  126. dsh web
  127. # (or from source)
  128. pnpm dsh web
  129. ```
  130. Open your browser, navigate to **Settings (⚙️)** ➔ **Tool Normalizer**, and observe real-time tool execution metrics and auto-healing in action!
  131. ---
  132. ## ⚙️ Configuration
  133. You can customize plugin behavior in your workspace's `cordis.patch.yml` or `cordis.yml`:
  134. ```yaml
  135. - insert:
  136. - id: tool-normalizer
  137. name: dsh-tool-normalizer
  138. config:
  139. autoWrapRunCode: true
  140. autoBridgeDirectTools: true
  141. autoObserveFiles: true
  142. autoClampRanges: true
  143. injectPrompt: true
  144. errorHints: true
  145. persistPassthrough: false
  146. ```
  147. | Option | Type | Default | Description |
  148. | :--- | :---: | :---: | :--- |
  149. | `autoWrapRunCode` | `boolean` | `true` | Auto-convert `command` -> `code`, supply missing descriptions, strip Markdown fences. |
  150. | `autoBridgeDirectTools` | `boolean` | `true` | Safely re-dispatch an `UNKNOWN_TOOL` result that reached `tools/execute`; host-level pre-dispatch denials cannot be intercepted by a plugin. |
  151. | `autoObserveFiles` | `boolean` | `true` | After `FS_NOT_OBSERVED`, read the target and retry one edit/write through the host dispatcher. |
  152. | `autoClampRanges` | `boolean` | `true` | Correct editor ranges; resolve relative paths against the session directory for `str_replace_editor` only. |
  153. | `injectPrompt` | `boolean` | `true` | Register prompt guidelines with `ctx.systemPrompt`: instructional guidance as a section, top-error diagnostics as runtime context (section fallback on older hosts). Static text only — never breaks prefix caching. |
  154. | `errorHints` | `boolean` | `true` | Append one actionable hint to unrecoverable PTC/syntax errors while preserving the original error text. |
  155. | `persistPassthrough` | `boolean` | `false` | Persist successful untouched pass-through calls as detailed JSONL events; failures and healing attempts are always retained. |
  156. Healing success rate is `healedSuccess / (healedSuccess + healedFailed)` and excludes untouched pass-through failures — including pre-execute/guard denials, which are counted in the totals but never attributed to healing. A pre-dispatch normalization whose final error belongs to a different failure class is attributed as an unrelated pass-through failure rather than a failed heal, so the rate measures real efficacy. Successful untouched calls are kept in aggregate counters and the compact `tool-normalizer-summary.json`, not one detail line per call.
  157. The token-savings KPI sums measured per-heal input tokens: each successful heal credits `skipped model round-trips × token-meter request pressure`, i.e. the prompt a further request would have re-submitted. It requires `@deepseek-ai/dsh-token-meter` in the composition; without it the figure stays `0` instead of using a hardcoded per-retry constant.
  158. ---
  159. ## 📦 Release & Publishing Guide
  160. ### Option 1: Automated Release via GitHub Actions (Recommended)
  161. 1. Set your npm access token as a secret in your GitHub repository:
  162. - Go to **GitHub Repository Settings ➔ Secrets and variables ➔ Actions ➔ New repository secret**.
  163. - Name: `NPM_TOKEN`, Value: `<your-npm-automation-token>` (Ensure 2FA bypass is enabled for write actions).
  164. 2. Bump the version and push a release tag:
  165. ```sh
  166. # Bump version (patch / minor / major)
  167. npm version patch
  168. # Push commit and tags to GitHub
  169. git push origin main --tags
  170. ```
  171. 3. Create a GitHub Release on the new tag. The GitHub Actions workflow (`.github/workflows/publish.yml`) will automatically run tests, build artifacts, and publish to npm!
  172. ### Option 2: Manual npm Publishing
  173. ```sh
  174. # 1. Ensure clean build & passing tests
  175. npm run check
  176. # 2. Login to npm (if not already logged in)
  177. npm login
  178. # 3. Publish to npm registry
  179. npm publish --access public
  180. ```
  181. ---
  182. ## Known Limitations
  183. - **No path normalization for `read`/`write`.** The `read`/`write`/`edit` tool family resolves relative paths against the session working directory by itself; only `str_replace_editor` rejects them. Normalizing those calls would count heals for invocations that would have succeeded anyway, so the plugin deliberately leaves them untouched to keep the healing rate honest.
  184. ---
  185. ## 🧪 Testing & Verification
  186. ```sh
  187. # Run all unit tests
  188. pnpm test
  189. # Run tests and compile build artifacts
  190. pnpm run check
  191. ```
  192. ---
  193. ## 📄 License
  194. MIT © [merenguesL](https://github.com/merenguesL)

Ratings and comments

Sign in to rate this skill.

Sign in

Free to use. If it’s helped, pass it on.