tokensave
The most comprehensive code intelligence MCP server for AI coding agents. 40+ tools, 30+ languages, 9 agent integrations. Pre-indexed semantic knowledge graphs for instant code understanding — fewer tokens, fewer tool calls, 100% local.
Safety scan
A static text scan, not a guarantee. Select the badge to see what was scanned and every finding.
Check it deeper on your machine
It has changed since its scan on 11 Nov 2026.
As a Claude Code plugin, install it and then ask Claude Code:
/plugin marketplace add keithmackay/mackayi/plugin install scanskillsafety@mackayiDo a deep scan of https://github.com/aovestdipaperino/tokensave with scanskillsafety
Or from a terminal, clone it and run it:
git clone https://github.com/keithmackay/scanskillsafety-skill ~/.claude/skills/scanskillsafetynode ~/.claude/skills/scanskillsafety/cli.cjs --deep 'https://github.com/aovestdipaperino/tokensave'
It checks the exact commit: every file, its packages against OSV (api.osv.dev), how it compares with this page's scan (findsafeskills.com receives the repository's address), and, if you agree, Claude reading every file with your own Claude subscription. It never runs the code.
Badge for your README
Markdown
[](/listings/aovestdipaperino--tokensave)HTML
<a href="/listings/aovestdipaperino--tokensave"><img src="/api/listings/aovestdipaperino--tokensave/badge.svg" alt="findsafeskills safety rating" /></a>Install
Install instructions from the README
As written by the repo's author. findsafeskills has not verified these steps.
1. Install
**Homebrew (macOS):**
brew install aovestdipaperino/tap/tokensave
**Scoop (Windows):**
scoop bucket add tokensave https://github.com/aovestdipaperino/scoop-bucket scoop install tokensave
**Cargo / cargo-binstall (any platform):**
# Fast install prebuilt binary without compiling: cargo binstall tokensave # Or compile from source: cargo install tokensave # full (50+ languages, default) cargo install tokensave --features medium # medium tier cargo install tokensave --no-default-features # lite (smallest binary)
**Prebuilt binaries (Linux, Windows, macOS):**
Download from the [latest release](https://github.com/aovestdipaperino/tokensave/releases/latest) and place the binary in your `PATH`.
| Platform | Archive | |---|---| | macOS (Apple Silicon) | `tokensave-vX.Y.Z-aarch64-macos.tar.gz` | | Linux (x86_64) | `tokensave-vX.Y.Z-x86_64-linux.tar.gz` | | Linux (ARM64) | `tokensave-vX.Y.Z-aarch64-linux.tar.gz` | | Windows (x86_64) | `tokensave-vX.Y.Z-x86_64-windows.zip` |
2. Configure your agent
tokensave install # auto-detects installed agents tokensave install --agent antigravity # Google Antigravity (formerly Windsurf) tokensave install --agent auggie # AugmentCode tokensave install --agent claude # Claude Code tokensave install --agent cline # Cline tokensave install --agent codex # OpenAI Codex CLI tokensave install --agent copilot # GitHub Copilot tokensave install --agent cursor # Cursor tokensave install --agent droid # Factory Droid tokensave install --agent gemini # Gemini CLI tokensave install --agent kilo # Kilo CLI tokensave install --agent kiro # AWS Kiro tokensave install --agent kimi # Moonshot Kimi CLI tokensave install --agent omp # Oh My Pi (OMP) tokensave install --agent opencode # OpenCode tokensave install --agent pi # Pi (pi.dev) tokensave install --agent plank # Plank (macOS only) tokensave install --agent qwen # Qwen Code tokensave install --agent roo-code # Roo Code tokensave install --agent vibe # Mistral Vibe tokensave install --agent zed # Zed tokensave install --agent grok # Grok Build (xAI) tokensave install --git-hook yes # auto-install the global post-commit and post-checkout hooks (no prompt) tokensave install --git-hook no # skip the post-commit and post-checkout hooks (no prompt) tokensave githooks # show which global git hooks tokensave owns tokensave githooks off # remove them, leaving any hook content you wrote
Each agent gets its MCP server registered in the native config format. Claude Code additionally gets a PreToolUse hook (blocks wasteful Explore agents), a UserPromptSubmit hook, a Stop hook, prompt rules in CLAUDE.md, and auto-allowed tool permissions. Kiro gets global MCP config, `tokensave.md` steering loaded as a resource, and a tokensave-managed default agent with permissive built-in/tokensave tool approval, delegation guardrail hooks, and post-write sync; user-managed Kiro agents are preserved.
Global OMP installs target the profile reported by bare `omp config path`, writing `<resolved-agent-dir>/mcp.json` and `<resolved-agent-dir>/rules/tokensave.md`. Export `OMP_PROFILE` or OMP's compatible `PI_PROFILE` when installing into a named profile; OMP's resolver also honors `PI_CONFIG_DIR` and `PI_CODING_AGENT_DIR`. Tokensave trusts that native resolver rather than duplicating OMP's profile logic. Tokensave installs MCP and advisory rules for OMP; it does not install OMP hook enforcement.
What was scanned
The files the safety scan read, as written by the repository's author and shown as plain text. Lines with a finding are marked.
README.md · README · 1,082 lines
View this file on GitHub (read at the repository's current version: the scan did not record which commit it read)
This file has changed since it was scanned, so this is not exactly the text that was scanned, and no line is marked. The findings are listed with the safety scan above.
- [](https://mcptoplist.com/server/io.github.aovestdipaperino%2Ftokensave)
- <p align="center">
- <img src="src/resources/logo.png" alt="TokenSave" width="300">
- </p>
- <h3 align="center">Semantic Code Intelligence for AI Coding Agents</h3>
- <p align="center"><strong>Fewer tokens • Fewer tool calls • 100% local</strong></p>
- <p align="center">
- <a href="https://github.com/aovestdipaperino/tokensave/stargazers"><img src="https://img.shields.io/github/stars/aovestdipaperino/tokensave?logo=github&color=brightgreen" alt="GitHub stars"></a>
- <a href="https://crates.io/crates/tokensave"><img src="https://img.shields.io/crates/v/tokensave.svg" alt="crates.io"></a>
- <a href="https://opensource.org/licenses/MIT"><img src="https://img.shields.io/badge/License-MIT-yellow.svg" alt="License: MIT"></a>
- <a href="https://www.rust-lang.org/"><img src="https://img.shields.io/badge/Rust-1.70+-orange.svg" alt="Rust"></a>
- <a href="https://ai.enzolombardi.net/"><img src="https://img.shields.io/badge/built%20with-AI-D97757?style=flat-square&labelColor=101010&logo=anthropic&logoColor=white" alt="Built with AI — part of Enzo Lombardi's AI portfolio"></a>
- </p>
- <p align="center">
- <img src="https://img.shields.io/badge/macOS-supported-blue.svg" alt="macOS">
- <img src="https://img.shields.io/badge/Linux-supported-blue.svg" alt="Linux">
- <img src="https://img.shields.io/badge/Windows-supported-blue.svg" alt="Windows">
- <a href="https://hypercommit.com/tokensave"><img src="https://img.shields.io/badge/Hypercommit-DB2475" alt="Hypercommit"></a>
- <!-- Listing badge only. Deliberately not wrapped in a link: the badge's own
- href points at a monetization page on the marketplace's side (#406). -->
- <img src="https://getlulu.dev/api/mcps/badge/tokensave" alt="Listed in the Lulu MCP marketplace">
- </p>
- ---
- ## Why tokensave?
- AI coding agents waste tokens exploring codebases. Every grep, glob, and file read costs money. On complex tasks, agents spawn multiple Explore sub-agents that scan hundreds of files just to build context.
- **tokensave gives agents a pre-indexed semantic knowledge graph.** Instead of scanning files, the agent queries the graph and gets instant, structured answers -- the right symbols, their relationships, and source code, in one call.
- ### How It Works
- ```mermaid
- flowchart TB
- subgraph agent["AI Coding Agent (Claude Code, Codex, Gemini, Cursor, ...)"]
- task["Implement user authentication"]
- first["Sub-agent"]
- second["Sub-agent"]
- task --> first
- task --> second
- first --- second
- end
- subgraph server["tokensave MCP Server"]
- search["Search: auth"]
- callers["Callers: login()"]
- context["Context for task"]
- db[("libSQL Graph DB<br/>Instant lookups<br/>FTS5 search")]
- search --> db
- callers --> db
- context --> db
- end
- first --> search
- first --> callers
- second --> context
- ```
- **Without tokensave:** Agents use `grep`, `glob`, and `Read` to scan files -- many API calls, high token usage.
- **With tokensave:** Agents query the graph via MCP tools -- instant results, local processing, fewer tokens.
- ---
- ## Key Features
- | | | |
- |---|---|---|
- | **Smart Context Building** | **Semantic Search** | **Impact Analysis** |
- | One tool call returns everything the agent needs -- entry points, related symbols, and code snippets. | Find code by meaning, not just text. Search for "authentication" and find `login`, `validateToken`, `AuthService`. | Know exactly what breaks before you change it. Trace callers, callees, and the full impact radius of any symbol. |
- | **80+ MCP Tools** | **50+ Languages** | **12+ Agent Integrations** |
- | From call graph traversal to dead code detection, atomic edit primitives, code-health metrics, test mapping, and complexity analysis. | Rust, Go, Java, Python, TypeScript, C, C++, Swift, Svelte, Astro, and 48 more including WGSL/HLSL/Metal shaders, CUDA/HIP, and Markdown. Three tiers (lite/medium/full) control binary size. | Claude Code, Codex CLI, Gemini CLI, Qwen Code, Kiro, Cursor, OpenCode, Copilot, Cline, Roo Code, Zed, Antigravity, Kilo CLI, Kimi CLI, Mistral Vibe, Grok Build, Factory Droid, OMP, Pi, Plank. |
- | **Multi-Branch Indexing** | **100% Local** | **Always Fresh** |
- | Per-branch databases, tracked automatically as you check out branches. Cross-branch diff and search without switching your checkout. | No data leaves your machine. No API keys. No external services. Everything runs on a local libSQL database. | On-demand staleness check on every MCP call (30 s cooldown) plus catch-up sync when the server connects. Multi-agent work is expected to use git worktrees — each agent gets its own checkout and the index diverges are merged by git, not by a file watcher. |
- | **Subprocess-Isolated Extraction** | **Code-Health Analytics** | **Atomic Edit Primitives** |
- | A native crash in any tree-sitter grammar (abort, segfault, anything) kills only the worker; the pool respawns it and sync continues. Sync never dies on a malformed file. | Composite health score (0-10000), Gini inequality, file-DAG depth, design-structure matrix, risk-weighted test gaps, and session deltas. | Edit files without regex or shell-quoting hazards: unique-anchor `str_replace`, atomic multi-replace, AST-rewrite, anchored insert. Auto re-indexes after writes. |
- ---
- ## Quick Start
- ### 1. Install
- Choose one installation method:
- | Platform | Command |
- |---|---|
- | Any platform (Cargo) | `cargo install tokensave`<br>or, with cargo-binstall, `cargo binstall tokensave` |
- | macOS (Homebrew) | `brew install aovestdipaperino/tap/tokensave` |
- | Windows (Scoop) | `scoop bucket add tokensave https://github.com/aovestdipaperino/scoop-bucket`<br>`scoop install tokensave` |
- You can also download a [prebuilt binary](https://github.com/aovestdipaperino/tokensave/releases/latest) and add it to your `PATH`.
- To build unreleased code from the upstream `master` branch, see
- [Installing from upstream master](docs/guide/INSTALLATION.md#installing-from-upstream-master).
- <details>
- <summary>Installation alternatives and language tiers</summary>
- The default Cargo build includes all supported languages. For a smaller build:
- ```bash
- cargo install tokensave --no-default-features --features medium
- cargo install tokensave --no-default-features # lite (smallest binary)
- ```
- Prebuilt release archives:
- | Platform | Archive |
- |---|---|
- | macOS (Apple Silicon) | `tokensave-vX.Y.Z-aarch64-macos.tar.gz` |
- | Linux (x86_64) | `tokensave-vX.Y.Z-x86_64-linux.tar.gz` |
- | Linux (ARM64) | `tokensave-vX.Y.Z-aarch64-linux.tar.gz` |
- | Windows (x86_64) | `tokensave-vX.Y.Z-x86_64-windows.zip` |
- </details>
- ### 2. Configure your agent
- Run the setup command to auto-detect your installed agents:
- ```bash
- tokensave install
- ```
- To choose an agent, use `--agent` (for example, `codex`, `claude`, or `cursor`):
- ```bash
- tokensave install --agent codex
- ```
- Setup registers the MCP server and the agent's integration for Claude Code,
- Codex CLI, Gemini CLI, Cursor, Copilot, Kiro, Pi, Zed, and
- [more than a dozen other agents](docs/guide/AGENT-INTEGRATIONS.md#configuring-your-agent).
- It also offers global Git hooks to keep initialized projects up to date. You can
- safely run setup again after upgrading. See
- [Agent Integrations](docs/guide/AGENT-INTEGRATIONS.md) for project-local installs,
- several agents at once, and what setup writes for each agent.
- ### 3. Index your project
- ```bash
- cd /path/to/your/project
- tokensave init
- ```
- This creates a `.tokensave/` directory with the knowledge graph database. A
- repository you create with `git init` is initialized automatically on its first
- commit or agent session (`auto_init`, on by default). After that,
- `tokensave sync` updates the index incrementally, and a connected agent keeps it fresh automatically. Commands work
- from any subdirectory of the project. See
- [Indexing and Freshness](docs/guide/INDEXING.md) for details.
- ---
- ## Documentation
- The full documentation lives in [`docs/guide/`](docs/guide/README.md) and is
- mirrored to the [GitHub Wiki](https://github.com/aovestdipaperino/tokensave/wiki).
- | Topic | What it covers |
- |---|---|
- | `TOKENSAVE_BENCH_REPOS_DIR` | **Required.** Root directory where each repo is cloned to `$DIR/<repo-name>/`. |
- | `TOKENSAVE_BENCH_REPOS` | Comma-separated subset of repo names to bench, e.g. `TOKENSAVE_BENCH_REPOS=emacs,scipy`. Defaults to all four. |
- | `TOKENSAVE_BENCH_SKIP_CLONE` | If set, the bench fails fast for any repo not already at its pinned ref instead of fetching. Useful in CI / offline runs. |
- **Filtering benchmarks** uses the standard criterion CLI — for example, only the `search` tool on `scipy`:
- ```bash
- cargo bench --bench large_repos -- 'scipy/tokensave_search'
- ```
- Reports (HTML + raw samples) land under `target/criterion/`.
- To change the pinned refs (e.g. to a newer release or a specific SHA), edit `REPOS` in `benches/repos.rs` and delete the corresponding `$TOKENSAVE_BENCH_REPOS_DIR/<repo>/.bench-ref` marker so the next run re-fetches. If you skip the post-run cleanup (e.g. you `Ctrl-C` mid-bench), running `git stash --include-untracked && git stash drop` inside each repo dir restores it manually.
- ### MCP test-matrix probe (scripts/mcp_probe)
- `scripts/mcp_probe/` is a Python harness that drives `tokensave serve` over stdio against a configurable set of real repos and exercises **every read-only MCP tool with 5 query variants per language**, producing a per-tool / per-repo status table. Same harness serves two purposes:
- - **Regression sweep.** New language support, new tool, or a refactor — re-run the matrix and any cell that newly errors, times out, or returns empty results stands out as a 🚩.
- - **Perf probe.** Per-call timings are logged in TSV; the same fixed corpus of repos doubles as a coarse cross-version comparison. The current `tokensave_inheritance_depth` cycle bug was found by this harness when a single tool on polkadot-sdk timed out at >60 s.
- **Layout** — `probe.py` is the driver (id-matched JSON-RPC so a slow tool can't poison subsequent calls), `isolated.py` re-runs a single tool with a fresh server per call (escapes server queueing), `build_matrix.py` reads the TSV and emits markdown, `tools/<lang>.py` modules contribute per-language query sets (Rust shipped; add Python/Go/… by dropping a new module), `repos.toml` lists target repos (override via `$TOKENSAVE_PROBE_REPOS`).
- **Quick run:**
- ```bash
- cargo build --release --bin tokensave
- python3 scripts/mcp_probe/probe.py
- python3 scripts/mcp_probe/build_matrix.py > matrix.md
- ```
- Output cells are `✓ 5/5` (clean), `🐛 e/N` (errors), `⏱ N/N` (timeouts), `∅ E/N` (empty), `🐢 ok/slow` (>10 s calls). Any cell carrying an error or timeout earns a 🚩 in the rightmost column. Per-call detail with the first 100 chars of each error landS in the TSV log for follow-up.
- Different from the criterion bench above: criterion measures per-iteration latency for a focused tool set on pinned refs and produces statistical reports under `target/criterion/`; `mcp_probe` exercises every tool with a broader query set on whatever repos you point it at, optimising for breadth of coverage rather than measurement precision.
- ---
- ## 80+ MCP Tools
- The server exposes more than 80 tools (one fewer when the optional `ast-grep` binary is not on `PATH`). Since 7.14.0 only the 11 core tools plus `tokensave_more` are listed by default; the rest are listed on demand through `tokensave_more`, stay callable by name, and are all listed with `"tools": "full"`. The tables below group the most commonly used ones by category. Most are read-only, safe to call in parallel, and annotated with `readOnlyHint`. The edit primitives are scoped to single files and re-index in place; session baseline and memory-recording tools also mutate local `.tokensave` state and are annotated as non-read-only. The three core tools (`tokensave_context`, `tokensave_search`, `tokensave_status`) are marked `anthropic/alwaysLoad` so they bypass the client's tool-search round-trip.
- ### Query another initialized project
- Semantic read tools can query an explicitly selected local graph without
- restarting the MCP server:
- ```json
- {
- "query": "screenGate",
- "graph_root": "/absolute/path/to/typewhisper"
- }
- ```
- Selected results include canonical root/branch provenance. Node IDs are
- namespaced to that graph, and the matching selectors must be repeated on
- follow-up calls. For example, a follow-up to a branch-selected query includes
- both values:
- ```json
- {
- "node_id": "graph:<fingerprint>:function:<raw-id>",
- "graph_root": "/absolute/path/to/typewhisper",
- "graph_branch": "feature/auth"
- }
- ```
- `graph_root` must be the exact absolute root of an already initialized project.
- `graph_branch` is optional and, when supplied, must name a tracked branch.
- Selected opens are read-only: they never initialize, sync, migrate, auto-track,
- or write graph/source data. They also do not contribute to savings accounting.
- Calls without selectors behave exactly as before.
- `tokensave_search` and `tokensave_files` also accept an array of roots to
- federate one request across several graphs. A multi-root array may include the
- project already served by the MCP server; that project participates as a
- read-only member of the federation. Unusable roots are reported as skipped
- when at least one root succeeds, and worktrees of the same repository are
- collapsed with the omitted paths reported. A scalar root or one-element array
- that names the served project remains invalid; omit `graph_root` for that
- single-project call.
- `graph_root` is only useful if you know the other project exists, so the server
- tells you: initialized projects sitting directly beside the served root are
- named in the MCP `instructions`, in `tokensave_status`, and in empty
- `tokensave_search` / `tokensave_context` results — the point at which a session
- would otherwise conclude a symbol does not exist rather than look next door
- (#375). Only immediate siblings are offered, at most five, and nothing is opened
- or indexed on their behalf; querying one still requires an explicit `graph_root`.
- Selectors are intentionally unavailable on tools that write, shell out, or
- depend on the current checkout: the edit primitives, VCS and branch tools,
- diagnostics and test execution, dependency and runtime introspection, workflow
- and session-memory tools, the persistent-cache tool (`tokensave_redundancy`),
- and server administration. Those tools reject a selector instead of silently
- ignoring it.
- ### Discovery
- | Tool | Purpose |
- |------|---------|
- | `tokensave_context` | Get relevant code context for a task -- entry points, related symbols, code snippets |
- | `tokensave_search` | Find symbols by name (functions, classes, types) |
- | `tokensave_node` | Get details + source code for a specific symbol |
- | `tokensave_files` | List indexed project files (source and tracked artifacts) with filtering |
- | `tokensave_module_api` | Public API surface of a file or directory |
- | `tokensave_similar` | Find symbols with similar names |
- | `tokensave_annotations` | Attribute/annotation/decorator introspection -- histogram of all annotations or per-site listings with target filters |
- | `tokensave_doc` | Companion Markdown documentation for a source file -- doc content, the files it covers, and a staleness signal |
- | `tokensave_dependencies` | Package-manifest introspection across 17 ecosystems -- workspace summary, per-package lookup, license surface, version drift |
- | `tokensave_status` | Index status, statistics, tokens saved |
- #### Custom source extensions
- Map extensions to an enabled extractor in `.tokensave/config.json`:
- ```json
- "language_aliases": { ".t": "perl", ".ep": "perl" }
- ```
- This parses legacy Perl tests without renaming them. Keys accept an optional
- leading dot and match case-insensitively; invalid, conflicting, unknown, or
- disabled mappings fail sync with a configuration error. Aliases override built-in
- extension associations and artifact tracking. Use existing `project.json`
- `entries` with `language` for file/glob overrides, which take priority over
- aliases. Run `tokensave sync` after editing mappings (`sync -f` also works).
- See the [configuration guide](docs/USER-GUIDE.md#custom-language-extensions).
- #### Non-code artifacts
- `tokensave_files` covers more than source. Files whose extension is listed in
- `artifact_extensions` (`.feature`, `.json`, `.yaml`, `.yml`, `.sql`, `.toml`,
- `.proto`, `.graphql`, `.md`, `.bnd`, `.bndrun` by default) are tracked by path
- so questions like "where are the `.feature` files for the login flow?" have a
- graph answer rather than a blocked `find` (#323). They are never parsed and contribute no symbols;
- `kind: "artifact"` and `kind: "code"` filter between the two, and analyses that
- mean "code" exclude them. An extension already handled by a language extractor
- is ignored in this list, so it cannot be used to stop a language being parsed.
- The list also decides what **literal search can look inside** (#442). A literal
- (`literal: true`) search over `tokensave_search` reads bytes rather than
- symbols, so it needs no parser -- but it iterates the indexed files, so it can
- only reach a file the index holds a row for. A tracked `.rst` document or
- `.tmpl` template has neither an extractor nor a default artifact entry, so its
- matches are missing; add the extension here and run `tokensave sync -f` and its
- lines are searched like any other, reported with `enclosing: null` since there
- is no symbol context. A literal response that could not reach every tracked
- file says so in an `unscanned` block naming the count and the extensions, so a
- partial answer is never presented as a complete one.
- ### Call Graph & Impact
- | Tool | Purpose |
- |------|---------|
- | `tokensave_callers` | Find what calls a function |
- | `tokensave_callees` | Find what a function calls |
- | `tokensave_impact` | See what's affected by changing a symbol |
- | `tokensave_affected` | Find test files affected by source changes |
- | `tokensave_rename` | Graph-based rename: every site with a confidence class (`exact`, `heuristic`, `ambiguous`, `text_only`), a diff preview, and an all-or-nothing apply. Not binding-aware |
- | `tokensave_hotspots` | Most connected symbols (highest call count) |
- ### Code Quality
- | Tool | Purpose |
- |------|---------|
- | `tokensave_complexity` | Rank functions by cyclomatic & cognitive complexity, nesting depth, Halstead metrics, maintainability index, CRAP, and safety metrics |
- | `tokensave_dead_code` | Find unreachable symbols (no incoming edges; symbols named as an ambiguity candidate are excluded) |
- | `tokensave_ambiguous_calls` | Call sites the resolver could not pin to one target, with every tied candidate |
- | `tokensave_god_class` | Find classes with too many members |
- | `tokensave_coupling` | Rank files by fan-in/fan-out |
- | `tokensave_inheritance_depth` | Find the deepest inheritance hierarchies |
- | `tokensave_circular` | Detect circular file dependencies |
- | `tokensave_imports` | Module-level import dependencies, cycles, and cut simulation |
- | `tokensave_recursion` | Detect recursive/mutually-recursive call cycles |
- | `tokensave_unused_imports` | Import statements never referenced |
- | `tokensave_doc_coverage` | Public symbols missing documentation |
- | `tokensave_simplify_scan` | Quality analysis of changed files (duplications, dead code, complexity) |
- ### Code-Health Analytics
- Five tools surface structural quality signals from the existing graph. The composite score uses a geometric mean over independent dimensions so no single one can be gamed.
- | Tool | Purpose |
- |------|---------|
- | `tokensave_health` | Composite quality signal (0-10000) from acyclicity, depth, equality, redundancy, and modularity |
- | `tokensave_gini` | Gini inequality coefficient for any metric (complexity, lines, fan-in/out, members) -- finds god files and uneven distribution |
- | `tokensave_dependency_depth` | Longest file-level dependency chains (Lakos levelization) with full chain reconstruction after Tarjan SCC cycle-breaking |
- | `tokensave_dsm` | Design Structure Matrix in `stats`, `clusters`, or `matrix` form -- reveals layering violations and hidden coupling |
- | `tokensave_test_risk` | Risk-weighted test-gap analysis combining complexity, fan-in, coverage, and 90-day git churn into a single score |
- ### Sessions
- Snapshot health metrics at the start of an AI coding session, then diff at the end to see what improved or regressed.
- | Tool | Purpose |
- |------|---------|
- | `tokensave_session_start` | Save current health metrics as a JSON baseline for later comparison |
- | `tokensave_session_end` | Recompute and diff against the baseline -- per-dimension deltas, pass/fail, automatic cleanup |
- ### Edit Primitives
- Four writer tools that let agents modify files without regex or shell-quoting hazards. Each is single-file, anchored, and triggers an in-place re-index after writing so the graph never goes stale.
- | Tool | Purpose |
- |------|---------|
- | `tokensave_str_replace` | Replace a unique `old_str` with `new_str`; fails if 0 or >1 matches (protects against multi-edit bugs) |
- | `tokensave_multi_str_replace` | Apply N `(old, new)` replacements atomically -- all-or-nothing transaction |
- | `tokensave_insert_at` | Insert content before or after a unique anchor string or line number |
- | `tokensave_ast_grep_rewrite` | Structural code rewrite via the `ast-grep` CLI in `--rewrite` mode |
- ### Git & Workflow
- | Tool | Purpose |
- |------|---------|
- | `tokensave_diff_context` | Semantic context for changed files -- modified symbols, dependencies, affected tests |
- | `tokensave_commit_context` | Semantic summary of uncommitted changes for commit message drafting |
- | `tokensave_pr_context` | Semantic diff between git refs for pull request descriptions |
- | `tokensave_changelog` | Semantic diff between two git refs |
- | `tokensave_test_map` | Source-to-test mapping at the symbol level, with uncovered symbol detection |
- | `tokensave_test_coverage` | Per-file/symbol/test-fn coverage rollup with transitive call-edge expansion |
- ### Type System
- | Tool | Purpose |
- |------|---------|
- | `tokensave_type_hierarchy` | Recursive type hierarchy tree for traits, interfaces, and classes |
- | `tokensave_rank` | Rank nodes by relationship count (most implemented interface, most extended class) |
- | `tokensave_distribution` | Node kind breakdown per file or directory |
- | `tokensave_largest` | Rank nodes by size -- largest classes, longest methods |
- ### Porting
- | Tool | Purpose |
- |------|---------|
- | `tokensave_port_status` | Compare symbols between source/target directories to track porting progress |
- | `tokensave_port_order` | Topological sort of symbols for porting -- port leaves first, then dependents |
- ### Multi-Branch
- | Tool | Purpose |
- |------|---------|
- | `tokensave_branch_search` | Search symbols in another branch's graph |
- | `tokensave_branch_diff` | Compare symbols between branches; `quality: true` compares structural quality |
- | `tokensave_branch_list` | List tracked branches with DB sizes and sync times |
- #### Branch quality regressions
- Compare two **existing branch indexes** at their Git tips:
- ```bash
- tokensave quality-diff --base master --target feature/example
- tokensave quality-diff --base master --target HEAD --json --fail-on regression
- tokensave quality-diff --base origin/master --target feature/example --cyclomatic 15 --min-increase 5
- ```
- Each reference must resolve to a commit and name a tracked Tokensave branch.
- `HEAD` resolves to the current symbolic branch; detached HEAD requires an explicit
- branch. Full `refs/heads/...` and `refs/remotes/...` names are accepted. Create and
- sync each index explicitly using the existing `tokensave branch add` / `sync`
- workflow before comparing. Comparison never checks out a branch or rebuilds an
- index. Missing indexes and empty analysis data are errors, including comparisons
- of a branch against itself.
- The baseline is **tip to tip**, not the merge base. On diverged branches, a
- reported difference can originate on either branch; this is not proof that a
- feature introduced the difference since branching.
- Supported diagnostics reuse extracted/indexed data:
- | Code | Metric / identity | Default threshold |
- |------|-------------------|-------------------|
- | `Q_COMPLEXITY_CYCLOMATIC` | Function branches + 1 | >10 |
- | `Q_COMPLEXITY_COGNITIVE` | Extracted cognitive complexity | >15 |
- | `Q_FUNCTION_LINES` | Inclusive function span | >100 |
- | `Q_FILE_FAN_OUT` / `Q_FILE_FAN_IN` | Distinct files reached by resolved calls/uses, or reaching this file | >10 |
- | `Q_CYCLIC_DEPENDENCY_EDGE` | Directed file edge inside a cyclic strongly connected component | Present |
- Change thresholds with `--cyclomatic`, `--cognitive`, `--function-lines`, and
- `--fan-out` (the latter applies to both fan-in and fan-out). Existing above-threshold
- issues become `worsened` when their metric increases by at least `--min-increase`
- (default 3); smaller increases remain `unchanged` for gating. Crossing a threshold
- is `new`; falling below it or deletion is `resolved`; a decrease above it is
- `improved`. Pre-existing cyclic edges remain unchanged. An existing edge becoming
- cyclic is new, even if the edge itself predates the comparison. Cycles are reported
- as edges in cyclic components, not as an enumerated cycle path.
- JSON includes deterministic diagnostic order, codes, categories, warning severity,
- identities, base/target values, locations (1-based for functions), confidence,
- classification counts and limitations. `--fail-on regression` exits **1** for new
- or worsened findings, **0** otherwise. Operational errors and explicitly accepted
- incomplete results exit **2**, independently of `--fail-on`. Without the flag,
- complete comparisons exit 0. JSON mode writes a single JSON document to stdout,
- including on operational errors.
- The existing MCP tool accepts the same comparison through:
- ```json
- {"base":"master","head":"feature/example","quality":true,"limit":20,"thresholds":{"cyclomatic":10,"min_increase":3}}
- ```
- The MCP `report` shares the CLI schema. Counts cover all findings; details default
- to 20 (maximum 500), with `truncated` and `total_findings` outside the report.
- Use `limit: 0` for counts only. `thresholds` accepts the option names in the table
- (`cyclomatic`, `cognitive`, `function_lines`, `fan_out`), plus `min_increase` and
- `allow_stale`; unknown options are rejected. File/kind filters are rejected in
- quality mode because cyclicity and coupling require global evidence. No additional
- MCP tool is exposed. Quality calls skip automatic per-call sync/version reindex;
- other independent activity in an already-running server retains its normal behavior.
- Freshness validation hashes indexed files against Git blobs with the same source
- decoder as indexing, accepting LF/CRLF line-ending conversion from Git checkout
- or commit normalization. A mismatched/deleted file or an unindexed source candidate is
- reported as stale/incomplete. Candidate discovery respects configured exclusion,
- hidden-file and size rules; branch-specific ignore or manifest policies may explain
- a candidate, so it is described as uncertain. `--allow-stale` (MCP
- `thresholds.allow_stale`) permits inspection with `complete: false` and exit 2;
- it never silently accepts stale evidence as a clean CI result. `complete` means
- these index-content checks passed, not that every language or dependency was
- analyzed. Configured exclusions and unsupported languages remain outside scope;
- extractor metrics left at zero do not prove low complexity. Indexes must use
- comparable analysis versions/configuration. Validation does not certify resolver
- completeness or detect arbitrary manual edits to graph rows.
- Locations do not participate in identity. Git-detected file renames normalize
- function and dependency identities, while qualified symbol names identify functions.
- Symbol renames, moves not detected by Git, and ambiguous overloads cannot be matched
- reliably: unmatched findings explicitly carry `confidence: "unmatched"`, and
- ambiguous same-name/kind/signature metrics are excluded from both snapshots and make the result incomplete. Overloads use their signatures to distinguish otherwise identical qualified names. Review unmatched new/resolved
- pairs before using them as defects; they can still affect CI. Function-size
- findings use source span, not executable statement count. Architecture policy,
- dead-code, test reachability/coverage and module-size regressions are deferred.
- Snapshots are read sequentially in database read transactions. Symbol bodies are
- loaded one indexed file at a time; only compact metric records survive into the
- comparison. File adjacency and the existing iterative SCC algorithm require
- O(files + resolved file dependencies) space/time for cycle analysis. No individual
- cycles are enumerated. Validation uses one Git batch process per snapshot and
- one decoded blob at a time (64 MiB maximum); it reads indexed content plus Git tree
- metadata, without scanning the working tree. Total cost also scales with indexed
- source bytes and symbols; this deliberately trades some I/O for verifiable
- freshness without commit provenance. No caches, worker pools or background jobs
- are created by the comparison. Large JSON reports can be substantial; use MCP
- counts or limited details when token budget matters.
- ### MCP Resources
- Four resources are exposed via `resources/list` and `resources/read`:
- - `tokensave://status` -- graph statistics as JSON
- - `tokensave://files` -- indexed file tree grouped by directory
- - `tokensave://overview` -- project summary with language distribution and symbol kinds
- - `tokensave://branches` -- tracked branches with DB sizes and parent info
- ---
- ## Token Tracking
- tokensave measures the tokens it saves on every MCP tool call. Each tool response includes a `tokensave_metrics: before=N after=M` line showing how many raw-file tokens were avoided by that specific call.
- **Turning the reporting off.** The metrics line, together with a sentence in the MCP `instructions`, asks the agent to report savings to you — which means the model spends *output* tokens narrating a saving tokensave made on *input* tokens. Output tokens are the more expensive kind, so if your agent mentions tokensave on nearly every turn, that narration can offset the win (#356). Set `report_savings` to `false` in `.tokensave/config.json`, or the `TOKENSAVE_REPORT_SAVINGS` environment variable to override it per-run (any value enables it except `0`, `false`, `no`, `off`, or empty). Both the metrics line and the instruction disappear; `tokensave install` likewise stops writing the reporting rule into agent prompt files. Measurement is untouched either way — every call still lands in the savings ledger, so `tokensave gain`, `tokensave list`, `status` and `monitor` keep reporting exactly as before. The default stays `true`.
- **Only the core tools are listed by default.** An MCP client sends the schema of every listed tool on every turn, before any tool is called. The full tokensave surface is about 89 KB of schema, which on a small-context model can be more than half of the window (#576). So `tools/list` sends only the core tools: `context`, `search`, `status`, `read`, `body`, `files`, `callers`, `callees`, `impact`, `str_replace` and `multi_str_replace` (about 17 KB with `tokensave_more`). The `initialize` instructions name these tools and the areas `tokensave_more` can list, so a client that defers tool schemas still knows the rest exist. The setting selects what the server lists, not what it can run: a tool that is not listed still answers a call by name, so agent permission lists and hooks keep working. To list every tool, set `"tools": "full"` in `.tokensave/config.json`, or `TOKENSAVE_TOOLS=full` in the environment of the MCP server; the environment variable wins over the file. A `"tools": "full"` written by 7.13.0, which wrote it into every config it saved, is read as the old default; set it again after upgrading to keep the full list.
- With the core toolset the server also lists `tokensave_more`. A call with an `area` (`analysis`, `edit`, `git`, `memory`, `navigate` or `all`) lists the tools of that area for the rest of the session, and the server sends `notifications/tools/list_changed` so the client fetches the list again. A session pays only for the areas it uses: the `git` area adds about 6 KB. A client that ignores `list_changed` keeps the core list. `tokensave install` grants `tokensave_more` along with every other tool, so it does not prompt; an install from before this release prompts once, until `tokensave reinstall` or the upgrade resync refreshes the permission list.
- ### Cost observability
- ```bash
- tokensave cost # 7-day cost summary (default)
- tokensave cost today # today only
- tokensave cost --by-model # breakdown by Claude model
- tokensave cost --by-task # breakdown by task category (coding, debugging, exploration, ...)
- tokensave cost --export json # JSON export to stdout
- tokensave cost --export csv # CSV export to stdout
- ```
- Parses Claude Code session transcripts (`~/.claude/projects/**/*.jsonl`), classifies each API turn into one of 13 task categories, computes dollar cost using model pricing, and stores results in `~/.tokensave/global.db` for fast aggregate queries. Pricing is refreshed from [LiteLLM](https://github.com/BerriAI/litellm) every 24 hours and falls back to an embedded table when offline.
- The `tokensave status` header includes a cost row showing today's spend, 7-day total, and efficiency ratio (tokens saved / total tokens). The `tokensave monitor` TUI shows a live cost panel alongside the savings feed. At the end of each Claude Code session, the `hook_stop` handler prints a one-line receipt to the terminal.
- Task classification categories: Coding, Debugging, Feature Dev, Refactoring, Testing, Exploration, Planning, Delegation, Git Ops, Build/Deploy, Brainstorming, Conversation, General. Classification is deterministic (pattern matching on tool names and Bash commands), requires no LLM calls, and is adapted from [AgentSeal/codeburn](https://github.com/AgentSeal/codeburn).
- ### Live monitor
- ```bash
- tokensave monitor
- ```
- A global TUI that shows MCP tool calls from all projects in real time, via a shared memory-mapped ring buffer at `~/.tokensave/monitor.mmap`. Each entry shows the project name, tool name, and token delta. A cost panel at the top shows today's spend, savings, efficiency, and top model (refreshed every 30 seconds).
- <p align="center">
- <img src="src/resources/monitor-screenshot.png" alt="tokensave monitor TUI" width="800">
- </p>
- ### Memory diagnostics
- ```bash
- tokensave memory [--clean]
- ```
- A machine-wide memory report for every tokensave process (MCP servers, syncs, index runs), via a shared memory-mapped table at `~/.tokensave/memory.mmap`. Each instance self-samples its RSS best-effort at startup, per MCP tool call, and around the sync/resolution phases, so the report shows current and peak RSS **with the phase that produced the peak** — the data needed to attribute high memory use (see #253). Rows are flagged `alive`, `dead` (an OOM-killed process leaves its peak/phase behind as a forensic record), or `orphan` (still running but reparented to init). `--clean` purges dead slots.
- `PEAK PHASE` names the highest *sample*, so it is only as precise as the sampling. Incremental sync records, in order: `sync:extract`, `sync:resolve:load_nodes`, `sync:resolve:build_caches`, `sync:resolve:refs`, `sync:variants`, `sync:done`. A full index records `index:extract`, `index:resolve:build_caches`, `index:resolve:refs`, `index:resolve:done`, `index:insert`, `index:done`.
- **Each is recorded after the work it names.** They used to be recorded before it, so every sample reported the previous step's RSS under the next step's label — which attributed 73 MiB to the node load that in fact belonged to loading the unresolved references, a step with no sample at all, and pointed a memory investigation at the wrong subsystem for months (#409). If you add a phase, sample after the work, not before it, and add one for any step large enough to hold the peak.
- ### Performance profiling
- ```bash
- tokensave profile enable # opt in (off by default; data stays local)
- tokensave profile report --mcp # latency by tool and phase, ranked by cumulative time
- tokensave profile export # sanitized aggregates to attach to an issue
- ```
- Opt-in latency attribution for database operations, MCP tool phases and indexing (#754). Aggregates are kept under `~/.tokensave/profile/` within a size and retention limit, and nothing is uploaded. See [docs/PROFILING.md](docs/PROFILING.md).
- You can also tune profiling for an individual repository: run `tokensave profile enable --local`, then edit `.tokensave/profile.toml` to adjust its slow-query threshold, retention period and storage limit. See [repository-local controls](docs/PROFILING.md#repository-local-controls) for examples. The `--local` option is hidden from CLI help.
- ### Session and lifetime counters
- ```bash
- tokensave current-counter # show per-project session counter
- tokensave reset-counter # reset the session counter
- tokensave status # shows project + global lifetime totals + cost
- ```
- `tokensave status` renders the project index stats, language breakdown, cost row (today / 7d / efficiency), and project + worldwide lifetime totals:
- <p align="center">
- <img src="src/resources/status-screenshot.png" alt="tokensave status output" width="640">
- </p>
- ### Worldwide counter
- All tokensave users contribute to an anonymous aggregate counter. `tokensave status` shows both your project total and the worldwide total. The upload sends only a single number (e.g. `4823`) with no identifying information. Opt out with `tokensave disable-upload-counter`.
- ---
- ## Index Freshness
- tokensave keeps the graph up to date without a background daemon or an OS-level file watcher.
- **On-demand staleness check.** Every MCP tool call checks whether any indexed files have been modified since the last sync. If stale files are found, they are re-extracted before the tool response is returned. A 30-second cooldown prevents back-to-back calls from re-walking the tree on every keystroke.
- **Catch-up sync on connect.** When the MCP server starts, it immediately runs a non-blocking catch-up sync that picks up any changes made while no agent was attached — a `git pull`, an IDE edit, a build step — so the very first tool call of a session sees a fresh index.
- **Multi-agent work and git worktrees.** When multiple agents work on the same project concurrently, the strong assumption is that each agent operates in its own git worktree. Worktrees are independent filesystem checkouts of the same repository: agent A and agent B each have their own copy of every file, so they never overwrite each other's in-flight edits. tokensave automatically detects when a query comes from a worktree nested inside the main checkout and serves results from the correct branch graph. Changes accumulate independently and are eventually reconciled via git merge or rebase — the same process used for any other parallel development. This design avoids the complexity and failure modes of cross-agent locking over a shared mutable directory.
- **CLI-only workflows.** If you run `tokensave` commands without an attached agent (no MCP server), the staleness check is not running between commands. Install git hooks to keep the index fresh automatically after every commit or clone:
- ```bash
- cp scripts/post-commit scripts/post-checkout .git/hooks/
- chmod +x .git/hooks/post-commit .git/hooks/post-checkout
- ```
- ### Upgrading from 5.x
- The standalone `tokensave daemon` command and its launchd/systemd/Windows Service autostart were removed in 6.0.0. The embedded OS-level file watcher that replaced the daemon was itself removed in 6.1.1 (it caused runaway CPU and memory on large monorepos with deep `node_modules` or `target` trees). The on-demand staleness model above is the current design.
- If you still have a daemon autostart from 5.x, remove it:
- - macOS: `launchctl unload ~/Library/LaunchAgents/com.tokensave.daemon.plist && rm ~/Library/LaunchAgents/com.tokensave.daemon.plist`
- - Linux: `systemctl --user disable --now tokensave-daemon && rm ~/.config/systemd/user/tokensave-daemon.service`
- - Windows: `sc.exe delete tokensave-daemon` (from an elevated terminal)
- If you don't recall the exact name: `launchctl list | grep tokensave` / `systemctl --user list-units | grep tokensave` / `sc.exe query state= all | findstr -i tokensave`.
- ---
- ## Self-Upgrade
- ```bash
- tokensave upgrade # upgrade to latest in current channel
- tokensave channel # show current channel (stable/beta)
- tokensave channel beta # switch to beta channel
- tokensave channel stable # switch back to stable
- ```
- `tokensave upgrade` downloads the correct platform binary from GitHub releases and replaces the running binary in place. Supports stable and beta channels independently.
- ---
- ## Versioning & upgrades
- tokensave version numbers look like SemVer but **do not follow it**: the component that changes encodes the maintenance the update requires, which tokensave performs automatically on the next launch — you never run a reinstall or reindex by hand.
- | Bump | Example | Update requires | Automatic action |
- | --- | --- | --- | --- |
- | **Patch** (`x.y.Z`) | `7.2.0 → 7.2.1` | Nothing | None — no reinstall, no reindex |
- | **Minor** (`x.Y.0`) | `7.2.0 → 7.3.0` | A reinstall (new harnesses, new tools, new config) | **Global reinstall** of every installed agent integration (refreshes permissions, hooks, and MCP config) |
- | **Major** (`X.0.0`) | `7.2.0 → 8.0.0` | A reinstall + full resync | Global reinstall **and** a per-project forced reindex (`sync -f` equivalent) |
- **Global reinstall.** On the first run of a new minor or major build, tokensave silently re-runs `install` for each agent it has registered, so the agent config always points at the current binary and exposes the current tool set. Patch bumps skip this — the running version marker is simply advanced.
- The reinstall is genuinely silent: the per-agent setup output you see from an explicit `tokensave install` is suppressed here, so it never appears in front of an ordinary `tokensave init` or `tokensave sync`. If an agent's config can't be refreshed — the app isn't installed, or its config lives somewhere read-only — you get one line naming the agents that failed:
- ```
- warning: could not refresh tokensave config for: copilot.
- Run tokensave install to see the error.
- ```
- Run `tokensave install` to see the underlying error. The version markers advance either way, so a config path that can never be written is reported once per upgrade rather than retried on every subsequent command.
- **Per-project forced reindex (major only).** A major bump means project indexes must be rebuilt. tokensave does this lazily and per project: on the **first MCP tool call** in a project after a major upgrade, it spawns a background full reindex (equivalent to `tokensave sync --force`) that never blocks the tool response.
- **Brew / cargo fallback.** External upgrades that replace the binary outside of `tokensave upgrade` — `brew upgrade tokensave` or `cargo install tokensave` — are detected the same way: if the running version is newer than the last version that performed an install, the reinstall runs on the next launch just as it would after a self-upgrade.
- See [TOKENSAVE-VERSIONING.md](TOKENSAVE-VERSIONING.md) for why tokensave diverges from SemVer (encoding maintenance in the version is what makes zero-touch upgrades possible), the marker mechanics, the independent database schema version, and the maintainer rules for cutting releases.
- ---
- ## CLI Reference
- ```bash
- tokensave init [path] # Initialize a new project (full index); without a path, the git root
- # (a no-op inside an existing project)
- tokensave init --cpu-usage low # Initialize with a fixed CPU profile (skip the benchmark)
- tokensave config benchmark # Run or reuse the machine-level hardware benchmark
- tokensave config benchmark --show # Show the saved benchmark and settings (never measures)
- tokensave config benchmark --force # Measure this machine again
- tokensave config benchmark --reset # Delete the saved benchmark
- tokensave tune [path] [--dry-run] # Re-benchmark, save globally, update a pinned cpu_usage
- tokensave sync [path] # Incremental sync (must be initialized first)
- tokensave sync --force [path] # Force a full re-index
- tokensave sync --doctor [path] # Sync and list added/modified/removed files
- tokensave verify [path] # Read-only index integrity and source-stat checks
- tokensave verify [path] --deep --json # Full SQLite checks and bounded source hashing
- tokensave status [path] # Show statistics + cost summary
- tokensave status [path] --json # Show statistics (JSON output)
- tokensave status --details # Include node-kind breakdown
- tokensave cost [range] # Token cost summary (default: 7d)
- tokensave cost --by-model # Cost grouped by model
- tokensave cost --by-task # Cost grouped by task category
- tokensave cost --export json|csv # Export cost data
- tokensave query <search> [path] # Search symbols
- tokensave files [--filter dir] [--pattern glob] [--json] # List indexed files
- tokensave affected <files...> [--stdin] [--depth N] # Find affected test files
- tokensave install [--agent NAME] # Configure agent integration
- tokensave reinstall # Refresh settings for all installed agents
- tokensave uninstall [--agent NAME] # Remove agent integration
- tokensave serve [--idle-timeout-secs N] # Start MCP server (N: exit after N idle seconds)
- tokensave servers [--json] # List running servers and the index each one holds
- tokensave monitor # Live TUI showing MCP calls across all projects
- tokensave memory [--clean] # Per-instance RSS report for all tokensave processes
- tokensave upgrade # Self-update to latest version
- tokensave channel [stable|beta] # Show or switch update channel
- tokensave doctor [--agent NAME] # Check installation health
- tokensave githooks [on|off] [--local] # Manage git hooks (--local: this repo only, no core.hooksPath)
- tokensave branch add|list|remove|removeall|gc # Multi-branch management
- tokensave current-counter # Show per-project token counter
- tokensave reset-counter # Reset per-project token counter
- tokensave disable-upload-counter # Opt out of worldwide counter uploads
- tokensave enable-upload-counter # Re-enable worldwide counter uploads
- ```
- ---
- ## `tokensave doctor`
- Run a comprehensive health check of your tokensave installation:
- ```bash
- tokensave doctor
- ```
- Checks: binary location, project index, global DB, user config, agent integration (MCP server, hooks, permissions, prompt rules), and network connectivity. If any tool permissions are missing after an upgrade, it tells you to run `tokensave install`. Use `--agent` to check a specific agent only.
- Doctor also validates that each installed hook uses the correct tokensave subcommand and auto-repairs broken hooks.
- ---
- ## How It Works with Claude Code
- Once configured, Claude Code automatically uses tokensave instead of reading raw files when it needs to understand your codebase. Three layers reinforce each other:
- | Layer | What it does | Why it matters |
- |-------|-------------|----------------|
- | **MCP server** | Exposes 80+ `tokensave_*` tools to Claude | Claude can query the graph directly |
- | **CLAUDE.md rules** | Tells Claude to prefer tokensave over agents/file reads | Prevents the model from falling back to expensive patterns |
- | **PreToolUse hook** | Native Rust hook blocks Explore agents | Catches cases where the model ignores the CLAUDE.md rules |
- | **UserPromptSubmit hook** | Runs at prompt submission | Lifecycle tracking for token accounting |
- | **Stop hook** | Runs when the session ends | Flushes token counters |
- The result: Claude gets the same code understanding with far fewer tokens. A typical Explore agent reads 20-50 files; tokensave returns the relevant symbols, relationships, and code snippets from its pre-built index.
- ---
- ## Network Calls & Privacy
- tokensave's core functionality (indexing, search, graph queries, MCP server) is **100% local** -- your code never leaves your machine.
- | Call | Data sent | When | Opt-out |
- |------|-----------|------|---------|
- | Worldwide counter upload | Token count (a number) + country (from IP) | sync, status, MCP sessions | `tokensave disable-upload-counter` |
- | Worldwide counter read | Nothing (GET request) | status | N/A (read-only, 1s timeout) |
- | Version check | Nothing (GET request) | status (cached 5m), sync (parallel) | N/A (1s timeout, no-op on failure) |
- | Model pricing refresh | Nothing (GET request) | `tokensave cost` (cached 24h) | N/A (5s timeout, falls back to embedded pricing) |
- The worldwide counter upload sends a single HTTP POST with a JSON body like `{"amount": 4823}`. No cookies, no tracking, no user ID. The Cloudflare Worker logs the country of your IP address (derived from request headers) for aggregate geographic statistics -- your actual IP address is not stored.
- Opt-in performance profiling (`tokensave profile`) is local only and makes no network calls.
- The model pricing refresh fetches a public JSON file from GitHub (`raw.githubusercontent.com/BerriAI/litellm/main/model_prices_and_context_window.json`) to keep Claude model pricing up to date for `tokensave cost`. No data is sent -- it is a plain HTTPS GET. The response is cached at `~/.tokensave/pricing.json` for 24 hours. If the fetch fails, tokensave uses its compiled-in pricing table.
- ---
- ## 50+ Languages
- tokensave supports more than 50 programming languages organized into three tiers controlled by Cargo feature flags. Each tier includes all languages from the tier below it. Markdown headers are extracted as `Module` nodes with hierarchical `Contains` edges so document structure participates in graph queries alongside source code.
- ### Lite -- `--no-default-features`
- Always compiled. The smallest binary for the most popular languages, plus Svelte and Astro (script-block extraction via the TypeScript extractor, no extra grammar dependency).
- | Language | Extensions |
- |----------|-----------|
- | Rust | `.rs` |
- | Go | `.go` |
- | Java | `.java` |
- | Scala | `.scala`, `.sc` |
- | TypeScript | `.ts`, `.tsx` |
- | JavaScript | `.js`, `.jsx` |
- | Python | `.py` |
- | C | `.c`, `.h` |
- | C++ | `.cpp`, `.hpp`, `.cc`, `.cxx`, `.hh` |
- | Kotlin | `.kt`, `.kts` |
- | C# | `.cs` |
- | Swift | `.swift` |
- | Svelte | `.svelte` |
- | Astro | `.astro` |
- ### Medium (Lite + 9 more) -- `--features medium`
- | Language | Extensions | Feature flag |
- |----------|-----------|-------------|
- | Dart | `.dart` | `lang-dart` |
- | Pascal | `.pas`, `.pp`, `.dpr` | `lang-pascal` |
- | PHP | `.php` | `lang-php` |
- | Ruby (including ERB and Slim templates) | `.rb`, `.rake`, `.erb`, `.slim` | `lang-ruby` |
- | Bash | `.sh`, `.bash` | `lang-bash` |
- | Protobuf | `.proto` | `lang-protobuf` |
- | PowerShell | `.ps1`, `.psm1` | `lang-powershell` |
- | Nix | `.nix` | `lang-nix` |
- | VB.NET | `.vb` | `lang-vbnet` |
- ERB and Slim templates use the Ruby grammar without requiring Ruby or template gems at indexing time. Template calls belong to the file node and retain their original line and byte column. ERB supports code/output tags, trim markers, comments, and escaped tags. Slim supports control/output lines, indentation-based blocks, attributes (including multiline attribute lists), text interpolation, and `ruby:` blocks.
- Custom Slim shortcuts and embedded-language compilation are not supported. Locals supplied by Rails at render time cannot be distinguished from bare helper calls; names assigned or bound in the template are excluded conservatively. Controller-to-view and partial-render relationships require separate Rails semantic wiring.
- ### Full (Medium + everything else) -- default
- | Language | Extensions | Feature flag |
- |----------|-----------|-------------|
- | ActionScript | `.as` | `lang-actionscript` |
- | Lua | `.lua` | `lang-lua` |
- | Zig | `.zig` | `lang-zig` |
- | Objective-C | `.m`, `.mm` | `lang-objc` |
- | Perl | `.pl`, `.pm` | `lang-perl` |
- | Batch/CMD | `.bat`, `.cmd` | `lang-batch` |
- | Fortran | `.f90`, `.f95`, `.f03`, `.f08`, `.f18`, `.f`, `.for` | `lang-fortran` |
- | COBOL | `.cob`, `.cbl`, `.cpy` | `lang-cobol` |
- | MS BASIC 2.0 | `.bas` | `lang-msbasic2` |
- | GW-BASIC | `.gw` | `lang-gwbasic` |
- | QBasic | `.qb` | `lang-qbasic` |
- | QuickBASIC 4.5 | `.bi`, `.bm` | `lang-qbasic` |
- | Dockerfile | `Dockerfile`, `.dockerfile` | `lang-dockerfile` |
- | GLSL | `.glsl`, `.vert`, `.frag`, `.comp` | `lang-glsl` |
- | Godot Shader | `.gdshader`, `.gdshaderinc` | `lang-glsl` |
- | Minecraft Function | `.mcfunction` | `lang-mcfunction` |
- | WGSL | `.wgsl` | `lang-wgsl` |
- | HLSL | `.hlsl`, `.fx` | `lang-hlsl` |
- | Verilog / SystemVerilog | `.v`, `.vh`, `.sv`, `.svh` | `lang-systemverilog` |
- | VHDL | `.vhd`, `.vhdl` | `lang-vhdl` |
- | Metal | `.metal` | `lang-metal` |
- | CUDA / HIP | `.cu`, `.cuh` | `lang-cuda` |
- | Markdown | `.md`, `.markdown` | `lang-markdown` |
- | R | `.r`, `.R` | `lang-r` |
- | SQL | `.sql` | `lang-sql` |
- | Julia | `.jl` | `lang-julia` |
- | Haskell | `.hs`, `.lhs` | `lang-haskell` |
- | OCaml | `.ml`, `.mli` | `lang-ocaml` |
- | Clojure | `.clj`, `.cljs`, `.cljc` | `lang-clojure` |
- | Erlang | `.erl`, `.hrl` | `lang-erlang` |
- | Elixir | `.ex`, `.exs` | `lang-elixir` |
- | F# | `.fs`, `.fsi`, `.fsx` | `lang-fsharp` |
- | F* | `.fst`, `.fsti` | `lang-fstar` |
- | Quint | `.qnt` | `lang-quint` |
- | Terraform | `.tf`, `.tfvars` | `lang-terraform` |
- | TOML | `.toml` | `lang-toml` |
- | Lean | `.lean` | `lang-lean` |
- | HTML | `.html`, `.htm` | `lang-html` |
- | CSS | `.css` | `lang-css` |
- Individual languages can also be cherry-picked without a full tier:
- ```bash
- cargo install tokensave --no-default-features --features lang-nix,lang-bash
- ```
- All extractors share the same depth: functions, classes, methods, fields, imports, call graphs, inheritance chains, docstrings, complexity metrics, decorator/annotation extraction, and cross-file dependency tracking.
- HTML and CSS are the exception, because neither language has callable symbols. HTML records elements carrying an `id`, custom elements, and the stylesheets and scripts a page pulls in; CSS records class and id selectors, custom properties, `@keyframes` names, and `@import`s. Neither resolves a `class="..."` attribute to a stylesheet rule: class names are ordinary words, and matching them by bare name across a whole project invents edges rather than finding them.
- ---
- ## tokensave vs CodeGraph
- tokensave is a ground-up Rust rewrite of [CodeGraph](https://www.npmjs.com/package/@colbymchenry/codegraph) (Node.js/TypeScript). Both build semantic code graphs for AI coding agents, but they diverge significantly in scope and capabilities.
- | | **tokensave** | **CodeGraph** |
- |---|---|---|
- | **Runtime** | Native binary (Rust) | Node.js 18+ |
- | **Install** | `brew install`, `cargo install`, `scoop install` | `npx @colbymchenry/codegraph` |
- | **Languages** | 50+ (3 tiers: lite/medium/full) | 19+ |
- | **MCP tools** | 80+ | 9 |
- | **Agent integrations** | 12+ (Claude, Codex, Gemini, Qwen, OpenCode, Cursor, Cline, Copilot, Roo Code, Zed, Antigravity, Kilo, Kiro, Kimi, Vibe, Grok, OMP, Pi, Plank, Factory Droid) | 1 (Claude Code) |
- | **Index freshness** | On-demand staleness check on every MCP call; catch-up sync on connect; multi-agent work expected to use git worktrees | Native OS-level file watcher (FSEvents/inotify/ReadDirectoryChangesW, 2 s debounce); catch-up sync on connect |
- | **Multi-branch indexing** | Yes, opt-in (per-branch DBs, cross-branch diff/search) | No |
- | **Complexity metrics** | AST-extracted (branches, loops, nesting depth, cyclomatic & cognitive complexity, Halstead, maintainability index, CRAP) | No |
- | **Porting tools** | Yes (`port_status`, `port_order`) | No |
- | **Graph visualizer** | Removed (v4.0.1) | Yes |
- | **Semantic search** | Agent-driven keyword expansion (zero-cost) | Local embeddings (nomic-embed-text-v1.5 via ONNX) |
- | **MCP resources** | 4 (status, files, overview, branches) | No |
- | **MCP annotations** | Yes (readOnlyHint, alwaysLoad) | No |
- | **Dead code detection** | Yes | No |
- | **Circular dependency detection** | Yes | No |
- | **Type hierarchy** | Yes | No |
- | **God class / coupling analysis** | Yes | No |
- | **Commit / PR context** | Yes | No |
- | **Test mapping** | Yes | No |
- | **Rename preview** | Yes | No |
- | **Token tracking** | Per-call metrics, live TUI monitor, session + lifetime counters | No |
- | **Code-health analytics** | Composite score, Gini, dependency depth, DSM, risk-weighted test gaps, session deltas | No |
- | **Edit primitives** | 4 atomic writers (`str_replace`, `multi_str_replace`, `insert_at`, `ast_grep_rewrite`) with auto re-indexing | No |
- | **Crash resilience** | Subprocess-isolated extraction; native grammar aborts skip the file, sync continues | No |
- | **Self-upgrade** | `tokensave upgrade` with stable/beta channels | `npm update` |
- | **DB engine** | libsql (SQLite fork, WAL, async) | better-sqlite3 / wa-sqlite (WASM) |
- | **Indexing speed** | ~1.2s for 1,782 files | ~4s for 1,782 files |
- | **Binary size** | ~25 MB compressed download (all grammars bundled) | ~80 MB (node_modules + WASM) |
- CodeGraph pioneered the approach and remains a solid choice if you prefer npm tooling and only need Claude Code integration. tokensave extends the concept with deeper analysis, more agents, multi-branch support, and a native binary with no runtime dependencies.
- For detailed comparisons against CodeGraph, Dual-Graph (GrapeRoot), code-review-graph, and OpenWolf, see [docs/COMPARABLE-TOOLS.md](docs/COMPARABLE-TOOLS.md).
- **Side-by-side comparison pages** (each one states where the other tool is better):
- [Serena](https://tokensave.dev/vs-serena) ·
- [code-review-graph](https://tokensave.dev/vs-code-review-graph) ·
- [Graphify](https://tokensave.dev/vs-graphify) ·
- [CodeGraph](https://tokensave.dev/vs-codegraph) ·
- [token-savior](https://tokensave.dev/vs-token-savior) ·
- [GrapeRoot](https://tokensave.dev/vs-graperoot) ·
- [LeanCTX](https://tokensave.dev/vs-leanctx) ·
- [OpenWolf](https://tokensave.dev/vs-openwolf) —
- index at [tokensave.dev/vs](https://tokensave.dev/vs).
- ---
- ## Why tokensave Over the Alternatives
- Several tools reduce token usage for AI coding agents. Here's why tokensave stands apart.
- ### Single native binary, zero dependencies
- Most alternatives require a runtime: Python, Node.js, or both. tokensave ships as a single Rust binary with all 50+ tree-sitter grammars bundled (~25 MB compressed download). Nothing else to install -- and unlike LSP-backed tools such as [Serena](https://tokensave.dev/vs-serena), no language server to install, start or warm up per language. Two tools now match tokensave on packaging: [CodeGraph](https://tokensave.dev/vs-codegraph) bundles its own runtime, and [LeanCTX](https://tokensave.dev/vs-leanctx) is also a single Rust binary.
- ### Deepest code intelligence
- tokensave works at the symbol level: functions, structs, fields, call edges, type hierarchies, complexity metrics. Alternatives like Dual-Graph (GrapeRoot) work at the file level -- they know which files exist but can't answer "who calls this function?" or "what breaks if I change this struct?" tokensave's 80+ specialized MCP tools cover call graph traversal, impact analysis, dead code detection, test mapping, rename preview, type hierarchies, circular dependency detection, complexity ranking, code-health analytics (Gini, DSM, dependency depth, risk-weighted test gaps), atomic edit primitives, and more. The closest comparable graph tool, [code-review-graph](https://tokensave.dev/vs-code-review-graph), exposes 30; several others expose fewer than 10 by deliberate design.
- ### Broadest agent support
- More than a dozen AI coding agent integrations with per-agent native configuration formats. No other tool covers as many agents with as deep an integration. Claude Code gets hooks, prompt rules, and auto-allowed tool permissions. Kiro gets global MCP config, `tokensave.md` steering loaded as a resource, a managed agent with permissive built-in/tokensave tool approval, and hooks for delegation guardrails plus post-write sync. Other agents get MCP server registration in their native config format.
- ### Multi-branch indexing
- The only tool in this space with optional per-branch graph databases and cross-branch diff and search. When enabled, switching branches is instant -- no re-indexing required.
- ### Per-call token tracking
- The only tool that reports exactly how many tokens each individual MCP tool call saved, plus a live TUI monitor across all projects and lifetime counters.
- ### Fully open source
- MIT-licensed Rust, auditable end to end. Dual-Graph's core engine (`graperoot` on PyPI) is proprietary -- you can't see what it does with your code graph. OpenWolf is AGPL-3.0, which requires derivative works to be open-sourced.
- ### Performance
- Full-index benchmark on a 1,782-file mixed Rust/Java/Scala codebase (57K nodes, 103K edges):
- | Tool | Time | Speedup |
- |---|---|---|
- | CodeGraph (TypeScript) | 31.2s | 1x |
- | **tokensave (Rust)** | **1.2s** | **26x** |
- ---
- ## Troubleshooting
- ### "tokensave not initialized"
- The `.tokensave/` directory doesn't exist in your project.
- ```bash
- tokensave init
- ```
- ### MCP server not connecting
- The AI agent doesn't see tokensave tools.
- 1. Ensure the agent config includes the tokensave MCP server (run `tokensave doctor`)
- 2. Restart the agent completely
- 3. Check that `tokensave` is in your PATH: `which tokensave`
- ### Missing symbols in search
- - Run `tokensave sync` to update the index
- - Check that the language is supported (see table above)
- - Verify the file isn't excluded by `.gitignore`
- ### Indexing is slow
- Large projects take longer on the first full index.
- - Subsequent runs use incremental sync and are much faster
- - Use `tokensave sync` (not `--force`) for day-to-day updates
- - Staleness is checked automatically on every MCP tool call while an agent is connected
- ### Disabling tokensave for specific projects
- If a project is too large and tokensave uses too much RAM, you can disable the MCP server per-project by setting `TOKENSAVE_DISABLE_SERVER=true` in its environment. The server exits cleanly without initializing.
- **Claude Code** — add to your project's `.claude/settings.json`:
- ```json
- {
- "mcpServers": {
- "tokensave": {
- "command": "tokensave",
- "args": ["serve"],
- "env": {
- "TOKENSAVE_DISABLE_SERVER": "true"
- }
- }
- }
- }
- ```
- **Other agents** — set the environment variable in whatever config your agent uses to launch MCP servers.
- You can also set it globally via the shell (`TOKENSAVE_DISABLE_SERVER=true claude`), but this disables the tokensave MCP server for every project in the session.
- `DISABLE_TOKENSAVE=true` remains supported as a deprecated compatibility alias for configurations created before this variable was namespaced.
- | [Installation and Upgrades](docs/guide/INSTALLATION.md) | Language tiers, self-upgrade and channels, versioning, building from source |
- | [Agent Integrations](docs/guide/AGENT-INTEGRATIONS.md) | Supported agents, project-local setup, what install writes |
- | [Indexing and Freshness](docs/guide/INDEXING.md) | Project discovery, index freshness, crash-resilient sync, custom extensions |
- | [MCP Tools](docs/guide/MCP-TOOLS.md) | The 80+ tools by category, cross-project queries, MCP resources |
- | [Multi-Branch Indexing](docs/guide/MULTI-BRANCH.md) | Per-branch graphs, automatic tracking, branch quality regressions |
- | [Token Tracking and Cost](docs/guide/TOKEN-TRACKING.md) | Savings ledger, cost reports, live monitor, memory diagnostics |
- | [Performance Profiling](docs/PROFILING.md) | Opt-in latency attribution for database operations, MCP tool phases and indexing |
- | [CLI Reference](docs/guide/CLI-REFERENCE.md) | Every command at a glance |
- | [Open Knowledge Format](docs/guide/OKF.md) | Validate, export and import OKF bundles, and search imported knowledge |
- | [Supported Languages](docs/guide/LANGUAGES.md) | 50+ languages and their feature tiers |
- | [Network Calls and Privacy](docs/guide/PRIVACY.md) | What leaves your machine and how to opt out |
- | [Troubleshooting](docs/guide/TROUBLESHOOTING.md) | `tokensave doctor` and common problems |
- | [Benchmarks](docs/guide/BENCHMARKS.md) | Retrieval-savings and performance benchmarks |
- | [Comparisons](docs/guide/COMPARISONS.md) | tokensave compared with CodeGraph and alternatives |
- The long-form [User Guide](docs/USER-GUIDE.md) walks through everyday use in more
- depth. To contribute, see [CONTRIBUTING.md](CONTRIBUTING.md).
- ---
- ## Origin
- This project is a Rust port of the original [CodeGraph](https://github.com/colbymchenry/codegraph) TypeScript implementation by [@colbymchenry](https://github.com/colbymchenry). The port maintains the same architecture and MCP tool interface while leveraging Rust for performance and native tree-sitter bindings.
- ---
- ## Building
- To build from source, see
- [Building from source](docs/guide/INSTALLATION.md#building-from-source) and
- [CONTRIBUTING.md](CONTRIBUTING.md).
- ## Star History
- <!-- Chart is rendered in CI by .github/workflows/star-history.yml (the hosted
- star-history.com embed broke with GitHub's 2026-06-30 stargazers API
- restriction). The chart images use the upstream master branch. -->
- <!-- star-history:start -->
- <picture>
- <source media="(prefers-color-scheme: dark)" srcset="https://raw.githubusercontent.com/aovestdipaperino/tokensave/master/assets/star-history/star-history-dark.svg">
- <img alt="Star history" src="https://raw.githubusercontent.com/aovestdipaperino/tokensave/master/assets/star-history/star-history-light.svg">
- </picture>
- <!-- star-history:end -->
- ## Sponsors
- <table>
- <tr>
- <td width="64" align="center" valign="middle">
- <img src="src/resources/signpath-logo.svg" alt="SignPath" width="40">
- </td>
- <td valign="middle">
- Free code signing on Windows provided by <a href="https://signpath.io">SignPath.io</a>, certificate by <a href="https://signpath.org">SignPath Foundation</a>
- </td>
- </tr>
- </table>
- ## License
- MIT License -- see [LICENSE](LICENSE) for details.
- **[tokensave.dev](https://tokensave.dev)**
Ratings and comments
Sign in to rate this skill.
Sign inFree to use. If it’s helped, pass it on.