jev-migration-analyzer
Scan a codebase to audit LLM calls (Claude, OpenAI, Bedrock, OpenRouter, LiteLLM, Vercel AI SDK) and evaluate which use cases can be migrated to TypeSafe AI's Jev System One decision model. Use when the user asks to analyze LLM usage for Jev migration, reduce frontier LLM costs or latency, replace LLM classifiers/routers/guardrails/rerankers with Jev, or generate a migration ROI and refactoring plan.
Safety scan
A static text scan, not a guarantee. Select the badge to see what was scanned and every finding.
Badge for your README
Markdown
[](/listings/rambmehta--jev-migration-analyzer)HTML
<a href="/listings/rambmehta--jev-migration-analyzer"><img src="/api/listings/rambmehta--jev-migration-analyzer/badge.svg" alt="findsafeskills safety rating" /></a>Install
Install instructions from the README
As written by the repo's author. findsafeskills has not verified these steps.
Requirements: Claude Code and Python 3.10+. Web search in Claude Code is optional and is used for live token prices.
**For all your projects:**
git clone <repo-url> ~/.claude/skills/jev-migration-analyzer
**For one project only:** place the folder at `<your-repo>/.claude/skills/jev-migration-analyzer/`.
On Windows, `~` is `C:\Users\<you>`. Start a new Claude Code session after installing; skills load when a session starts.
What was scanned
The files the safety scan read, as written by the repository's author and shown as plain text. Lines with a finding are marked.
SKILL.md · Manifest · 119 lines
- ---
- name: jev-migration-analyzer
- description: Scan a codebase to audit LLM calls (Claude, OpenAI, Bedrock, OpenRouter, LiteLLM, Vercel AI SDK) and evaluate which use cases can be migrated to TypeSafe AI's Jev System One decision model. Use when the user asks to analyze LLM usage for Jev migration, reduce frontier LLM costs or latency, replace LLM classifiers/routers/guardrails/rerankers with Jev, or generate a migration ROI and refactoring plan.
- ---
- # Jev Migration Analyzer
- Scan a codebase to inventory every frontier LLM call, disqualify unsuitable generative or high-risk workloads, evaluate trade-offs (latency, cost, observability, zero-trust safety), compute ROI, and generate copy-pasteable refactoring diffs and shadow-mode rollout wrappers for TypeSafe AI's **Jev** (`jev-latest` / `jev-1.14.0`).
- ## When to Use
- - The user wants to audit a codebase for Claude, OpenAI, Bedrock, OpenRouter, or gateway LLM calls that can be offloaded to Jev.
- - The user asks how to cut LLM latency or token costs on classification, routing, scoring, guardrail checks, or RAG reranking.
- - The user wants a migration roadmap, trade-off analysis, code diffs, or shadow-mode validation wrapper for adopting Jev.
- ## Core Execution Workflow
- Follow these six steps in order whenever auditing a repository.
- ### Step 1: Inventory All LLM Call Sites
- Run `scripts/scan_llm_calls.py` against the target directory (or use targeted search if Python execution is unavailable) to locate all direct SDK calls, cloud gateways, and abstractions:
- ```bash
- python3 scripts/scan_llm_calls.py <target_path>
- ```
- On Windows, use `python` or `py -3` instead of `python3` (the default install does not provide `python3`). The scripts need Python 3.10+.
- The scanner is line-based. If the codebase routes calls through its own wrapper (e.g. `callLLM()`), the scanner reports only the wrapper: search for the wrapper's callers and treat each as a separate call site.
- Inspect every discovered call site for:
- - **Provider & Model:** Anthropic, OpenAI, Google GenAI, AWS Bedrock (`InvokeModel`/`Converse`), OpenRouter, LiteLLM, Vercel AI SDK (`generateObject`/`generateText`), LangChain, LlamaIndex.
- - **Input & Context Shape:** State payload size (tokens), multimodal inputs (PNG/JPEG vs. audio/video), and whether adjacent calls share the same input context.
- - **Output Schema & Control Flow:** Zod schemas, Pydantic models, JSON mode, tool calls used for structured extraction, and downstream `if`/`switch` branching or side effects.
- ### Step 2: Apply the "When NOT to Use Jev" Disqualification Gate
- Before recommending Jev, test every call site against these 8 disqualification rules. If any rule triggers, classify the call as **Tier 3: Keep on Frontier LLM / Deterministic Code** and explain why:
- 1. **Generative or Open-Ended Text Output:** The app requires newly written prose, replies, summaries, code, explanations, or extraction of unknown strings (arbitrary names, addresses, custom text). Jev emits zero prose.
- 2. **Explainability or Regulatory Audit Proof:** Compliance, legal review, or UX requires a human-readable rationale or chain of evidence explaining *why* a decision was made. Jev returns calibrated probabilities (`97%`), not reasoning or proof.
- 3. **Exact Arithmetic, Counting, or Date Math:** The prompt asks the model to count items, check numeric thresholds (`amount > 500`), or compare dates. Jev treats numbers and dates as text patterns; move these to deterministic application code.
- 4. **Identity, Permissions, or Sole Security Boundaries:** The call enforces authentication, RBAC permissions, or prompt-injection defense on untrusted inputs. Upstream poisoned data can trigger "confident garbage out."
- 5. **Unbounded High-Blast-Radius Actions:** The decision directly executes irreversible side effects (financial transfers, account deletion, server shutdown) without human-in-the-loop escalation or multi-signal verification.
- 6. **Multi-Hop Reasoning ("System 2" Logic):** The task chains multiple inferential steps or plans across open-ended unknowns. Jev is a single-hop System One model.
- 7. **Context >64k Tokens or Audio/Video Inputs:** Requests exceed 64k total tokens (32k per question) without chunking/long-context opt-in, or require audio/video understanding. (Note: `jev-1.14.0` supports PNG/JPEG images and `Box` localization with the opt-in header, not audio/video.)
- 8. **Compound "Kitchen-Sink" Logic Without Decomposition:** A single prompt packs multiple unrelated conditions into one boolean check without splitting them into separate atomic Jev questions.
- ### Step 3: Classify Candidates & Detect High-Value Patterns
- Assign every surviving call site to a migration tier and actively scan for four optimization patterns (see `references/jev_primitives_guide.md`):
- - **Tier 1 (Direct Replacement):** Bounded, low-blast-radius decisions using `Choice` (up to 255 options), `Score` (ordered 2-10 level rubric), `Noul` (calibrated 0-1 yes/no probability), or `Box` (image region localization on `jev-1.14.0`).
- - **Tier 2 (Hybrid, Cascade, or Guarded Migration):**
- - **Confidence-Gated Cascade:** Route 100% of traffic to Jev first (~70-500ms). Auto-resolve when `confidence >= 0.85` (or calibrated `noul` threshold); escalate ambiguous cases (`< 0.85`) to Claude/GPT-4o.
- - **RAG Chunk Pre-Filter / Reranker:** Insert a batch Jev `Score`/`Noul` pass over retrieved vector DB chunks (Pinecone, pgvector, Weaviate, Qdrant, Chroma) to filter irrelevant or adversarial passages before LLM synthesis.
- - **High-Blast-Radius Guarded Flow:** Decompose macro-decisions into granular sub-questions (`Noul`/`Choice`) evaluated in parallel so every contributing signal is logged, paired with human escalation on uncertain or high-impact outcomes.
- - **Anti-Pattern Detectors (Flag Explicitly):**
- - **Multi-Call Compaction:** Group sequential or `Promise.all`/`asyncio.gather` LLM calls operating on the same input context and merge them into a single `POST /v1/systemone` request with multiple parallel questions.
- - **Chain-of-Thought (CoT) Waste:** Flag Zod/Pydantic fields like `reasoning`, `thought_process`, or `explanation` added solely to improve classification accuracy and discarded afterward. Jev uses RLCD-calibrated probabilities and needs zero CoT output tokens.
- ### Step 4: Evaluate Advantages vs. Drawbacks Per Call Site
- Document both sides concretely for each candidate:
- - **Advantages:** 40x-200x lower latency (70-500ms vs. multi-second generation), $0.042/1M input tokens with **$0 output token cost**, native RLCD-calibrated probabilities, 100% type-safe responses (zero schema/JSON repair retries), and parallel multi-question batching over shared state.
- - **Drawbacks & Risks:** Black-box lack of natural-language rationale ("confidence is not proof"), vulnerability to upstream input poisoning ("confident garbage out"), silent behavioral drift if unmonitored, closed-world option constraints (always include an `other` / `none_of_the_above` bucket in `Choice`), and `jev-latest` alias drift unless pinned to a specific version ID like `jev-1.14.0`.
- ### Step 5: Calculate Monetary & Time ROI
- #### 5a. Look up current token prices (before running the calculator)
- The price table built into `scripts/roi_calculator.py` is a dated fallback. Before calculating, look up live list prices with your web search / web fetch tools:
- 1. **List the distinct models** found in Step 1 (e.g. `claude-sonnet-4-20250514`, `gpt-4o-mini`, a Bedrock model ID), plus Jev.
- 2. **Search each provider's official pricing page**, in this order of preference:
- - Anthropic: `claude.com/pricing` or the pricing page in the Claude API docs
- - OpenAI: `openai.com/api/pricing`
- - Google: the Gemini API or Vertex AI pricing page
- - AWS Bedrock: `aws.amazon.com/bedrock/pricing`. Bedrock and Azure prices can differ from the model vendor's own price, so use the platform the code actually calls.
- - OpenRouter: the model's page on `openrouter.ai`
- - Jev: TypeSafe's own docs/site first, then OpenRouter or AI/ML API's Jev page
- Fall back to a reputable third-party price list only if the official page cannot be reached, and say so in the report.
- 3. **Record for each model** in the report's "Token Prices Used" table: input and output USD per 1M tokens, the exact source URL, and the date checked. Use standard on-demand prices; note but do not apply prompt-caching, batch, or long-context tiers unless the code uses them (e.g. `cache_control` blocks or a batch API). Check that the page's model name matches the code's exact model or version, not just the same family.
- 4. **Pass the prices to the calculator** with `--price-in`, `--price-out` and `--jev-price-in`.
- 5. **If web tools are unavailable or a lookup fails**, run with the built-in table, and label those figures in the report as "fallback price, as of the date in `roi_calculator.py`, unverified". Never present fallback prices as current.
- Send only public model names in search queries, never code, repository names, or customer data.
- If the user can share actual spend (billing dashboard or invoices), prefer it over list prices: real costs include caching, batch, and negotiated discounts that list prices miss.
- #### 5b. Run the calculator
- Run `scripts/roi_calculator.py` (or apply its exact formulas) across 10k, 100k, and 1M monthly call tiers, using the prices from 5a:
- ```bash
- python3 scripts/roi_calculator.py --calls 100000 --in-tokens 1200 --out-tokens 150 --model claude-sonnet-4 --price-in 3.00 --price-out 15.00 --jev-price-in 0.042 --latency-ms 1800 --mode direct
- ```
- Use `--compaction N` when N calls are merged into one Jev request (applies to both modes).
- Report monthly dollar savings, percentage cost reduction, per-request mean latency drop (the calculator models averages, not p50), and total synchronous compute/user-wait hours saved per month across Direct Replacement, Multi-Call Compaction, and 85%-Confidence Cascade modes.
- ### Step 6: Generate Refactoring Diffs, Observability & Shadow Rollout
- Using `references/refactoring_patterns.md` and `references/shadow_mode_wrappers.md`, deliver:
- 1. **Before/After Code Diffs:** Convert the existing Zod/Pydantic schemas and prompts into `@typesafe-ai/sdk` (TypeScript) or `typesafe-sdk` (Python) calls against `client.system_one()`. Always remind engineers that question IDs are not visible to the model—put full instructions inside `instructions` and clear descriptions in `criteria`.
- 2. **Zero-Trust Observability Wrapper:** Include structured audit logging for input hashes, pinned `model` version (`jev-1.14.0`), all sub-question probabilities, and rolling approval-rate drift alerts.
- 3. **Non-Blocking Shadow Mode Wrapper:** Provide a drop-in async wrapper that runs Jev alongside the existing frontier LLM call in production, logging agreement rates and latency deltas before traffic cutover.
- Format the final deliverable using `assets/audit_report_template.md`.
- ## Gotchas
- - **Question IDs Are Invisible to Jev:** Never rely on a question key like `"is_refund_eligible"` to convey meaning. Jev only reads `instructions` and `criteria`; write self-contained instructions for every question.
- - **Always Include a Fallback Option in `Choice`:** Without an `"other"` or `"none_of_the_above"` option, Jev must distribute 100% of probability mass across your listed keys even when none apply.
- - **Pin Model Versions in Production:** Using `jev-latest` can shift probability distributions when TypeSafe ships a new release. Pin to `jev-1.14.0` once confidence thresholds are tuned.
- - **Never Treat Confidence as Proof:** For security, financial, or destructive workflows, enforce input validation, decompose into atomic sub-questions for auditability, and keep human-in-the-loop escalation.
README.md · README · 130 lines
- # Jev Migration Analyzer
- A [Claude Code](https://claude.com/claude-code) skill that audits a codebase's LLM calls and identifies which ones can move to TypeSafe AI's **Jev** decision model, with cost and latency estimates and ready-to-use code changes.
- > **Disclaimer**
- > - This project is not affiliated with or endorsed by TypeSafe AI. Jev, Claude, OpenAI, Gemini and other product names belong to their respective owners.
- > - Jev is in early access and its API may change. Facts in this skill were checked against public sources as of November 2026; claims that could not be verified are listed under [Known limitations](#known-limitations).
- > - Every recommendation needs human review before it reaches production. See the [LICENSE](LICENSE): the software is provided as is, without warranty.
- ## What it does
- When you ask Claude Code to audit a repository, the skill walks through six steps:
- 1. **Inventory**: finds LLM calls (Anthropic, OpenAI, Google, AWS Bedrock, Azure, OpenRouter, LiteLLM, LangChain, Vercel AI SDK and more).
- 2. **Disqualify**: screens out calls Jev should not handle, such as prose generation, multi-step reasoning, exact arithmetic, security boundaries and irreversible actions.
- 3. **Classify**: sorts the remaining calls into tiers (direct replacement, confidence-gated cascade, or keep as is) and flags waste such as unused chain-of-thought fields or several calls on the same input.
- 4. **Trade-offs**: lists the advantages and drawbacks for each call.
- 5. **ROI**: looks up current token prices on the web and estimates monthly cost and latency savings.
- 6. **Code changes**: produces before/after diffs plus a shadow-mode wrapper to compare Jev with your current model in production before switching.
- The result is a report like this:
- | Priority | File & Line | Current Model | Tier & Pattern | Jev Primitives |
- | :--- | :--- | :--- | :--- | :--- |
- | P1 | `src/tickets.ts:42` | Claude Sonnet | Tier 1: direct replace + CoT removal | `choice` + `noul` |
- | P2 | `app/router.py:88` | GPT-4o via OpenRouter | Tier 2: confidence cascade | `choice` |
- | Keep | `src/writer.ts:15` | Claude Sonnet | Tier 3: generates prose | none |
- ## What it doesn't do
- - It **never edits your code**. It only suggests changes.
- - It **never calls Jev** or any LLM API itself.
- - It **doesn't prove accuracy**. Validate every migration on your own labelled data or with the included shadow-mode wrapper.
- ## Install
- Requirements: Claude Code and Python 3.10+. Web search in Claude Code is optional and is used for live token prices.
- **For all your projects:**
- ```bash
- git clone <repo-url> ~/.claude/skills/jev-migration-analyzer
- ```
- **For one project only:** place the folder at `<your-repo>/.claude/skills/jev-migration-analyzer/`.
- On Windows, `~` is `C:\Users\<you>`. Start a new Claude Code session after installing; skills load when a session starts.
- ## Usage
- In Claude Code, type `/jev-migration-analyzer`, or ask something like:
- > Audit this repo's LLM calls for Jev migration.
- ### Running the scripts directly
- Both scripts work on their own and print JSON. On Windows, use `python` or `py -3` instead of `python3`.
- **Scanner:** lists LLM call sites and anti-patterns:
- ```bash
- python3 scripts/scan_llm_calls.py <path-to-repo>
- ```
- **ROI calculator:** compares current cost and latency with Jev:
- ```bash
- python3 scripts/roi_calculator.py --calls 100000 --in-tokens 1200 --out-tokens 150 \
- --model claude-sonnet-4 --price-in 3.00 --price-out 15.00 --jev-price-in 0.042 --mode direct
- ```
- | Option | Meaning | Default |
- | :--- | :--- | :--- |
- | `--calls` | Monthly request volume | 100000 |
- | `--in-tokens` / `--out-tokens` | Average tokens per call | 1000 / 120 |
- | `--model` | Baseline model from the built-in table | `claude-sonnet-4` |
- | `--price-in` / `--price-out` | Current baseline price, USD per 1M tokens | built-in table |
- | `--jev-price-in` | Current Jev input price, USD per 1M tokens | 0.042 |
- | `--latency-ms` | Measured baseline latency | built-in estimate |
- | `--mode` | `direct` (full replacement) or `cascade` (Jev first, fall back when unsure) | `direct` |
- | `--offload-rate` | Share of requests Jev resolves in cascade mode | 0.80 |
- | `--compaction` | Number of separate LLM calls merged into one Jev request | 1 |
- The built-in price table is a fallback and may be out of date. The output always shows whether each price was supplied or came from the table.
- ## Privacy and safety
- - **Scanner:** reads local files only and makes no network calls.
- - **ROI calculator:** makes no network calls.
- - **Price lookups:** Claude searches the web using public model names only, never your code, repository names or data.
- - **Shadow-mode wrappers:** log a hash of each input by default. Logging raw input for disagreements is opt-in; redact personal data before turning it on.
- ## Known limitations
- - **Scanner:** line-based pattern matching rather than full parsing. Calls routed through your own wrapper function are reported once at the wrapper. Covers Python and JavaScript/TypeScript only, and doesn't detect raw HTTP calls.
- - **Routing:** code changes target TypeSafe's native SDK. Routing Jev through OpenRouter or the Vercel AI SDK isn't covered yet.
- - **Alternatives:** the skill doesn't suggest cheaper options outside Jev, such as smaller models, prompt caching or batch APIs.
- - **Prices:** list prices exclude caching, batch and negotiated discounts. Your own billing data is more accurate.
- - **Unverified Jev claims:** these couldn't be confirmed in public sources and should be checked against TypeSafe's documentation:
- - the term "RLCD-calibrated"
- - the `TypeSafe-Danger-Zone-Images` header
- - the `jev-1.14.0` version string
- - `box` as a direct-replacement primitive (documented as experimental)
- - the 40x–200x latency figure
- - how `score` values are numbered
- ## Folder layout
- ```text
- jev-migration-analyzer/
- ├── SKILL.md # Instructions Claude follows
- ├── scripts/
- │ ├── scan_llm_calls.py # Finds LLM call sites
- │ └── roi_calculator.py # Cost and latency estimates
- ├── references/
- │ ├── jev_primitives_guide.md # Jev API and question types
- │ ├── refactoring_patterns.md # Before/after code examples
- │ └── shadow_mode_wrappers.md # Production comparison wrappers
- ├── assets/
- │ └── audit_report_template.md # Report format
- └── LICENSE
- ```
- ## Contributing
- Pull requests are welcome. The most useful ones correct Jev facts as the API changes: please include a link to the source.
- ## License
- MIT, see [LICENSE](LICENSE).
Ratings and comments
Sign in to rate this skill.
Sign inFree to use. If it’s helped, pass it on.