jarvis
JARVIS — a voice assistant for Claude Code. Talk to your Mac and he brainstorms a project with you, builds it, and tells you out loud when a Claude Code session needs you. macOS, on your Claude subscription, no API key.
Safety scan
A static text scan, not a guarantee. Select the badge to see what was scanned and every finding.
Badge for your README
Markdown
[](/listings/ethanplusai--jarvis)HTML
<a href="/listings/ethanplusai--jarvis"><img src="/api/listings/ethanplusai--jarvis/badge.svg" alt="findsafeskills safety rating" /></a>Install
Install instructions from the README
As written by the repo's author. findsafeskills has not verified these steps.
git clone <your fork of this repo> jarvis cd jarvis cp .env.example .env pip install -r requirements.txt python -m playwright install chromium # for read_page / look_at_page cd frontend && npm install && cd ..
**Fill in the `.env`.** `.env.example` documents the lot; the short version is one required key and three optional ones:
FISH_API_KEY=... # required, no fallback # JARVIS_BRAIN_MODEL=sonnet # optional: the brain's model # FISH_VOICE_ID=... # optional: a different voice # USER_NAME=Tony # optional: what he calls you
**Generate the certificates.** These are not optional:
openssl req -x509 -newkey rsa:2048 -keyout key.pem -out cert.pem -days 365 -nodes -subj '/CN=localhost'
`server.py` serves HTTPS whenever `cert.pem` and `key.pem` sit beside it, and `frontend/vite.config.ts` proxies `/api` and `/ws` to `https://localhost:8340`. Without the pair the backend serves plain HTTP, the dev server's proxy cannot reach it, and every API and WebSocket call through the front end fails with a 500 while the page itself still loads — a confusing way to spend an evening. Generate them once and forget about them.
Then, in two terminals:
python server.py --host 127.0.0.1 # terminal 1 cd frontend && npm run dev # terminal 2
Open **Chrome** at `http://localhost:5173`, click the page once to allow audio, and speak. The dashboard is at `http://localhost:5173/dashboard.html`.
> **Who can reach it.** `--host` defaults to `127.0.0.1` — this machine only. > Everything JARVIS serves acts with your full authority: `POST /api/runs` > spawns `claude --dangerously-skip-permissions`, `/api/sessions` reads every > Claude Code conversation you have. A strict `Origin` check stands in front > of every WebSocket and every state-changing route, so a web page you happen > to be visiting cannot open `/ws/voice` and speak as you. There is nothing > to configure: the page is same-origin, so the browser sends an `Origin` no > page can forge. > > An `Origin` is only unforgeable when a *browser* sets it, though. Anything > speaking raw HTTP can claim to be the dashboard, so a client with no > `Origin` — a script, the brain's own MCP child — must instead present the > token in `<data-dir>/jarvis/tool-token`, and on the network the loopback > bind is the rest of the answer. `--host 0.0.0.0` still works if you mean > it; set `JARVIS_ALLOWED_ORIGINS` to the address you will actually open the > page at, or the browser will be turned away too. > > This is one of five deliberate trust decisions in this design — see > [What this trusts, and why](#what-this-trusts-and-why) before you run it.
One more thing about Chrome: the microphone permission is scoped to the **origin including the port**. If you restart Vite and it lands on 5174, the grant does not follow, and Chrome will not re-prompt — it just stays denied, silently. If the mic stops working after everything else looks right, check that the port has not moved.
What was scanned
The files the safety scan read, as written by the repository's author and shown as plain text. Lines with a finding are marked.
README.md · README · 478 lines
View this file on GitHub (read at the repository's current version: the scan did not record which commit it read)
- # JARVIS
- **Just A Rather Very Intelligent System — a voice for Claude Code.**
- JARVIS is a British butler who sits on top of the Claude Code you already pay
- for. You talk to him. He brainstorms a project with you out loud, one question
- at a time; when you have settled on something he writes the design down as a
- file in your project; then he starts a real Claude Code session on it and
- drives it through plan → review → execute. While it runs he watches every
- Claude Code session on your machine, and when one of them is stuck waiting on
- a human he tells you which one, out loud, without you having to look.
- > "Will do, sir."
- 
- *What you actually look at while you talk to him: `frontend/src/orb.ts`,
- rendered live. The audio driving the pulse is synthetic — a speech-shaped
- envelope fitted to a measurement of the real analyser, not a recording of his
- voice — but every pixel is that file running. Regenerate with
- `scripts/make_orb_loop.py`.*
- 
- *The whole dashboard, clicked through. Fictional sample data throughout — the
- projects, prompts, people and figures in every screenshot on this page are
- invented.*
- ---
- ## What it costs
- **No AI API usage, and none is possible.** JARVIS's brain is a Claude Code
- process running on *your* Claude subscription — the same login you use in the
- terminal. There is no Anthropic API key anywhere on the voice path, and there
- is no way to accidentally put one there:
- ```python
- # claude_env.py
- SCRUBBED_ENV_PREFIXES = ("CLAUDE_CODE_", "ANTHROPIC_")
- SCRUBBED_ENV_KEYS = {"CLAUDECODE"}
- ```
- Every Claude Code process JARVIS spawns — the brain and every build — is
- launched through `claude_env.child_env()`, which strips **every** `ANTHROPIC_*`
- variable out of the environment first. This is deliberate and it is not a
- nicety: the CLI silently *prefers* an inherited `ANTHROPIC_API_KEY` over your
- login, and `claude auth status` goes on reporting `loggedIn: true` while
- billing quietly moves onto the key. So JARVIS removes the key rather than
- trusting itself not to pass it. Leave one in your `.env` if you like — the
- startup check will warn you it is there, and the brain will still never see it.
- 
- *So the number that matters is not a dollar figure — it is how much of your
- subscription's two windows is gone. Fictional sample data.*
- **The one thing you pay for is [Fish Audio](https://fish.audio/)**, which
- gives JARVIS his voice. Be aware there is no fallback: `tts.py` returns
- nothing without `FISH_API_KEY`, so JARVIS goes silent and his replies appear
- as text in the browser instead. If you would rather use a different TTS, that
- is a small, well-isolated file to replace — see *Make it yours* below.
- ## What he does
- - **Brainstorms out loud.** The conversation is the design phase. He asks one
- question at a time, offers two or three approaches, and does not start
- anything until you have agreed on one.
- - **Writes the design down.** What you agreed goes on disk as
- `docs/superpowers/specs/YYYY-MM-DD-<topic>-design.md` *inside the project
- being built* — before a single process is spawned. You can read it back by
- numbered section and approve it by voice, or open it on the dashboard.
- - **Drives the build.** A build is a real `claude -p` session handed a brief
- that tells it to write a phased plan, review that plan against the spec,
- then execute it task by task under test-driven development, ticking the
- plan's checkboxes as it goes. "How far has it got" is answered by reading
- those checkboxes, not by guessing.
- - **Watches every Claude Code session on the machine** — not just his own. Ask
- "which of my sessions are waiting on me?" and he checks live. He can post a
- message into one, and answer a permission prompt for one running in
- Terminal.app by pressing a single key.
- - **Interrupts you when it matters.** A session that needs a human gets said
- out loud immediately; a session that merely finished gets batched into one
- sentence at the next pause. If nobody has the browser tab open, it becomes a
- macOS notification instead.
- - **Remembers.** Long-term memory is a folder of plain Markdown files, one
- fact per file, with an index the brain always sees. You can read and edit it
- in any text editor.
- - **Records everything.** Every Claude Code process JARVIS starts is a *run*:
- a row in SQLite with its prompt, project, status, token usage and the full
- event stream. Watch them live at `/dashboard`.
- 
- *The Runs view. Every Claude Code process JARVIS starts is a row here, with
- the prompt that started it. Fictional sample data.*
- The dashboard has six tabs — Runs, Sessions, Memory, Specs, Projects and
- Usage. Usage shows what your subscription's five-hour and seven-day windows
- have left, and who spent it.
- 
- *Sessions, with a blocked one open beside the list. The reason a session is
- stuck is the CLI's own words, not a guess. Fictional sample data.*
- ## Requirements
- - **macOS.** Terminal control, window listing, screenshots and notifications
- all go through AppleScript. There is no Linux or Windows path today.
- - **Google Chrome.** Not a preference — a constraint. The microphone uses the
- Web Speech API (`SpeechRecognition` / `webkitSpeechRecognition`, see
- `frontend/src/voice.ts`), which Firefox has never implemented. There is no
- server-side transcription to fall back on.
- - **Claude Code, installed and logged in.** `npm install -g
- @anthropic-ai/claude-code` (2.1.224 or newer), then run `claude` once and
- log in. This is what JARVIS runs on.
- - **Python 3.11+** and **Node.js 18+**.
- - **A Fish Audio API key.** Required; there is no fallback voice.
- ## Setup
- ```bash
- git clone <your fork of this repo> jarvis
- cd jarvis
- cp .env.example .env
- pip install -r requirements.txt
- python -m playwright install chromium # for read_page / look_at_page
- cd frontend && npm install && cd ..
- ```
- **Fill in the `.env`.** `.env.example` documents the lot; the short version is
- one required key and three optional ones:
- ```env
- FISH_API_KEY=... # required, no fallback
- # JARVIS_BRAIN_MODEL=sonnet # optional: the brain's model
- # FISH_VOICE_ID=... # optional: a different voice
- # USER_NAME=Tony # optional: what he calls you
- ```
- **Generate the certificates.** These are not optional:
- ```bash
- openssl req -x509 -newkey rsa:2048 -keyout key.pem -out cert.pem -days 365 -nodes -subj '/CN=localhost'
- ```
- `server.py` serves HTTPS whenever `cert.pem` and `key.pem` sit beside it, and
- `frontend/vite.config.ts` proxies `/api` and `/ws` to
- `https://localhost:8340`. Without the pair the backend serves plain HTTP, the
- dev server's proxy cannot reach it, and every API and WebSocket call through
- the front end fails with a 500 while the page itself still loads — a confusing
- way to spend an evening. Generate them once and forget about them.
- Then, in two terminals:
- ```bash
- python server.py --host 127.0.0.1 # terminal 1
- cd frontend && npm run dev # terminal 2
- ```
- Open **Chrome** at `http://localhost:5173`, click the page once to allow
- audio, and speak. The dashboard is at `http://localhost:5173/dashboard.html`.
- > **Who can reach it.** `--host` defaults to `127.0.0.1` — this machine only.
- > Everything JARVIS serves acts with your full authority: `POST /api/runs`
- > spawns `claude --dangerously-skip-permissions`, `/api/sessions` reads every
- > Claude Code conversation you have. A strict `Origin` check stands in front
- > of every WebSocket and every state-changing route, so a web page you happen
- > to be visiting cannot open `/ws/voice` and speak as you. There is nothing
- > to configure: the page is same-origin, so the browser sends an `Origin` no
- > page can forge.
- >
- > An `Origin` is only unforgeable when a *browser* sets it, though. Anything
- > speaking raw HTTP can claim to be the dashboard, so a client with no
- > `Origin` — a script, the brain's own MCP child — must instead present the
- > token in `<data-dir>/jarvis/tool-token`, and on the network the loopback
- > bind is the rest of the answer. `--host 0.0.0.0` still works if you mean
- > it; set `JARVIS_ALLOWED_ORIGINS` to the address you will actually open the
- > page at, or the browser will be turned away too.
- >
- > This is one of five deliberate trust decisions in this design — see
- > [What this trusts, and why](#what-this-trusts-and-why) before you run it.
- One more thing about Chrome: the microphone permission is scoped to the
- **origin including the port**. If you restart Vite and it lands on 5174, the
- grant does not follow, and Chrome will not re-prompt — it just stays denied,
- silently. If the mic stops working after everything else looks right, check
- that the port has not moved.
- ## Connections: bring your own
- JARVIS ships connected to nothing, because the useful thing is not our guess at
- what you use — it is the door. Any MCP server works. Put its `mcpServers` block
- into `data/jarvis/connections.json`:
- ```json
- {
- "mcpServers": {
- "notion": {
- "command": "npx",
- "args": ["-y", "@notionhq/notion-mcp-server"],
- "env": { "NOTION_TOKEN": "secret_..." }
- }
- }
- }
- ```
- That is the block from the server's own README, unchanged. Restart JARVIS and
- ask him **"what are you connected to?"** — he answers from what actually
- started, names anything that would not start and why, and tells you what it
- costs him. (About 250 tokens of his context per tool, on every turn — a
- measured figure, and counted against the tools that actually loaded rather than
- the ones you meant to load. It is not charged against how much of your
- conversation he remembers.)
- Your other MCP servers are deliberately ignored. JARVIS runs with
- `--strict-mcp-config`, so nothing in `~/.claude.json` or Claude Desktop reaches
- him unless you put it in that file yourself — connecting something to JARVIS
- should be a thing you did, not a thing you inherited.
- Whatever a connected server returns is treated the way a web page is: reported,
- never obeyed. You vouched for the server's code when you installed it; you did
- not write the ticket, the email, or the shared page it hands back.
- See `skills/jarvis-setup/SKILL.md` for the longer walkthrough — including what
- to do when a server does not start.
- Earlier versions of JARVIS came wired to Apple Calendar, Mail and Notes
- instead. Those are gone: they cost three AppleScript permission prompts on
- first launch, before you had heard him say anything, for an assistant whose job
- is building software. A door you choose is worth more than three we picked.
- ## How it works
- ```
- Microphone → Chrome Web Speech API → WebSocket → FastAPI (server.py)
- │
- ▼
- the brain (brain.py) — ONE long-lived
- `claude -p` process on your subscription
- │
- ┌─────────────────────────────────┼──────────────────────────────┐
- ▼ ▼ ▼
- speech.py → Fish Audio → speaker MCP tools (jarvis_mcp.py session_watch.py
- → POST /internal/tool) (every Claude Code
- │ session on the machine)
- ▼
- RunExecutor → run store (SQLite)
- │
- ▼
- /api/runs + /ws/runs → /dashboard
- ```
- The brain is **one persistent process**, not a request per turn. Your
- transcript is written to its stdin; its reply streams back and is split into
- sentences as it arrives, so the first words are already being spoken while the
- rest is still being written. It reaches JARVIS's own capabilities as MCP tools
- over a stdio channel that forwards to `POST /internal/tool`. The exact set
- is the allowlist at the top of `brain.py`, which is the list to read rather
- than a number to quote here.
- What it does *not* get is a way to act on something it has just read off the
- web. A turn that used `WebSearch`, `WebFetch`, `read_page` or `look_at_page`
- is refused every acting tool for the rest of that turn, and no acting tool
- runs at all unless the turn began with you speaking. Text that arrives from a
- web page, a screenshot or another session's transcript is untrusted, and is
- treated that way.
- Anything spawned as real work goes through one recorded pipeline, and two
- invariants hold throughout it:
- 1. **A run always reaches a terminal state** — `succeeded`, `failed`,
- `timed_out` or `cancelled`. Never stuck in `running`.
- 2. **Every state transition is a database write before it is a
- notification.** The WebSocket is a cache-invalidation hint, never a source
- of truth; the dashboard reconciles against `/api/runs`.
- | Layer | Technology |
- |-------|-----------|
- | Backend | FastAPI + Python (`server.py`) |
- | Frontend | Vite + TypeScript + Three.js (voice UI), vanilla TS (dashboard) |
- | Communication | WebSocket — JSON messages, base64 MP3 audio |
- | Brain | One long-lived `claude -p` process, Sonnet by default, on your subscription |
- | Voice | Fish Audio, one request per sentence |
- | System | AppleScript — Terminal, Chrome, notifications, screenshots |
- | Storage | SQLite for runs and usage; plain Markdown for memory |
- ### Key files
- | File | Purpose |
- |------|---------|
- | `server.py` | The server: WebSocket handler, HTTP API, every `/internal/tool` handler |
- | `brain.py` | The voice brain — spawning, turns, restarts, context rotation |
- | `claude_env.py` | The environment every spawned child gets, including the `ANTHROPIC_*` scrub |
- | `jarvis_mcp.py` | Stdio MCP server exposing JARVIS's tools to the brain |
- | `speech.py` | Sentence splitting, echo rejection, barge-in, and the queue of everything JARVIS says |
- | `tts.py` | Fish Audio synthesis — the whole voice, in one small file |
- | `builds.py` | Spec, brief and plan: the pipeline behind a real multi-hour build |
- | `specs.py` | The review surface — reading a design back by numbered section, and approving it |
- | `run_store.py` | SQLite `runs` / `run_events`, and the six-value status enum |
- | `run_executor.py` | Spawns runs and drives each to a terminal state |
- | `stream_parser.py` | Pure parsing of Claude Code's stream-json output (no I/O) |
- | `session_watch.py` | Watches every Claude Code session on the machine |
- | `session_steer.py` | Posts a message into a running session's inbox socket |
- | `dialog.py` | Presses one key in the Terminal tab that owns a session |
- | `jarvis_memory.py` | Memory as a folder of Markdown files, not a database |
- | `jarvis_home/CLAUDE.md` | JARVIS's persona and rules — the file to edit to change who he is |
- | `preflight.py` | First-run checks: CLI version, login, Accessibility, Fish key |
- | `data_paths.py` | The single source of truth for where JARVIS writes |
- | `frontend/src/voice.ts` | Web Speech API, audio playback |
- | `frontend/src/orb.ts` | The Three.js particle orb |
- | `frontend/src/dashboard/` | The dashboard (vanilla TS, no framework) |
- ## What this trusts, and why
- Five things below are true on purpose. Each is a trade-off JARVIS's design
- made deliberately, not a bug waiting for a fix — know them before you run it.
- **A directory name can put words in JARVIS's mouth.** To say *which* session
- he means — "hammer in Desktop," not "one of them" — JARVIS composes a voice
- name out of directory names and a conversation's title, and accepts up to
- ~60 characters of letters, digits, spaces and light punctuation (`,.-/+'`)
- in it (`_said_name` in `server.py`; `_plain_phrase` accepts the same class
- for a session's `waitingFor`). Anyone who can create a directory on this
- machine, or write a Claude Code roster file's `waitingFor` field, can put
- those words into a sentence JARVIS speaks aloud. There is no separator, no
- `<`, `>`, `"` or `=` in the allowed set, so it cannot forge a whole line or
- close a wrapper — only words, never a fake instruction with structure. The
- alternative is an assistant that cannot name what it is looking at.
- **The taint gate stops action, not persuasion.** Anything JARVIS reads from
- a web page, a file, another session, a run, the screen or a connected MCP
- server is marked untrusted, and every acting tool is refused for the rest
- of that turn — for the two tools that write memory, the rest of that
- generation (`_untrusted_content_refusal`, `TAINTING_TOOLS`, `MEMORY_WRITERS`
- in `server.py`). This does **not** mean text an attacker suggested is never
- acted on — nothing tracks where a sentence in the brain's context came from,
- so a suggestion planted by something JARVIS read is still sitting there when
- you next speak. It means you have to ask again, in your own words, on a turn
- that opened clean. That is the strongest guarantee this design can actually
- keep, not a claim that the suggestion is gone.
- **Every spawned run has full privileges.** `JARVIS_SKIP_PERMISSIONS`
- defaults on, so every run passes `--dangerously-skip-permissions`
- (`run_executor.py`) — there is no TTY to answer a permission prompt, so
- without it a run hangs forever instead of doing its job. A run is therefore
- a full-privilege process on your machine, in whatever directory you pointed
- it at, scrubbed only of `ANTHROPIC_*`/`CLAUDE_CODE_*` environment variables
- (`claude_env.py`), not of filesystem access. It can write anything you could
- write by hand, including the Claude Code session roster and transcripts that
- JARVIS himself later reads back as another session's words.
- **Loopback is trusted.** Anything on this machine that can present an
- `Origin` of `http://localhost:5173` through `5180`, or the API port itself
- (`DEV_SERVER_PORTS`, `JARVIS_DEFAULT_HOST` in `web_auth.py`), gets the same
- access a browser tab gets — including starting a run — because that is the
- same window Vite's own dev-server restarts land in. Any other process on
- your machine bound to one of those ports gets it too. `--host 0.0.0.0`
- widens the same trust to your LAN and prints a warning when you do it. The
- other door is the bearer token in `<data-dir>/jarvis/tool-token`, written
- `0600` (`data_paths.ensure_tool_token`) so only your user account can read
- it.
- **A dropped audio ack degrades hearing for up to 45 seconds.** The browser
- acks each chunk of speech as it finishes playing; JARVIS uses that to relax
- echo rejection for the one- and two-word replies people actually interrupt
- with ("now," "yes"). If the tab crashes or the socket drops, no more acks
- arrive, that relaxation stays off, and a short reply matching JARVIS's last
- sentence can be discarded as an echo of himself — until the `ack_timeout`
- watchdog (45 seconds, `speech.py`) gives up on the chunk and settles it.
- Reload the tab if a short answer seems to be getting ignored.
- ## Make it yours
- This is a starting point, not an appliance. The whole idea is that you clone
- it and bend it to what you do. The seams are deliberately obvious:
- - **His personality** lives in `jarvis_home/CLAUDE.md` — how he speaks, what
- he refuses to say, what he does when he is unsure. It is copied into his
- data directory on first run; edit that copy and it is never overwritten, or
- edit the template and it ships to a fresh install. If you want something
- other than a British butler, this is the only file you need to touch.
- - **His voice** is `tts.py`: one HTTP call, sixty-odd lines. Swap in a
- different provider, or a local model, without touching anything else.
- - **His tools** are the `TOOL_HANDLERS` table in `server.py`, exposed to the
- brain through `jarvis_mcp.py` and gated by an allowlist in `brain.py`.
- Adding one means writing a handler and naming it in those two places.
- - **The look** is `frontend/src/dashboard/theme/` — four CSS files, tokens
- first. `frontend/dashboard-preview.html` renders every component on one page
- against the real stylesheet, so you can redesign without running the
- backend. The screenshots above are renders of that page.
- - **The orb** is `frontend/src/orb.ts`, self-contained Three.js.
- Contributions are welcome, and the most useful ones are the ones this cannot
- do yet: non-macOS system integration, alternative TTS engines, and a mobile
- client. Please open an issue before a large PR.
- ## Development
- ```bash
- pytest
- ```
- That runs the whole suite — around 1,640 tests — and touches neither the
- network nor your screen. Nine browser tests are deselected by default because
- they visit real URLs and `browser.py` launches Chromium with `headless=False`
- on purpose, so they open windows on your desk. Run those deliberately:
- ```bash
- pytest -m browser
- ```
- No test spawns a real `claude` process, and none should. `tests/conftest.py`
- sets `JARVIS_BRAIN_AUTOSTART=0` for the whole suite.
- Everything JARVIS writes — the SQLite database, the memory folder, usage
- tracking, the internal tool token — lives under `data/`, which is ignored in
- full. Point `JARVIS_DATA_DIR` somewhere else to run an instance without
- touching your real data:
- ```bash
- JARVIS_DATA_DIR=/tmp/jarvis-scratch python server.py --host 127.0.0.1
- ```
- ## License
- Free for personal, non-commercial use. Commercial use requires a license —
- visit [ethanplus.ai](https://ethanplus.ai) for inquiries. See
- [LICENSE](LICENSE) for details.
- ## Credits
- Built by [Ethan](https://ethanplus.ai). Runs on
- [Claude Code](https://claude.com/claude-code) and
- [Fish Audio](https://fish.audio).
- Inspired by the AI that started it all — Tony Stark's JARVIS.
- > **Disclaimer:** This is an independent fan project and is not affiliated
- > with, endorsed by, or connected to Marvel Entertainment, The Walt Disney
- > Company, or any related entities. The JARVIS name and character are property
- > of Marvel Entertainment.
Ratings and comments
Sign in to rate this skill.
Sign inFree to use. If it’s helped, pass it on.