Skip to content
Back to search

openclaw-voice

SkillWorks with Claude CodeGitHub

🦞 Open-source browser-based voice chat for AI assistants. Self-hosted, private, free. Whisper STT + ElevenLabs TTS. Works with OpenAI, Claude, or custom agents.

Safety scan

A static text scan, not a guarantee. Select the badge to see what was scanned and every finding.

Checked today
Badge for your README
findsafeskills safety rating

Markdown

[![findsafeskills safety rating](/api/listings/purple-horizons--openclaw-voice/badge.svg)](/listings/purple-horizons--openclaw-voice)

HTML

<a href="/listings/purple-horizons--openclaw-voice"><img src="/api/listings/purple-horizons--openclaw-voice/badge.svg" alt="findsafeskills safety rating" /></a>

Install

i

Install instructions from the README

As written by the repo's author. findsafeskills has not verified these steps.

# Clone
git clone https://github.com/Purple-Horizons/openclaw-voice.git
cd openclaw-voice

# Install
python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt

# Optional: Install VAD support (better noise handling)
pip install torch torchaudio

# Configure (create .env from example)
cp .env.example .env
# Edit .env with your API keys

# Run
PYTHONPATH=. python -m src.server.main

# Open http://localhost:8765

What was scanned

The files the safety scan read, as written by the repository's author and shown as plain text. Lines with a finding are marked.

SKILL.md Β· Manifest Β· 113 lines

View this file on GitHub (read at the repository's current version: the scan did not record which commit it read)

  1. # OpenClaw Voice Skill
  2. Browser-based voice chat for OpenClaw agents.
  3. ## What It Does
  4. Adds voice chat capability to your OpenClaw agent. Users can speak to your agent via a web browser and hear responses in real-time.
  5. ## Stack
  6. - **STT**: faster-whisper (local, no API costs)
  7. - **TTS**: ElevenLabs (cloud, high quality) or Chatterbox (self-hosted)
  8. - **Transport**: WebSocket
  9. - **Backend**: OpenClaw gateway (chatCompletions endpoint)
  10. ## Quick Setup
  11. ### 1. Enable Gateway Endpoint
  12. Add to your `openclaw.json`:
  13. ```json
  14. {
  15. "gateway": {
  16. "http": {
  17. "endpoints": {
  18. "chatCompletions": {
  19. "enabled": true
  20. }
  21. }
  22. }
  23. }
  24. }
  25. ```
  26. ### 2. Add Voice Agent
  27. ```json
  28. {
  29. "agents": {
  30. "list": [
  31. {
  32. "id": "voice",
  33. "workspace": "/path/to/your/workspace",
  34. "model": "anthropic/claude-sonnet-4-5"
  35. }
  36. ]
  37. }
  38. }
  39. ```
  40. ### 3. Run the Server
  41. ```bash
  42. # Clone and setup
  43. git clone https://github.com/Purple-Horizons/openclaw-voice.git
  44. cd openclaw-voice
  45. uv sync # or pip install -r requirements.txt
  46. # Configure
  47. cp .env.example .env
  48. # Edit .env with your OPENCLAW_GATEWAY_URL, OPENCLAW_GATEWAY_TOKEN, ELEVENLABS_API_KEY
  49. # Run
  50. PYTHONPATH=. python -m src.server.main
  51. ```
  52. ### 4. Access
  53. Open `http://localhost:8765` in your browser.
  54. For HTTPS (required for mobile mic), use Tailscale Funnel or your own SSL.
  55. ## Environment Variables
  56. | Variable | Required | Description |
  57. |----------|----------|-------------|
  58. | `OPENCLAW_GATEWAY_URL` | Yes* | OpenClaw gateway URL (e.g., `http://localhost:18789`) |
  59. | `OPENCLAW_GATEWAY_TOKEN` | Yes* | Gateway auth token |
  60. | `ELEVENLABS_API_KEY` | Recommended | For high-quality TTS |
  61. | `OPENAI_API_KEY` | Fallback | Used if gateway not configured |
  62. *Required for full agent integration. Falls back to direct OpenAI if not set.
  63. ## Features
  64. - **Push-to-talk**: Hold button to speak
  65. - **Continuous mode**: Hands-free, auto-listens after responses
  66. - **Keyboard shortcut**: Spacebar toggles recording
  67. - **Mobile support**: Works on phones (requires HTTPS)
  68. ## API Key Auth (Optional)
  69. For production, enable auth:
  70. ```bash
  71. OPENCLAW_REQUIRE_AUTH=true
  72. OPENCLAW_MASTER_KEY=your-secret-key
  73. ```
  74. Then generate user keys via `POST /api/keys`.
  75. ## Files
  76. - `src/server/main.py` - FastAPI server
  77. - `src/server/stt.py` - Speech-to-text (Whisper)
  78. - `src/server/tts.py` - Text-to-speech (ElevenLabs/Chatterbox)
  79. - `src/server/backend.py` - AI backend (gateway or OpenAI)
  80. - `src/client/index.html` - Browser UI
  81. ## License
  82. MIT β€” Built by [Purple Horizons](https://purplehorizons.io)
README.md Β· README Β· 233 lines

View this file on GitHub (read at the repository's current version: the scan did not record which commit it read)

  1. # OpenClaw Voice
  2. **Open-source browser-based voice interface for AI assistants.**
  3. Talk to your AI like you talk to Alexa β€” but self-hosted, private, and connected to your own agent.
  4. ![License](https://img.shields.io/badge/license-MIT-blue.svg)
  5. ![Python](https://img.shields.io/badge/python-3.10+-green.svg)
  6. 🌐 **Website:** [openclawvoice.com](https://openclawvoice.com)
  7. ## Features
  8. | Feature | Description |
  9. |---------|-------------|
  10. | 🎀 **Local STT** | Whisper runs locally via faster-whisper. Your voice never leaves your machine. |
  11. | πŸ”Š **Streaming TTS** | ElevenLabs with sentence-by-sentence streaming. Hear responses while they generate. |
  12. | 🎯 **Voice Activity Detection** | Silero VAD filters background noise. Works in noisy environments. |
  13. | 🧹 **Smart Text Cleaning** | Strips markdown, hashtags, URLs before TTS. No more "hash hash". |
  14. | πŸ”Œ **Any AI Backend** | OpenAI, Claude, or full OpenClaw agent with memory and tools. |
  15. | 🌐 **Browser-Based** | No app install. Works on desktop and mobile. |
  16. | πŸš— **Continuous Mode** | Hands-free conversation. Auto-listens after each response. |
  17. ## Quick Start
  18. ```bash
  19. # Clone
  20. git clone https://github.com/Purple-Horizons/openclaw-voice.git
  21. cd openclaw-voice
  22. # Install
  23. python3 -m venv .venv
  24. source .venv/bin/activate
  25. pip install -r requirements.txt
  26. # Optional: Install VAD support (better noise handling)
  27. pip install torch torchaudio
  28. # Configure (create .env from example)
  29. cp .env.example .env
  30. # Edit .env with your API keys
  31. # Run
  32. PYTHONPATH=. python -m src.server.main
  33. # Open http://localhost:8765
  34. ```
  35. ## For OpenClaw Agents (Auto-Install)
  36. One-liner for AI agents to self-install:
  37. ```bash
  38. cd /tmp && \
  39. git clone https://github.com/Purple-Horizons/openclaw-voice.git && \
  40. cd openclaw-voice && \
  41. python3 -m venv .venv && source .venv/bin/activate && \
  42. pip install -r requirements.txt torch torchaudio && \
  43. PYTHONPATH=. ELEVENLABS_API_KEY="$ELEVENLABS_API_KEY" OPENAI_API_KEY="$OPENAI_API_KEY" \
  44. nohup python -m src.server.main > /tmp/voice-server.log 2>&1 &
  45. ```
  46. ## Configuration
  47. ### Environment Variables
  48. | Variable | Required | Default | Description |
  49. |----------|----------|---------|-------------|
  50. | `ELEVENLABS_API_KEY` | Yes* | β€” | ElevenLabs API key for TTS |
  51. | `OPENAI_API_KEY` | Yes* | β€” | OpenAI API key (if not using gateway) |
  52. | `OPENCLAW_GATEWAY_URL` | No | β€” | OpenClaw gateway URL for full agent |
  53. | `OPENCLAW_GATEWAY_TOKEN` | No | β€” | Gateway auth token |
  54. | `OPENCLAW_PORT` | No | `8765` | Server port |
  55. | `OPENCLAW_STT_MODEL` | No | `base` | Whisper model size |
  56. | `OPENCLAW_STT_DEVICE` | No | `auto` | Device: `auto`, `cpu`, `cuda`, `mps` |
  57. | `OPENCLAW_REQUIRE_AUTH` | No | `false` | Require API keys for clients |
  58. *One of `OPENAI_API_KEY` or `OPENCLAW_GATEWAY_URL` required.
  59. ### Whisper Model Sizes
  60. | Model | Speed | Quality | VRAM | Best For |
  61. |-------|-------|---------|------|----------|
  62. | `tiny` | Fastest | Fair | ~400MB | Quick testing |
  63. | `base` | Fast | Good | ~1GB | **Default. Good balance.** |
  64. | `small` | Medium | Better | ~2GB | Clearer transcription |
  65. | `medium` | Slower | Great | ~5GB | Accuracy priority |
  66. | `large-v3-turbo` | Slow | Best | ~6GB | Maximum accuracy |
  67. ### TTS Options
  68. | Backend | Type | Quality | Latency | Notes |
  69. |---------|------|---------|---------|-------|
  70. | **ElevenLabs** | Cloud | Excellent | ~500ms | Default. Streaming supported. |
  71. | Chatterbox | Local | Very Good | ~1s | MIT license, voice cloning |
  72. | XTTS-v2 | Local | Excellent | ~1s | Voice cloning supported |
  73. | Mock | Local | None | 0ms | For testing (silence) |
  74. ElevenLabs uses `eleven_turbo_v2_5` for fastest response.
  75. ## OpenClaw Gateway Integration
  76. Connect to your full OpenClaw agent (same memory, tools, and persona as text chat):
  77. ```bash
  78. # .env
  79. OPENCLAW_GATEWAY_URL=http://localhost:18789
  80. OPENCLAW_GATEWAY_TOKEN=your-token
  81. ELEVENLABS_API_KEY=your-key
  82. ```
  83. Add to your `openclaw.json`:
  84. ```json
  85. {
  86. "gateway": {
  87. "http": {
  88. "endpoints": {
  89. "chatCompletions": { "enabled": true }
  90. }
  91. }
  92. },
  93. "agents": {
  94. "list": [
  95. {
  96. "id": "voice",
  97. "workspace": "/path/to/workspace",
  98. "model": "anthropic/claude-sonnet-4-5"
  99. }
  100. ]
  101. }
  102. }
  103. ```
  104. ## Architecture
  105. ```
  106. β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” WebSocket β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
  107. β”‚ Browser │◄────────────►│ Voice Server β”‚
  108. β”‚ (mic/spk) β”‚ β”‚ β”‚
  109. β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚
  110. β”‚ β”‚ Whisper β”‚β†’β”‚ AI β”‚β†’β”‚ElevenLabsβ”‚ β”‚
  111. β”‚ β”‚ (STT) β”‚ β”‚ β”‚ β”‚ (TTS) β”‚ β”‚
  112. β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚
  113. β”‚ ↑ β”‚ β”‚
  114. β”‚ [VAD] [streaming] β”‚
  115. β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
  116. ```
  117. **Streaming Flow:**
  118. 1. User speaks β†’ Whisper transcribes locally
  119. 2. AI responds (streamed) β†’ buffer sentences
  120. 3. First sentence complete β†’ TTS starts immediately
  121. 4. Audio streams to browser while AI continues
  122. 5. Result: ~50% faster perceived response
  123. ## HTTPS for Mobile
  124. Mobile browsers require HTTPS for microphone access. Options:
  125. **Tailscale Funnel (easiest):**
  126. ```bash
  127. tailscale funnel 8765
  128. # Access via https://your-machine.tailnet-name.ts.net
  129. ```
  130. **nginx + Let's Encrypt:**
  131. ```nginx
  132. server {
  133. listen 443 ssl;
  134. server_name voice.yourdomain.com;
  135. location / {
  136. proxy_pass http://127.0.0.1:8765;
  137. proxy_http_version 1.1;
  138. proxy_set_header Upgrade $http_upgrade;
  139. proxy_set_header Connection "upgrade";
  140. }
  141. }
  142. ```
  143. ## API
  144. ### WebSocket Protocol
  145. Connect to `ws://localhost:8765/ws`:
  146. ```javascript
  147. // Start recording
  148. { "type": "start_listening" }
  149. // Send audio (base64 PCM float32, 16kHz)
  150. { "type": "audio", "data": "base64..." }
  151. // Stop recording
  152. { "type": "stop_listening" }
  153. // Receive events:
  154. { "type": "transcript", "text": "...", "final": true }
  155. { "type": "response_chunk", "text": "..." } // Streaming text
  156. { "type": "audio_chunk", "data": "...", "sample_rate": 24000 } // Streaming audio
  157. { "type": "response_complete", "text": "..." } // Full response
  158. { "type": "vad_status", "speech_detected": true } // VAD feedback
  159. ```
  160. ## Roadmap
  161. - [x] WebSocket voice gateway
  162. - [x] Whisper STT (local)
  163. - [x] ElevenLabs TTS
  164. - [x] Streaming TTS (sentence-by-sentence)
  165. - [x] Voice Activity Detection (Silero)
  166. - [x] Text cleaning (markdown/hashtags/URLs)
  167. - [x] Continuous conversation mode
  168. - [x] OpenClaw gateway integration
  169. - [ ] WebRTC for lower latency
  170. - [ ] Voice cloning UI
  171. - [ ] Docker support
  172. ## License
  173. MIT License β€” see [LICENSE](LICENSE).
  174. ## Credits
  175. - [faster-whisper](https://github.com/guillaumekln/faster-whisper) β€” Local STT
  176. - [ElevenLabs](https://elevenlabs.io) β€” Text-to-Speech
  177. - [Silero VAD](https://github.com/snakers4/silero-vad) β€” Voice Activity Detection
  178. - Built for [OpenClaw](https://openclaw.ai)
  179. ---
  180. **Made with 🦞 by [Purple Horizons](https://purplehorizons.io)**

Ratings and comments

Sign in to rate this skill.

Sign in

Free to use. If it’s helped, pass it on.