Skip to content
Back to search

dsh-voice

Full-duplex voice plugin for DeepSeek Harness: mic → SenseVoice ASR (sherpa-onnx) → LLM → Edge TTS with true barge-in. 全双工语音插件:麦克风语音输入,SenseVoice 本地识别(简体中文+标点+ITN),流式 TTS 朗读,支持语音打断。Zero API key.

Safety scan

A static text scan, not a guarantee. Select the badge to see what was scanned and every finding.

Checked today
Badge for your README
findsafeskills safety rating

Markdown

[![findsafeskills safety rating](/api/listings/haoku123--dsh-voice/badge.svg)](/listings/haoku123--dsh-voice)

HTML

<a href="/listings/haoku123--dsh-voice"><img src="/api/listings/haoku123--dsh-voice/badge.svg" alt="findsafeskills safety rating" /></a>

Install

i

Install instructions from the README

As written by the repo's author. findsafeskills has not verified these steps.

dsh plugin --profile web add <repo-url-or-path>
dsh --profile web

Note: needs Node ≥ 22.19 or ≥ 24 (`node:zlib` zstd APIs). Works with dsh 0.1.0-rc.6 and 0.2.x — the `@deepseek-ai/dsh-web` peer range covers both.

What was scanned

The files the safety scan read, as written by the repository's author and shown as plain text. Lines with a finding are marked.

README.md · README · 154 lines
  1. # dsh-voice
  2. English | [简体中文](./README.zh-CN.md)
  3. Full-duplex voice mode for DeepSeek Harness: streamed ASR → LLM → TTS with barge-in.
  4. ## Status
  5. **v0.8.2 — press-to-talk with a live caption.**
  6. Speak into the composer mic: the assistant silences itself (playback stops,
  7. host synthesis queue drops), the running turn is cancelled (the stop-button
  8. route), and your speech is transcribed by the host (SenseVoice, CN-native
  9. simplified-Chinese ASR with punctuation + ITN) and submitted. The reply
  10. streams back as spoken audio with live captions.
  11. Three ways to dictate:
  12. | gesture | behaviour |
  13. | --- | --- |
  14. | tap the mic | continuous dictation; the VAD segments on trailing silence |
  15. | **hold the send key** (or the mic) | records until release, slide up to discard |
  16. | **hold `Ctrl`** (configurable `asr.hotkey`) | same, without leaving the keyboard; `Esc` discards |
  17. While a hold is open the overlay shows a **live caption** — the interim
  18. transcript of what has been said so far — and keeps a spinner up after
  19. release until the authoritative transcript lands.
  20. Barge-in detection is triggered by the mic's leading speech edge. The ASR
  21. engine runs an **NLMS acoustic echo canceller** (see `src/aec.ts`) using the
  22. page's own TTS playback as the echo reference, so loud assistant audio is
  23. subtracted from the mic before the VAD — the browser-level
  24. `echoCancellation` constraint is kept only as a fallback when no echo
  25. reference is available.
  26. Live caption interims are **incremental**: each pass sends only the audio
  27. recorded since the last pass (correlated by session header), and the host
  28. decodes a bounded sliding window per session instead of re-decoding the
  29. whole hold. Preview cost therefore stops growing with hold length.
  30. ## Demo
  31. ![dsh-voice demo](docs/demo.gif)
  32. The loop: hold the composer's send key (its arrow is covered by a mic glyph),
  33. watch the live caption fill in while you speak, release into a spinner until
  34. the final transcript lands, then the reply streams back as spoken audio
  35. sentence-by-sentence — until the user's voice interrupts playback and stops
  36. the running turn mid-line (true barge-in). `Ctrl` does the same without
  37. leaving the keyboard.
  38. ## How it works
  39. ```
  40. input: mic ──RMS endpoint detection──▶ POST /asr (raw f32 PCM)
  41. │ text (SenseVoice)
  42. ▼
  43. composer draft ──submit──▶ model stream ──llm/stream tap──▶ SentenceSegmenter
  44. │
  45. browser ◀── SSE /dsh-voice-api/stream ── TtsQueue (msedge-tts) ◀──┘
  46. (base64 MP3 frames + caption text)
  47. barge-in: speech edge ──▶ engine.skip() + POST /cancel (epoch bump)
  48. + session.cancel() when a turn is running
  49. ```
  50. - The `llm/stream` tap is **lossless**: every chunk is yielded unchanged, the
  51. segmenter only observes. The model stream is never blocked by synthesis.
  52. - ASR runs **host-side** with [sherpa-onnx](https://github.com/k2-fsa/sherpa-onnx)
  53. (Apache-2.0) running **SenseVoice** — the CN-native speech model that
  54. outperforms whisper on Chinese: native simplified output, punctuation,
  55. inverse text normalization (ITN) and 50+ language auto-detection. The
  56. browser only records and posts raw little-endian f32 PCM.
  57. - Model files stream through a **cache-through proxy** at
  58. `/dsh-voice-api/hf` and are mirrored to disk
  59. (`~/.cache/dsh-voice/models/`, configurable via `cacheDir`), so every
  60. browser/recognizer load after the first is served from local disk.
  61. Downloads resume from partial `.part` files when interrupted. Use
  62. `npm run prefetch` to warm the cache once.
  63. - RMS endpoint detection: 16kHz getUserMedia, 2s trailing-silence cutoff,
  64. max 30s segment, pre/post padding. Zero dependencies.
  65. - **Press-to-talk bypasses the VAD entirely.** Holding the key is already the
  66. intent, so every buffer between press and release is kept — gating on
  67. loudness there only drops quiet speech, which is indistinguishable from a
  68. broken button. Only captures below 250ms are discarded (mis-taps).
  69. - **Live caption**: while a hold is open the engine re-decodes the buffer
  70. every ~900ms and shows the interim text. SenseVoice is not a streaming
  71. model, so this is only requested while the overlay is actually on screen,
  72. and stops past 12s of audio. Interims are strictly previews: they never
  73. reach the composer draft, and an interim that lands after the release is
  74. dropped (epoch check) so it can never overwrite the final transcript.
  75. - Barge-in is three-layered: local playback queue cleared, host `TtsQueue`
  76. epoch bumped (queued AND in-flight synthesis dropped), and the running
  77. turn cancelled when `session.running` is true. An aborted turn never
  78. flushes its trailing half-sentence — exactly what the user interrupted.
  79. - `modelHost` accepts any HF-compatible mirror (e.g. `https://hf-mirror.com`
  80. for CN networks).
  81. ## API
  82. | Route | Purpose |
  83. |-------|---------|
  84. | `GET /dsh-voice-api/stream` | SSE; `event: audio` frames `{sessionId, seq, text, audio(base64 MP3)}` |
  85. | `POST /dsh-voice-api/asr` | raw little-endian f32 PCM body → `{text}` via SenseVoice |
  86. | `POST /dsh-voice-api/cancel` | `{sessionId}` drops queued + in-flight synthesis (epoch bump) |
  87. | `GET /dsh-voice-api/config` | ASR runtime config `{asr, basePath}`. Always served at this fixed path, so a client can discover a custom `basePath` |
  88. | `GET /dsh-voice-api/hf/*` | cache-through HF model proxy (mirrors to `cacheDir`) |
  89. | `GET /dsh-voice-api/*` | ping: `{ok, name, enabled}` |
  90. Config (bundle patch row):
  91. ```yaml
  92. - id: voice
  93. name: '@haoku123/dsh-voice'
  94. config:
  95. voice: zh-CN-XiaoxiaoNeural
  96. cacheDir: ~/.cache/dsh-voice/models # optional, on-disk model cache
  97. asr:
  98. model: csukuangfj/sherpa-onnx-sense-voice-zh-en-ja-ko-yue-2024-07-17
  99. modelHost: https://huggingface.co # or https://hf-mirror.com
  100. language: auto # auto | zh | en | ja | ko | yue
  101. useItn: true # inverse text normalization
  102. autoSend: false
  103. mode: toggle # toggle | hold
  104. hotkey: Control # keyboard press-to-talk; '' disables
  105. ```
  106. Model files are fetched through the proxy on first use; warm the cache once
  107. with the dsh host running:
  108. ```sh
  109. npm run prefetch # uses http://127.0.0.1:3080 by default
  110. ```
  111. ## Install
  112. ```sh
  113. dsh plugin --profile web add <repo-url-or-path>
  114. dsh --profile web
  115. ```
  116. Note: needs Node ≥ 22.19 or ≥ 24 (`node:zlib` zstd APIs). Works with dsh
  117. 0.1.0-rc.6 and 0.2.x — the `@deepseek-ai/dsh-web` peer range covers both.
  118. ## Tests
  119. ```sh
  120. npm test # host suites + client component tests
  121. npm run test:host # segmenter / TTS backoff / barge-in / integration
  122. npm run test:client # vitest + jsdom
  123. node test/host.integration.test.mjs # llm/stream tap + real Edge TTS + SSE + /config
  124. node test/tts-retry.test.mjs # failed synthesis backs off instead of hot-looping
  125. node verify-client.mjs # client bundle registration/exports/slots/dynamic-import
  126. ```

Ratings and comments

Sign in to rate this skill.

Sign in

Free to use. If it’s helped, pass it on.