Skip to content
Back to search

violin

Open-source Video Translation Skill

Safety scan

A static text scan, not a guarantee. Select the badge to see what was scanned and every finding.

Checked today
Badge for your README
findsafeskills safety rating

Markdown

[![findsafeskills safety rating](/api/listings/shang-zhu--violin/badge.svg)](/listings/shang-zhu--violin)

HTML

<a href="/listings/shang-zhu--violin"><img src="/api/listings/shang-zhu--violin/badge.svg" alt="findsafeskills safety rating" /></a>

Install

i

Install instructions from the README

As written by the repo's author. findsafeskills has not verified these steps.

These steps run a script from the network that findsafeskills does not scan.

Try it without installing anything

The live demo runs at **<https://www.violin-ai.com>** — drop a short clip in, get a dubbed video out in a few minutes.

Run locally

Requires **Python 3.10+** and **ffmpeg** on PATH.

curl -LsSf https://astral.sh/uv/install.sh | sh   # install uv if you don't have it
uv tool install violin                            # recommended — faster, isolated
# or: pip install violin                          # if you'd rather install into your current Python env

export TOGETHER_API_KEY=...                       # get one at https://api.together.ai (add to ~/.zshrc to persist)

Three ways to use it:

**1. CLI** — translate one file:

violin lecture.mp4 lecture_zh.mp4 --language Chinese

**2. Web app** — full REST API + browser UI:

violin-api
# → http://127.0.0.1:8000           (browser UI)
# → http://127.0.0.1:8000/docs      (interactive API docs)

**3. Claude Code skill** — invoke from any Claude Code session:

violin --install-skill          # one-time: copies the skill into ~/.claude/skills/
claude
> please use the violin skill to translate path/to/video.mp4 into Chinese

<details><summary>Run from source (for hacking on the pipeline)</summary>

git clone https://github.com/shang-zhu/violin.git
cd violin
uv sync
cp .env.example .env             # then fill in TOGETHER_API_KEY
uv run main.py lecture.mp4 lecture_zh.mp4 --language Chinese

To use the `violin` / `violin-api` commands globally while edits to your local source reflect immediately, install editable:

uv tool uninstall violin     # if you've installed the PyPI version
uv tool install --editable .

After this, `violin` / `violin-api` run from your local checkout — edit any file and the next invocation picks it up; no rebuild needed. To switch back to PyPI: `uv tool uninstall violin && uv tool install violin`.

</details>

---

What was scanned

The files the safety scan read, as written by the repository's author and shown as plain text. Lines with a finding are marked.

README.md · README · 301 lines

View this file on GitHub (read at the repository's current version: the scan did not record which commit it read)

  1. # 🎻 Violin
  2. **Open-source Video Translation Skill.**
  3. [🌐 Live demo](https://www.violin-ai.com) · [📝 Blog post](https://www.together.ai/blog/violin-open-source-translation-skill) · [📜 MIT License](https://github.com/shang-zhu/violin/blob/main/LICENSE)
  4. <p align="left">
  5. <img src="assets/logo.png" alt="Violin logo" width="256">
  6. </p>
  7. <!-- ![demo](assets/outcome.png) -->
  8. Upload a video. Violin transcribes the speech, translates it, synthesizes a native-sounding voice-over in the target language, and remuxes it back into the video — fully aligned, with optional SRT subtitles.
  9. Available as a **CLI**, a **FastAPI web app**, and a **Claude Code skill**.
  10. ---
  11. ## ✨ Features
  12. - **33 target languages** with handpicked native-speaker voices for the 16 most-used ones (Cartesia Sonic 3 + ElevenLabs)
  13. - **In-video Q&A** — ask questions about any moment in the dubbed video; answers use nearby subtitles plus sampled frames
  14. - **Natural-language voice picker** — describe the voice you want, an LLM picks from the catalog
  15. - **6 style profiles** *(experimental)* — standard / kids / academic / casual / storyteller / news
  16. - **Pluggable stack** — Together / OpenAI / ElevenLabs interchangeable for every stage, one YAML
  17. ---
  18. ## 🚀 Quick start
  19. ### Try it without installing anything
  20. The live demo runs at **<https://www.violin-ai.com>** — drop a short clip in, get a dubbed video out in a few minutes.
  21. ### Run locally
  22. Requires **Python 3.10+** and **ffmpeg** on PATH.
  23. ```bash
  24. curl -LsSf https://astral.sh/uv/install.sh | sh # install uv if you don't have it
  25. uv tool install violin # recommended — faster, isolated
  26. # or: pip install violin # if you'd rather install into your current Python env
  27. export TOGETHER_API_KEY=... # get one at https://api.together.ai (add to ~/.zshrc to persist)
  28. ```
  29. Three ways to use it:
  30. **1. CLI** — translate one file:
  31. ```bash
  32. violin lecture.mp4 lecture_zh.mp4 --language Chinese
  33. ```
  34. **2. Web app** — full REST API + browser UI:
  35. ```bash
  36. violin-api
  37. # → http://127.0.0.1:8000 (browser UI)
  38. # → http://127.0.0.1:8000/docs (interactive API docs)
  39. ```
  40. **3. Claude Code skill** — invoke from any Claude Code session:
  41. ```bash
  42. violin --install-skill # one-time: copies the skill into ~/.claude/skills/
  43. claude
  44. > please use the violin skill to translate path/to/video.mp4 into Chinese
  45. ```
  46. <details><summary>Run from source (for hacking on the pipeline)</summary>
  47. ```bash
  48. git clone https://github.com/shang-zhu/violin.git
  49. cd violin
  50. uv sync
  51. cp .env.example .env # then fill in TOGETHER_API_KEY
  52. uv run main.py lecture.mp4 lecture_zh.mp4 --language Chinese
  53. ```
  54. To use the `violin` / `violin-api` commands globally while edits to your local source reflect immediately, install editable:
  55. ```bash
  56. uv tool uninstall violin # if you've installed the PyPI version
  57. uv tool install --editable .
  58. ```
  59. After this, `violin` / `violin-api` run from your local checkout — edit any file and the next invocation picks it up; no rebuild needed. To switch back to PyPI: `uv tool uninstall violin && uv tool install violin`.
  60. </details>
  61. ---
  62. ## 📝 To Do List
  63. - [-] support voice cloning.
  64. - [-] lip sync generation.
  65. ---
  66. ## 🎬 How Violin works
  67. ```
  68. Video
  69. │
  70. ├─ ffmpeg ─────────────────────► Extract audio (16 kHz WAV)
  71. │
  72. ├─ Whisper Large v3 ────────────► Word-level timestamps → sentence segments
  73. │
  74. ├─ LLM (DeepSeek V4 Pro by default) ──► Translate each segment, respecting style profile
  75. │
  76. ├─ TTS (Cartesia Sonic 3 by default) ─► Synthesize dubbed audio per segment
  77. │
  78. └─ ffmpeg ─────────────────────► Speed-align video to dubbed audio,
  79. concat with freeze-frame fallback,
  80. single-pass AAC encode the audio track,
  81. write output mp4 + optional SRT
  82. ```
  83. ---
  84. ## ⚙️ Configuration
  85. Override any default by writing your own YAML and passing it with `--config my.yaml` — only the keys you want to change need to appear; values deep-merge with the [built-in defaults](https://github.com/shang-zhu/violin/blob/main/config/default.yaml).
  86. ### Switch providers
  87. ```yaml
  88. # config/default.yaml — pick the stack you want
  89. models:
  90. transcription:
  91. provider: together # together | openai
  92. model: openai/whisper-large-v3 # together → openai/whisper-large-v3 | openai → whisper-1
  93. translation:
  94. provider: together # together | openai
  95. model: deepseek-ai/DeepSeek-V4-Pro-0813 # together → deepseek-ai/DeepSeek-V4-Pro-0813 | openai → gpt-5.5
  96. tts:
  97. provider: together # together | elevenlabs | openai
  98. model: cartesia/sonic-3 # together → cartesia/sonic-3 | elevenlabs → eleven_v3 | openai → tts-1-hd
  99. chat:
  100. provider: together # together | openai — must be vision-capable
  101. model: zai-org/GLM-5.3-Flash # together → zai-org/GLM-5.3-Flash | openai → gpt-5.5
  102. ```
  103. > The `chat` stage is a reasoning model by default, and it spends part of `chat.max_tokens` on thinking before it answers. If you point it at a different model, keep that budget generous (the shipped default is `4096`) — too tight and the API returns an empty answer with `finish_reason: length`.
  104. ### Production overrides
  105. A starter `config/prod.yaml` is included for public deployments. It adds upload limits, serializes jobs, and caps ffmpeg concurrency. The included `Dockerfile` + `docker-compose.yml` + `Caddyfile` are how the live demo is hosted — `docker compose up -d --build` after filling `.env` is enough to put a copy of Violin behind auto-HTTPS on any Docker host.
  106. ### Environment variables
  107. | Variable | When required | Description |
  108. |----------|---------------|-------------|
  109. | `TOGETHER_API_KEY` | **Recommended** — covers every stage with the default config | Together AI API key |
  110. | `OPENAI_API_KEY` | Any stage uses `provider: openai` | Covers `whisper-1`, GPT models, and `tts-1` |
  111. | `ELEVENLABS_API_KEY` | TTS uses `provider: elevenlabs` | ElevenLabs API key |
  112. | `CORS_ORIGINS` | Optional | Comma-separated allowed origins (default: `*`) |
  113. > You only need keys for the providers you actually pick. Pure-OpenAI deployments (all stages on `openai`) work too — `OPENAI_API_KEY` alone is enough. Same idea for ElevenLabs.
  114. ---
  115. ## 🎭 Style profiles
  116. Six built-in profiles tune both the translation LLM prompt and the TTS delivery. Use `--style <name>` on the CLI or pass `style` in API requests.
  117. | Style | Tone | TTS speed | Emotion |
  118. |-------|------|-----------|---------|
  119. | `standard` | Faithful translation, natural voice | 1.0× | — |
  120. | `kids` | Rewritten for a 7-year-old, plain language | 1.0× | excited |
  121. | `academic` | Formal register, preserves jargon and honorifics | 0.95× | calm |
  122. | `casual` | Spoken slang, contractions, friendly | 1.1× | content |
  123. | `storyteller` | Vivid, dramatic narration | 0.9× | enthusiastic |
  124. | `news` | Concise, declarative, broadcast-style | 1.0× | neutral |
  125. Add your own by editing `prompts/styles.yaml`.
  126. See all available styles: `violin --style list`.
  127. ---
  128. ## 💻 CLI usage
  129. > Examples use the PyPI-installed `violin` command. If you're running from a git checkout, substitute `uv run main.py` for `violin` (and `uv run run_api.py` for `violin-api`).
  130. ```bash
  131. # Basic
  132. violin lecture.mp4 lecture_es.mp4 --language Spanish
  133. # Pick a style
  134. violin talk.mp4 talk_zh.mp4 --language Chinese --style kids
  135. # Pick a specific voice
  136. violin lecture.mp4 lecture_fr.mp4 --language French --voice "french narrator man"
  137. # Skip SRT
  138. violin lecture.mp4 lecture_ja.mp4 --language Japanese --no-subtitles
  139. # Full replacement (no original audio underneath)
  140. violin lecture.mp4 lecture_ko.mp4 --language Korean --no-voiceover
  141. # Custom config (e.g. switch to OpenAI/ElevenLabs)
  142. violin lecture.mp4 lecture_it.mp4 --language Italian --config config/other_api.yaml
  143. ```
  144. ### CLI flags
  145. | Flag | Default | Description |
  146. |------|---------|-------------|
  147. | `--language` / `-l` | *(required)* | Target language name (e.g. `Spanish`, `Japanese`) |
  148. | `--voice` / `-v` | auto | TTS voice. Defaults to the primary native voice for the target language |
  149. | `--source-language` | `auto-detect` | Source language hint for translation |
  150. | `--no-subtitles` | off | Skip SRT generation |
  151. | `--voiceover` / `--no-voiceover` | voiceover on | Keep original audio underneath the dub, or full replacement |
  152. | `--style` / `-s` | `standard` | Style profile name. Use `--style list` to see all |
  153. | `--config` / `-c` | `config/default.yaml` | Path to a YAML override file |
  154. | `--timings-out` | off | Write per-step wall-clock timings + cost as JSON |
  155. ---
  156. ## 🛰️ Web app & REST API
  157. ```bash
  158. violin-api # default dev mode
  159. violin-api --host 0.0.0.0 --port 8080 # bind everywhere
  160. violin-api --config config/prod.yaml # production overrides (requires a git checkout for config/prod.yaml)
  161. ```
  162. Core flow: `POST /jobs` to start, `GET /jobs/{id}` to poll, `GET /jobs/{id}/video` and `/srt` to download, `POST /jobs/{id}/chat` for in-video Q&A. Full list with request/response schemas at **`/docs`**.
  163. ### Example
  164. ```bash
  165. # Submit
  166. JOB=$(curl -s -X POST http://localhost:8000/jobs \
  167. -F "file=@lecture.mp4" \
  168. -F "language=Spanish" \
  169. -F "style=academic" | jq -r .id)
  170. # Poll
  171. curl -s http://localhost:8000/jobs/$JOB | jq '{status, progress}'
  172. # Download
  173. curl -OJ http://localhost:8000/jobs/$JOB/video
  174. curl -OJ http://localhost:8000/jobs/$JOB/srt
  175. ```
  176. Job data lives under `jobs/{id}/`. Set `api.job_ttl_hours` to auto-delete jobs older than N hours (default `0` = disabled; `config/prod.yaml` uses 24h for the public demo).
  177. ---
  178. ## 🌍 Supported languages
  179. Violin supports **33 target languages**. The 16 below ship with handpicked native-speaker voices for each provider; the rest fall back to the English voice catalog (which is multilingual under both Cartesia Sonic 3 and ElevenLabs `eleven_v3`).
  180. Ordered by native-speaker population.
  181. | Language | Cartesia native voice (M / F) | ElevenLabs native voice (M / F) |
  182. |----------|-------------------------------|---------------------------------|
  183. | Chinese | chinese commercial man / chinese female conversational | Lin / Lingyue |
  184. | Spanish | spanish narrator man / spanish narrator lady | Carlos / Valeria |
  185. | English | tutorial man / helpful woman | Adam / Sarah |
  186. | Hindi | hindi narrator man / hindi narrator woman | Yatin / Madhusmita |
  187. | Arabic | middle eastern woman | Faris / Haneen |
  188. | Portuguese | friendly brazilian man / pleasant brazilian lady | Medeiros / Luna |
  189. | Russian | russian narrator man 1 / russian narrator woman | Ivo / Xenia |
  190. | Japanese | japanese male conversational / japanese woman conversational | Shohei / Maiko |
  191. | Turkish | turkish narrator man / turkish calm man | Sinan / Aura |
  192. | German | german reporter man / german conversational woman | Daniel / Sina |
  193. | Korean | korean narrator man / korean calm woman | Joon-ho / Soo |
  194. | French | french narrator man / french narrator lady | Lior / Virginie |
  195. | Italian | italian narrator man / italian narrator woman | Raffaele / Chiara |
  196. | Polish | polish confident man / polish narrator woman | Gregor / Jola |
  197. | Dutch | dutch confident man / dutch man | Ronald / Jolanda |
  198. | Swedish | swedish narrator man / swedish calm lady | Andreas / Louise |
  199. The 17 fallback languages (using the English voice catalog), also ordered by native speakers: Vietnamese, Tamil, Indonesian, Malay, Ukrainian, Romanian, Thai, Greek, Hungarian, Catalan, Czech, Bulgarian, Danish, Slovak, Croatian, Finnish, Norwegian.
  200. ---
  201. ## 🤝 Contributing
  202. PRs welcome. Got questions or hit a bug? Email **<heyviolinai@gmail.com>** or open an issue.
  203. ---
  204. ## ⚠️ Disclaimer
  205. This is a personal open-source project, not a Together AI product. Users are responsible for ensuring they have the right to download and translate any content they process. Designed for Creative Commons, public domain, your own recordings, and other content you have permission to use.
  206. ---
  207. ## 📜 License
  208. [MIT](https://github.com/shang-zhu/violin/blob/main/LICENSE) — use it freely, including commercially.
  209. ---
  210. ## 🙏 Acknowledgements
  211. Built on top of [Together AI](https://together.ai), [Whisper](https://github.com/openai/whisper), [Cartesia Sonic 3](https://cartesia.ai), [ElevenLabs](https://elevenlabs.io), [FastAPI](https://fastapi.tiangolo.com/), and [ffmpeg](https://ffmpeg.org).

Ratings and comments

Sign in to rate this skill.

Sign in

Free to use. If it’s helped, pass it on.