Skip to content
Back to search

openshorts

Open source AI clip generator: turns long videos into viral 9:16 shorts with AI moment detection, face tracking, subtitles and dubbing. Self-host free with Docker (MIT), or use the cloud with GPU speed from $12/mo. MCP server and API for AI agents.

Safety scan

A static text scan, not a guarantee. Select the badge to see what was scanned and every finding.

Checked today

Check it deeper on your machine

It has changed since its scan on 11 Nov 2026.

As a Claude Code plugin, install it and then ask Claude Code:

/plugin marketplace add keithmackay/mackayi/plugin install scanskillsafety@mackayiDo a deep scan of https://github.com/mutonby/openshorts with scanskillsafety

Or from a terminal, clone it and run it:

git clone https://github.com/keithmackay/scanskillsafety-skill ~/.claude/skills/scanskillsafetynode ~/.claude/skills/scanskillsafety/cli.cjs --deep 'https://github.com/mutonby/openshorts'

It checks the exact commit: every file, its packages against OSV (api.osv.dev), how it compares with this page's scan (findsafeskills.com receives the repository's address), and, if you agree, Claude reading every file with your own Claude subscription. It never runs the code.

Badge for your README
findsafeskills safety rating

Markdown

[![findsafeskills safety rating](/api/listings/mutonby--openshorts/badge.svg)](/listings/mutonby--openshorts)

HTML

<a href="/listings/mutonby--openshorts"><img src="/api/listings/mutonby--openshorts/badge.svg" alt="findsafeskills safety rating" /></a>

Install

i

Install instructions from the README

As written by the repo's author. findsafeskills has not verified these steps.

1. Clone

git clone https://github.com/mutonby/openshorts.git
cd openshorts

2. Configure (optional)

cp .env.example .env
# Edit .env with your AWS keys for S3 backup

3. Launch

docker compose up --build

4. Open Dashboard

Navigate to **`http://localhost:5175`**

1. Go to **Settings** and enter your API keys (Gemini, fal.ai, ElevenLabs, Upload-Post) 2. **Clip Generator**: Upload a long-form video to generate viral shorts 3. **AI Shorts**: Describe your product or paste a URL to generate UGC marketing videos 4. **YouTube Studio**: Generate thumbnails, titles, and descriptions for YouTube 5. **UGC Gallery**: Browse all generated videos and avatars

5. GPU acceleration (optional, NVIDIA)

The default image is CPU-only. With an NVIDIA card (any card with NVENC, e.g. RTX 4060) an 8-minute video clips in about a minute instead of 5 to 8. Nothing is passed through in the VM sense — the container just gets access to the host GPU.

**Host:** install the NVIDIA driver (`nvidia-smi` must work) and the [NVIDIA Container Toolkit](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html):

sudo nvidia-ctk runtime configure --runtime=docker && sudo systemctl restart docker
docker run --rm --gpus all nvidia/cuda:12.4.0-base-ubuntu22.04 nvidia-smi   # sanity check

On Windows use Docker Desktop with the WSL2 backend and the Windows NVIDIA driver; no driver inside WSL.

**Compose:** create `docker-compose.override.yml` next to `docker-compose.yml` (picked up automatically). `GPU: "1"` adds cuBLAS/cuDNN and onnxruntime-gpu to the image (~2 GB); `video` is required for NVENC.

services:
  backend:
    build:
      context: .
      args:
        GPU: "1"
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: all
              capabilities: [gpu, video]

**`.env`:**

WHISPER_MODEL=large-v3-turbo
WHISPER_DEVICE=cuda
WHISPER_COMPUTE=float16
FFMPEG_ENCODER=auto           # probes h264_nvenc at startup, falls back to x264
TRANSCRIBE_BACKEND=parakeet   # optional: ~2x faster than whisper, 25 European languages, auto-falls back to whisper
ASR_GPU_CONCURRENCY=1

**Verify:**

docker compose up --build -d
docker exec openshorts-backend nvidia-smi -L
docker exec openshorts-backend ffmpeg -hide_banner -f lavfi -i testsrc=size=256x256:rate=1 -frames:v 1 -c:v h264_nvenc -f null -

The backend log on the first job reports the chosen encoder and transcription device. A CUDA error in whisper (e.g. VRAM exhausted) retries once on CPU automatically. 8 GB of VRAM is enough for `large-v3-turbo` fp16 plus the detection models.

---

6. Run without a Google key (local LLM, optional)

The only cloud call in the clip pipeline is the moment picker: it sends the transcript (never the video) to Gemini. Point it at any OpenAI-compatible server instead and the whole pipeline stays on your box:

# .env
LLM_BASE_URL=http://host.docker.internal:11434/v1   # Ollama on the host
LLM_MODEL=qwen2.5:14b                                # any chat model that follows instructions
# LLM_API_KEY=...                                    # only if your server checks one (vLLM --api-key, OpenRouter)

Works with Ollama, LM Studio, vLLM, llama.cpp server, LocalAI and OpenRouter. The dashboard stops asking for a Gemini key when this is set. Two things to know:

- **Context length.** A scoring call carries three transcript windows (~2-3k tokens) and the detail call up to ten (~5k on a long podcast). Ollama defaults to a 4096-token context and truncates silently, so run it with `OLLAMA_CONTEXT_LENGTH=16384` (or set `num_ctx` in a Modelfile); raise `LLM_SCORE_BATCH` above 3 only if your context allows it. 7-8B models return valid JSON reliably, 3B ones do not. - **What still needs Gemini.** Anything that has to look at frames: the

What was scanned

The files the safety scan read, as written by the repository's author and shown as plain text. Lines with a finding are marked.

README.md · README · 501 lines

View this file on GitHub (read at the repository's current version: the scan did not record which commit it read)

This file has changed since it was scanned, so this is not exactly the text that was scanned, and no line is marked. The findings are listed with the safety scan above.

  1. # OpenShorts.app
  2. **The open-source OpusClip alternative.** Turn long videos into viral 9:16 Shorts, TikToks and Reels. Self-host free (MIT) or use the hosted cloud.
  3. | | OpenShorts | OpusClip | Vizard | Klap |
  4. |---|---|---|---|---|
  5. | Open source | Yes (MIT) | No | No | No |
  6. | Self-hostable | Yes (Docker) | No | No | No |
  7. | Free without watermark | Yes, self-hosted | No | No | No |
  8. | API + MCP server | Yes | API | API | API |
  9. [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
  10. [![Open Source](https://badges.frapsoft.com/os/v1/open-source.svg?v=103)](https://opensource.org/)
  11. [![PRs Welcome](https://img.shields.io/badge/PRs-welcome-brightgreen.svg)](http://makeapullrequest.com)
  12. [![Docker](https://img.shields.io/badge/Docker-Ready-2496ED?logo=docker&logoColor=white)](https://docs.docker.com/compose/)
  13. [![GitHub stars](https://img.shields.io/github/stars/mutonby/openshorts?style=social)](https://github.com/mutonby/openshorts)
  14. [![Last Commit](https://img.shields.io/github/last-commit/mutonby/openshorts)](https://github.com/mutonby/openshorts/commits/main)
  15. **Open source AI video platform** with 3 tools in one: **Clip Generator**, **AI Shorts (UGC videos with AI actors)**, and **YouTube Studio**.
  16. ![Your podcast, and the vertical clip OpenShorts makes of it: both speakers stacked, captions on the seam](screenshots/split-before-after.gif)
  17. <sub>Real OpenShorts output, not a mock-up. Source: [Turn the Tables with Dan and Shoshana Jordan](https://www.youtube.com/watch?v=-KbQj_vboOU), Heritage of Faith, licensed CC BY.</sub>
  18. Two people on camera? OpenShorts stacks them instead of shrinking the wide shot, puts the captions on the seam where they cover nobody, and switches back to a face-tracked crop when the cut goes to one person. The AI picks the layout per video; nothing to configure.
  19. **Cloud version:** don't want to run it yourself? [OpenShorts Cloud](https://www.openshorts.app/?ref=github) is the same software on a GPU. Free plan: your first video is clipped whole up to 60 minutes, then 20 minutes a month, with a watermark and no credit card. Paid plans from $12/mo remove the watermark and add **auto-posting** and **Autopilot** (below).
  20. **Two ways to run it, same software either way:**
  21. | | Self-hosted (this repo) | Hosted on [openshorts.app](https://www.openshorts.app/) |
  22. |---|---|---|
  23. | **Price** | Free forever, MIT | Free plan, paid from $12/mo |
  24. | **Speed** | 5 to 8 min per 8-min video on CPU | About 50s on our NVIDIA GPU |
  25. | **API keys** | Bring your own Gemini, ElevenLabs, fal.ai | Gemini included, nothing to set up |
  26. | **Watermark / limits** | None, ever | Watermark and 20 min/mo on the free plan, neither on paid |
  27. | **Auto-posting** | With your own Upload-Post account | Paid plans: connect TikTok, Instagram, YouTube, Facebook, LinkedIn, Threads and Bluesky once, post or schedule from the app |
  28. | **Autopilot** | Not included | Paid plans: every new video on your channel becomes scheduled shorts |
  29. | **Setup** | Docker, 8GB+ RAM, model downloads | Sign in and paste a link |
  30. | **MCP / API for agents** | Same `/mcp` endpoint, but only while your machine is on | Always-on endpoint at [mcp.openshorts.app](https://www.openshorts.app/mcp), API keys in one click |
  31. | **Your data** | Your server | Ours |
  32. Self-hosting is genuinely free and always will be. It costs you a machine, your own API keys and the time to keep it running. The hosted plans exist to cover that hardware and those keys.
  33. **Compared to Opus Clip, Klap, Vizard** (public pricing pages, checked October 2026; full breakdown at [openshorts.app/alternatives](https://www.openshorts.app/alternatives)):
  34. | | Free plan | Paid from | Open source |
  35. |---|---|---|---|
  36. | **OpenShorts** | Self-hosted: free, no watermark, no cap. Cloud: first video up to 60 min, then 20 min/mo, watermarked | $12/mo (Cloud, 100 min) | **Yes, MIT** |
  37. | **Opus Clip** | 60 min/mo, watermarked, export within 3 days | $15/mo (Starter, 150 min), billed per minute of source | No |
  38. | **Klap** | 1 video to try, no ongoing free plan | $14/mo billed yearly (100 clips), billed per clip | No |
  39. | **Vizard** | 60 min/mo, 720p, watermarked | $14.50/mo billed yearly ($29 monthly), billed per minute uploaded | No |
  40. ### Video: the story behind OpenShorts Cloud
  41. [![Launching the paid plan of an open source app](https://img.youtube.com/vi/Bf5z-Wh9QxM/maxresdefault.jpg)](https://www.youtube.com/watch?v=Bf5z-Wh9QxM "Watch on YouTube")
  42. *In Spanish. Click the image to watch on YouTube.*
  43. ---
  44. ## What's Inside
  45. ### 1. Clip Generator
  46. Paste a YouTube link or upload a file. OpenShorts finds the strongest moments of a podcast, webinar, stream or interview and returns finished 9:16 clips: reframed, captioned, with a hook line, a virality score and titles for each platform.
  47. ![A clip with its viral score, titles and edit actions](dashboard/public/screens/app-clipcard.webp)
  48. Every clip opens in a real editor: source and program monitors side by side with the full transcript. Click a word to set a cut, add segments, change the framing and re-render in seconds.
  49. ![The clip editor with source monitor, program monitor, transcript and segments](dashboard/public/screens/app-editor.webp)
  50. ### 2. Autopilot (Cloud)
  51. Connect your YouTube channel and publish as usual. Every new video is clipped on its own, you get an email when the clips are ready, and with autopublish on the best ones go out to your socials (TikTok, Instagram, YouTube, Facebook, LinkedIn, Threads and Bluesky), one a day at the hour you pick. Paid plans only.
  52. ![Autopilot settings](dashboard/public/screens/autopilot-settings.webp)
  53. ### 3. AI Shorts (UGC Video Creator)
  54. Generate marketing videos with AI actors for **any product or business**. No camera, no studio, no influencer budget. Describe your product or paste a URL.
  55. ![The UGC gallery with AI actor videos](dashboard/public/screens/app-ugc.webp)
  56. - **Two cost modes**: Low Cost (~$0.65/video) and Premium (~$2/video)
  57. - AI actors with lip-sync, voiceover, b-roll and TikTok-style subtitles
  58. - Pick an avatar from the shared gallery or upload your own photo
  59. - Every video lands in the public gallery ([openshorts.app/gallery](https://openshorts.app/gallery)) with its own SEO page
  60. ### 4. YouTube Studio
  61. AI titles (10 options with a refinement chat), thumbnails with your face over a background, descriptions with chapter timestamps, and one-click publishing to YouTube.
  62. ### 5. Inside Claude and ChatGPT
  63. Add OpenShorts to Claude or ChatGPT with one URL and ask for clips in plain language: same pipeline, same minutes, no API key to manage. See [Automate it](#automate-it-mcp-server-rest-api-and-webhooks).
  64. ![Connecting OpenShorts to an AI agent](dashboard/public/screens/connect-an-agent.webp)
  65. ---
  66. ## Key Features
  67. ### Clip Generator
  68. - **Viral Moment Detection**: Google Gemini 3.1 Flash-Lite ranks the transcript and scene boundaries to pick 3-15 moments of 15-60 seconds, each with a virality score
  69. - **Runs fully local if you want**: point `LLM_BASE_URL` at Ollama, LM Studio, vLLM or any OpenAI-compatible server and the moment picker runs on your own model, no Google key needed (see [Run without a Google key](#6-run-without-a-google-key-local-llm-optional))
  70. - **Smart 9:16 Reframing**: per scene: TRACK (MediaPipe + YOLOv8 face tracking), GENERAL (blurred background), SPLIT (two speakers stacked, captions on the seam) and SCREENCAST (screen over presenter); the layout is picked per video by Gemini or forced from the dashboard
  71. - **Captions**: faster-whisper (or Parakeet on GPU) with word-level timestamps, several caption styles, burned into the clip
  72. - **Clip editor**: word-level cuts, extra segments, framing changes and fast re-renders
  73. - **Hook overlays**: an opening line written from what is said in the clip, editable in text, style, font and position
  74. - **AI Voice Dubbing**: ElevenLabs, 30+ languages with voice cloning (bring your own ElevenLabs key, on the Cloud too)
  75. - **AI Video Effects**: Gemini-generated FFmpeg filters
  76. ### Social Auto-Publishing (via [Upload-Post](https://www.upload-post.com))
  77. - **Post a clip to TikTok, Instagram, YouTube, Facebook, LinkedIn, Threads and Bluesky at once**, or schedule it for any date and time
  78. - **Schedule a week**: spread a batch of clips across the coming days in one go
  79. - **Post analytics**: views of the clips you published from OpenShorts, in the app
  80. - On the Cloud it is wired up on every paid plan; self-hosted, paste your own Upload-Post key
  81. - **Only want to post videos you already have, without clipping?** Use [Upload-Post](https://www.upload-post.com) directly: TikTok, Instagram, YouTube, LinkedIn, Facebook, X, Threads, Pinterest, Bluesky and more, with a free plan, a REST API and an MCP server
  82. ### AI Shorts Pipeline
  83. 1. **Analyze**: Scrape website URL + web research, or generate from manual description
  84. 2. **Script**: AI writes viral scripts (hook - problem - solution - CTA format)
  85. 3. **Actor**: Generate AI actors with Flux 2 Pro or select from shared gallery
  86. 4. **Voice**: ElevenLabs TTS voiceover (English/Spanish, male/female)
  87. 5. **Video**: Talking head generation (Hailuo 2.3 Fast img2video + VEED Lipsync, or Kling Avatar v2 in Premium)
  88. 6. **B-roll**: AI-generated visuals with Ken Burns effect
  89. 7. **Composite**: FFmpeg final assembly with subtitles and hook overlays
  90. 8. **Publish**: Direct posting to TikTok, Instagram Reels, YouTube Shorts via Upload-Post
  91. ### Infrastructure
  92. - S3-compatible storage for clip backup and the public gallery
  93. - SEO gallery pages served by FastAPI with JSON-LD structured data
  94. - Async job queue with configurable concurrency, and optional NVIDIA GPU rendering (NVENC)
  95. ---
  96. ## Who Is This For?
  97. - **Content creators** — Turn long videos into shorts automatically, publish to all platforms at once, or let Autopilot do it for every new upload
  98. - **Marketing agencies** — Generate UGC videos for clients at scale, no actors or studios needed
  99. - **SaaS founders** — Create product demos and marketing shorts from just a URL
  100. - **E-commerce brands** — Product videos with AI actors for TikTok Shop, Instagram, YouTube
  101. - **Local businesses** — Restaurants, gyms, real estate, coaching — affordable video marketing
  102. - **Developers** — Self-host, customize the pipeline, integrate via API
  103. ---
  104. ## AI Shorts Showcase
  105. Videos generated with OpenShorts AI Shorts — no camera, no studio, no actors:
  106. | | | |
  107. |:---:|:---:|:---:|
  108. | [![Biohacking for Investors](https://test-videos-upload-post.s3.eu-west-3.amazonaws.com/videos/cdceec1b/actor.png)](https://openshorts.app/video/cdceec1b) | [![Secret Weapon for Devs](https://test-videos-upload-post.s3.eu-west-3.amazonaws.com/videos/d3a80b6b/actor.png)](https://openshorts.app/video/d3a80b6b) | [![El Secreto de los Agentes de IA](https://test-videos-upload-post.s3.eu-west-3.amazonaws.com/videos/8ab7de92/actor.png)](https://openshorts.app/video/8ab7de92) |
  109. | **Biohacking for Investors** · LOW COST | **Secret Weapon for Devs** · LOW COST | **El Secreto de los Agentes de IA** · PREMIUM |
  110. > Browse all videos at [openshorts.app/gallery](https://openshorts.app/gallery)
  111. ---
  112. ## OpenShorts vs Competitors
  113. | Feature | OpenShorts | Opus Clip | CapCut | Vizard | Klap | Descript |
  114. |---------|:---:|:---:|:---:|:---:|:---:|:---:|
  115. | **Price** | **Free self-hosted**<br>from $12/mo hosted | $15-29/mo | $8/mo | $14.50/mo yearly<br>$29 monthly | from $14/mo yearly | $24-65/mo |
  116. | **Self-hosted** | **Yes** | No | No | No | No | No |
  117. | **Open source** | **Yes** | No | No | No | No | No |
  118. | **Watermark** | **Never self-hosted**<br>free plan only when hosted | Free tier | Some | Free tier | No free plan<br>(1 trial video) | Free tier |
  119. | **Upload limits** | **None self-hosted**<br>by plan when hosted | 10-30GB | Credit-based | 60min-10hr | 10-100 vids/mo | 60min-40hr |
  120. | **AI clip detection** | Yes | Yes | Yes | Yes | Yes | Yes |
  121. | **Smart 9:16 reframing** | Yes | Yes | Yes | Yes | Yes | No |
  122. | **Auto subtitles** | Yes | Yes | Yes | Yes | Yes | Yes |
  123. | **Voice dubbing (30+ langs)** | Yes, with your ElevenLabs key | No | Pro only | No | Pro only | Business only |
  124. | **AI UGC actors** | **Yes** | No | No | No | No | No |
  125. | **AI video effects** | Yes | No | Yes | No | No | No |
  126. | **Hook text overlays** | Yes | No | No | No | No | No |
  127. | **YouTube Studio (titles, thumbnails)** | **Yes** | No | No | No | No | No |
  128. | **Social auto-publishing** | Yes | Pro only | TikTok only | Paid only | Paid only | No |
  129. | **Schedule uploads** | Yes | Pro only | No | Paid only | Paid only | No |
  130. | **Data privacy** | **Your server** | Their cloud | Their cloud | Their cloud | Their cloud | Their cloud |
  131. | **Works with a local LLM (Ollama)** | **Yes** | No | No | No | No | No |
  132. ---
  133. ## How Much Does It Cost?
  134. Self-hosting OpenShorts is free. You provide the machine and you only pay for the AI APIs you use, and most have generous free tiers:
  135. | Service | Free Tier | Paid Cost | Used For |
  136. |---------|-----------|-----------|----------|
  137. | **Google Gemini** | Free trial with generous limits | < $0.01 per 10-min video | Viral moment detection, script generation, web research |
  138. | **Local LLM (Ollama, LM Studio, vLLM...)** | **Free, your hardware** | $0 | Viral moment detection instead of Gemini (`LLM_BASE_URL`) |
  139. | **fal.ai** | Pay-per-use | ~$0.50-1.50 per AI Short | Actor generation, talking head video, lip-sync |
  140. | **ElevenLabs** | Free tier available | Pay-per-use | Voiceover, voice dubbing |
  141. | **Upload-Post** | **10 free uploads/month** to all networks (no credit card) | Pay-per-use | Auto-publishing to TikTok, Instagram, YouTube |
  142. | **AWS S3** | Optional | ~$0.023/GB | Cloud backup for clips and gallery |
  143. **Bottom line:** You can clip videos for practically free with Gemini, and publish 10 videos/month to all social networks at zero cost with Upload-Post.
  144. **Don't want to run any of that?** [openshorts.app](https://www.openshorts.app/) is the same software on our hardware: our NVIDIA GPU clips an 8-minute video in about 50 seconds instead of the 5 to 8 minutes it takes on a typical CPU, the Gemini key is included, and paid plans add auto-posting and Autopilot with nothing to set up. The free plan clips your first video whole up to 60 minutes, then 20 minutes a month, with a watermark and no credit card; paid plans start at $12/mo for 100 minutes without watermark.
  145. ---
  146. ## Requirements
  147. - **Docker & Docker Compose**
  148. - **Google Gemini API Key** ([Free — get it here](https://aistudio.google.com/app/apikey)) — required for all AI features
  149. - **fal.ai API Key** ([Pay-per-use](https://fal.ai)) — required for AI Shorts (actor generation, video, lip-sync)
  150. - **ElevenLabs API Key** ([Free tier](https://elevenlabs.io)) — required for voiceover/dubbing
  151. - **Upload-Post API Key** ([free tier](https://upload-post.com)) — required for direct social posting
  152. ---
  153. ## Getting Started
  154. ### 1. Clone
  155. ```bash
  156. git clone https://github.com/mutonby/openshorts.git
  157. cd openshorts
  158. ```
  159. ### 2. Configure (optional)
  160. ```bash
  161. cp .env.example .env
  162. # Edit .env with your AWS keys for S3 backup
  163. ```
  164. ### 3. Launch
  165. ```bash
  166. docker compose up --build
  167. ```
  168. ### 4. Open Dashboard
  169. Navigate to **`http://localhost:5175`**
  170. 1. Go to **Settings** and enter your API keys (Gemini, fal.ai, ElevenLabs, Upload-Post)
  171. 2. **Clip Generator**: Upload a long-form video to generate viral shorts
  172. 3. **AI Shorts**: Describe your product or paste a URL to generate UGC marketing videos
  173. 4. **YouTube Studio**: Generate thumbnails, titles, and descriptions for YouTube
  174. 5. **UGC Gallery**: Browse all generated videos and avatars
  175. 6. **AI Agent**: Drop vertical videos in a folder and an agent skill picks, edits and queues clips for your approval
  176. ### 5. GPU acceleration (optional, NVIDIA)
  177. The default image is CPU-only. With an NVIDIA card (any card with NVENC, e.g. RTX 4060) an 8-minute video clips in about a minute instead of 5 to 8. Nothing is passed through in the VM sense — the container just gets access to the host GPU.
  178. **Host:** install the NVIDIA driver (`nvidia-smi` must work) and the [NVIDIA Container Toolkit](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html):
  179. ```bash
  180. sudo nvidia-ctk runtime configure --runtime=docker && sudo systemctl restart docker
  181. docker run --rm --gpus all nvidia/cuda:12.4.0-base-ubuntu22.04 nvidia-smi # sanity check
  182. ```
  183. On Windows use Docker Desktop with the WSL2 backend and the Windows NVIDIA driver; no driver inside WSL.
  184. **Compose:** create `docker-compose.override.yml` next to `docker-compose.yml` (picked up automatically). `GPU: "1"` adds cuBLAS/cuDNN and onnxruntime-gpu to the image (~2 GB); `video` is required for NVENC.
  185. ```yaml
  186. services:
  187. backend:
  188. build:
  189. context: .
  190. args:
  191. GPU: "1"
  192. deploy:
  193. resources:
  194. reservations:
  195. devices:
  196. - driver: nvidia
  197. count: all
  198. capabilities: [gpu, video]
  199. ```
  200. **`.env`:**
  201. ```
  202. WHISPER_MODEL=large-v3-turbo
  203. WHISPER_DEVICE=cuda
  204. WHISPER_COMPUTE=float16
  205. FFMPEG_ENCODER=auto # probes h264_nvenc at startup, falls back to x264
  206. TRANSCRIBE_BACKEND=parakeet # optional: ~2x faster than whisper, 25 European languages, auto-falls back to whisper
  207. ASR_GPU_CONCURRENCY=1
  208. ```
  209. **Verify:**
  210. ```bash
  211. docker compose up --build -d
  212. docker exec openshorts-backend nvidia-smi -L
  213. docker exec openshorts-backend ffmpeg -hide_banner -f lavfi -i testsrc=size=256x256:rate=1 -frames:v 1 -c:v h264_nvenc -f null -
  214. ```
  215. The backend log on the first job reports the chosen encoder and transcription device. A CUDA error in whisper (e.g. VRAM exhausted) retries once on CPU automatically. 8 GB of VRAM is enough for `large-v3-turbo` fp16 plus the detection models.
  216. ---
  217. ### 6. Run without a Google key (local LLM, optional)
  218. The only cloud call in the clip pipeline is the moment picker: it sends the
  219. transcript (never the video) to Gemini. Point it at any OpenAI-compatible
  220. server instead and the whole pipeline stays on your box:
  221. ```bash
  222. # .env
  223. LLM_BASE_URL=http://host.docker.internal:11434/v1 # Ollama on the host
  224. LLM_MODEL=qwen2.5:14b # any chat model that follows instructions
  225. # LLM_API_KEY=... # only if your server checks one (vLLM --api-key, OpenRouter)
  226. ```
  227. Works with Ollama, LM Studio, vLLM, llama.cpp server, LocalAI and OpenRouter.
  228. The dashboard stops asking for a Gemini key when this is set. Two things to
  229. know:
  230. - **Context length.** A scoring call carries three transcript windows
  231. (~2-3k tokens) and the detail call up to ten (~5k on a long podcast).
  232. Ollama defaults to a 4096-token context and truncates silently, so run it
  233. with `OLLAMA_CONTEXT_LENGTH=16384` (or set `num_ctx` in a Modelfile); raise
  234. `LLM_SCORE_BATCH` above 3 only if your context allows it. 7-8B models
  235. return valid JSON reliably, 3B ones do not.
  236. - **What still needs Gemini.** Anything that has to look at frames: the
  237. automatic layout picker (`AUTO_LAYOUT`), the on-screen content detector
  238. and silent videos (no speech to clip by). Without a Gemini key those fall
  239. back to the plain face-tracking crop, and a silent video fails with a
  240. message that says so. Add a key alongside `LLM_BASE_URL` and you get both.
  241. ## Technical Pipeline
  242. ### Clip Generator
  243. 1. **Ingest** — YouTube URL (yt-dlp) or local upload
  244. 2. **Transcribe** — faster-whisper (or Parakeet on GPU) with word-level timestamps
  245. 3. **Detect** — PySceneDetect for scene boundaries
  246. 4. **Analyze** — Gemini identifies 3-15 viral moments (15-60s each)
  247. 5. **Extract** — FFmpeg precise clip cutting
  248. 6. **Reframe** — AI vertical cropping with subject tracking
  249. 7. **Effects** — Subtitles, hooks, AI video effects
  250. 8. **Publish** — S3 backup + Upload-Post social distribution
  251. ### AI Shorts
  252. 1. **Analyze** — Website scraping + Gemini web research (or manual description)
  253. 2. **Script** — Gemini generates viral scripts with segments
  254. 3. **Actor** — Flux 2 Pro portrait generation (or gallery/upload)
  255. 4. **Voice** — ElevenLabs TTS voiceover
  256. 5. **Video** — Hailuo 2.3 Fast img2video + VEED Lipsync (Low Cost) or Kling Avatar v2 (Premium)
  257. 6. **B-roll** — Flux 2 Pro image generation + Ken Burns effect
  258. 7. **Composite** — FFmpeg assembly with ASS subtitles and hook overlays
  259. 8. **Gallery** — Upload to public S3 with metadata for SEO pages
  260. 9. **Publish** — Upload-Post to TikTok, Instagram, YouTube
  261. ---
  262. ## Automate It: MCP Server, REST API and Webhooks
  263. You don't need the dashboard. The whole pipeline is callable by AI agents and scripts.
  264. ### MCP server (`/mcp`)
  265. OpenShorts ships a built-in [MCP](https://modelcontextprotocol.io) server, so Claude, ChatGPT, Cursor or any MCP client can clip and publish videos for you:
  266. **claude.ai and ChatGPT**: paste `https://mcp.openshorts.app/mcp` as a custom connector (Settings → Connectors) and approve the access on openshorts.app. The server does OAuth 2.1 with dynamic client registration, so there is no key to copy; the connection shows up under Account → API keys, where revoking it disconnects the app.
  267. ```bash
  268. # Claude Code / Cursor / n8n (hosted): create an API key in your account page
  269. claude mcp add --transport http openshorts https://mcp.openshorts.app/mcp \
  270. --header "Authorization: Bearer osk_..."
  271. # Self-hosted (no key needed, BYOK rules apply):
  272. claude mcp add --transport http openshorts http://localhost:8000/mcp
  273. # Self-hosted without running the web server: same tools over stdio
  274. claude mcp add openshorts -- python mcp_stdio.py
  275. ```
  276. Tools: `process_video` (URL or `upload_id`; `captions: false` when the source already has subtitles, `auto_hook: false` to skip the hook line, burned by default like the dashboard), `create_upload` (hand the agent a local file: PUT the bytes, then process), `get_job_status`, `list_clips`, `get_quota`, `add_subtitles`, `recut_clip`, `publish_clip`. A prompt like *"clip this podcast and schedule the best 3 to TikTok"* is now a one-liner in your agent of choice.
  277. ### REST API + API keys
  278. Hosted accounts can mint `osk_...` API keys (account page). A key authenticates as you everywhere — same plan, same minutes, same job ownership:
  279. ```bash
  280. curl -X POST https://api.openshorts.app/api/process \
  281. -H "Authorization: Bearer osk_..." -H "Content-Type: application/json" \
  282. -d '{"url": "https://youtube.com/watch?v=...", "acknowledged": true,
  283. "webhook_url": "https://your-server.com/hooks/openshorts"}'
  284. ```
  285. Interactive docs at `/docs` (OpenAPI) on any instance.
  286. ### Completion webhooks
  287. Pass `webhook_url` (and optionally `webhook_secret`) to `POST /api/process` and you get exactly one `POST` when the job reaches a terminal state — no polling loops in your n8n / Zapier / cron pipelines:
  288. ```json
  289. {"event": "job.completed", "job_id": "…",
  290. "clips": [{"index": 0, "title": "…", "video_url": "…", "download_url": "…"}]}
  291. ```
  292. With a secret, the body is signed: `X-OpenShorts-Signature: sha256=<hmac-sha256(body)>`.
  293. ### CLI
  294. The same API from the terminal, zero dependencies (`cli/`):
  295. ```bash
  296. pip install openshorts # or: uvx openshorts
  297. export OPENSHORTS_API_KEY=osk_... # hosted
  298. # export OPENSHORTS_API_URL=http://localhost:8000 # self-hosted, no key
  299. openshorts process "https://youtube.com/watch?v=..." --wait
  300. openshorts clips <job_id>
  301. openshorts publish <job_id> 0 --platforms tiktok,youtube
  302. ```
  303. ### Agent skill
  304. `skills/openshorts/SKILL.md` follows the open
  305. [Agent Skills](https://agentskills.io) standard, so it works in any
  306. skill-capable agent:
  307. ```bash
  308. # Claude Code (and most agents): copy the folder into the skills directory
  309. cp -r skills/openshorts ~/.claude/skills/
  310. # Hermes Agent: install straight from this repo
  311. hermes skills install mutonby/openshorts/skills/openshorts
  312. # OpenClaw: from ClawHub
  313. openclaw skills install @mutonby/openshorts
  314. ```
  315. ### n8n
  316. An importable workflow (video URL in, published-ready clips out, no polling)
  317. lives in [`examples/n8n/`](examples/n8n/).
  318. ---
  319. ## Tech Stack
  320. | Layer | Technology |
  321. |-------|-----------|
  322. | Backend | Python 3.11, FastAPI, google-genai, faster-whisper, ultralytics (YOLOv8), mediapipe, opencv-python, yt-dlp, FFmpeg, httpx |
  323. | Frontend | React 18, Vite 4, Tailwind CSS 3.4 |
  324. | AI APIs | Google Gemini, fal.ai (Flux, Hailuo, VEED, Kling), ElevenLabs |
  325. | Infrastructure | Docker + Docker Compose, AWS S3 |
  326. | Publishing | Upload-Post API (TikTok, Instagram, YouTube) |
  327. ---
  328. ## Environment Variables
  329. **Server-side (.env):**
  330. | Variable | Description |
  331. |----------|------------|
  332. | `AWS_ACCESS_KEY_ID` | AWS access key for S3 |
  333. | `AWS_SECRET_ACCESS_KEY` | AWS secret key |
  334. | `AWS_REGION` | AWS region (default: us-east-1) |
  335. | `AWS_S3_BUCKET` | Private bucket for clip backup |
  336. | `AWS_S3_PUBLIC_BUCKET` | Public bucket for gallery/avatars |
  337. | `MAX_CONCURRENT_JOBS` | Concurrent processing limit (default: 5) |
  338. | `LLM_BASE_URL` | OpenAI-compatible server for the moment picker (Ollama, vLLM, LM Studio...). Set it and the Gemini key becomes optional |
  339. | `LLM_MODEL` | Model name on that server (default `llama3.1:8b`) |
  340. | `LLM_API_KEY` | Bearer token for that server, if it checks one |
  341. | `LLM_SCORE_BATCH` | Transcript windows per scoring call (default 3 local, 8 Gemini) |
  342. **Client-side (encrypted in localStorage):**
  343. | Key | Description |
  344. |-----|------------|
  345. | `GEMINI_API_KEY` | Google Gemini — required unless `LLM_BASE_URL` is set (then only for layout picking and silent videos) |
  346. | `FAL_KEY` | fal.ai — required for AI Shorts |
  347. | `ELEVENLABS_API_KEY` | ElevenLabs — required for voiceover/dubbing |
  348. | `UPLOAD_POST_API_KEY` | Upload-Post — required, for social posting |
  349. ---
  350. ## Security & Performance
  351. - **Non-Root Execution**: Containers run as dedicated `appuser`
  352. - **Concurrency Control**: Semaphore-based job queue (`MAX_CONCURRENT_JOBS`)
  353. - **Auto-Cleanup**: Old jobs are purged automatically (24h by default when self-hosted, `JOB_RETENTION_SECONDS`)
  354. - **Encrypted Keys**: API keys encrypted client-side, never stored server-side
  355. - **Upload Validation**: Image uploads validated for format and minimum size
  356. - **File Limits**: 2GB upload limit protection
  357. ---
  358. ## Social Media Setup (Upload-Post)
  359. 1. **Register**: [app.upload-post.com/login](https://app.upload-post.com/login)
  360. 2. **Create Profile**: Go to [Manage Users](https://app.upload-post.com/manage-users)
  361. 3. **Connect Accounts**: Link TikTok, Instagram, and/or YouTube
  362. 4. **Get API Key**: Navigate to [API Keys](https://app.upload-post.com/api-keys)
  363. 5. **Use in OpenShorts**: Paste the key in Settings
  364. On [OpenShorts Cloud](https://www.openshorts.app/) none of this is needed: paid plans connect your socials from the app.
  365. ---
  366. ## Star History
  367. [![Star History Chart](https://api.star-history.com/svg?repos=mutonby/openshorts&type=Date)](https://star-history.com/#mutonby/openshorts&Date)
  368. ## Contributions
  369. Contributions are welcome! Whether it's adding new AI models, improving the lip-sync pipeline, or building new features — feel free to open a PR.
  370. ## License
  371. MIT License for the core application — OpenShorts is yours to use, modify, and scale.
  372. **Exception:** the [`cloud/`](cloud/LICENSE) directory (billing, managed keys, and the hosted-service infrastructure behind the optional `BILLING_ENABLED` flag) is source-available under the OpenShorts Commercial License. You can read it, modify it, and self-host it for personal or internal use, but you can't offer it to third parties as a paid/hosted service. Self-hosting the core app never requires this directory.

Ratings and comments

Sign in to rate this skill.

Sign in

Free to use. If it’s helped, pass it on.