Skip to content
Back to search

short-video-maker

Creates short videos for TikTok, Instagram Reels, and YouTube Shorts using the Model Context Protocol (MCP) and a REST API.

Safety scan

A static text scan, not a guarantee. Select the badge to see what was scanned and every finding.

Checked today
Badge for your README
findsafeskills safety rating

Markdown

[![findsafeskills safety rating](/api/listings/gyoridavid--short-video-maker/badge.svg)](/listings/gyoridavid--short-video-maker)

HTML

<a href="/listings/gyoridavid--short-video-maker"><img src="/api/listings/gyoridavid--short-video-maker/badge.svg" alt="findsafeskills safety rating" /></a>

Install

i

Install instructions from the README

As written by the repo's author. findsafeskills has not verified these steps.

Docker (recommended)

There are three docker images, for three different use cases. Generally speaking, most of the time you want to spin up the `tiny` one.

Tiny

- Uses the `tiny.en` whisper.cpp model - Uses the `q4` quantized kokoro model - `CONCURRENCY=1` to overcome OOM errors coming from Remotion with limited resources - `VIDEO_CACHE_SIZE_IN_BYTES=2097152000` (2gb) to overcome OOM errors coming from Remotion with limited resources

docker run -it --rm --name short-video-maker -p 3123:3123 -e LOG_LEVEL=debug -e PEXELS_API_KEY= gyoridavid/short-video-maker:latest-tiny

Normal

- Uses the `base.en` whisper.cpp model - Uses the `fp32` kokoro model - `CONCURRENCY=1` to overcome OOM errors coming from Remotion with limited resources - `VIDEO_CACHE_SIZE_IN_BYTES=2097152000` (2gb) to overcome OOM errors coming from Remotion with limited resources

docker run -it --rm --name short-video-maker -p 3123:3123 -e LOG_LEVEL=debug -e PEXELS_API_KEY= gyoridavid/short-video-maker:latest

Cuda

If you own an Nvidia GPU and you want use a larger whisper model with GPU acceleration, you can use the CUDA optimised Docker image.

- Uses the `medium.en` whisper.cpp model (with GPU acceleration) - Uses `fp32` kokoro model - `CONCURRENCY=1` to overcome OOM errors coming from Remotion with limited resources - `VIDEO_CACHE_SIZE_IN_BYTES=2097152000` (2gb) to overcome OOM errors coming from Remotion with limited resources

docker run -it --rm --name short-video-maker -p 3123:3123 -e LOG_LEVEL=debug -e PEXELS_API_KEY= --gpus=all gyoridavid/short-video-maker:latest-cuda

Docker compose

You might use Docker Compose to run n8n or other services, and you want to combine them. Make sure you add the shared network to the service configuration.

version: "3"

services:
  short-video-maker:
    image: gyoridavid/short-video-maker:latest-tiny
    environment:
      - LOG_LEVEL=debug
      - PEXELS_API_KEY=
    ports:
      - "3123:3123"
    volumes:
	    - ./videos:/app/data/videos # expose the generated videos

If you are using the [Self-hosted AI starter kit](https://github.com/n8n-io/self-hosted-ai-starter-kit) you want to add `networks: ['demo']` to the\*\* `short-video-maker` service so you can reach it with http://short-video-maker:3123 in n8n.

What was scanned

The files the safety scan read, as written by the repository's author and shown as plain text. Lines with a finding are marked.

README.md · README · 506 lines

View this file on GitHub (read at the repository's current version: the scan did not record which commit it read)

  1. ## [📚 Join our Skool community for support, premium content and more!](https://www.skool.com/ai-agents-az/about?s1m)
  2. ### Be part of a growing community and help us create more content like this
  3. # Description
  4. An open source automated video creation tool for generating short-form video content. Short Video Maker combines text-to-speech, automatic captions, background videos, and music to create engaging short videos from simple text inputs.
  5. This project is meant to provide a free alternative to heavy GPU-power hungry video generation (and a free alternative to expensive, third-party API calls). It doesn't generate a video from scratch based on an image or an image prompt.
  6. The repository was open-sourced by the [AI Agents A-Z Youtube Channel](https://www.youtube.com/channel/UCloXqLhp_KGhHBe1kwaL2Tg). We encourage you to check out the channel for more AI-related content and tutorials.
  7. The server exposes an [MCP](https://github.com/modelcontextprotocol) and a REST server.
  8. While the MCP server can be used with an AI Agent (like n8n) the REST endpoints provide more flexibility for video generation.
  9. You can find example n8n workflows created with the REST/MCP server [in this repository](https://github.com/gyoridavid/ai_agents_az/tree/main/episode_7).
  10. # TOC
  11. ## Getting started
  12. - [Requirements](#general-requirements)
  13. - [How to run the server](#getting-started-1)
  14. - [Web UI](#web-ui)
  15. - [Tutorial](#tutorial-with-n8n)
  16. - [Examples](#examples)
  17. ## Usage
  18. - [Environment variables](#environment-variables)
  19. - [REST API](#rest-api)
  20. - [Configuration options](#configuration-options)
  21. - [MCP](#mcp-server)
  22. ## Info
  23. - [Features](#features)
  24. - [How it works](#how-it-works)
  25. - [Limitations](#limitations)
  26. - [Concepts](#concepts)
  27. - [Troubleshooting](#troubleshooting)
  28. - [Deploying in the cloud](#deploying-to-the-cloud)
  29. - [FAQ](#faq)
  30. - [Dependencies](#dependencies-for-the-video-generation)
  31. - [Contributing](#how-to-contribute)
  32. - [License](#license)
  33. - [Acknowledgements](#acknowledgments)
  34. # Tutorial with n8n
  35. [![Automated faceless video generation (n8n + MCP) with captions, background music, local and 100% free](https://img.youtube.com/vi/jzsQpn-AciM/0.jpg)](https://www.youtube.com/watch?v=jzsQpn-AciM)
  36. # Examples
  37. <table>
  38. <tr>
  39. <td>
  40. <video src="https://github.com/user-attachments/assets/1b488e7d-1b40-439d-8767-6ab51dbc0922" width="480" height="270"></video>
  41. </td>
  42. <td>
  43. <video src="https://github.com/user-attachments/assets/bb7ce80f-e6e1-44e5-ba4e-9b13d917f55b" width="270" height="480"></video>
  44. </td>
  45. <td>
  46. </tr>
  47. </table>
  48. # Features
  49. - Generate complete short videos from text prompts
  50. - Text-to-speech conversion
  51. - Automatic caption generation and styling
  52. - Background video search and selection via Pexels
  53. - Background music with genre/mood selection
  54. - Serve as both REST API and Model Context Protocol (MCP) server
  55. # How It Works
  56. Shorts Creator takes simple text inputs and search terms, then:
  57. 1. Converts text to speech using Kokoro TTS
  58. 2. Generates accurate captions via Whisper
  59. 3. Finds relevant background videos from Pexels
  60. 4. Composes all elements with Remotion
  61. 5. Renders a professional-looking short video with perfectly timed captions
  62. # Limitations
  63. - The project only capable generating videos with English voiceover (kokoro-js doesn’t support other languages at the moment)
  64. - The background videos are sourced from Pexels
  65. # General Requirements
  66. - internet
  67. - free pexels api key
  68. - ≥ 3 gb free RAM, my recommendation is 4gb RAM
  69. - ≥ 2 vCPU
  70. - ≥ 5gb disc space
  71. # Concepts
  72. ## Scene
  73. Each video is assembled from multiple scenes. These scenes consists of
  74. 1. Text: Narration, the text the TTS will read and create captions from.
  75. 2. Search terms: The keywords the server should use to find videos from Pexels API. If none can be found, joker terms are being used (`nature`, `globe`, `space`, `ocean`)
  76. # Getting started
  77. ## Docker (recommended)
  78. There are three docker images, for three different use cases. Generally speaking, most of the time you want to spin up the `tiny` one.
  79. ### Tiny
  80. - Uses the `tiny.en` whisper.cpp model
  81. - Uses the `q4` quantized kokoro model
  82. - `CONCURRENCY=1` to overcome OOM errors coming from Remotion with limited resources
  83. - `VIDEO_CACHE_SIZE_IN_BYTES=2097152000` (2gb) to overcome OOM errors coming from Remotion with limited resources
  84. ```jsx
  85. docker run -it --rm --name short-video-maker -p 3123:3123 -e LOG_LEVEL=debug -e PEXELS_API_KEY= gyoridavid/short-video-maker:latest-tiny
  86. ```
  87. ### Normal
  88. - Uses the `base.en` whisper.cpp model
  89. - Uses the `fp32` kokoro model
  90. - `CONCURRENCY=1` to overcome OOM errors coming from Remotion with limited resources
  91. - `VIDEO_CACHE_SIZE_IN_BYTES=2097152000` (2gb) to overcome OOM errors coming from Remotion with limited resources
  92. ```jsx
  93. docker run -it --rm --name short-video-maker -p 3123:3123 -e LOG_LEVEL=debug -e PEXELS_API_KEY= gyoridavid/short-video-maker:latest
  94. ```
  95. ### Cuda
  96. If you own an Nvidia GPU and you want use a larger whisper model with GPU acceleration, you can use the CUDA optimised Docker image.
  97. - Uses the `medium.en` whisper.cpp model (with GPU acceleration)
  98. - Uses `fp32` kokoro model
  99. - `CONCURRENCY=1` to overcome OOM errors coming from Remotion with limited resources
  100. - `VIDEO_CACHE_SIZE_IN_BYTES=2097152000` (2gb) to overcome OOM errors coming from Remotion with limited resources
  101. ```jsx
  102. docker run -it --rm --name short-video-maker -p 3123:3123 -e LOG_LEVEL=debug -e PEXELS_API_KEY= --gpus=all gyoridavid/short-video-maker:latest-cuda
  103. ```
  104. ## Docker compose
  105. You might use Docker Compose to run n8n or other services, and you want to combine them. Make sure you add the shared network to the service configuration.
  106. ```bash
  107. version: "3"
  108. services:
  109. short-video-maker:
  110. image: gyoridavid/short-video-maker:latest-tiny
  111. environment:
  112. - LOG_LEVEL=debug
  113. - PEXELS_API_KEY=
  114. ports:
  115. - "3123:3123"
  116. volumes:
  117. - ./videos:/app/data/videos # expose the generated videos
  118. ```
  119. If you are using the [Self-hosted AI starter kit](https://github.com/n8n-io/self-hosted-ai-starter-kit) you want to add `networks: ['demo']` to the\*\* `short-video-maker` service so you can reach it with http://short-video-maker:3123 in n8n.
  120. # NPM
  121. While Docker is the recommended way to run the project, you can run it with npm or npx.
  122. On top of the general requirements, the following are necessary to run the server.
  123. ## Supported platforms
  124. - Ubuntu ≥ 22.04 (libc 2.5 for Whisper.cpp)
  125. - Required packages: `git wget cmake ffmpeg curl make libsdl2-dev libnss3 libdbus-1-3 libatk1.0-0 libgbm-dev libasound2 libxrandr2 libxkbcommon-dev libxfixes3 libxcomposite1 libxdamage1 libatk-bridge2.0-0 libpango-1.0-0 libcairo2 libcups2`
  126. - Mac OS
  127. - ffmpeg (`brew install ffmpeg`)
  128. - node.js (tested on 22+)
  129. Windows is **NOT** supported at the moment (whisper.cpp installation fails occasionally).
  130. # Web UI
  131. @mushitori made a Web UI to generate the videos from your browser.
  132. <table>
  133. <tr>
  134. <td>
  135. <img width="1088" alt="Screenshot 2025-05-12 at 1 45 11 PM" src="https://github.com/user-attachments/assets/2ab64aea-f639-41b0-bd19-2fcf73bb1a3d" />
  136. </td>
  137. <td>
  138. <img width="1075" alt="Screenshot 2025-05-12 at 1 45 44 PM" src="https://github.com/user-attachments/assets/0ff568fe-ddcb-4dad-ae62-2640290aef1e" />
  139. </td>
  140. <td>
  141. <img width="1083" alt="Screenshot 2025-05-12 at 1 45 51 PM" src="https://github.com/user-attachments/assets/d3c1c826-3cb3-4313-b17c-605ff612fb63" />
  142. </td>
  143. <td>
  144. <img width="1070" alt="Screenshot 2025-05-12 at 1 46 42 PM" src="https://github.com/user-attachments/assets/18edb1a0-9fc2-48b3-8896-e919e7dc57ff" />
  145. </td>
  146. </tr>
  147. </table>
  148. You can load it on http://localhost:3123
  149. # Environment variables
  150. ## 🟢 Configuration
  151. | key | description | default |
  152. | --------------- | --------------------------------------------------------------- | ------- |
  153. | PEXELS_API_KEY | [your (free) Pexels API key](https://www.pexels.com/api/) | |
  154. | LOG_LEVEL | pino log level | info |
  155. | WHISPER_VERBOSE | whether the output of whisper.cpp should be forwarded to stdout | false |
  156. | PORT | the port the server will listen on | 3123 |
  157. ## ⚙️ System configuration
  158. | key | description | default |
  159. | ------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------- |
  160. | KOKORO_MODEL_PRECISION | The size of the Kokoro model to use. Valid options are `fp32`, `fp16`, `q8`, `q4`, `q4f16` | depends, see the descriptions of the docker images above ^^ |
  161. | CONCURRENCY | [concurrency refers to how many browser tabs are opened in parallel during a render. Each Chrome tab renders web content and then screenshots it.](https://www.remotion.dev/docs/terminology/concurrency). Tweaking this value helps with running the project with limited resources. | depends, see the descriptions of the docker images above ^^ |
  162. | VIDEO_CACHE_SIZE_IN_BYTES | Cache for [<OffthreadVideo>](https://remotion.dev/docs/offthreadvideo) frames in Remotion. Tweaking this value helps with running the project with limited resources. | depends, see the descriptions of the docker images above ^^ |
  163. ## ⚠️ Danger zone
  164. | key | description | default |
  165. | ------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------- |
  166. | WHISPER_MODEL | Which whisper.cpp model to use. Valid options are `tiny`, `tiny.en`, `base`, `base.en`, `small`, `small.en`, `medium`, `medium.en`, `large-v1`, `large-v2`, `large-v3`, `large-v3-turbo` | Depends, see the descriptions of the docker images above. For npm, the default option is `medium.en` |
  167. | DATA_DIR_PATH | the data directory of the project | `~/.ai-agents-az-video-generator` with npm, `/app/data` in the Docker images |
  168. | DOCKER | whether the project is running in a Docker container | `true` for the docker images, otherwise `false` |
  169. | DEV | guess! :) | `false` |
  170. # Configuration options
  171. | key | description | default |
  172. | ---------------------- | -------------------------------------------------------------------------------------------------------------- | ---------- |
  173. | paddingBack | The end screen, for how long the video should keep playing after the narration has finished (in milliseconds). | 0 |
  174. | music | The mood of the background music. Get the available options from the GET `/api/music-tags` endpoint. | random |
  175. | captionPosition | The position where the captions should be rendered. Possible options: `top`, `center`, `bottom`. Default value | `bottom` |
  176. | captionBackgroundColor | The background color of the active caption item. | `blue` |
  177. | voice | The Kokoro voice. | `af_heart` |
  178. | orientation | The video orientation. Possible options are `portrait` and `landscape` | `portrait` |
  179. | musicVolume | Set the volume of the background music. Possible options are `low` `medium` `high` and `muted` | `high` |
  180. # Usage
  181. ## MCP server
  182. ## Server URLs
  183. `/mcp/sse`
  184. `/mcp/messages`
  185. ## Available tools
  186. - `create-short-video` Creates a short video - the LLM will figure out the right configuration. If you want to use specific configuration, you need to specify those in you prompt.
  187. - `get-video-status` Somewhat useless, it’s meant for checking the status of the video, but since the AI agents aren’t really good with the concept of time, you’ll probably will end up using the REST API for that anyway.
  188. # REST API
  189. ### GET `/health`
  190. Healthcheck endpoint
  191. ```bash
  192. curl --location 'localhost:3123/health'
  193. ```
  194. ```bash
  195. {
  196. "status": "ok"
  197. }
  198. ```
  199. ### POST `/api/short-video`
  200. ```bash
  201. curl --location 'localhost:3123/api/short-video' \
  202. --header 'Content-Type: application/json' \
  203. --data '{
  204. "scenes": [
  205. {
  206. "text": "Hello world!",
  207. "searchTerms": ["river"]
  208. }
  209. ],
  210. "config": {
  211. "paddingBack": 1500,
  212. "music": "chill"
  213. }
  214. }'
  215. ```
  216. ```bash
  217. {
  218. "videoId": "cma9sjly700020jo25vwzfnv9"
  219. }
  220. ```
  221. ### GET `/api/short-video/{id}/status`
  222. ```bash
  223. curl --location 'localhost:3123/api/short-video/cm9ekme790000hysi5h4odlt1/status'
  224. ```
  225. ```bash
  226. {
  227. "status": "ready"
  228. }
  229. ```
  230. ### GET `/api/short-video/{id}`
  231. ```bash
  232. curl --location 'localhost:3123/api/short-video/cm9ekme790000hysi5h4odlt1'
  233. ```
  234. Response: the binary data of the video.
  235. ### GET `/api/short-videos`
  236. ```bash
  237. curl --location 'localhost:3123/api/short-videos'
  238. ```
  239. ```bash
  240. {
  241. "videos": [
  242. {
  243. "id": "cma9wcwfc0000brsi60ur4lib",
  244. "status": "processing"
  245. }
  246. ]
  247. }
  248. ```
  249. ### DELETE `/api/short-video/{id}`
  250. ```bash
  251. curl --location --request DELETE 'localhost:3123/api/short-video/cma9wcwfc0000brsi60ur4lib'
  252. ```
  253. ```bash
  254. {
  255. "success": true
  256. }
  257. ```
  258. ### GET `/api/voices`
  259. ```bash
  260. curl --location 'localhost:3123/api/voices'
  261. ```
  262. ```bash
  263. [
  264. "af_heart",
  265. "af_alloy",
  266. "af_aoede",
  267. "af_bella",
  268. "af_jessica",
  269. "af_kore",
  270. "af_nicole",
  271. "af_nova",
  272. "af_river",
  273. "af_sarah",
  274. "af_sky",
  275. "am_adam",
  276. "am_echo",
  277. "am_eric",
  278. "am_fenrir",
  279. "am_liam",
  280. "am_michael",
  281. "am_onyx",
  282. "am_puck",
  283. "am_santa",
  284. "bf_emma",
  285. "bf_isabella",
  286. "bm_george",
  287. "bm_lewis",
  288. "bf_alice",
  289. "bf_lily",
  290. "bm_daniel",
  291. "bm_fable"
  292. ]
  293. ```
  294. ### GET `/api/music-tags`
  295. ```bash
  296. curl --location 'localhost:3123/api/music-tags'
  297. ```
  298. ```bash
  299. [
  300. "sad",
  301. "melancholic",
  302. "happy",
  303. "euphoric/high",
  304. "excited",
  305. "chill",
  306. "uneasy",
  307. "angry",
  308. "dark",
  309. "hopeful",
  310. "contemplative",
  311. "funny/quirky"
  312. ]
  313. ```
  314. # Troubleshooting
  315. ## Docker
  316. The server needs at least 3gb free memory. Make sure to allocate enough RAM to Docker.
  317. If you are running the server from Windows and via wsl2, you need to set the resource limits from the [wsl utility 2](https://learn.microsoft.com/en-us/windows/wsl/wsl-config#configure-global-options-with-wslconfig) - otherwise set it from Docker Desktop. (Ubuntu is not restricting the resources unless specified with the run command).
  318. ## NPM
  319. Make sure all the necessary packages are installed.
  320. # n8n
  321. Setting up the MCP (or REST) server depends on how you run n8n and the server. Please follow the examples from the matrix below.
  322. | | n8n is running locally, using `n8n start` | n8n is running locally using Docker | n8n is running in the cloud |
  323. | ------------------------------------------------- | ------------------------------------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------ |
  324. | `short-video-maker` is running in Docker, locally | `http://localhost:3123` | It depends. You can technically use `http://host.docker.internal:3123` as it points to the host, but you could configure to use the same network and use the service name to communicate like `http://short-video-maker:3123` | won’t work - deploy `short-video-maker` to the cloud |
  325. | `short-video-maker` is running with npm/npx | `http://localhost:3123` | `http://host.docker.internal:3123` | won’t work - deploy `short-video-maker` to the cloud |
  326. | `short-video-maker` is running in the cloud | You should use your IP address `http://{YOUR_IP}:3123` | You should use your IP address `http://{YOUR_IP}:3123` | You should use your IP address `http://{YOUR_IP}:3123` |
  327. # Deploying to the cloud
  328. While each VPS provider is different, and it’s impossible to provide configuration to all of them, here are some tips.
  329. - Use Ubuntu ≥ 22.04
  330. - Have ≥ 4gb RAM, ≥ 2vCPUs and ≥5gb storage
  331. - Use [pm2](https://pm2.keymetrics.io/) to run/manage the server
  332. - Put the environment variables to the `.bashrc` file (or similar)
  333. # FAQ
  334. ## Can I use other languages? (French, German etc.)
  335. Unfortunately, it’s not possible at the moment. Kokoro-js only supports English.
  336. ## Can I pass in images and videos and can it stitch it together
  337. No
  338. ## Should I run the project with `npm` or `docker`?
  339. Docker is the recommended way to run the project.
  340. ## How much GPU is being used for the video generation?
  341. Honestly, not a lot - only whisper.cpp can be accelerated.
  342. Remotion is CPU-heavy, and [Kokoro-js](https://github.com/hexgrad/kokoro) runs on the CPU.
  343. ## Is there a UI that I can use to generate the videos
  344. No (t yet)
  345. ## Can I select different source for the videos than Pexels, or provide my own video
  346. No
  347. ## Can the project generate videos from images?
  348. No
  349. ## Dependencies for the video generation
  350. | Dependency | Version | License | Purpose |
  351. | ------------------------------------------------------ | -------- | --------------------------------------------------------------------------------- | ------------------------------- |
  352. | [Remotion](https://remotion.dev/) | ^4.0.286 | [Remotion License](https://github.com/remotion-dev/remotion/blob/main/LICENSE.md) | Video composition and rendering |
  353. | [Whisper CPP](https://github.com/ggml-org/whisper.cpp) | v1.5.5 | MIT | Speech-to-text for captions |
  354. | [FFmpeg](https://ffmpeg.org/) | ^2.1.3 | LGPL/GPL | Audio/video manipulation |
  355. | [Kokoro.js](https://www.npmjs.com/package/kokoro-js) | ^1.2.0 | MIT | Text-to-speech generation |
  356. | [Pexels API](https://www.pexels.com/api/) | N/A | [Pexels Terms](https://www.pexels.com/license/) | Background videos |
  357. ## How to contribute?
  358. PRs are welcome.
  359. See the [CONTRIBUTING.md](CONTRIBUTING.md) file for instructions on setting up a local development environment.
  360. ## License
  361. This project is licensed under the [MIT License](LICENSE).
  362. ## Acknowledgments
  363. - ❤️ [Remotion](https://remotion.dev/) for programmatic video generation
  364. - ❤️ [Whisper](https://github.com/ggml-org/whisper.cpp) for speech-to-text
  365. - ❤️ [Pexels](https://www.pexels.com/) for video content
  366. - ❤️ [FFmpeg](https://ffmpeg.org/) for audio/video processing
  367. - ❤️ [Kokoro](https://github.com/hexgrad/kokoro) for TTS

Ratings and comments

Sign in to rate this skill.

Sign in

Free to use. If it’s helped, pass it on.