Skip to content
Back to search

watch-video

SkillWorks with Claude CodeGitHub

Watches a video properly (an X, YouTube, TikTok or Instagram link, or a local file) and works out its edit, words and music. Use when the user asks to watch, analyse, break down or learn from a video, a set of videos or a whole account. Gives the shot list, a frame per shot, on-screen text, the song (Shazam), tempo and beats against the cuts, loudness, silences and a voiceover transcript, and has a batch runner for many videos. Works on macOS, Windows and Linux, including sandboxed Linux environments with a lighter kit.

Safety scan

A static text scan, not a guarantee. Select the badge to see what was scanned and every finding.

Badge for your README
findsafeskills safety rating

Markdown

[![findsafeskills safety rating](/api/listings/pratyushjain98--watch-video/badge.svg)](/listings/pratyushjain98--watch-video)

HTML

<a href="/listings/pratyushjain98--watch-video"><img src="/api/listings/pratyushjain98--watch-video/badge.svg" alt="findsafeskills safety rating" /></a>

Install

i

Install instructions from the README

As written by the repo's author. findsafeskills has not verified these steps.

As a Claude Code skill

Put this folder at `~/.claude/skills/watch-video`:

git clone https://github.com/PratyushJain98/watch-video ~/.claude/skills/watch-video

or copy the folder there by hand. Then run the setup script for your OS (below) once. After that, ask Claude Code to watch or break down a video and it will follow `SKILL.md`.

As a command-line tool

Clone or copy the folder anywhere, run the setup script for your OS, then call `watch.py` with the Python that setup created (see **Use** below). Nothing depends on Claude.

What was scanned

The files the safety scan read, as written by the repository's author and shown as plain text. Lines with a finding are marked.

SKILL.md · Manifest · 107 lines
  1. ---
  2. name: watch-video
  3. description: Watches a video properly (an X, YouTube, TikTok or Instagram link, or a local file) and works out its edit, words and music. Use when the user asks to watch, analyse, break down or learn from a video, a set of videos or a whole account. Gives the shot list, a frame per shot, on-screen text, the song (Shazam), tempo and beats against the cuts, loudness, silences and a voiceover transcript, and has a batch runner for many videos. Works on macOS, Windows and Linux, including sandboxed Linux environments with a lighter kit.
  4. ---
  5. # Watch a video
  6. Claude can't play video or hear sound, so this kit turns a video into things Claude can read and look at.
  7. The scripts are in the `scripts` folder next to this SKILL.md. Installed as a personal skill that is `~/.claude/skills/watch-video/scripts`; if it is somewhere else, `find / -path '*watch-video/scripts/watch.py' 2>/dev/null` shows it. Results go under `~/.cache/watch-video` (set `WATCH_VIDEO_CACHE` to move it).
  8. ## Which machine am I on?
  9. Run `uname -s` and take the first that fits:
  10. 1. **macOS** (`Darwin`): follow **Setup** and **Watch** below.
  11. 2. **Windows** (starts with `MINGW` or `MSYS`, that is Git Bash): follow **Windows** below.
  12. 3. **Linux** (`Linux`), including a sandbox that starts empty each session: follow **Linux** below. It's a lighter kit.
  13. Then read the results the same way everywhere.
  14. ## Setup on macOS (once per machine)
  15. `zsh <scripts>/setup.sh`
  16. It needs `uv`, `ffmpeg`, `yt-dlp` and `swiftc` (Xcode Command Line Tools), and Apple Silicon for mlx-whisper. It builds `~/.cache/watch-video` with a Python 3.12 env (librosa, shazamio, mlx-whisper, matplotlib) and an Apple Vision text-recognition tool. The first transcription downloads Whisper large-v3-turbo, about 1.6 GB. Every setup script keeps a 7-day supply-chain cooldown: it skips any package published in the last 7 days.
  17. ## Watch (macOS)
  18. ```
  19. C=${WATCH_VIDEO_CACHE:-~/.cache/watch-video}; S=<the scripts folder>
  20. yt-dlp -f "bv*[height<=1080]+ba/b" --merge-output-format mp4 -o "$C/vids/<name>.%(ext)s" "<url>"
  21. $C/venv/bin/python $S/watch.py $C/vids/<name>.mp4 # writes $C/runs/<name>/
  22. $C/venv/bin/python $S/align.py $C/runs/<name> # optional: do cuts land on the music's hits?
  23. ```
  24. Public X posts download without logging in. Each video takes from about 30 s to 2 min. If a download fails, retry it with `-f "bv*+ba/b"`. If `ffmpeg` or `ffprobe` isn't found, the shell hasn't loaded Homebrew (common over SSH): run `export PATH=/opt/homebrew/bin:$PATH` first.
  25. ## Many videos (an account, a list of links)
  26. Use the batch runner rather than a hand-written loop. It works on every OS, after that OS's setup:
  27. ```
  28. bash $S/watch-batch.sh <list.txt> # one link or file path per line; run it in the background
  29. ```
  30. - It downloads each link into `vids/`, watches it into `runs/<name>/`, skips anything already watched, and prints one line per video: `OK`, `SKIP` or `FAIL` with the reason.
  31. - Each video gets a time limit (`WATCH_TIMEOUT`, default 300 s), so one stuck video can't hold the rest. On macOS this needs `gtimeout` from coreutils; without it the limit is off and the runner says so.
  32. - Only one runner can work at a time: a second one refuses to start while the lock folder `.batch.lock` exists. If you stop a run, check its processes are really gone before starting another; a stopped loop can survive and process the same videos twice.
  33. - The song lookup is off in batches: Shazam rate-limits after a couple of dozen lookups. Add `--shazam` if the music matters. Each lookup gives up after 20 s, and the first refused or timed-out one skips the rest for that video (`song_matches_note` says why).
  34. - Instagram names come out as `ig_<shortcode>`; the downloaded `<name>.info.json` beside each video holds the caption, likes and comments.
  35. - If you write your own loop instead, give yt-dlp and watch.py `</dev/null`: ffmpeg reads standard input and will otherwise swallow the rest of the list.
  36. ## Windows
  37. Run everything from Git Bash. Setup, once: `bash <scripts>/setup-windows.sh`. It needs only `uv` and `ffmpeg` on PATH, installs nothing system-wide, and prints whether OCR and the GPU are working.
  38. ```
  39. C=${WATCH_VIDEO_CACHE:-~/.cache/watch-video}; S=<the scripts folder>
  40. $C/venv/Scripts/yt-dlp.exe --js-runtimes node -f "bv*[height<=1080]+ba/b" --merge-output-format mp4 -o "$C/vids/<name>.%(ext)s" "<url>"
  41. $C/venv/Scripts/python.exe $S/watch.py $C/vids/<name>.mp4 # writes $C/runs/<name>/
  42. $C/venv/Scripts/python.exe $S/align.py $C/runs/<name>
  43. ```
  44. What's different from macOS:
  45. - **yt-dlp** lives in the venv. `--js-runtimes node` lets it use Node for YouTube, which otherwise warns and may miss formats.
  46. - **On-screen text** comes from the built-in Windows OCR via `scripts/ocr.ps1` (Windows PowerShell 5.1), in the profile's OCR language. It's weaker than Apple Vision on stylised captions, so check the frames when the text matters. If OCR fails, `summary.json` gets an `on_screen_text_note` and you read the frames yourself.
  47. - **Whisper** is faster-whisper: large-v3-turbo in float16 on an NVIDIA GPU, using the cuBLAS/cuDNN wheels inside the venv. With no usable GPU it falls back to `small` in int8 on the CPU, which is slower and less accurate. `summary.json` → `whisper` says which one ran.
  48. ## Linux
  49. Run `setup-linux.sh` once (once per session in a sandbox that starts empty). It needs only `python3` and PyPI, installs nothing system-wide, keeps the 7-day cooldown, and ends with a short report: ffmpeg, OCR, Whisper, Shazam and the video sites, each yes or no. If it times out, run it again; downloads already made are reused. The scripts folder may be read-only; nothing is written there.
  50. ```
  51. C=${WATCH_VIDEO_CACHE:-~/.cache/watch-video}; S=<the scripts folder>
  52. bash $S/setup-linux.sh
  53. $C/venv/bin/yt-dlp --ffmpeg-location $C/bin -f "bv*[height<=1080]+ba/b" --merge-output-format mp4 -o "$C/vids/<name>.%(ext)s" "<url>"
  54. $C/venv/bin/python $S/watch.py <video file> # writes $C/runs/<name>/
  55. $C/venv/bin/python $S/align.py $C/runs/<name>
  56. ```
  57. What's weaker here:
  58. - **Whisper** runs on the CPU: faster-whisper `small` in int8, slower and less accurate than on macOS or a Windows GPU. `WATCH_WHISPER_MODEL=medium` (or a local model folder) changes it. It only transcribes stretches with a voice, so a music-only track gives an empty transcript in seconds instead of minutes of junk; sung words may be missed. If the model can't be downloaded, there is no `transcript.txt` and `summary.json` has a `whisper_note`.
  59. - **On-screen text** comes from `tesseract` if it is installed. If not, `summary.json` has an `on_screen_text_note`; read the text from `shots.jpg` and `frames/` yourself.
  60. - **Shazam** is often blocked or rate-limited from sandboxes and cloud machines. Then `summary.json` has a `song_matches_note`. Don't guess the song. For many videos in a row, pass `--no-shazam` unless the music matters.
  61. - **ffmpeg** is the system's, else the copy inside the imageio-ffmpeg package. That copy has no ffprobe; watch.py manages without it.
  62. If the video can't be downloaded here (the report says the sites are blocked, or yt-dlp fails), don't hunt for workarounds. Ask the user to attach the file or put it in the working folder, then run watch.py on that path. If that isn't possible, say plainly that this environment can't fetch the video, and that the full kit (GPU Whisper, OCR, Shazam, direct downloads) needs a machine where setup finished with everything reported as working.
  63. ## Read the results, in this order
  64. 1. **`shots.jpg`**: one frame from the middle of every shot, labelled `#n start (length)`. This is the edit at a glance. Use `sheet.jpg` (30 evenly spaced frames) when there are few shots, or the camera moves continuously.
  65. 2. **`summary.json`**:
  66. - the edit: shots, `shot_lengths`, `cuts_per_5s` (pace over time)
  67. - the music: `tempo_bpm`, `silences`, `biggest_jumps` (drops and hits), and `song_matches` (the actual track, from Shazam)
  68. - the words: `on_screen_text` per shot
  69. 3. **`timeline.png`**: a mel spectrogram over loudness, with cuts in red and beats as white ticks. It shows builds, drops and silences against the edit.
  70. 4. **`transcript.txt`**: the voiceover. It's only real if the lines read as speech. On music-only tracks, Whisper invents filler ("Music", "Thank you.", stray foreign words), so ignore it then.
  71. With one or two shots, the on-screen text was read from only one or two frames and misses anything that changes over time; read `sheet.jpg` for those. When handing results to another session, say which limits apply (filler transcripts, few-shot videos, song lookup off).
  72. ## Honesty rules
  73. - Never claim cuts are "on the beat" from the chart alone. The `cuts_on_beat` and `align.py` figures compare against chance. Most edits are cut to phrases, not frame-locked to beats, so say what the numbers show.
  74. - Never quote song lyrics, since they are copyrighted. Name songs by title and artist only.
  75. - Sound is understood through measurement and identification, not listening. Say so when it matters, and leave judgements of taste to the user.
  76. - Only download videos the user has the right to download, and respect each platform's terms.
README.md · README · 258 lines
  1. # watch-video
  2. A Claude Code skill and command-line tool that turns a video into things a language model (or a person) can read: a shot list with one frame per shot, the on-screen text, the song, the tempo and beats against the cuts, loudness, silences and a voiceover transcript.
  3. Claude can't play video or hear sound. This kit measures the video instead, so Claude can talk about how it was edited, what it says and what music it uses, from evidence rather than guesses.
  4. It takes a link (X, YouTube, TikTok, Instagram, or anything else yt-dlp supports) or a local file, works on macOS, Windows and Linux, and has a batch runner for whole lists or accounts.
  5. ## What you get
  6. For each video, a folder `runs/<name>/` with:
  7. | File | What it holds |
  8. | --- | --- |
  9. | `shots.jpg` | One frame from the middle of every shot, labelled `#n start (length)`. The edit at a glance. |
  10. | `sheet.jpg` | 30 evenly spaced frames, for videos with few cuts or a moving camera. |
  11. | `timeline.png` | A mel spectrogram over a loudness curve, with cuts in red and beats as white ticks. |
  12. | `frames/` | The per-shot frames at full size. |
  13. | `summary.json` | Cut times, shot lengths, cuts per 5 seconds, tempo, cuts on the beat against chance, loudness, silences, the biggest loudness jumps, song matches, on-screen text per shot, and speech totals. |
  14. | `transcript.txt` | The Whisper transcript with timestamps. |
  15. | `audio.wav` | The mono soundtrack, kept for `align.py`. |
  16. `align.py` adds one more figure to `summary.json`: how many cuts land within 50 ms of a strong musical hit, against what chance would give.
  17. ## Sample output
  18. A 12-second test clip: four coloured shots with a caption each, a click track at 120 BPM, and a spoken line.
  19. ```
  20. $ python watch.py test-clip.mp4 --threshold 0.12
  21. test-clip {"duration": 12.0, "fps": 30.0, "shots": 4, "mean_shot": 3.0, "cuts_per_5s": [1, 2, 0], "tempo_bpm": 117.5, "cuts_on_beat": "3/3 (100%), chance about 23%", ...}
  22. ```
  23. `summary.json` (silences and loudness jumps left out):
  24. ```json
  25. {
  26. "file": "test-clip.mp4", "duration": 12.0, "fps": 30.0, "size": [1280, 720],
  27. "cuts": [3.0, 6.0, 9.0], "shots": 4, "mean_shot": 3.0, "shot_lengths": [3.0, 3.0, 3.0, 3.0],
  28. "cuts_per_5s": [1, 2, 0],
  29. "on_screen_text": [
  30. {"shot": "#1 0.0s (3.0)", "text": "OPENING"},
  31. {"shot": "#2 3.0s (3.0)", "text": "SECOND SHOT"},
  32. {"shot": "#3 6.0s (3.0)", "text": "THIRD SHOT"},
  33. {"shot": "#4 9.0s (3.0)", "text": "THE END"}
  34. ],
  35. "tempo_bpm": 117.5, "beats": 23, "cuts_on_beat": "3/3 (100%), chance about 23%",
  36. "loudness_db": {"median": -18.8, "max": -10.3},
  37. "song_matches": [{"at": 0.0, "title": null}],
  38. "speech_segments": 1, "speech_seconds": 7.0, "language": "en",
  39. "cuts_on_hits": "2/3 (67%), chance about 19%"
  40. }
  41. ```
  42. `transcript.txt`:
  43. ```
  44. [ 0.0- 7.0] This is a short test of the watch video tool, four colored shots, a click track, and this voice.
  45. ```
  46. `shots.jpg`:
  47. ![Four frames, one per shot, each labelled with its shot number, start time and length](docs/sample-shots.jpg)
  48. `timeline.png`:
  49. ![Mel spectrogram and loudness curve with cuts marked in red and beats as white ticks](docs/sample-timeline.png)
  50. The song lookup returned no match, as expected for a synthetic click track.
  51. ## How it works
  52. - **Cuts**: ffmpeg's scene-change score, with a threshold of 0.25 by default.
  53. - **Frames**: ffmpeg takes one frame from the middle of each shot (up to 72, sampled evenly beyond that) and 30 evenly spaced frames; Pillow tiles them into contact sheets.
  54. - **On-screen text**: Apple's Vision framework on macOS, the built-in Windows OCR on Windows, and tesseract on Linux if it is installed.
  55. - **Music**: librosa estimates tempo and beats, loudness, silences and the largest jumps, and draws the spectrogram with matplotlib. Cuts are compared with beat times against what chance would give.
  56. - **Song**: fingerprints of up to three 12-second clips are looked up on Shazam through the shazamio library.
  57. - **Words**: Whisper large-v3-turbo through mlx-whisper on macOS, faster-whisper on Windows (GPU if available) and Linux (CPU).
  58. ## Install
  59. ### As a Claude Code skill
  60. Put this folder at `~/.claude/skills/watch-video`:
  61. ```
  62. git clone https://github.com/PratyushJain98/watch-video ~/.claude/skills/watch-video
  63. ```
  64. or copy the folder there by hand. Then run the setup script for your OS (below) once. After that, ask Claude Code to watch or break down a video and it will follow `SKILL.md`.
  65. ### As a command-line tool
  66. Clone or copy the folder anywhere, run the setup script for your OS, then call `watch.py` with the Python that setup created (see **Use** below). Nothing depends on Claude.
  67. ## Setup by OS
  68. Every setup script builds a Python env in `~/.cache/watch-video` (set `WATCH_VIDEO_CACHE` to put it elsewhere) and installs nothing system-wide. Each one applies a 7-day supply-chain cooldown (the scripts call it a quarantine): it only installs package versions published at least 7 days ago, using uv's `--exclude-newer` (or pip's `--uploaded-prior-to` on Linux when uv is not available).
  69. ### macOS
  70. Needs Apple Silicon (mlx-whisper has no Intel build, so setup fails on an Intel Mac), plus uv, ffmpeg, yt-dlp and the Xcode Command Line Tools:
  71. ```
  72. xcode-select --install
  73. brew install uv ffmpeg yt-dlp coreutils # coreutils gives gtimeout, used by the batch runner
  74. zsh scripts/setup.sh
  75. ```
  76. Recent yt-dlp releases need a JavaScript runtime (Deno or Node) for full YouTube support; see yt-dlp's documentation.
  77. This installs librosa, shazamio, mlx-whisper, soundfile and matplotlib into a Python 3.12 env, and compiles `scripts/ocr.swift` into a small Apple Vision OCR tool. The first transcription downloads the Whisper large-v3-turbo model, about 1.6 GB.
  78. ### Windows
  79. Run everything from Git Bash. Needs uv and ffmpeg on PATH; Node is recommended so yt-dlp can get every YouTube format:
  80. ```
  81. winget install Git.Git
  82. winget install astral-sh.uv
  83. winget install Gyan.FFmpeg
  84. winget install OpenJS.NodeJS.LTS
  85. bash scripts/setup-windows.sh
  86. ```
  87. This installs librosa, shazamio, faster-whisper, soundfile, matplotlib and yt-dlp into a Python 3.12 env. If `nvidia-smi` works, it also installs NVIDIA's cuBLAS and cuDNN runtime wheels into the env so Whisper runs on the GPU without a system-wide CUDA install. It finishes by checking Windows OCR (which needs an OCR-capable language in Settings, Time & language, Language) and saying whether Whisper will use the GPU or the CPU.
  88. ### Linux
  89. Needs `python3` and access to PyPI. ffmpeg and tesseract are optional but recommended:
  90. ```
  91. sudo apt install ffmpeg tesseract-ocr # optional; Debian or Ubuntu shown
  92. bash scripts/setup-linux.sh
  93. ```
  94. This uses uv if it is installed, otherwise downloads uv from PyPI (checking its SHA-256 against PyPI's record), otherwise falls back to pip. It installs librosa, shazamio, faster-whisper, soundfile, matplotlib, pillow and yt-dlp. If there is no system ffmpeg it uses the one bundled in the imageio-ffmpeg package. It ends with a short report of what works: ffmpeg, OCR, Whisper, Shazam and each video site, yes or no.
  95. It is written to also work in locked-down sandboxes (no root, no GPU, a read-only skill folder, a network that may only reach package registries). In a sandbox that starts empty each session, run it once per session.
  96. ## Use
  97. The commands below use `C` for the cache folder and `S` for this repo's `scripts` folder.
  98. macOS:
  99. ```
  100. C=~/.cache/watch-video; S=~/.claude/skills/watch-video/scripts
  101. yt-dlp -f "bv*[height<=1080]+ba/b" --merge-output-format mp4 -o "$C/vids/<name>.%(ext)s" "<url>"
  102. $C/venv/bin/python $S/watch.py $C/vids/<name>.mp4 # writes $C/runs/<name>/
  103. $C/venv/bin/python $S/align.py $C/runs/<name> # optional: do the cuts land on musical hits?
  104. ```
  105. Windows (Git Bash): yt-dlp lives in the env, and the Python is `venv/Scripts/python.exe`:
  106. ```
  107. $C/venv/Scripts/yt-dlp.exe --js-runtimes node -f "bv*[height<=1080]+ba/b" --merge-output-format mp4 -o "$C/vids/<name>.%(ext)s" "<url>"
  108. $C/venv/Scripts/python.exe $S/watch.py $C/vids/<name>.mp4
  109. ```
  110. Linux: yt-dlp lives in the env and needs to be told where ffmpeg is:
  111. ```
  112. $C/venv/bin/yt-dlp --ffmpeg-location $C/bin -f "bv*[height<=1080]+ba/b" --merge-output-format mp4 -o "$C/vids/<name>.%(ext)s" "<url>"
  113. $C/venv/bin/python $S/watch.py $C/vids/<name>.mp4
  114. ```
  115. `watch.py` options:
  116. | Option | Effect |
  117. | --- | --- |
  118. | `--out <folder>` | Write results here instead of `$C/runs/<video name>/`. |
  119. | `--threshold <n>` | Scene-change threshold for cuts (default 0.25). Lower finds more cuts, including false ones. |
  120. | `--no-whisper` | Skip the transcript. |
  121. | `--no-shazam` | Skip the song lookup. |
  122. Environment variables:
  123. | Variable | Effect |
  124. | --- | --- |
  125. | `WATCH_VIDEO_CACHE` | Where the env, downloads and results live (default `~/.cache/watch-video`). |
  126. | `WATCH_WHISPER_MODEL` | Linux only: the faster-whisper model name (default `small`) or a local model folder. |
  127. | `WATCH_TIMEOUT` | Batch runner: seconds allowed per video (default 300). |
  128. | `WATCH_PYTHON` | Linux setup: which `python3` to build the env from. |
  129. | `WATCH_SETUP_PIP` | Linux setup: set to `1` to use pip instead of uv. |
  130. ## Batch mode
  131. ```
  132. bash scripts/watch-batch.sh list.txt [extra watch.py options]
  133. ```
  134. `list.txt` holds one link or file path per line. The runner:
  135. - downloads each link into `vids/` (with yt-dlp's `.info.json` beside it, which holds the caption and counts), then watches it into `runs/<name>/`;
  136. - skips videos that already have a `summary.json`;
  137. - prints one line per video, `OK`, `SKIP` or `FAIL` with the reason, and a total at the end; it exits non-zero if anything failed;
  138. - gives each video a time limit (`WATCH_TIMEOUT`), using `timeout` or `gtimeout` if one is installed;
  139. - refuses to start while another batch holds the lock folder `$C/.batch.lock`;
  140. - turns the song lookup off unless you pass `--shazam`, because Shazam starts refusing requests after a couple of dozen lookups.
  141. Names come from the link: `yt_<id>`, `x_<id>`, `tt_<id>`, `ig_<shortcode>`.
  142. ## Dependencies and licences
  143. This repo is MIT-licensed. The setup scripts pull in the following, each under its own licence:
  144. | Dependency | Used for | Licence |
  145. | --- | --- | --- |
  146. | [uv](https://github.com/astral-sh/uv) | Creating the env and installing packages | MIT or Apache-2.0 |
  147. | [Python](https://www.python.org) 3.12 (uv downloads a standalone build if none is installed) | Running the scripts | PSF License |
  148. | [FFmpeg](https://ffmpeg.org) | Cut detection, frames, audio extraction | LGPL-2.1+, or GPL if built with GPL parts (Homebrew's and most Linux builds are) |
  149. | [yt-dlp](https://github.com/yt-dlp/yt-dlp) | Downloading videos | Unlicense |
  150. | [librosa](https://github.com/librosa/librosa) | Tempo, beats, onsets, loudness, spectrogram | ISC |
  151. | [NumPy](https://numpy.org) | Numerics | BSD-3-Clause |
  152. | [matplotlib](https://matplotlib.org) | `timeline.png` | Matplotlib licence (PSF-style, BSD-compatible) |
  153. | [Pillow](https://python-pillow.org) | Contact sheets | MIT-CMU (HPND) |
  154. | [python-soundfile](https://github.com/bastibe/python-soundfile) | Reading audio (bundles libsndfile, LGPL-2.1) | BSD-3-Clause |
  155. | [shazamio](https://github.com/shazamio/ShazamIO) | Song identification | MIT |
  156. | [mlx-whisper](https://github.com/ml-explore/mlx-examples) (macOS) | Transcription on Apple Silicon | MIT |
  157. | [faster-whisper](https://github.com/SYSTRAN/faster-whisper) and CTranslate2 (Windows, Linux) | Transcription | MIT |
  158. | Whisper model weights ([OpenAI](https://github.com/openai/whisper); downloaded from Hugging Face as `mlx-community/whisper-large-v3-turbo` or `Systran/faster-whisper-*` conversions) | Transcription | MIT |
  159. | NVIDIA cuBLAS and cuDNN wheels (Windows with an NVIDIA GPU only) | GPU transcription | NVIDIA proprietary licence: free to use, not open source |
  160. | [imageio-ffmpeg](https://github.com/imageio/imageio-ffmpeg) (Linux without ffmpeg only) | A bundled ffmpeg binary | BSD-2-Clause wrapper; the bundled ffmpeg is GPL |
  161. | [Tesseract](https://github.com/tesseract-ocr/tesseract) (Linux, optional) | OCR | Apache-2.0 |
  162. | Apple Vision (macOS) and Windows.Media.Ocr (Windows) | OCR | Part of the operating system |
  163. Each of these brings its own dependencies (for example numba, scipy and scikit-learn through librosa, and aiohttp through shazamio), all from PyPI.
  164. Python packages come from PyPI and the Whisper models from Hugging Face. System tools come from your OS package manager. If no Python 3.12 is installed, uv downloads one from Astral's [python-build-standalone](https://github.com/astral-sh/python-build-standalone) releases on GitHub. Nothing is fetched from any other location.
  165. ## Network services
  166. - **Video sites**, through yt-dlp, when you give a link.
  167. - **PyPI** during setup, and **Hugging Face** once, to download the Whisper model.
  168. - **Shazam**, for the song lookup. shazamio is an unofficial client that talks to Shazam's servers the way Shazam's own apps do; it is not an official public API. It needs no account or key and costs nothing, but it can rate-limit, refuse requests or stop working without notice, and its use is subject to Shazam's terms. Each lookup sends an audio fingerprint of a 12-second clip. Use `--no-shazam` to turn it off.
  169. No part of the tool needs a paid service or an API key.
  170. ## Limitations
  171. - **Cuts between shots of similar brightness can be missed.** ffmpeg's scene score is driven mostly by brightness. In the sample clip above, the cut from the red shot to the green one scored 0.16 and was missed at the default threshold of 0.25; `--threshold 0.12` found it. Lower thresholds also add false cuts on fast motion or flashes, so check `shots.jpg`.
  172. - **On-screen text is read from one frame per shot.** Text that changes within a shot, and anything in videos with only one or two shots, can be missed; `sheet.jpg` helps there. Windows OCR is weaker than Apple Vision on stylised captions.
  173. - **Transcripts of music-only tracks are filler.** Whisper invents lines such as "Music" or "Thank you." when there is no speech. On Linux, voice-activity filtering avoids most of this, but sung words may be missed.
  174. - **Linux transcription runs on the CPU only**, with the `small` model by default, even if the machine has a GPU. It is slower and less accurate than macOS or a Windows GPU.
  175. - **macOS needs Apple Silicon.** mlx-whisper has no Intel build, so Intel Macs are not supported.
  176. - **Tempo is an estimate.** The 120 BPM click track above came out as 117.5 BPM. Cuts "on the beat" are reported against chance; most edits follow musical phrases rather than individual beats.
  177. - **Song identification is best-effort** (see Network services above).
  178. - **Downloads depend on yt-dlp** keeping up with each site. Private or login-only videos are not handled; download those yourself and pass the file.
  179. - **Very long videos** get at most 72 frames in `shots.jpg`, sampled evenly across the shots.
  180. ## Using it responsibly
  181. Downloading a video can break a platform's terms of service or infringe copyright. Only download and analyse videos you have the right to, follow each platform's terms, and do not redistribute what you download. Song lyrics are copyrighted: the skill tells Claude to name songs by title and artist and never to quote lyrics from transcripts.
  182. ## Licence
  183. MIT. See [LICENSE](LICENSE).

Ratings and comments

Sign in to rate this skill.

Sign in

Free to use. If it’s helped, pass it on.