Skip to content
Back to search

clips-kitty-nebius-basketball

Basketball highlights with NVIDIA Nemotron on Nebius Token Factory: turn a game into vertical clips. Nemotron picks the plays and writes the titles; E-BARD, YOLO, OpenCV and Whisper read the game. One NBA game cost under a cent. Clips Kitty's Nebius x NVIDIA hackathon edition, open source (AGPL-3.0).

Safety scan

A static text scan, not a guarantee. Select the badge to see what was scanned and every finding.

Checked today
Badge for your README
findsafeskills safety rating

Markdown

[![findsafeskills safety rating](/api/listings/colingpt9--clips-kitty-nebius-basketball/badge.svg)](/listings/colingpt9--clips-kitty-nebius-basketball)

HTML

<a href="/listings/colingpt9--clips-kitty-nebius-basketball"><img src="/api/listings/colingpt9--clips-kitty-nebius-basketball/badge.svg" alt="findsafeskills safety rating" /></a>

Install

i

Install instructions from the README

As written by the repo's author. findsafeskills has not verified these steps.

The full, tested steps are in **[docs/hackathon/JUDGE_SETUP.md](docs/hackathon/JUDGE_SETUP.md)**. You need Python 3.11, FFmpeg, and a Nebius Token Factory API key from <https://tokenfactory.nebius.com/project/api-keys>.

git clone https://github.com/ColinGPT9/clips-kitty-nebius-basketball
cd clips-kitty-nebius-basketball
pip install -r requirements.txt
python scripts/fetch_basketball_model.py && python scripts/fetch_panns.py
export NEBIUS_API_KEY=...                   # Windows PowerShell: $env:NEBIUS_API_KEY="..."
python scripts/hackathon/nebius_check.py    # one real Nemotron call, made the app's way
cd ui && npm install && npm run dev         # the desktop app

In the app:

1. Open **Settings → AI → Direct provider API → Nebius Token Factory**. 2. Paste the key, choose **Nemotron 3.5 Lightning** and press **Test connection**. 3. Go to **Add videos → Sports → Basketball**, add a game, and press **▶ Start queue**.

Use footage you have the rights to. The benchmark uses a public-domain 1995 game from the Internet Archive.

**Replaying a game with another model.** Run the engine through the benchmark once, and the game's evidence is saved. Any model can then be replayed on it in minutes, without processing the video again:

python scripts/hackathon/llm_bench.py serve --freeze bench/frozen -- --port 8765
python scripts/hackathon/llm_bench.py replay bench/frozen/<video>.pkl \
    --backend nebius/nvidia/Nemotron-3_5-Lightning --label nemotron --out bench/runs
python scripts/hackathon/llm_bench.py compare bench/runs/*.json

What was scanned

The files the safety scan read, as written by the repository's author and shown as plain text. Lines with a finding are marked.

README.md · README · 239 lines
  1. <div align="center">
  2. # Clips Kitty — Basketball Highlights with NVIDIA Nemotron
  3. **Turn a basketball game into vertical highlight clips. NVIDIA Nemotron on Nebius Token Factory picks the plays and writes the titles. E-BARD, YOLO, OpenCV and Whisper read the game.**
  4. Nebius × NVIDIA Global AI Hackathon 2026 · Track: **Best Apps and Agents** · Licence: **AGPL-3.0**
  5. [What changed](docs/hackathon/NEBIUS_CHANGES.md) · [Benchmarks](docs/hackathon/BENCHMARKS.md) · [Judge setup](docs/hackathon/JUDGE_SETUP.md) · [Demo script](docs/hackathon/DEMO_SCRIPT.md) · [Feedback](docs/hackathon/FEEDBACK.md)
  6. <a href="https://youtu.be/g1dMqrq-YaI"><img src="https://github.com/user-attachments/assets/2a1bdd85-723e-46d6-b74b-00579c71b6a7" alt="Clips Kitty turning an NBA broadcast into a 9:16 clip that follows the ball" width="100%"></a>
  7. </div>
  8. This repository is the hackathon edition of **[Clips Kitty](https://github.com/ColinGPT9/clips-studio)**,
  9. a free, open-source AI video clipper. It covers only what the hackathon adds:
  10. NVIDIA Nemotron, served by Nebius Token Factory, as the language model of the
  11. **Sports → Basketball** pipeline. Everything else about the app is documented in
  12. the original repository.
  13. ## What it does
  14. You give it a basketball game, as a file or a URL. It:
  15. 1. **Reads the game:** the commentary (Whisper), the crowd, whistle and buzzer
  16. (PANNs), the score bug and game clock (RapidOCR), and the shots and cutaways
  17. (YOLO and OpenCV).
  18. 2. **Asks Nemotron which plays are worth posting.** Nemotron scores the
  19. commentary with the sound and picture events beside it, scores the moments
  20. the basketball profile proposes, and reranks the finalists.
  21. 3. **Writes the titles with Nemotron**, then checks each one against the
  22. scoreboard. A title that names the wrong play or scorer is written again.
  23. 4. **Cuts each play to 9:16.** E-BARD finds the ball, hoop, players and
  24. referees in every frame, and the crop follows the play. Captions come from
  25. Whisper's word timings.
  26. 5. Leaves the clips in the **Clip Editor** for review. **WoopSocial** posts
  27. them only when the creator presses Publish.
  28. The language model is called on the creator's own Nebius key, with no server in
  29. between. One game cost under a cent (see [Results](#results)).
  30. ## What's new for the hackathon
  31. - **A Nebius Token Factory provider** (`llm/providers/nebius.py`). Settings → AI
  32. lists it beside the other providers, with its live model list and prices.
  33. **Nemotron 3.5 Lightning** is preferred when Nebius offers it.
  34. - **Nemotron answers without thinking.** With thinking on, it spent its whole
  35. 8,192-token output limit reasoning on the first chunk of a game, and the job
  36. failed. The provider now asks for `enable_thinking: false`.
  37. - **Reasoning-model handling and usage reporting** in the shared
  38. OpenAI-compatible adapter: thinking written into the answer is removed before
  39. the JSON is read, and every job logs the tokens the provider billed.
  40. - **A frozen-evidence benchmark** (`scripts/hackathon/llm_bench.py`). It saves
  41. everything the pipeline knows about a game when the language model is first
  42. asked, so other models can be replayed on the same evidence.
  43. - **A one-call check** (`scripts/hackathon/nebius_check.py`) that makes a real
  44. Basketball request the app's way, on your key.
  45. Nothing was trained, fine-tuned or replaced. Every changed file is listed in
  46. [NEBIUS_CHANGES.md](docs/hackathon/NEBIUS_CHANGES.md).
  47. ## Architecture
  48. ```mermaid
  49. flowchart TD
  50. V["Basketball game<br/>a video file or a URL"] --> W & P & O & Y
  51. W["Whisper (faster-whisper)<br/>commentary with word timings"] --> N1
  52. P["PANNs<br/>crowd · whistle · buzzer"] --> N1
  53. O["RapidOCR + OpenCV<br/>score bug and game clock"] --> N1
  54. Y["YOLOv8 (COCO) + OpenCV<br/>shots, cutaways, scene cuts"] --> N1
  55. N1["NVIDIA Nemotron on Nebius<br/>scores each commentary chunk,<br/>with the sound and picture events beside it"] --> B
  56. B["Basketball profile<br/>moments from the score bug, the commentary's calls,<br/>the crowd and the clock"] --> N2
  57. N2["NVIDIA Nemotron on Nebius<br/>scores the moment windows,<br/>then reranks the finalists"] --> R
  58. R["Window refinement<br/>YOLO + clock OCR: start on the court,<br/>end when the play is over"] --> N3
  59. N3["NVIDIA Nemotron on Nebius<br/>titles, descriptions and hashtags;<br/>rewrites any the scoreboard contradicts"] --> E
  60. E["E-BARD (YOLOv8n)<br/>ball · hoop · players · referees in every frame"] --> T
  61. T["Clips Kitty tracker<br/>the 9:16 crop follows the play"] --> F
  62. F["FFmpeg<br/>cut, captions, encode"] --> C
  63. C["Clip Editor<br/>review, trim, retitle"] --> S
  64. S["WoopSocial<br/>posts when the creator presses Publish"]
  65. classDef nemotron fill:#76b900,stroke:#4a7400,color:#000
  66. classDef ebard fill:#f58426,stroke:#a8551a,color:#000
  67. class N1,N2,N3 nemotron
  68. class E ebard
  69. ```
  70. | Component | Role | Changed for the hackathon? |
  71. |---|---|---|
  72. | **NVIDIA Nemotron** (3.5 Lightning) | Scores commentary chunks and moment windows, reranks the finalists, writes titles and descriptions, and rewrites titles the scoreboard contradicts | **New backend.** Same prompts, schemas and parsers as before |
  73. | **Nebius Token Factory** | Hosts Nemotron behind an OpenAI-compatible API, called directly on the creator's key | **New provider** |
  74. | **[E-BARD](https://github.com/GabrieleGiudic/E-BARD)** | A YOLOv8n detector for basketball broadcasts: the ball, hoop, players and referees, for the 9:16 framing. Used unmodified under CC BY 4.0 | No |
  75. | **YOLOv8** (Ultralytics, COCO) | Shot types, cutaways to the crowd and bench, close-ups, where clips start | No |
  76. | **OpenCV** | Frame decoding, motion and cut signals, camera pan | No |
  77. | **Whisper** (faster-whisper) | Transcript, word timings, captions | No |
  78. | **PANNs**, **RapidOCR** | Crowd, whistle and buzzer; the score bug and game clock | No |
  79. | **FFmpeg** | Cutting, captions, crop and encoding | No |
  80. | **WoopSocial** | Posting to the creator's connected accounts | No |
  81. **What Nemotron sees.** The same evidence the app's default model always got,
  82. through the same prompts: the Whisper transcript chunk by chunk, the events
  83. beside it (crowd and buzzer sounds, audio spikes, motion, scene cuts), the
  84. windows the basketball moments proposed, and, for titles, each clip's
  85. score-bug note (who scored, the score, the quarter and clock). No cloud model is
  86. sent the picture.
  87. **Why it stays modular.** The Basketball code never names a provider or model.
  88. A job gets whatever backend `llm.backend` names, through one interface
  89. (`llm/base.py`), so the next model is a configuration change, not a
  90. computer-vision rebuild.
  91. ## Results
  92. ### A 2026 NBA game, end to end on Nemotron
  93. Run on 28 October 2026: Toronto Raptors at Washington Wizards, a 43-minute
  94. extended-highlights broadcast with a score bug. The footage and clips are not in
  95. this repository, only the measurements.
  96. | | |
  97. |---|---|
  98. | Model | `nvidia/Nemotron-3_5-Lightning` on Nebius, thinking off |
  99. | Machine | Windows PC, RTX 3060 12 GB (Whisper `large-v3-turbo` on CUDA, NVENC encoding) |
  100. | Whole job, from the file to rendered clips | **30.1 minutes**, 62 clips at 1080×1920 |
  101. | Nemotron calls | 43, every answer usable JSON |
  102. | Tokens and cost | 65,834 in, 19,537 out: **about $0.009** for the game |
  103. | Nemotron's own time | about 90 seconds of the job (measured on a replay of the same evidence) |
  104. | E-BARD framing | 49 clips tracked, 12 assisted, 1 uncertain |
  105. | Titles | The scoreboard check caught 37 of 57 titles naming the wrong play or scorer: 23 were written again, 14 were written from the scoreboard |
  106. **What Nemotron did worse.** Many titles needed the scoreboard check, and one
  107. wrong title still got through ("Bulls's Three Puts the Raptors Up 7", in a
  108. Wizards game). Titles written from the scoreboard are accurate but plain, such
  109. as "Wizards trail by 4". The first attempt, with thinking on, failed outright
  110. (see [What's new](#whats-new-for-the-hackathon)).
  111. ### Gemma and Nemotron on the same evidence
  112. The benchmark freezes one game's evidence and replays only the language-model
  113. steps per model, so the model is the only variable. It uses a public-domain 1995
  114. high-school game.
  115. | Run | Must-have plays found | Titles right | Language-model time | Cost |
  116. |---|---|---|---|---|
  117. | Gemma (`gemma:7b`, local, 4-core CPU), app run | 2 of 7 | 2 of 6 | about 60 min | none (local) |
  118. | Gemma, replay on the same evidence | 2 of 7 | 1 of 7, 2 partly | 56.8 min | none (local) |
  119. | Nemotron 3.5 Lightning (Nebius), replay | 3 of 7 | 4 of 9, 4 partly | 30 s | $0.0032 |
  120. | Nemotron, second replay | 3 of 7 | 4 of 9, 5 partly | 35 s | $0.0034 |
  121. Nemotron found the last three seconds and the court storm, which Gemma missed
  122. both times, and it never gave the game to the wrong team. Its title mistakes were
  123. details, such as the time left on the game-winner. Both models missed the same
  124. four early plays. Gemma's time is a 4-core CPU with no GPU, so the comparison
  125. shows what a creator without a GPU gains, not which model is faster.
  126. Method, reference plays and every clip are in
  127. **[docs/hackathon/BENCHMARKS.md](docs/hackathon/BENCHMARKS.md)**.
  128. ## Quick start
  129. The full, tested steps are in **[docs/hackathon/JUDGE_SETUP.md](docs/hackathon/JUDGE_SETUP.md)**.
  130. You need Python 3.11, FFmpeg, and a Nebius Token Factory API key from
  131. <https://tokenfactory.nebius.com/project/api-keys>.
  132. ```bash
  133. git clone https://github.com/ColinGPT9/clips-kitty-nebius-basketball
  134. cd clips-kitty-nebius-basketball
  135. pip install -r requirements.txt
  136. python scripts/fetch_basketball_model.py && python scripts/fetch_panns.py
  137. export NEBIUS_API_KEY=... # Windows PowerShell: $env:NEBIUS_API_KEY="..."
  138. python scripts/hackathon/nebius_check.py # one real Nemotron call, made the app's way
  139. cd ui && npm install && npm run dev # the desktop app
  140. ```
  141. In the app:
  142. 1. Open **Settings → AI → Direct provider API → Nebius Token Factory**.
  143. 2. Paste the key, choose **Nemotron 3.5 Lightning** and press **Test connection**.
  144. 3. Go to **Add videos → Sports → Basketball**, add a game, and press **▶ Start queue**.
  145. Use footage you have the rights to. The benchmark uses a public-domain 1995
  146. game from the Internet Archive.
  147. **Replaying a game with another model.** Run the engine through the benchmark
  148. once, and the game's evidence is saved. Any model can then be replayed on it in
  149. minutes, without processing the video again:
  150. ```bash
  151. python scripts/hackathon/llm_bench.py serve --freeze bench/frozen -- --port 8765
  152. python scripts/hackathon/llm_bench.py replay bench/frozen/<video>.pkl \
  153. --backend nebius/nvidia/Nemotron-3_5-Lightning --label nemotron --out bench/runs
  154. python scripts/hackathon/llm_bench.py compare bench/runs/*.json
  155. ```
  156. ## Where the hackathon code is
  157. | Path | What |
  158. |---|---|
  159. | [`llm/providers/nebius.py`](llm/providers/nebius.py) | The Nebius provider: endpoint, live model list and prices, Nemotron preferred, thinking off |
  160. | [`llm/providers/adapters/chat_completions.py`](llm/providers/adapters/chat_completions.py) | The shared OpenAI-compatible adapter: thinking removed from answers, usage recorded |
  161. | [`llm/providers/usage.py`](llm/providers/usage.py) | Adds up the tokens each model was billed for (counts only, never prompts or keys) |
  162. | [`scripts/hackathon/llm_bench.py`](scripts/hackathon/llm_bench.py) | The frozen-evidence benchmark: `serve`, `replay`, `compare` |
  163. | [`scripts/hackathon/nebius_check.py`](scripts/hackathon/nebius_check.py) | One real Nemotron request through the app's own code |
  164. | [`tests/test_nebius.py`](tests/test_nebius.py) | Offline tests: every documented Nebius error, rate limits, thinking, usage |
  165. | [`sports/basketball/`](sports/basketball/) | The Basketball pipeline Nemotron serves (unchanged) |
  166. | [`docs/hackathon/`](docs/hackathon/) | Changes, benchmarks, judge setup, demo script, feedback |
  167. ## New versus existing work
  168. Clips Kitty started in June 2026. During the hackathon's submission period
  169. (from 2026-08-26), Colin built most of what this entry demonstrates:
  170. - **2026-09-25:** cloud AI on the user's own key, with OpenRouter, OpenAI, Gemini,
  171. Claude, xAI and Meta.
  172. - **2026-09-30:** the Sports framework.
  173. - **2026-10-05 to 2026-10-21:** Basketball, including the E-BARD 9:16 crop
  174. (#151, #156) and release 2.1.0 (#168).
  175. 331 of the 752 commits up to `34f0c23`, where this edition starts, were made
  176. inside the submission period. The WoopSocial publisher, the Clip Editor, Whisper
  177. and the YOLO and OpenCV work predate it or were built alongside.
  178. **New in this hackathon edition**, 2026-10-28 onward: the Nebius provider,
  179. Nemotron with thinking off, the reasoning-model handling and usage reporting in
  180. the shared adapter, the benchmark harness and the check script, the guards that
  181. stop this copy publishing over the original's image or website, and this
  182. documentation.
  183. ## Licensing
  184. AGPL-3.0, as the original, in [LICENSE](LICENSE). Copyright (C) 2026 ColinGPT9,
  185. as in [NOTICE](NOTICE). Third-party components and model weights keep their own
  186. licences, listed in NOTICE:
  187. - the [E-BARD](https://github.com/GabrieleGiudic/E-BARD) weights are CC BY 4.0, used unmodified;
  188. - Ultralytics YOLOv8 is AGPL-3.0;
  189. - faster-whisper and the Whisper weights are MIT.
  190. Nemotron is called as a hosted API on Nebius, and no model weights are
  191. distributed here. Nebius lists Nemotron 3.5 Lightning under OpenMDW-1.1.

Ratings and comments

Sign in to rate this skill.

Sign in

Free to use. If it’s helped, pass it on.