Skip to content
Back to search

adforge

Product page → 9:16 video ad. Agentic pipeline with a self-critiquing VLM loop and an MCP server.

Safety scan

A static text scan, not a guarantee. Select the badge to see what was scanned and every finding.

Checked today
Badge for your README
findsafeskills safety rating

Markdown

[![findsafeskills safety rating](/api/listings/coobeliues--adforge/badge.svg)](/listings/coobeliues--adforge)

HTML

<a href="/listings/coobeliues--adforge"><img src="/api/listings/coobeliues--adforge/badge.svg" alt="findsafeskills safety rating" /></a>

Install

i

Install instructions from the README

As written by the repo's author. findsafeskills has not verified these steps.

git clone <this repo> && cd adforge
python -m venv .venv && source .venv/bin/activate
pip install -e ".[tts,dev]"          # core + local TTS
pip install -e ".[gpu]"              # optional: local image generation (torch + diffusers)

ollama pull qwen3.5:4b               # LLM + vision judge in one 3 GB model
cp .env.example .env                 # pick providers
adforge doctor                       # checks ollama, model, ffmpeg, fonts, GPU

Make an ad:

# from a product page
adforge make "https://…/product" --lang en --duration 15

# or manually
adforge make bottle --name "HydroPeak 750" --desc "Keeps drinks cold 24h, leak-proof" \
  --image photo.jpg --price "12 990" --currency KZT --lang ru --music track.mp3

# plan first, review, tweak, render
adforge make "https://…" --plan-only
adforge edit <run_id> 1 --caption "Still drinking warm water?"
adforge render <run_id>

Talk to the agent:

adforge chat
you › make a 15s ad in Russian for https://… aimed at students
you › shot 2 feels generic — make it a rainy bus stop at night and re-render

What was scanned

The files the safety scan read, as written by the repository's author and shown as plain text. Lines with a finding are marked.

README.md · README · 204 lines
  1. # AdForge
  2. **Product page in → 9:16 video ad out.** An agentic pipeline that reads a product page, writes a
  3. creative brief and storyboard, generates keyframes, **reviews its own frames with a vision model and
  4. rewrites prompts until they pass**, then animates, voices, captions and cuts a TikTok/Reels-ready ad.
  5. Runs fully local on a laptop (Ollama + a 6 GB GPU), every capability is a swappable provider, and the
  6. whole thing is drivable by **Claude Code / Claude Desktop over MCP** or by its own from-scratch agent.
  7. ```bash
  8. adforge make https://shop.example.com/product/123 --lang ru --duration 15
  9. ```
  10. <p align="center">
  11. <img src="docs/demo.gif" width="240" alt="Generated ad">
  12. &nbsp;
  13. <img src="docs/storyboard.png" width="560" alt="Storyboard keyframes">
  14. </p>
  15. <p align="center"><sub>A 15 s Russian-language ad generated fully locally (qwen3.5:4b + SDXL-Turbo + Piper on an
  16. RTX 4050 laptop) from a product name, a description and one product photo. Left: final cut. Right: the
  17. storyboard's keyframes — real product photo shots (hero, close-up, CTA) and AI-generated scenes.</sub></p>
  18. ---
  19. ## Why it's interesting
  20. | | |
  21. |---|---|
  22. | **Self-critiquing quality loop** | Each generated keyframe goes through `generate → judge → revise`. Cheap pixel heuristics (blur, exposure, contrast) gate first, then a VLM scores prompt adherence, artifacts and ad suitability against a rubric. Failing frames get their prompt rewritten by the LLM *from the judge's feedback* and are regenerated; the best attempt wins. Every attempt, score and revision is visible in the run report. |
  23. | **MCP server** | 8 tools (`plan_ad`, `edit_shot`, `regenerate_shot`, `render_ad`, `storyboard_sheet`, …) with progress notifications. Claude can plan an ad, *look at* the storyboard image, give a director's note on shot 3 and re-render only what changed. |
  24. | **Agent loop from scratch** | `adforge chat` — a ~100-line tool-calling loop (no framework): JSON-schema tool specs generated from Python signatures, tool errors fed back to the model for self-correction, context compaction of old tool outputs. Works with a 4B local model. |
  25. | **Provider abstraction** | Image / video / TTS / judge / LLM are interfaces. Today: Ollama, any OpenAI-compatible endpoint (vLLM, Groq, OpenRouter, Gemini…), diffusers on a local GPU, Piper TTS. Cloud video models drop in as another provider. |
  26. | **Resilient cloud client** | Cloud LLMs get overloaded or hang. The OpenAI-compatible client walks a model fallback chain on 429/5xx/timeouts and sticks to the model that answered; permanent errors fail fast. Provider-specific tool-call fields (Gemini 3's `thought_signature`) are round-tripped verbatim, which tool calling requires. |
  27. | **Structured output that survives small models** | Pydantic schemas → Ollama `format` / `json_schema`; lenient coercion of sloppy values; on validation failure the error is sent back and the model fixes its own JSON. |
  28. | **Real product, not hallucinated** | The product is only ever shown from its real photos (enforced for the reveal and CTA shots); generated scenes show people and situations. Photos are cut out from studio backgrounds (corner-detected, border-connected morphological reconstruction — keeps white details *inside* the product, works on tightly cropped shop photos) and staged on a brand-colored set. A single photo yields several distinct shots: hero, punch-in close-ups on different parts, CTA layout. |
  29. | **Claims review** | A compliance pass flags voiceover/caption claims that the product data doesn't support (invented numbers, comparisons, awards). Flags are surfaced in the CLI, the report and `get_run` for a human or Claude to resolve with `edit_shot` — never silently rewritten, because a small model spots suspicious lines far more reliably than it rewrites them. |
  30. | **Observability** | Each run is a directory: `state.json` (resumable), `trace.jsonl` (every LLM call with latency + tokens, every judge verdict and revision), `report.html`, `storyboard.png`. |
  31. | **Steps are idempotent** | Edit one shot → only its keyframe/clip is re-rendered. |
  32. | **Fits a 6 GB GPU / 16 GB laptop** | The LLM and SDXL can't share 6 GB of VRAM, so they time-share it: the LLM is unloaded before each image, and SDXL runs in three short phases (text encoders → UNet → VAE), each loaded straight from safetensors into VRAM and freed right after. Weights never sit in system RAM — a full run stays under a 6 GB RAM cgroup cap (vs. >8.5 GB with diffusers' standard CPU offload, which OOM-killed the laptop). |
  33. ## Architecture
  34. ```mermaid
  35. flowchart LR
  36. URL[Product URL / manual input] --> ING[Ingest<br/>JSON-LD · OpenGraph · photos]
  37. ING --> BRIEF[Creative brief<br/>LLM sees product photo]
  38. BRIEF --> SB[Storyboard<br/>hook → … → CTA<br/>+ claims review]
  39. SB --> VO[Voiceover<br/>Piper TTS]
  40. SB --> KF{Keyframes}
  41. KF -- product_photo --> HERO[Cut-out + brand set]
  42. KF -- generated --> GEN[Image provider]
  43. GEN --> J[Judge<br/>heuristics → VLM rubric]
  44. J -- fail --> REV[LLM rewrites prompt<br/>from judge feedback] --> GEN
  45. J -- pass / best of N --> ANIM
  46. HERO --> ANIM[Animate<br/>Ken Burns motion]
  47. VO --> CUT
  48. ANIM --> CUT[Final cut<br/>crossfades · karaoke captions<br/>CTA end card · ducked music]
  49. CUT --> OUT[final.mp4 + report.html]
  50. subgraph Drivers
  51. CLI[CLI]
  52. MCP[MCP server<br/>Claude Code / Desktop]
  53. AG[Own agent loop]
  54. end
  55. Drivers -.-> ING & SB & KF & CUT
  56. ```
  57. ```
  58. src/adforge/
  59. pipeline.py orchestrator; each stage is a resumable method on a Run
  60. creative.py brief / storyboard / prompt-revision LLM steps + invariants
  61. llm/ provider-agnostic chat: tools, vision, structured output w/ self-repair
  62. providers/ image (diffusers, placeholder) · video (Ken Burns) · tts (Piper, silent) · judge
  63. render/ cut-out & hero staging · captions & end card · audio mix · final assembly
  64. tools.py the tool surface shared by MCP and the agent (+ schema from signatures)
  65. agent.py from-scratch tool-calling loop
  66. mcp_server.py MCP server with progress notifications
  67. report.py HTML report + storyboard contact sheet
  68. ```
  69. ## Quickstart
  70. ```bash
  71. git clone <this repo> && cd adforge
  72. python -m venv .venv && source .venv/bin/activate
  73. pip install -e ".[tts,dev]" # core + local TTS
  74. pip install -e ".[gpu]" # optional: local image generation (torch + diffusers)
  75. ollama pull qwen3.5:4b # LLM + vision judge in one 3 GB model
  76. cp .env.example .env # pick providers
  77. adforge doctor # checks ollama, model, ffmpeg, fonts, GPU
  78. ```
  79. Make an ad:
  80. ```bash
  81. # from a product page
  82. adforge make "https://…/product" --lang en --duration 15
  83. # or manually
  84. adforge make bottle --name "HydroPeak 750" --desc "Keeps drinks cold 24h, leak-proof" \
  85. --image photo.jpg --price "12 990" --currency KZT --lang ru --music track.mp3
  86. # plan first, review, tweak, render
  87. adforge make "https://…" --plan-only
  88. adforge edit <run_id> 1 --caption "Still drinking warm water?"
  89. adforge render <run_id>
  90. ```
  91. Talk to the agent:
  92. ```bash
  93. adforge chat
  94. you › make a 15s ad in Russian for https://… aimed at students
  95. you › shot 2 feels generic — make it a rainy bus stop at night and re-render
  96. ```
  97. ## Use it from Claude Code / Claude Desktop (MCP)
  98. Opening this repo in Claude Code picks up the bundled [`.mcp.json`](.mcp.json) automatically. Elsewhere:
  99. ```bash
  100. claude mcp add adforge -- /abs/path/to/.venv/bin/adforge mcp
  101. ```
  102. Claude Desktop (`claude_desktop_config.json`):
  103. ```json
  104. {
  105. "mcpServers": {
  106. "adforge": {
  107. "command": "/abs/path/to/.venv/bin/adforge",
  108. "args": ["mcp"],
  109. "env": { "ADFORGE_RUNS_DIR": "/abs/path/to/runs" }
  110. }
  111. }
  112. }
  113. ```
  114. Streamable HTTP: `adforge mcp --http --port 8765`.
  115. ## Configuration
  116. All settings are env vars (`ADFORGE_*`) or `.env` — see [.env.example](.env.example).
  117. | Setting | Options | Default |
  118. |---|---|---|
  119. | `LLM_PROVIDER` / `LLM_MODEL` | `ollama`, `openai` (any OpenAI-compatible URL via `LLM_BASE_URL`) | `ollama` / `qwen3.5:4b` |
  120. | `LLM_FALLBACK_MODELS` | comma-separated models tried when the main one is overloaded | — |
  121. | `VISION_MODEL` | separate judge model | same as LLM |
  122. | `IMAGE_PROVIDER` | `diffusers` (local GPU), `placeholder` | `placeholder` |
  123. | `I2V_PROVIDER` | image-to-video model for *generated* scenes only (real product shots always stay on `VIDEO_PROVIDER`) | `none` |
  124. | `DIFFUSERS_MODEL` | `stabilityai/sdxl-turbo` (~6.5 GB, best), `stabilityai/sd-turbo` (~2.6 GB, for 16 GB RAM machines) | `stabilityai/sdxl-turbo` |
  125. | `TTS_PROVIDER` | `piper` (en/ru/kk voices), `silent` | `silent` |
  126. | `JUDGE` | `vlm`, `heuristic`, `none` | `vlm` |
  127. | `MAX_ATTEMPTS` / `PASS_SCORE` | quality loop budget / threshold | `3` / `7.0` |
  128. ## Performance (laptop: Ryzen 5 8645HS, RTX 4050 6 GB, 16 GB RAM)
  129. | Stage | Time |
  130. |---|---|
  131. | Brief + storyboard (qwen3.5:4b, local) | ~30 s |
  132. | SDXL-Turbo keyframe 576×1024 → 1080×1920 (4.9 GB peak VRAM) | ~7–10 s |
  133. | VLM judge, per frame | ~4 s |
  134. | Animate 5 shots, 1080×1920@30 (parallel) | ~8 s |
  135. | Final cut with captions + audio | ~15 s |
  136. | **Full 15 s ad, 5 shots, quality loop on** | **~2–3 min** |
  137. ## Tests
  138. ```bash
  139. pytest # unit tests + full pipeline e2e with a scripted LLM (no network, no GPU)
  140. ```
  141. Covers JSON repair, schema coercion, storyboard invariants, JSON-LD/OpenGraph parsing, price
  142. formatting, cut-out (inner whites, tightly cropped photos, busy backgrounds), caption timing and
  143. contrast, crossfade timeline, the RAM guard, the agent's error recovery, and the full pipeline:
  144. `generate → judge(fail) → revise → judge(pass)`, claim flags, resume without regeneration,
  145. partial re-render after edits (a longer voiceover re-voices and re-renders only that shot),
  146. shot-index validation.
  147. ```bash
  148. ruff check src tests && ruff format --check src tests
  149. ```
  150. ## Known limitations
  151. - A text-to-image model can't reproduce a specific product, so the real product appears only in
  152. `product_photo` shots (enforced for the solution and CTA shots); generated shots are meant to show
  153. people and situations. A small local LLM sometimes still writes product-centric prompts for
  154. generated shots — the judge catches and scores them down, but a stronger LLM (any
  155. OpenAI-compatible endpoint) follows the rule far more reliably.
  156. - The cut-out treats a near-uniform studio background as background; white parts of the product
  157. that touch its silhouette are removed with it.
  158. - With a 4B local LLM, claim flags have false positives/negatives — they are review hints, not a guarantee.
  159. - Karaoke word timings are estimated from word length, not aligned to the audio.
  160. ## Roadmap
  161. - Cloud providers: fal.ai / Replicate (FLUX, Kling, Wan), Modal-hosted Wan/LTX image-to-video, Gemini as LLM + video judge
  162. - Neural background removal (rembg/BiRefNet) for product photos with white parts on the silhouette edge
  163. - Word-level TTS alignment for exact karaoke captions
  164. - A/B variants: N hooks per product, VLM-ranked
  165. - Gradio UI on Hugging Face Spaces

Ratings and comments

Sign in to rate this skill.

Sign in

Free to use. If it’s helped, pass it on.