Back to search
adforge
MCP serverGitHub
Product page → 9:16 video ad. Agentic pipeline with a self-critiquing VLM loop and an MCP server.
Safety scan
A static text scan, not a guarantee. Select the badge to see what was scanned and every finding.
Checked today
Badge for your README
Markdown
[](/listings/coobeliues--adforge)HTML
<a href="/listings/coobeliues--adforge"><img src="/api/listings/coobeliues--adforge/badge.svg" alt="findsafeskills safety rating" /></a>Install
i
Install instructions from the README
As written by the repo's author. findsafeskills has not verified these steps.
git clone <this repo> && cd adforge python -m venv .venv && source .venv/bin/activate pip install -e ".[tts,dev]" # core + local TTS pip install -e ".[gpu]" # optional: local image generation (torch + diffusers) ollama pull qwen3.5:4b # LLM + vision judge in one 3 GB model cp .env.example .env # pick providers adforge doctor # checks ollama, model, ffmpeg, fonts, GPU
Make an ad:
# from a product page adforge make "https://…/product" --lang en --duration 15 # or manually adforge make bottle --name "HydroPeak 750" --desc "Keeps drinks cold 24h, leak-proof" \ --image photo.jpg --price "12 990" --currency KZT --lang ru --music track.mp3 # plan first, review, tweak, render adforge make "https://…" --plan-only adforge edit <run_id> 1 --caption "Still drinking warm water?" adforge render <run_id>
Talk to the agent:
adforge chat you › make a 15s ad in Russian for https://… aimed at students you › shot 2 feels generic — make it a rainy bus stop at night and re-render
What was scanned
The files the safety scan read, as written by the repository's author and shown as plain text. Lines with a finding are marked.
README.md · README · 204 lines
- # AdForge
- **Product page in → 9:16 video ad out.** An agentic pipeline that reads a product page, writes a
- creative brief and storyboard, generates keyframes, **reviews its own frames with a vision model and
- rewrites prompts until they pass**, then animates, voices, captions and cuts a TikTok/Reels-ready ad.
- Runs fully local on a laptop (Ollama + a 6 GB GPU), every capability is a swappable provider, and the
- whole thing is drivable by **Claude Code / Claude Desktop over MCP** or by its own from-scratch agent.
- ```bash
- adforge make https://shop.example.com/product/123 --lang ru --duration 15
- ```
- <p align="center">
- <img src="docs/demo.gif" width="240" alt="Generated ad">
-
- <img src="docs/storyboard.png" width="560" alt="Storyboard keyframes">
- </p>
- <p align="center"><sub>A 15 s Russian-language ad generated fully locally (qwen3.5:4b + SDXL-Turbo + Piper on an
- RTX 4050 laptop) from a product name, a description and one product photo. Left: final cut. Right: the
- storyboard's keyframes — real product photo shots (hero, close-up, CTA) and AI-generated scenes.</sub></p>
- ---
- ## Why it's interesting
- | | |
- |---|---|
- | **Self-critiquing quality loop** | Each generated keyframe goes through `generate → judge → revise`. Cheap pixel heuristics (blur, exposure, contrast) gate first, then a VLM scores prompt adherence, artifacts and ad suitability against a rubric. Failing frames get their prompt rewritten by the LLM *from the judge's feedback* and are regenerated; the best attempt wins. Every attempt, score and revision is visible in the run report. |
- | **MCP server** | 8 tools (`plan_ad`, `edit_shot`, `regenerate_shot`, `render_ad`, `storyboard_sheet`, …) with progress notifications. Claude can plan an ad, *look at* the storyboard image, give a director's note on shot 3 and re-render only what changed. |
- | **Agent loop from scratch** | `adforge chat` — a ~100-line tool-calling loop (no framework): JSON-schema tool specs generated from Python signatures, tool errors fed back to the model for self-correction, context compaction of old tool outputs. Works with a 4B local model. |
- | **Provider abstraction** | Image / video / TTS / judge / LLM are interfaces. Today: Ollama, any OpenAI-compatible endpoint (vLLM, Groq, OpenRouter, Gemini…), diffusers on a local GPU, Piper TTS. Cloud video models drop in as another provider. |
- | **Resilient cloud client** | Cloud LLMs get overloaded or hang. The OpenAI-compatible client walks a model fallback chain on 429/5xx/timeouts and sticks to the model that answered; permanent errors fail fast. Provider-specific tool-call fields (Gemini 3's `thought_signature`) are round-tripped verbatim, which tool calling requires. |
- | **Structured output that survives small models** | Pydantic schemas → Ollama `format` / `json_schema`; lenient coercion of sloppy values; on validation failure the error is sent back and the model fixes its own JSON. |
- | **Real product, not hallucinated** | The product is only ever shown from its real photos (enforced for the reveal and CTA shots); generated scenes show people and situations. Photos are cut out from studio backgrounds (corner-detected, border-connected morphological reconstruction — keeps white details *inside* the product, works on tightly cropped shop photos) and staged on a brand-colored set. A single photo yields several distinct shots: hero, punch-in close-ups on different parts, CTA layout. |
- | **Claims review** | A compliance pass flags voiceover/caption claims that the product data doesn't support (invented numbers, comparisons, awards). Flags are surfaced in the CLI, the report and `get_run` for a human or Claude to resolve with `edit_shot` — never silently rewritten, because a small model spots suspicious lines far more reliably than it rewrites them. |
- | **Observability** | Each run is a directory: `state.json` (resumable), `trace.jsonl` (every LLM call with latency + tokens, every judge verdict and revision), `report.html`, `storyboard.png`. |
- | **Steps are idempotent** | Edit one shot → only its keyframe/clip is re-rendered. |
- | **Fits a 6 GB GPU / 16 GB laptop** | The LLM and SDXL can't share 6 GB of VRAM, so they time-share it: the LLM is unloaded before each image, and SDXL runs in three short phases (text encoders → UNet → VAE), each loaded straight from safetensors into VRAM and freed right after. Weights never sit in system RAM — a full run stays under a 6 GB RAM cgroup cap (vs. >8.5 GB with diffusers' standard CPU offload, which OOM-killed the laptop). |
- ## Architecture
- ```mermaid
- flowchart LR
- URL[Product URL / manual input] --> ING[Ingest<br/>JSON-LD · OpenGraph · photos]
- ING --> BRIEF[Creative brief<br/>LLM sees product photo]
- BRIEF --> SB[Storyboard<br/>hook → … → CTA<br/>+ claims review]
- SB --> VO[Voiceover<br/>Piper TTS]
- SB --> KF{Keyframes}
- KF -- product_photo --> HERO[Cut-out + brand set]
- KF -- generated --> GEN[Image provider]
- GEN --> J[Judge<br/>heuristics → VLM rubric]
- J -- fail --> REV[LLM rewrites prompt<br/>from judge feedback] --> GEN
- J -- pass / best of N --> ANIM
- HERO --> ANIM[Animate<br/>Ken Burns motion]
- VO --> CUT
- ANIM --> CUT[Final cut<br/>crossfades · karaoke captions<br/>CTA end card · ducked music]
- CUT --> OUT[final.mp4 + report.html]
- subgraph Drivers
- CLI[CLI]
- MCP[MCP server<br/>Claude Code / Desktop]
- AG[Own agent loop]
- end
- Drivers -.-> ING & SB & KF & CUT
- ```
- ```
- src/adforge/
- pipeline.py orchestrator; each stage is a resumable method on a Run
- creative.py brief / storyboard / prompt-revision LLM steps + invariants
- llm/ provider-agnostic chat: tools, vision, structured output w/ self-repair
- providers/ image (diffusers, placeholder) · video (Ken Burns) · tts (Piper, silent) · judge
- render/ cut-out & hero staging · captions & end card · audio mix · final assembly
- tools.py the tool surface shared by MCP and the agent (+ schema from signatures)
- agent.py from-scratch tool-calling loop
- mcp_server.py MCP server with progress notifications
- report.py HTML report + storyboard contact sheet
- ```
- ## Quickstart
- ```bash
- git clone <this repo> && cd adforge
- python -m venv .venv && source .venv/bin/activate
- pip install -e ".[tts,dev]" # core + local TTS
- pip install -e ".[gpu]" # optional: local image generation (torch + diffusers)
- ollama pull qwen3.5:4b # LLM + vision judge in one 3 GB model
- cp .env.example .env # pick providers
- adforge doctor # checks ollama, model, ffmpeg, fonts, GPU
- ```
- Make an ad:
- ```bash
- # from a product page
- adforge make "https://…/product" --lang en --duration 15
- # or manually
- adforge make bottle --name "HydroPeak 750" --desc "Keeps drinks cold 24h, leak-proof" \
- --image photo.jpg --price "12 990" --currency KZT --lang ru --music track.mp3
- # plan first, review, tweak, render
- adforge make "https://…" --plan-only
- adforge edit <run_id> 1 --caption "Still drinking warm water?"
- adforge render <run_id>
- ```
- Talk to the agent:
- ```bash
- adforge chat
- you › make a 15s ad in Russian for https://… aimed at students
- you › shot 2 feels generic — make it a rainy bus stop at night and re-render
- ```
- ## Use it from Claude Code / Claude Desktop (MCP)
- Opening this repo in Claude Code picks up the bundled [`.mcp.json`](.mcp.json) automatically. Elsewhere:
- ```bash
- claude mcp add adforge -- /abs/path/to/.venv/bin/adforge mcp
- ```
- Claude Desktop (`claude_desktop_config.json`):
- ```json
- {
- "mcpServers": {
- "adforge": {
- "command": "/abs/path/to/.venv/bin/adforge",
- "args": ["mcp"],
- "env": { "ADFORGE_RUNS_DIR": "/abs/path/to/runs" }
- }
- }
- }
- ```
- Streamable HTTP: `adforge mcp --http --port 8765`.
- ## Configuration
- All settings are env vars (`ADFORGE_*`) or `.env` — see [.env.example](.env.example).
- | Setting | Options | Default |
- |---|---|---|
- | `LLM_PROVIDER` / `LLM_MODEL` | `ollama`, `openai` (any OpenAI-compatible URL via `LLM_BASE_URL`) | `ollama` / `qwen3.5:4b` |
- | `LLM_FALLBACK_MODELS` | comma-separated models tried when the main one is overloaded | — |
- | `VISION_MODEL` | separate judge model | same as LLM |
- | `IMAGE_PROVIDER` | `diffusers` (local GPU), `placeholder` | `placeholder` |
- | `I2V_PROVIDER` | image-to-video model for *generated* scenes only (real product shots always stay on `VIDEO_PROVIDER`) | `none` |
- | `DIFFUSERS_MODEL` | `stabilityai/sdxl-turbo` (~6.5 GB, best), `stabilityai/sd-turbo` (~2.6 GB, for 16 GB RAM machines) | `stabilityai/sdxl-turbo` |
- | `TTS_PROVIDER` | `piper` (en/ru/kk voices), `silent` | `silent` |
- | `JUDGE` | `vlm`, `heuristic`, `none` | `vlm` |
- | `MAX_ATTEMPTS` / `PASS_SCORE` | quality loop budget / threshold | `3` / `7.0` |
- ## Performance (laptop: Ryzen 5 8645HS, RTX 4050 6 GB, 16 GB RAM)
- | Stage | Time |
- |---|---|
- | Brief + storyboard (qwen3.5:4b, local) | ~30 s |
- | SDXL-Turbo keyframe 576×1024 → 1080×1920 (4.9 GB peak VRAM) | ~7–10 s |
- | VLM judge, per frame | ~4 s |
- | Animate 5 shots, 1080×1920@30 (parallel) | ~8 s |
- | Final cut with captions + audio | ~15 s |
- | **Full 15 s ad, 5 shots, quality loop on** | **~2–3 min** |
- ## Tests
- ```bash
- pytest # unit tests + full pipeline e2e with a scripted LLM (no network, no GPU)
- ```
- Covers JSON repair, schema coercion, storyboard invariants, JSON-LD/OpenGraph parsing, price
- formatting, cut-out (inner whites, tightly cropped photos, busy backgrounds), caption timing and
- contrast, crossfade timeline, the RAM guard, the agent's error recovery, and the full pipeline:
- `generate → judge(fail) → revise → judge(pass)`, claim flags, resume without regeneration,
- partial re-render after edits (a longer voiceover re-voices and re-renders only that shot),
- shot-index validation.
- ```bash
- ruff check src tests && ruff format --check src tests
- ```
- ## Known limitations
- - A text-to-image model can't reproduce a specific product, so the real product appears only in
- `product_photo` shots (enforced for the solution and CTA shots); generated shots are meant to show
- people and situations. A small local LLM sometimes still writes product-centric prompts for
- generated shots — the judge catches and scores them down, but a stronger LLM (any
- OpenAI-compatible endpoint) follows the rule far more reliably.
- - The cut-out treats a near-uniform studio background as background; white parts of the product
- that touch its silhouette are removed with it.
- - With a 4B local LLM, claim flags have false positives/negatives — they are review hints, not a guarantee.
- - Karaoke word timings are estimated from word length, not aligned to the audio.
- ## Roadmap
- - Cloud providers: fal.ai / Replicate (FLUX, Kling, Wan), Modal-hosted Wan/LTX image-to-video, Gemini as LLM + video judge
- - Neural background removal (rembg/BiRefNet) for product photos with white parts on the silhouette edge
- - Word-level TTS alignment for exact karaoke captions
- - A/B variants: N hooks per product, VLM-ranked
- - Gradio UI on Hugging Face Spaces
Ratings and comments
Sign in to rate this skill.
Sign inFree to use. If it’s helped, pass it on.