Skip to content
Back to search

cdaf

CDAF (Cached Descriptive Asset Files) - open sidecar format for video so AI agents stop re-analyzing the same footage. Spec, CLI, agent skill, reproducible benchmark.

Safety scan

A static text scan, not a guarantee. Select the badge to see what was scanned and every finding.

Checked today
Badge for your README
findsafeskills safety rating

Markdown

[![findsafeskills safety rating](/api/listings/uditakhourii--cdaf/badge.svg)](/listings/uditakhourii--cdaf)

HTML

<a href="/listings/uditakhourii--cdaf"><img src="/api/listings/uditakhourii--cdaf/badge.svg" alt="findsafeskills safety rating" /></a>

Install

i

What was scanned

The files the safety scan read, as written by the repository's author and shown as plain text. Lines with a finding are marked.

README.md · README · 231 lines

View this file on GitHub (read at the repository's current version: the scan did not record which commit it read)

  1. # CDAF — Cached Descriptive Asset Files
  2. [![tests](https://github.com/UditAkhourii/cdaf/actions/workflows/tests.yml/badge.svg)](https://github.com/UditAkhourii/cdaf/actions/workflows/tests.yml)
  3. [![npm](https://img.shields.io/npm/v/cdaf-skill.svg)](https://www.npmjs.com/package/cdaf-skill)
  4. [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)
  5. [![Spec: v1.0](https://img.shields.io/badge/spec-v1.0-blue.svg)](SPEC.md)
  6. **Stop making AI agents watch the same video twice.**
  7. CDAF is an open sidecar format for video: a plain-text, timestamped description that
  8. lives next to the video file with the same basename. Generate it once, and every AI
  9. agent that touches that footage afterward reads a few hundred text tokens instead of
  10. running a full video-understanding pass.
  11. ### Teach your agent the format — one command
  12. ```bash
  13. npx cdaf-skill
  14. ```
  15. Works on Windows, macOS, and Linux. That installs the [agent skill](#-agent-skill), so
  16. your coding agent checks for a `.cdaf` sidecar and verifies it against the video's hash
  17. *before* it ever spends tokens watching. Then describe your footage once:
  18. ```bash
  19. pip install "cdaf[generate] @ git+https://github.com/UditAkhourii/cdaf.git#subdirectory=cli"
  20. cdaf generate ./footage
  21. ```
  22. ```
  23. footage/
  24. ├── sunset-drone.mp4 ← plays everywhere, untouched
  25. └── sunset-drone.cdaf ← what agents read instead of watching
  26. ```
  27. ![CDAF turns repeated per-task video analysis into one per-asset description pass followed by verified text reads](figures/concept.svg)
  28. **Measured** (reproducible benchmark, `gemini-2.5-flash`, 20 questions): answering
  29. from the sidecar matched direct video analysis on accuracy — **20/20 vs 19/20** — at
  30. **10.1× fewer prompt tokens per question** (303 vs 3,066) and ~35% lower latency. The
  31. ratio grows linearly with clip length (~50× for 60-second footage). In production use
  32. at scale, this pattern cut video-workflow AI costs to roughly **1/25th**.
  33. ---
  34. ## Components
  35. | Component | Where | What it does |
  36. |---|---|---|
  37. | **[Spec](#-spec)** | [SPEC.md](SPEC.md) | The normative format definition (v1.0) |
  38. | **[Engine](#-engine-core-library)** | [cli/cdaf/](cli/cdaf/) | Python library: parse, validate, hash, generate |
  39. | **[CLI](#-cli)** | [cli/](cli/) | `cdaf` command: generate / validate / read / status |
  40. | **Local provider** | [cli/cdaf/local.py](cli/cdaf/local.py) | `--local`: generate with a local model, no API key |
  41. | **[Agent Skill](#-agent-skill)** | [skills/](skills/claude-code/cdaf/SKILL.md) | Teaches agents the sidecar-first protocol · `npx cdaf-skill` |
  42. | **[Benchmarks](#-benchmarks)** | [benchmarks/](benchmarks/) | Reproducible eval: sidecar vs direct video |
  43. | **[Paper](#-paper)** | [paper/PREPRINT.md](paper/PREPRINT.md) | arXiv preprint draft built on the benchmark |
  44. ---
  45. ## 📄 Spec
  46. A CDAF asset is `clip.mp4` + `clip.cdaf`: a UTF-8 sidecar with a minimal `key: value`
  47. header and a markdown body. The header carries the **SHA-256 and byte size of the
  48. exact video described** — edit the video and every conforming tool detects the sidecar
  49. as stale and refuses to use it. The body is optimized for the actual reader (an LLM):
  50. ```
  51. --- CDAF/1.0
  52. video: sunset-drone.mp4
  53. sha256: 4a7d1ed4…
  54. bytes: 48211394
  55. duration: 00:00:31.500
  56. generator: gemini-2.5-flash
  57. created: 2026-08-26T14:03:12Z
  58. ---
  59. ## Summary
  60. A 31-second aerial drone clip of a coastal highway at golden hour…
  61. ## Segments
  62. [00:00.0-00:05.2] Wide aerial establishing shot: a two-lane coastal highway…
  63. [00:05.2-00:12.8] The drone pushes in and descends toward the highway…
  64. ## Transcript
  65. (no speech)
  66. ## On-screen Text
  67. (none)
  68. ## Tags
  69. drone, aerial, coastal highway, golden hour, sunset, b-roll, …
  70. ```
  71. Full normative rules (freshness semantics, versioning, section grammar):
  72. **[SPEC.md](SPEC.md)**. A complete example:
  73. [examples/sunset-drone.cdaf](examples/sunset-drone.cdaf).
  74. ## ⚙️ Engine (core library)
  75. The `cdaf` Python package is split by dependency weight:
  76. - **Zero-dependency core** ([sidecar.py](cli/cdaf/sidecar.py)) — parse, serialize,
  77. chunked SHA-256, freshness checking, segment extraction. Agents and CI can validate
  78. sidecars with nothing but the standard library.
  79. - **Generator** ([generate.py](cli/cdaf/generate.py)) — Gemini Files API,
  80. bring-your-own-key, `brief`/`standard`/`rich` detail profiles, token-usage capture.
  81. Only imported when you generate.
  82. - **Probe** ([probe.py](cli/cdaf/probe.py)) — optional ffprobe metadata
  83. (duration/resolution/fps), degrades gracefully when ffprobe is absent.
  84. ```python
  85. from cdaf import load, check_freshness, sidecar_path_for
  86. sc = load(sidecar_path_for("footage/sunset-drone.mp4"))
  87. if check_freshness("footage/sunset-drone.mp4", sc) == "fresh":
  88. context_for_llm = sc.body
  89. ```
  90. The format is model-agnostic: the engine's Gemini backend is one implementation, and
  91. Claude/GPT/local-VLM backends slot in behind the same `Sidecar` type.
  92. ## 💻 CLI
  93. Requires Python ≥ 3.10. Bring your own Gemini API key
  94. ([free tier available](https://aistudio.google.com/apikey)) — your footage goes to
  95. your key, not to us.
  96. ```bash
  97. pip install "cdaf[generate] @ git+https://github.com/UditAkhourii/cdaf.git#subdirectory=cli"
  98. export GEMINI_API_KEY=your-key # PowerShell: $env:GEMINI_API_KEY="your-key"
  99. cdaf generate ./footage # describe every video, skip fresh sidecars
  100. cdaf status ./footage # FRESH / STALE / MISSING report
  101. cdaf read ./footage/sunset-drone.mp4 # print the description (verifies hash first)
  102. cdaf validate ./footage/clip.mp4 # exit 0 iff sidecar is well-formed and fresh
  103. ```
  104. Working from a clone instead? `pip install ./cli[generate]`. A PyPI release
  105. (`pip install cdaf`) is on the roadmap.
  106. Flags: `--detail brief|standard|rich`, `--model <gemini-model>`, `--force`.
  107. `validate`/`read`/`status` need **no dependencies and no API key**. Safety lives in
  108. the tool, not just in guidance: `cdaf read` refuses to print a stale sidecar.
  109. ## 🤖 Agent Skill
  110. The skill turns the format into behavior: **before analyzing any video, check for the
  111. sidecar; verify freshness (size check for exploration, full hash for consequential
  112. decisions); read it instead of watching; grep `.cdaf` files to search whole libraries;
  113. regenerate when stale.**
  114. Install it with `npx cdaf-skill` (see [above](#teach-your-agent-the-format--one-command)),
  115. or pick a scope:
  116. | Command | Installs to |
  117. |---|---|
  118. | `npx cdaf-skill` | `~/.claude/skills/cdaf` (all your projects) |
  119. | `npx cdaf-skill --project` | `./.claude/skills/cdaf` (this project only) |
  120. | `npx cdaf-skill --dir <path>` | anywhere you want |
  121. | `npx cdaf-skill --print` | stdout — for pasting into other agent frameworks |
  122. Re-running is safe: an unchanged skill is left alone, and a modified one is backed up
  123. to `SKILL.md.bak` before updating. Prefer to copy it by hand? The file is
  124. [skills/claude-code/cdaf/SKILL.md](skills/claude-code/cdaf/SKILL.md).
  125. Any other agent framework — the whole contract is one system-prompt paragraph:
  126. > Before analyzing a video file, look for a `.cdaf` file with the same basename. If
  127. > its `bytes`/`sha256` header matches the video file, read it instead of processing
  128. > the video.
  129. ### Why agentic video editors care
  130. Programmatic and agent-native editors (Remotion, HyperFrames, prompt-to-edit tools)
  131. already do everything in text — compositions are code, cut decisions are data. Video
  132. understanding is the one step that forces them out of the text domain, priced per
  133. exposure (~263 tokens per second of footage on Gemini-class models). With CDAF:
  134. - **Selection becomes retrieval** — "sunset coastal shots, no people" is a grep over
  135. `.cdaf` files, not a multimodal sweep of the library.
  136. - **Cut lists come from segment timestamps** — `[00:05.2-00:12.8]` lines map straight
  137. to Remotion `<Sequence>`s or HyperFrames clips.
  138. - **Captions and audio sync for free** — the Transcript section aligns speech to the
  139. timeline.
  140. - **The library appreciates** — 40 candidate clips ≈ 473k tokens to watch, ~24k to
  141. read — every session after the first.
  142. ## 📊 Benchmarks
  143. Fully reproducible: [bench.py](benchmarks/bench.py) synthesizes test videos with
  144. ffmpeg from scripted recipes (known colors, hard cuts, timed word overlays), so ground
  145. truth is exact and grading is objective — no LLM judge, no dataset license.
  146. ```bash
  147. python benchmarks/bench.py make # synthesize testset (needs ffmpeg)
  148. python benchmarks/bench.py run # both conditions via Gemini (needs GEMINI_API_KEY)
  149. python benchmarks/bench.py report # writes benchmarks/RESULTS.md
  150. ```
  151. ![Accuracy, prompt-token cost, and latency for 20 questions in each condition](figures/benchmark.svg)
  152. | Condition | Accuracy | Mean prompt tokens/question | Mean latency |
  153. |---|---|---|---|
  154. | Direct video | 19/20 (95%) | 3,066 | 3.46 s |
  155. | **CDAF sidecar** | **20/20 (100%)** | **303** | **2.24 s** |
  156. Each sidecar-answered question saves 2,763 prompt tokens, so generating a sidecar
  157. (3,601 tokens) **breaks even after ≈1.3 questions per video** — everything after that
  158. is the ~10× saving. Per-clip and per-question detail:
  159. [benchmarks/RESULTS.md](benchmarks/RESULTS.md).
  160. ## 📝 Paper
  161. **[https://zenodo.org/records/22110594](https://zenodo.org/records/22110594)** — *Cached Descriptive Asset Files (CDAF):
  162. A Sidecar Format for Token-Efficient Video Understanding in Agentic Pipelines* —
  163. arXiv draft (cs.MM / cs.AI): format rationale, benchmark methodology and results,
  164. production case study, integration analysis for agentic editors, limitations.
  165. ## Roadmap
  166. - PyPI release of the `cdaf` CLI (`pip install cdaf`)
  167. - MCP server exposing `cdaf_read` / `cdaf_status` / `cdaf_generate` to any MCP client
  168. - Hosted free converter (no local install; bring-your-own-key or free quota)
  169. - Additional generator backends (Claude, GPT, local VLMs)
  170. - Node/TypeScript port of the engine
  171. - Natural-footage benchmark extension with human-verified QA
  172. - Signed sidecars (`signature` header) for cross-organization trust
  173. ## License
  174. MIT — see [LICENSE](LICENSE).

Ratings and comments

Sign in to rate this skill.

Sign in

Free to use. If it’s helped, pass it on.