Skip to content
Back to search

AutoDesign

SkillWorks with CodexGitHub

The open-source design agent and harness, better than Claude Design on academic communication artifacts production. This DesignHarness can also be used with any coding harness you like ( Codex/Claude Code/Kimi Code/Pi/OpenCode etc..) and any agentic model you want.

Safety scan

A static text scan, not a guarantee. Select the badge to see what was scanned and every finding.

Checked today

Check it deeper on your machine

Its scripts were only partly scanned.

As a Claude Code plugin, install it and then ask Claude Code:

/plugin marketplace add keithmackay/mackayi/plugin install scanskillsafety@mackayiDo a deep scan of https://github.com/Yaxin9Luo/AutoDesign with scanskillsafety

Or from a terminal, clone it and run it:

git clone https://github.com/keithmackay/scanskillsafety-skill ~/.claude/skills/scanskillsafetynode ~/.claude/skills/scanskillsafety/cli.cjs --deep 'https://github.com/Yaxin9Luo/AutoDesign'

It checks the exact commit: every file, its packages against OSV (api.osv.dev), how it compares with this page's scan (findsafeskills.com receives the repository's address), and, if you agree, Claude reading every file with your own Claude subscription. It never runs the code.

Badge for your README
findsafeskills safety rating

Markdown

[![findsafeskills safety rating](/api/listings/yaxin9luo--autodesign/badge.svg)](/listings/yaxin9luo--autodesign)

HTML

<a href="/listings/yaxin9luo--autodesign"><img src="/api/listings/yaxin9luo--autodesign/badge.svg" alt="findsafeskills safety rating" /></a>

Install

i

Install instructions from the README

As written by the repo's author. findsafeskills has not verified these steps.

uv sync
uv run python scripts/install_playwright_browsers.py
cd runtime/video && npm ci --omit=dev
cd ../../web && npm install

Configure provider keys in `.env` or enter them in the Web UI Settings drawer. Do not replace an existing `.env` during an update.

What was scanned

The files the safety scan read, as written by the repository's author and shown as plain text. Lines with a finding are marked.

README.md · README · 580 lines · 1 finding marked

View this file on GitHub (read at the repository's current version: the scan did not record which commit it read)

  1. <p align="center">
  2. <strong>English</strong> ·
  3. <a href="./README.zh-CN.md">简体中文</a> ·
  4. <a href="./README.ko.md">한국어</a>
  5. </p>
  6. ### News
  7. | Date | Update |
  8. | :--- | :--- |
  9. | **2026-08-19** | [Poster canvas controls: prompt-first templates, aspect ratios, and exact pixel sizes, plus academic presets in the Web UI](https://github.com/Yaxin9Luo/AutoDesign/compare/51dc4212c86f850bb7915d0e1e7096c1a29cd040...55b18da6bff0e541e6d204c525b0f9d715fc17da) |
  10. | **2026-08-19** | [Agent Skills v0.2.0: Poster Agent-first PDF ingestion and white primary canvases across all four Skills](./agent_skills/README.md#agent-skills-v0-2-0) |
  11. | **2026-08-18** | [Poster Skill Agent-first v2: direct PDF curation, revision-bound attempts, and read-only DOM QA](./agent_skills/README.md#poster-agent-first-v2) |
  12. | **2026-08-17** | [Standalone Agent Skills for Poster, PPT, Webpage, and Video are now installable](./agent_skills/README.md) |
  13. | **2026-08-15** | [Added official DeepSeek Harness support for coding agents](https://github.com/Yaxin9Luo/AutoDesign/pull/2) |
  14. | **2026-08-14** | [Initial public release](https://github.com/Yaxin9Luo/AutoDesign/commit/55586f66fa4a126997f0d252e070701c4ae68920) |
  15. <p align="center">
  16. <img src="./assets/readme/hero-research-product.webp" width="100%" alt="AutoDesign improves the harness around a fixed model and ships editable posters, slides, webpages, and videos">
  17. </p>
  18. <h1 align="center">AutoDesign: Meta-Harness Optimization for Long-Horizon Agentic Design</h1>
  19. <p align="center">
  20. Learn a reusable DesignHarness around fixed models, then turn one paper into an editable poster, slides, webpage, and narrated + captioned video.
  21. </p>
  22. <p align="center">
  23. <a href="https://arxiv.org/abs/2608.13560"><kbd>Paper · arXiv:2608.13560 ↗</kbd></a>
  24. &nbsp;&nbsp;·&nbsp;&nbsp;
  25. <a href="https://huggingface.co/datasets/YaxinLuo/PosterBench"><kbd>Dataset · PosterBench ↗</kbd></a>
  26. &nbsp;&nbsp;·&nbsp;&nbsp;
  27. <a href="https://huggingface.co/datasets/YaxinLuo/PosterBench-mini"><kbd>Dataset · PosterBench-mini ↗</kbd></a>
  28. </p>
  29. <p align="center">
  30. <a href="https://autodesign.designanything.ai/"><strong>✦ Explore the AutoDesign story ↗</strong></a>
  31. &nbsp;&nbsp;·&nbsp;&nbsp;
  32. <a href="https://designanything.ai/"><strong>Open the Demo Page ↗</strong></a>
  33. </p>
  34. <p align="center">
  35. <a href="#user-content-demos"><strong>Demos</strong></a> ·
  36. <a href="#user-content-agent-skills">Agent Skills</a> ·
  37. <a href="#user-content-quickstart">Quickstart</a> ·
  38. <a href="#user-content-paper-suite">Paper Suite</a> ·
  39. <a href="#user-content-methodology">Methodology</a> ·
  40. <a href="#user-content-benchmark">PosterBench</a> ·
  41. <a href="#user-content-human-evaluation">Human Evaluation</a> ·
  42. <a href="#user-content-interfaces">Outputs</a> ·
  43. <a href="#user-content-citation">Citation</a>
  44. </p>
  45. <a id="demos"></a>
  46. ## <img src="./assets/readme/icons/gallery.svg" width="26" alt="" align="absmiddle"> AutoDesign for AutoDesign · One Paper → Four Artifacts
  47. These are real outputs, not mockups. AutoDesign turned its own paper into the
  48. paper's Figure 2 poster, a 24-slide formal academic talk, a complete editorial
  49. research webpage, and a six-minute 1080p conference video.
  50. <table width="100%">
  51. <tr>
  52. <td width="50%" valign="top">
  53. <a href="./assets/readme/demo/artifacts/autodesign-for-autodesign-poster.pdf"><img src="./assets/readme/demo/poster-autodesign.webp" width="100%" alt="AutoDesign for AutoDesign poster from Figure 2 of the paper"></a><br>
  54. <strong>Poster · AutoDesign</strong><br>
  55. Figure 2 of the paper: an information-dense, editable academic poster made by AutoDesign for itself.<br>
  56. <a href="./assets/readme/demo/artifacts/autodesign-for-autodesign-poster.pdf"><strong>Open full poster PDF ↗</strong></a>
  57. </td>
  58. <td width="50%" valign="top">
  59. <a href="./assets/readme/demo/artifacts/autodesign-slides-formal-academic.pdf"><img src="./assets/readme/demo/slides-autodesign-formal-academic.webp" width="100%" alt="Selected slides from the 24-slide formal AutoDesign academic conference talk"></a><br>
  60. <strong>Slides · AutoDesign</strong><br>
  61. A complete 24-slide formal academic conference talk made by AutoDesign for itself.<br>
  62. <a href="./assets/readme/demo/artifacts/autodesign-slides-formal-academic.pdf"><strong>Open full slide deck PDF ↗</strong></a>
  63. </td>
  64. </tr>
  65. <tr>
  66. <td width="50%" valign="top">
  67. <a href="./assets/readme/demo/artifacts/autodesign-landing-page.html"><img src="./assets/readme/demo/webpage-autodesign.webp" width="100%" alt="Editorial AutoDesign research webpage generated from the AutoDesign paper"></a><br>
  68. <strong>Webpage · AutoDesign</strong><br>
  69. An editorial research experience that turns the paper's method, evidence, results, and limitations into an interactive story.<br>
  70. <a href="./assets/readme/demo/artifacts/autodesign-landing-page.html"><strong>Download full landing page ↗</strong></a>
  71. </td>
  72. <td width="50%" valign="top">
  73. <a href="./assets/readme/demo/artifacts/autodesign-conference-video-6min.mp4"><img src="./assets/readme/demo/video-autodesign-conference.webp" width="100%" alt="Contact sheet from the six-minute AutoDesign conference video"></a><br>
  74. <strong>Video · AutoDesign</strong><br>
  75. A six-minute 1080p conference video introducing Meta-Harness Optimization, DesignHarness, and PosterBench.<br>
  76. <a href="./assets/readme/demo/artifacts/autodesign-conference-video-6min.mp4"><strong>Watch MP4 ↗</strong></a>
  77. </td>
  78. </tr>
  79. </table>
  80. <a id="agent-skills"></a>
  81. ## <img src="./assets/readme/icons/blocks.svg" width="26" alt="" align="absmiddle"> Agent Skills · Use AutoDesign without the server
  82. Install the standalone Poster, PPT, Webpage, or Video Skill directly in Codex,
  83. Claude Code, or DeepSeek Harness. Each Skill carries its own lightweight local
  84. harness and keeps editable artifacts, evidence, attempts, and review state in
  85. your chosen output directory—no AutoDesign application server required.
  86. Poster is the first Skill to receive the new Agent-first v2 workflow: the host
  87. agent can inspect the paper PDF, request deterministic crops, review the source
  88. catalog, and repair the correct workflow stage while a read-only browser audit
  89. checks both screen and print output. [See what changed →](./agent_skills/README.md#poster-agent-first-v2)
  90. [Open the Agent Skills install guide →](./agent_skills/README.md)
  91. <a id="quickstart"></a>
  92. ## <img src="./assets/readme/icons/bolt.svg" width="26" alt="" align="absmiddle"> Start locally
  93. ### <img src="./assets/readme/icons/terminal.svg" width="20" alt="" align="absmiddle"> One-command local launch
  94. Prerequisites: Node.js 22+ and `ffmpeg`/`ffprobe`.
  95. ```bash
  96. curl -fsSL https://designanything.ai/install.sh | bash
    • Note: Pipes a downloaded script directly into a shell (curl | sh / wget | bash)
    • Line 118 on GitHub
  97. autodesign start
  98. ```
  99. The launcher installs under `~/.local/share/autodesign`, keeps state under
  100. `~/.autodesign`, serves the bundled `web/dist`, and opens a browser. Run
  101. `autodesign doctor` to check the installed runtime. Existing
  102. `~/.designanything` state is migrated with a compatibility symlink. If the
  103. hosted endpoint is unavailable, use the source setup below.
  104. ### <img src="./assets/readme/icons/code.svg" width="20" alt="" align="absmiddle"> Build from source
  105. Requirements: Python 3.10+, [`uv`](https://docs.astral.sh/uv/), Node.js 22+,
  106. npm, and `ffmpeg`/`ffprobe` for Video.
  107. #### 1. Install
  108. ```bash
  109. uv sync
  110. uv run python scripts/install_playwright_browsers.py
  111. cd runtime/video && npm ci --omit=dev
  112. cd ../../web && npm install
  113. ```
  114. Configure provider keys in `.env` or enter them in the Web UI Settings drawer.
  115. Do not replace an existing `.env` during an update.
  116. #### 2. Launch the workbench
  117. Start the backend:
  118. ```bash
  119. uv run uvicorn scripts.web_server:app --reload --port 8000
  120. ```
  121. In another terminal, start the frontend:
  122. ```bash
  123. cd web
  124. npm run dev
  125. ```
  126. Open [localhost:5173](http://localhost:5173). Backend health is available at
  127. [`/api/health`](http://127.0.0.1:8000/api/health).
  128. Upload one PDF and choose **Paper All-in-One** to launch the poster, slides,
  129. webpage, and narrated-video tracks together.
  130. #### 3. Generate a paper poster
  131. ```bash
  132. uv --cache-dir .uv-cache run python -m autodesign run \
  133. "Create a dense academic conference poster from the attached paper." \
  134. --from-file /absolute/path/to/paper.pdf \
  135. --template cvpr-landscape
  136. ```
  137. Inspect `final/poster.html`, `final/preview.png`, the final manifest, and
  138. `run_events.jsonl`. A file can exist after a fallback, so terminal status and
  139. validation feedback remain part of the result.
  140. <details>
  141. <summary><strong>Use a visual reference</strong></summary>
  142. ```bash
  143. uv --cache-dir .uv-cache run python -m autodesign run \
  144. "Create a paper poster using the reference's visual system." \
  145. --from-file /absolute/path/to/paper.pdf \
  146. --reference-poster /absolute/path/to/reference.png
  147. ```
  148. Reference posters transfer visual systems only. Their text, claims, logos, QR
  149. codes, figures, tables, and links never become paper evidence.
  150. </details>
  151. <a id="paper-suite"></a>
  152. ## <img src="./assets/readme/icons/file-output.svg" width="26" alt="" align="absmiddle"> One paper. Every artifact you need next.
  153. <p align="center">
  154. <img src="./assets/readme/paper-suite.svg" width="100%" alt="A paper PDF flows through AutoDesign into a webpage, slides, poster, and narrated video">
  155. </p>
  156. Finish the paper once. **Paper All-in-One** packages the same source into
  157. everything that usually comes next: a promotional webpage, conference slides,
  158. an academic poster, and a narrated video with timed subtitles. No need to
  159. rebuild the paper's story for each format.
  160. <p align="center">
  161. <a href="https://designanything.ai/"><strong>Generate the complete paper suite ↗</strong></a>
  162. </p>
  163. ## <img src="./assets/readme/icons/gallery.svg" width="26" alt="" align="absmiddle"> Watch AutoDesign in action
  164. Follow a guided local walkthrough: configure the Workbench, launch Paper
  165. All-in-One, inspect the run, and enter each editable canvas. You can also try
  166. the [online demo](https://designanything.ai/) in your browser; for the complete,
  167. most reliable experience, we recommend installing AutoDesign locally.
  168. <p align="center">
  169. <strong>Guided local walkthrough · Paper All-in-One → editable canvases</strong>
  170. </p>
  171. https://github.com/user-attachments/assets/69c25973-fedf-4273-aa33-6bd3e409c692
  172. <details>
  173. <summary><strong>Open the academic poster wall</strong></summary>
  174. <br>
  175. <p align="center"><strong>Claude 4.8 authoring route</strong></p>
  176. <p align="center">
  177. <img src="./assets/readme/demo/poster-longcat-next-claude.webp" width="32%" alt="LongCat-Next academic poster generated through the Claude 4.8 authoring route">
  178. <img src="./assets/readme/demo/poster-underwater-sam-claude.webp" width="32%" alt="Underwater SAM academic poster generated through the Claude 4.8 authoring route">
  179. <img src="./assets/readme/demo/poster-m87-claude.webp" width="32%" alt="M87 Event Horizon Telescope academic poster generated through the Claude 4.8 authoring route">
  180. </p>
  181. <p align="center"><strong>Codex GPT-5.5 xhigh authoring route</strong></p>
  182. <p align="center">
  183. <img src="./assets/readme/demo/poster-ddpm-codex.webp" width="32%" alt="Denoising diffusion probabilistic models poster generated through the Codex GPT-5.5 xhigh authoring route">
  184. <img src="./assets/readme/demo/poster-lung-adenocarcinoma-codex.webp" width="32%" alt="Lung adenocarcinoma poster generated through the Codex GPT-5.5 xhigh authoring route">
  185. <img src="./assets/readme/demo/poster-economic-complexity-codex.webp" width="32%" alt="Economic complexity poster generated through the Codex GPT-5.5 xhigh authoring route">
  186. </p>
  187. </details>
  188. ## <img src="./assets/readme/icons/sparkles.svg" width="26" alt="" align="absmiddle"> Why AutoDesign
  189. - **The whole paper journey, in one workflow.** Build the promotional webpage,
  190. talk deck, conference poster, and narrated + captioned video from the same
  191. source instead of restarting four times.
  192. - **Editable by default.** HTML, native text, tables, and named assets remain
  193. available for revision instead of being flattened into one image.
  194. - **Source-grounded.** Claims, figures, and tables retain provenance beside the
  195. run; a reference can transfer style, never evidence.
  196. - **Optimizes the system, not model weights.** Complete trajectories expose
  197. recurring failures, while meta-harness optimization improves one reusable
  198. DesignHarness component at a time.
  199. - **Inspectable and local-first.** Events, manifests, candidates, validation
  200. feedback, and final files stay available on your machine.
  201. <a id="methodology"></a>
  202. ## <img src="./assets/readme/icons/route.svg" width="26" alt="" align="absmiddle"> Method: meta-harness optimization
  203. A **design harness** is the system around a fixed LLM or MLLM that turns a
  204. multimodal source into a human-facing artifact through an execution trajectory.
  205. A **meta-harness** improves that surrounding system. AutoDesign therefore learns
  206. from complete rollouts while keeping the underlying model weights fixed. Before
  207. autonomous optimization, an evaluator coding agent uses human-annotated
  208. reference artifacts across seven quality dimensions to implement a fixed
  209. optimization-time evaluator. It combines rule-based checks with VLM judgments
  210. and remains distinct from the frozen PosterBench protocol used for final system
  211. comparison.
  212. <p align="center">
  213. <img src="./assets/readme/research/research-overview.webp" width="100%" alt="Latest paper Figure 1: meta-harness optimization trajectory and DesignHarness gains across seven fixed configurations">
  214. </p>
  215. <p align="center">
  216. <img src="./assets/readme/research/designharness-evolution.webp" width="100%" alt="Three stages of autonomous DesignHarness evolution followed by human-in-the-loop refinement">
  217. </p>
  218. Autonomous outer-loop iterations evolve the harness through rollout,
  219. evaluation, one-component update proposals, and acceptance. After autonomous
  220. optimization reaches a plateau, optional Human-in-the-loop guidance can
  221. redirect the search and further improve production poster quality.
  222. ### <img src="./assets/readme/icons/repeat.svg" width="20" alt="" align="absmiddle"> Two nested feedback loops
  223. | Loop | What it improves | Evidence and update |
  224. |---|---|---|
  225. | **Inner loop · artifact generation** | One editable artifact under a fixed design harness | A **Designer** revises the artifact; a **Critic** returns feedback; their interactions form an execution trajectory |
  226. | **Outer loop · harness optimization** | The reusable design harness across tasks | The **MetaHarnessOptimizer** analyzes trajectories, evaluator scores, the persistent optimization record, and optional human guidance |
  227. Every outer-loop iteration follows four stages: **rollout → evaluation → update
  228. proposal → acceptance**. The optimizer acts as a planner and code editor,
  229. updates exactly one harness component, and retains the candidate only when
  230. training performance improves without reducing performance on an independent
  231. development set. Development trajectories are hidden from the update proposer.
  232. <p align="center">
  233. <img src="./assets/readme/research/meta-harness-overview.webp" width="100%" alt="AutoDesign meta-harness method with rollout evidence, five harness components, optimizer roles, optional human guidance, and a train-development acceptance gate">
  234. </p>
  235. Human-in-the-loop guidance is optional. A user can give the planner
  236. observations or high-level directions to redirect a stalled search; explicit
  237. human input can also correct a systematic evaluator bias. Without guidance,
  238. the outer loop runs autonomously.
  239. ### <img src="./assets/readme/icons/blocks.svg" width="20" alt="" align="absmiddle"> Five design-harness components
  240. | Component | Elements optimized by the meta-harness |
  241. |---|---|
  242. | **Context and Memory** | Multimodal source management, task prompts, skills, reusable assets, and persistent revision state |
  243. | **Tools and Specifications** | Tools and editable-artifact specifications for layout, typography, and provenance |
  244. | **Execution Runtime** | The workspace and runtime for authoring, rendering, validating, and exporting |
  245. | **Orchestration** | Task routing, attempt budgets, loop control, candidate selection, fallback, and finalization |
  246. | **Evaluation and Feedback** | Rule-based validation, model-based critique, and localized revision feedback |
  247. ### <img src="./assets/readme/icons/gear.svg" width="20" alt="" align="absmiddle"> The optimized DesignHarness
  248. Meta-harness optimization yields **DesignHarness**, the reusable
  249. artifact-producing system. Its four stages are **source ingestion**, **iterative
  250. artifact generation and revision**, **validation with dual critics**, and
  251. **finalization**. Paper metadata, claims, figures, tables, and source locations
  252. become provenance-aware context; a coding-agent Designer edits native HTML; a
  253. rule-based validator and VLM critic return localized feedback; and the best
  254. valid candidate is made self-contained for delivery.
  255. The current implementation permits up to 12 refinement attempts. Blocking
  256. checks cover unsafe or missing assets, broken provenance, severe overflow or
  257. overlap, and required typography or layout constraints. If no candidate passes
  258. within the budget, the retained attempt history supports a constrained fallback
  259. before the same finalization stage.
  260. <p align="center">
  261. <img src="./assets/readme/research/poster-harness.webp" width="100%" alt="DesignHarness stages from provenance-aware paper ingestion through editable generation, rule validation, VLM critique, and finalization">
  262. </p>
  263. <p align="center">
  264. <img src="./assets/readme/research/qualitative-trajectory.webp" width="100%" alt="Five selected attempts from one AutoDesign poster trajectory, from a clipped first draft to the accepted ninth attempt">
  265. </p>
  266. The latest paper traces one poster run through five selected attempts. The
  267. critic identifies a clipped analysis lane at A1; A3 restores the fit, A5 refits
  268. the header, A6 rescales evidence, and A9 preserves the repaired composition and
  269. is accepted. The trajectory shows that diagnostics drive localized edits while
  270. valid layout and source-derived content survive across revisions.
  271. <a id="benchmark"></a>
  272. ## <img src="./assets/readme/icons/trophy.svg" width="26" alt="" align="absmiddle"> PosterBench leaderboard
  273. **PosterBench** evaluates a 100-paper large set and a fixed 10-paper small set
  274. across AI/ML, biomedicine and health, climate and earth environment, economics
  275. and policy, and physics and astronomy. Every output is rendered to a common
  276. poster format before scoring.
  277. The metadata-only manifests are released on Hugging Face as
  278. [`YaxinLuo/PosterBench`](https://huggingface.co/datasets/YaxinLuo/PosterBench)
  279. and
  280. [`YaxinLuo/PosterBench-mini`](https://huggingface.co/datasets/YaxinLuo/PosterBench-mini).
  281. They can be downloaded or loaded directly with `datasets` without
  282. redistributing the underlying paper PDFs.
  283. The seven dimensions are **Faithfulness, Coverage, Density, Visual Evidence,
  284. Layout, Readability, and Aesthetics**, weighted
  285. **10/10/15/10/20/25/10**. Programmatic evidence and source-conditioned VLM
  286. judgments are aggregated first; then the strictest active ceiling for severe
  287. layout damage, insufficient presentation viability, confirmed visible failure,
  288. or protected render integrity is applied to each poster.
  289. <p align="center">
  290. <img src="./assets/readme/research/evaluation-protocol.webp" width="100%" alt="PosterBench evaluation protocol with localized programmatic audits, seven source-conditioned dimensions, and a protected render-integrity gate">
  291. </p>
  292. ### <img src="./assets/readme/icons/chart-bars.svg" width="20" alt="" align="absmiddle"> Full-Scale Benchmark Main Track · 100 papers
  293. AutoDesign achieves the two highest PosterBench Scores. With Claude Code and
  294. Claude 4.8 fixed, it scores **78.32**, exceeding Claude Design by **7.45**
  295. points and OpenDesign by **8.87** points.
  296. <p align="center">
  297. <img src="./assets/readme/research/posterbench-main.webp" width="72%" alt="PosterBench full-scale comparison of design agents and coding-agent model configurations">
  298. </p>
  299. | Rank | Score | System | Design harness | Coding agent | Model |
  300. |---:|---:|---|---|---|---|
  301. | **1** | **78.32** | **AutoDesign** | **DesignHarness** | **Claude Code** | **Claude 4.8** |
  302. | **2** | **77.97** | **AutoDesign** | **DesignHarness** | **Codex** | **GPT-5.5** |
  303. | 3 | 73.37 | Codex | — | Codex | GPT-5.5 |
  304. | 4 | 70.87 | Claude Design | Claude Design | Claude Code | Claude 4.8 |
  305. | 5 | 70.01 | Claude Code | — | Claude Code | Claude 4.8 |
  306. | 6 | 69.45 | OpenDesign | OpenDesign | Claude Code | Claude 4.8 |
  307. | 7 | 62.17 | OpenDesign | OpenDesign | Codex | GPT-5.5 |
  308. | 8 | 61.14 | Doubao | — | Claude Code | Seed 2.1 |
  309. | 9 | 56.71 | PosterGen | — | — | Claude 4.8 |
  310. | 10 | 52.22 | GLM | — | Claude Code | GLM 5.2 |
  311. | 11 | 51.46 | Kimi | — | Claude Code | Kimi K2.7 |
  312. | 12 | 49.09 | Any2Poster | — | — | Claude 4.8 |
  313. | 13 | 46.01 | DeepSeek | — | Claude Code | DeepSeek V4 Pro |
  314. | 14 | 44.61 | Paper2Poster | — | — | Claude 4.8 |
  315. <details>
  316. <summary><strong>Open the Small-Scale Benchmark Main Track · fixed 10-paper subset</strong></summary>
  317. <br>
  318. | Rank | Score | System | Design harness | Coding agent | Model |
  319. |---:|---:|---|---|---|---|
  320. | **1** | **81.46** | **AutoDesign** | **DesignHarness** | **Codex** | **GPT-5.5** |
  321. | 2 | 75.87 | Codex | — | Codex | GPT-5.5 |
  322. | **3** | **74.56** | **AutoDesign** | **DesignHarness** | **Claude Code** | **Claude 4.8** |
  323. | 4 | 70.36 | OpenDesign | OpenDesign | Claude Code | Claude 4.8 |
  324. | 5 | 69.55 | Claude Code | — | Claude Code | Claude 4.8 |
  325. | 6 | 66.83 | Claude Design | Claude Design | Claude Code | Claude 4.8 |
  326. | 7 | 60.58 | OpenDesign | OpenDesign | Codex | GPT-5.5 |
  327. | 8 | 57.20 | Kimi | — | Claude Code | Kimi K2.7 |
  328. | 9 | 54.01 | Doubao | — | Claude Code | Seed 2.1 |
  329. | 10 | 51.82 | PosterGen | — | — | Claude 4.8 |
  330. | 11 | 50.32 | GLM | — | Claude Code | GLM 5.2 |
  331. | 12 | 46.88 | Any2Poster | — | — | Claude 4.8 |
  332. | 13 | 42.06 | Paper2Poster | — | — | Claude 4.8 |
  333. | 14 | 34.73 | DeepSeek | — | Claude Code | DeepSeek V4 Pro |
  334. </details>
  335. ### <img src="./assets/readme/icons/sliders.svg" width="20" alt="" align="absmiddle"> Controlled tracks · fixed 10-paper subset
  336. Each controlled track varies one factor while holding the others fixed.
  337. | Rank | Design Harness Track<br><sub>Fixed: Claude Code + Claude 4.8</sub> | Score | Coding Harness Track<br><sub>Fixed: AutoDesign + GLM 5.2</sub> | Score | Model Track<br><sub>Fixed: AutoDesign + Claude Code</sub> | Score |
  338. |---:|---|---:|---|---:|---|---:|
  339. | **1** | **AutoDesign** | **74.56** | **Kimi Code** | **82.31** | **Claude 4.8** | **74.56** |
  340. | 2 | OpenDesign | 70.36 | ZCode | 69.53 | Seed 2.1 Pro | 71.83 |
  341. | 3 | Claude Design | 66.83 | OpenCode | 67.87 | Kimi K2.7 | 70.12 |
  342. | 4 | — | — | Claude Code | 64.33 | GLM 5.2 | 64.33 |
  343. | 5 | — | — | — | — | LongCat 2.0 | 55.13 |
  344. | 6 | — | — | — | — | DeepSeek V4 Pro | 54.29 |
  345. ### <img src="./assets/readme/icons/trending-up.svg" width="20" alt="" align="absmiddle"> DesignHarness benefit
  346. Across seven matched model–coding-agent configurations, attaching DesignHarness
  347. improves every PosterBench Score by **+5.01 to +19.56 points**. Native
  348. Codex–GPT-5.5 rises from **75.87 to 81.46 (+5.59)**; Claude Code–Kimi K2.7
  349. rises from **57.20 to 70.12 (+12.92)**; and the largest gain is
  350. **+19.56** for Claude Code–DeepSeek V4 Pro.
  351. <p align="center">
  352. <img src="./assets/readme/research/harness-gains.webp" width="72%" alt="PosterBench gains from attaching DesignHarness to seven fixed coding-agent and model configurations">
  353. </p>
  354. ### <img src="./assets/readme/icons/balance.svg" width="20" alt="" align="absmiddle"> Cost–performance trade-off
  355. On the fixed 10-paper subset, the observed Pareto frontier runs from LongCat 2.0
  356. (**55.13 at $0.27/poster**) through Doubao Seed 2.1 Pro
  357. (**71.83 at $2.75**) and Claude 4.8 (**74.56 at $7.63**) to GPT-5.5
  358. (**81.46 at $10.02**). Doubao reaches 88% of the GPT-5.5 score at 27%
  359. of its normalized designer-only API cost.
  360. <p align="center">
  361. <img src="./assets/readme/research/cost-performance.webp" width="78%" alt="PosterBench score, normalized designer-only API cost, median runtime, and empirical Pareto frontier">
  362. </p>
  363. For the executable protocol, data preparation, score ownership, record-level
  364. ceilings, and reproduction commands, see the
  365. [PosterBench evaluation guide](eval/README.md).
  366. <a id="human-evaluation"></a>
  367. ## <img src="./assets/readme/icons/users.svg" width="26" alt="" align="absmiddle"> Human evaluation
  368. The fully system-blind study collected **936 responses** from **11 volunteer
  369. reviewers**: 933 ranking judgments and three skips. AutoDesign has the highest
  370. Bradley–Terry estimate at **64.0%**, with a **55.2–77.8%** 95% interval. Its
  371. tie-adjusted empirical preference is 61.3% against Claude Code, 63.1% against
  372. OpenDesign, and 67.6% against Claude Design.
  373. <p align="center">
  374. <img src="./assets/readme/research/human-evaluation.webp" width="100%" alt="System-blind Bradley-Terry estimates and AutoDesign head-to-head outcomes">
  375. </p>
  376. PosterBench is positively, though imperfectly, associated with human preference
  377. (**r = 0.34**, 95% interval **0.22–0.44**). Agreement with the
  378. PosterBench-preferred poster rises from **51.9%** for 0–3-point gaps to
  379. **74.4%** when the score gap is at least 20 points.
  380. <p align="center">
  381. <img src="./assets/readme/research/benchmark-human-alignment.webp" width="100%" alt="PosterBench score association with system-blind human preference and agreement by score margin">
  382. </p>
  383. ## <img src="./assets/readme/icons/compass.svg" width="26" alt="" align="absmiddle"> Future directions
  384. The current DesignHarness already produces pilot **paper-to-slide,
  385. paper-to-webpage, and paper-to-conference-video** artifacts, but PosterBench
  386. formally validates academic posters only. Slides, webpages, and videos still
  387. need medium-specific source–output data, evaluators, rendering and validation
  388. gates, and communication objectives before their research claims match the
  389. poster pipeline.
  390. <p align="center">
  391. <img src="./assets/readme/research/multiformat-pilots.webp" width="100%" alt="Paper poster, slide, webpage, and conference-video pilots produced by the current DesignHarness">
  392. </p>
  393. Longer term, AutoDesign aims toward multimodal-in, multimodal-out agentic design:
  394. integrating papers, visual evidence, code, data, and human guidance to create
  395. medium-specific outputs. Open research problems include better component
  396. selection, evaluator evolution anchored by frozen tasks and human audits, and
  397. combining harness optimization with model post-training.
  398. <p align="center">
  399. <img src="./assets/readme/research/future-multimodal-system.webp" width="80%" alt="Future multimodal-in multimodal-out agentic design system">
  400. </p>
  401. We welcome researchers, designers, and engineers to contribute new design
  402. harnesses, refinement workflows, evaluators, and artifact capabilities.
  403. <p align="center">
  404. <a href="https://github.com/Yaxin9Luo/AutoDesign"><strong>Contribute on GitHub ↗</strong></a> ·
  405. <a href="https://autodesign.designanything.ai/"><strong>Explore the project ↗</strong></a>
  406. </p>
  407. <a id="interfaces"></a>
  408. ## <img src="./assets/readme/icons/terminal.svg" width="26" alt="" align="absmiddle"> Interfaces and outputs
  409. The Web UI provides Paper All-in-One generation, model and provider settings,
  410. progress streaming, cancel and retry, server-backed history, and direct editing
  411. for supported HTML-first artifacts.
  412. Start the interactive CLI with:
  413. ```bash
  414. uv --cache-dir .uv-cache run python -m autodesign
  415. ```
  416. | Use case | Primary output |
  417. |---|---|
  418. | Academic paper poster | `final/poster.html`, `final/preview.png`, optional PDF |
  419. | Slide deck | `final/deck.html`, `final/deck.pdf`, slide previews |
  420. | Landing or project page | `final/index.html`, `final/preview.png` |
  421. | Video | Editable HyperFrames project, narrated MP4 with AAC audio, transcript, and timed SRT/VTT subtitles |
  422. | Creative poster | HTML/PNG, with legacy PSD/SVG paths where supported |
  423. | Research reproduction handoff | OpenResearch project, session, and report links |
  424. Single-run output lives under `out/runs/<run_id>/`; EvaData batch output lives
  425. under `out/eva_poster_batches/<batch_id>/`. Both locations are ignored by Git.
  426. The canonical Python module and installed launcher are `autodesign`. The
  427. `design_anything` module, `design-anything` console command, `designanything`
  428. launcher, and `DESIGN_ANYTHING_*` environment variables are deprecated
  429. compatibility aliases. New configuration and automation should use
  430. `AUTODESIGN_*`.
  431. ## <img src="./assets/readme/icons/file-output.svg" width="26" alt="" align="absmiddle"> Citation
  432. ```bibtex
  433. @article{luo2026autodesign,
  434. title={AutoDesign: Meta-Harness Optimization for Long-Horizon Agentic Design},
  435. author={Luo, Yaxin and Jiang, Haobin and Zou, Jialv and Huang, Xu and Yan, Wenhao and Li, Haodong and Yue, Zhengrong and Li, Jing and Chen, Xiaofu and Zhao, Xiaohan and Liu, Jiacheng and Cui, Jiacheng and Shen, Zhiqiang and Li, Xiaotong},
  436. journal={arXiv preprint arXiv:2608.13560},
  437. year={2026}
  438. }
  439. ```
  440. ## <img src="./assets/readme/icons/users.svg" width="26" alt="" align="absmiddle"> Acknowledgements
  441. AutoDesign is made possible by the open-source community. We are especially
  442. grateful to:
  443. - [HyperFrames](https://github.com/heygen-com/hyperframes) for the HTML-first
  444. video runtime, composition linting, and MP4 rendering.
  445. - [KaTeX](https://katex.org/) for offline mathematical typesetting in portable
  446. HTML artifacts.
  447. - [html-ppt-skill](https://github.com/lewislulu/html-ppt-skill) for the
  448. MIT-licensed deck-authoring reference assets adapted in this repository.
  449. ## <img src="./assets/readme/icons/shield-check.svg" width="26" alt="" align="absmiddle"> License
  450. MIT. Bundled third-party assets retain their own licenses; see
  451. [Third-Party Notices](./THIRD_PARTY_NOTICES.md) for details.

Ratings and comments

Sign in to rate this skill.

Sign in

Free to use. If it’s helped, pass it on.