Skip to content
Back to search

vibesys

MarketplaceWorks with Claude CodeGitHub

Skills that run VibeSys-style, measurement-driven optimization inside Claude Code, with domain skills for kernels, LLM serving, databases, microservices and native command-line tools.

Safety scan

A static text scan, not a guarantee. Select the badge to see what was scanned and every finding.

Checked yesterday
Badge for your README
findsafeskills safety rating

Markdown

[![findsafeskills safety rating](/api/listings/uw-syfi--vibesys/badge.svg)](/listings/uw-syfi--vibesys)

HTML

<a href="/listings/uw-syfi--vibesys"><img src="/api/listings/uw-syfi--vibesys/badge.svg" alt="findsafeskills safety rating" /></a>

Install

Install instructions from the README

As written by the repo's author. findsafeskills has not verified these steps.

VibeSys is available as a plugin for Claude Code and Codex. In Claude Code, for example:

claude plugin marketplace add uw-syfi/vibesys
claude plugin install vibesys@vibesys
# Install additional plugins for domain-specific knowledge
claude plugin install vibesys-llm-serving@vibesys

Then ask your agent to create a task describing your optimization goal and correctnes criteria, and start the optimization:

/vibesys:create-task
/vibesys:optimize <task>

See the [skills installation guide](skills/README.md#install) for Codex, `npx skills`, and domain plugins for databases, microservices, native command-line tools, and kernels.

To run managed optimizations with `vibesys-client` instead, see [Install for managed runs](orchestrator/README.md#install-for-managed-runs). To remove it, see [Removing VibeSys](orchestrator/README.md#removing-vibesys).

What was scanned

The files the safety scan read, as written by the repository's author and shown as plain text. Lines with a finding are marked.

.claude-plugin/marketplace.json · Manifest · 134 lines
  1. {
  2. "name": "vibesys",
  3. "description": "Skills that run VibeSys-style, measurement-driven optimization inside Claude Code, with domain skills for kernels, LLM serving, databases, microservices and native command-line tools.",
  4. "owner": {
  5. "name": "uw-syfi"
  6. },
  7. "plugins": [
  8. {
  9. "name": "vibesys",
  10. "source": "./skills",
  11. "description": "Resumable, measurement-driven optimization loop with parallel hypotheses, measurements and independent judging; the domain skills are in the vibesys-llm-serving, vibesys-database, vibesys-microservices, vibesys-cli-tools and vibesys-kernels-<backend> plugins.",
  12. "version": "0.14.0",
  13. "strict": false,
  14. "skills": [
  15. "./search/allocations",
  16. "./search/combine-candidates",
  17. "./search/commit",
  18. "./search/create-task",
  19. "./search/implement-hypothesis",
  20. "./search/judge-candidate",
  21. "./search/measure",
  22. "./search/optimize",
  23. "./search/plan-hypotheses",
  24. "./search/profile-revision",
  25. "./search/report-progress",
  26. "./search/setup",
  27. "./search/state",
  28. "./search/visualize-progress"
  29. ]
  30. },
  31. {
  32. "name": "vibesys-llm-serving",
  33. "source": "./skills",
  34. "description": "Domain skills for optimizing LLM serving systems, including predictions from the ServingStudio simulator, for use with the vibesys optimization loop.",
  35. "version": "0.3.0",
  36. "strict": false,
  37. "skills": [
  38. "./domains/serving-studio",
  39. "./domains/serving-systems"
  40. ]
  41. },
  42. {
  43. "name": "vibesys-microservices",
  44. "source": "./skills",
  45. "description": "Domain skills for optimizing microservices, including Tokio services profiled with dial9, for use with the vibesys optimization loop.",
  46. "version": "0.1.1",
  47. "strict": false,
  48. "skills": [
  49. "./domains/custom-rpc-optimization",
  50. "./domains/dial9-tokio/dial9-rank-bottlenecks",
  51. "./domains/dial9-tokio/skills/dial9-diagnose-long-poll",
  52. "./domains/dial9-tokio/skills/dial9-red-flags",
  53. "./domains/dial9-tokio/skills/dial9-runtime",
  54. "./domains/dial9-tokio/skills/dial9-toolkit",
  55. "./domains/dial9-tokio/skills/dial9-trace-analysis",
  56. "./domains/dial9-tokio/skills/dial9-trace-loading",
  57. "./domains/dial9-tokio/skills/dial9-trace-recipes",
  58. "./domains/dial9-tokio/skills/dial9-zoom-window",
  59. "./domains/microservice-optimization"
  60. ]
  61. },
  62. {
  63. "name": "vibesys-database",
  64. "source": "./skills",
  65. "description": "Tokio runtime diagnosis for database and other Tokio-based services.",
  66. "version": "0.1.1",
  67. "strict": false,
  68. "skills": [
  69. "./domains/dial9-tokio/dial9-rank-bottlenecks",
  70. "./domains/dial9-tokio/skills/dial9-diagnose-long-poll",
  71. "./domains/dial9-tokio/skills/dial9-red-flags",
  72. "./domains/dial9-tokio/skills/dial9-runtime",
  73. "./domains/dial9-tokio/skills/dial9-toolkit",
  74. "./domains/dial9-tokio/skills/dial9-trace-analysis",
  75. "./domains/dial9-tokio/skills/dial9-trace-loading",
  76. "./domains/dial9-tokio/skills/dial9-trace-recipes",
  77. "./domains/dial9-tokio/skills/dial9-zoom-window"
  78. ]
  79. },
  80. {
  81. "name": "vibesys-kernels-cuda",
  82. "source": "./skills",
  83. "description": "Domain skills for correct and fast CUDA kernels, with Nsight Compute and timeline analysis, for use with the vibesys optimization loop.",
  84. "version": "0.1.0",
  85. "strict": false,
  86. "skills": [
  87. "./domains/kernel-correctness",
  88. "./domains/kernel-ncu-analysis",
  89. "./domains/kernel-performance",
  90. "./domains/kernel-timeline-analysis"
  91. ]
  92. },
  93. {
  94. "name": "vibesys-kernels-rocm",
  95. "source": "./skills",
  96. "description": "Domain skills for correct and fast ROCm kernels, with rocprof analysis, for use with the vibesys optimization loop.",
  97. "version": "0.2.1",
  98. "strict": false,
  99. "skills": [
  100. "./domains/kernel-correctness",
  101. "./domains/kernel-performance",
  102. "./domains/kernel-rocprof-analysis"
  103. ]
  104. },
  105. {
  106. "name": "vibesys-kernels-trainium",
  107. "source": "./skills",
  108. "description": "Domain skills for correct and fast Trainium kernels, with the vendored Neuron NKI pack, for use with the vibesys optimization loop.",
  109. "version": "0.1.1",
  110. "strict": false,
  111. "skills": [
  112. "./domains/kernel-correctness",
  113. "./domains/kernel-performance",
  114. "./domains/neuron-agentic-development/neuron-safeguards",
  115. "./domains/neuron-agentic-development/skills/neuron-explorer-profile-schema",
  116. "./domains/neuron-agentic-development/skills/neuron-nki-debugging",
  117. "./domains/neuron-agentic-development/skills/neuron-nki-docs",
  118. "./domains/neuron-agentic-development/skills/neuron-nki-profile-querying",
  119. "./domains/neuron-agentic-development/skills/neuron-nki-profiling",
  120. "./domains/neuron-agentic-development/skills/neuron-nki-writing"
  121. ]
  122. },
  123. {
  124. "name": "vibesys-cli-tools",
  125. "source": "./skills",
  126. "description": "Domain skill for optimizing locally built native command-line programs (text, filesystem, sorting and repository tools) without changing their behavior, for use with the vibesys optimization loop.",
  127. "version": "0.1.3",
  128. "strict": false,
  129. "skills": [
  130. "./domains/cli-tool-optimization"
  131. ]
  132. }
  133. ]
  134. }
README.md · README · 143 lines
  1. # VibeSys: Generating Bespoke Systems with AI Agents
  2. [![arXiv](https://img.shields.io/badge/arXiv-2605.06068-b31b1b.svg)](https://arxiv.org/abs/2605.06068)
  3. **An agentic framework that generates bespoke systems from application requirements, workload characteristics, and the underlying hardware.**
  4. ## News
  5. - **2026-11-04** — [VibeSys on Trainium](https://syfi.cs.washington.edu/blog/2026-11-04-vibesys-trainium/): agents build and tune Qwen3.8 serving engines on AWS Trainium2.
  6. - **2026-10-08** — [VibeSys on AMD MI300A](https://syfi.cs.washington.edu/blog/2026-10-08-qwen35-mi300a/): an agent-built Qwen3.5-397B engine achieves 2.3× the goodput of tuned SGLang.
  7. - **2026-05-12** — [Introducing VibeServe](https://syfi.cs.washington.edu/blog/2026-05-12-introducing-vibeserve/): let AI agents write your serving stack. [Paper](https://arxiv.org/abs/2605.06068).
  8. ## About VibeSys
  9. Today's systems software is built for generality. A serving engine, operating
  10. system, or database supports many workloads and hardware platforms, but its
  11. shared abstractions can leave opportunities in any one deployment unexploited.
  12. Specialization can recover large gains; building and maintaining a separate
  13. system for each deployment has rarely justified the engineering cost. Coding
  14. agents are beginning to change that calculation.
  15. **VibeSys uses AI agents to build and optimize bespoke systems for a specific
  16. application, workload, and hardware target.** You provide the requirements,
  17. correctness checks, and performance benchmark. Agents explore system designs,
  18. implement candidates, and use measurements to guide the next change. Persistent
  19. search history and independent review let that process continue across many
  20. experiments, while domain skills supply the knowledge needed for serving
  21. engines, databases, microservices, and kernels. VibeSys can start from an existing
  22. codebase or build a system from scratch.
  23. [**VibeServe**](https://arxiv.org/abs/2605.06068) demonstrated this approach for
  24. LLM serving: agents generated complete serving systems that exploit deployment
  25. knowledge across scheduling, memory management, and kernels. On predicted-output
  26. code editing, a generated system achieved up to **5.95× the speed of vLLM's
  27. standard decoding**. VibeSys extends this approach to systems software more
  28. broadly, with each task defining what must remain correct and what to optimize.
  29. <p align="center">
  30. <img src="docs/figures/idea.png" width="85%" alt="Generic serving today vs. VibeServe's per-target bespoke systems">
  31. </p>
  32. ## Quickstart
  33. VibeSys is available as a plugin for Claude Code and Codex. In Claude Code, for example:
  34. ```bash
  35. claude plugin marketplace add uw-syfi/vibesys
  36. claude plugin install vibesys@vibesys
  37. # Install additional plugins for domain-specific knowledge
  38. claude plugin install vibesys-llm-serving@vibesys
  39. ```
  40. Then ask your agent to create a task describing your optimization goal and correctnes criteria, and start the optimization:
  41. ```text
  42. /vibesys:create-task
  43. /vibesys:optimize <task>
  44. ```
  45. See the [skills installation guide](skills/README.md#install) for Codex,
  46. `npx skills`, and domain plugins for databases, microservices, native command-line tools, and kernels.
  47. To run managed optimizations with `vibesys-client` instead, see
  48. [Install for managed runs](orchestrator/README.md#install-for-managed-runs). To remove it, see
  49. [Removing VibeSys](orchestrator/README.md#removing-vibesys).
  50. ## Performance highlights
  51. Below are some recent optimization results from running VibeSys. Click on the images to see blog posts detailing the results.
  52. <table>
  53. <tr>
  54. <td colspan="2" align="center">
  55. <a href="https://syfi.cs.washington.edu/blog/2026-10-08-qwen35-mi300a/">
  56. <img src="docs/figures/qwen35-397b-mi300a.png" width="900" alt="Qwen3.5-397B on four AMD MI300As: goodput improves across optimization rounds and exceeds tuned SGLang">
  57. </a>
  58. <br><strong>Qwen3.5-397B · 4× AMD MI300A</strong><br>
  59. 2,242 tok/s goodput · <strong>2.3× tuned SGLang</strong>
  60. </td>
  61. </tr>
  62. <tr>
  63. <td width="50%" align="center">
  64. <a href="https://syfi.cs.washington.edu/blog/2026-11-04-vibesys-trainium/">
  65. <img src="docs/figures/qwen38-27b-trainium2.png" width="450" alt="Qwen3.8-27B on one Trainium2 chip: throughput rises from 195 to 861 tokens per second across optimization rounds">
  66. </a>
  67. <br><strong>Qwen3.8-27B · 1× Trainium2</strong><br>
  68. 195 → 861 tok/s · <strong>4.4× the first working engine</strong>
  69. </td>
  70. <td width="50%" align="center">
  71. <a href="https://syfi.cs.washington.edu/blog/2026-11-04-vibesys-trainium/">
  72. <img src="docs/figures/qwen38-flash-next-trainium2.png" width="450" alt="Qwen3.8-Flash-Next on sixteen Trainium2 chips: throughput improves across twelve optimization rounds">
  73. </a>
  74. <br><strong>Qwen3.8-Flash-Next · 16× Trainium2</strong><br>
  75. <strong>3.1× the starting engine</strong>
  76. </td>
  77. </tr>
  78. </table>
  79. ## Architecture
  80. <p align="center">
  81. <img src="docs/figures/architecture.png" width="90%" alt="VibeServe architecture: outer loop dispatches per-round tasks to an inner loop of Implementer / Accuracy Judge / Performance Evaluator agents">
  82. </p>
  83. The framework factors the work along two axes:
  84. - **Outer loop** — a fresh designer selects one falsifiable causal hypothesis
  85. from git history, profiling evidence, and durable roadmap/progress memory, then
  86. hands it off until it is proven, disproven, or otherwise terminated.
  87. - **Inner loop** — a hypothesis-scoped implementer session edits the candidate,
  88. chooses targeted experiments and parameter ranges, and reports whether to
  89. continue or nominate the result.
  90. - **Independent judge** — a fresh, read-only reviewer checks the implementation,
  91. activation evidence, invariants, and reward-hacking risks at a sparse cadence.
  92. After a PASS, the framework—not an agent—runs and records the canonical
  93. accuracy and benchmark commands.
  94. - **Performance evaluator** — profiles the implementation (Nsight Systems,
  95. PyTorch profiler) and feeds bottleneck hints into future design decisions.
  96. - **Skills library** — Agent Skills entries distilled from existing serving engines and research literature (continuous batching, paged-KV, FlashInfer/FlashAttention, MLX, hybrid-cache management, …). New model families, hardware platforms, and optimization techniques are added by writing a skill, not by modifying the framework.
  97. - **Execution environment**: an isolated runtime view where candidate source
  98. is writable while evaluator-owned inputs and framework metadata are read-only
  99. and integrity-checked. It exposes the target hardware (local CUDA, Modal,
  100. Docker, or Apple Silicon) plus profilers.
  101. Each round is recorded in git and a framework-owned audit. Provisional rounds
  102. remain explicitly unreviewed; only judge-approved candidates receive official
  103. accuracy and performance results.
  104. ## Citation
  105. If you use the VibeServe initiative in your research, please cite:
  106. ```bibtex
  107. @misc{kamahori2026vibeserveaiagentsbuild,
  108. title={VibeServe: Can AI Agents Build Bespoke LLM Serving Systems?},
  109. author={Keisuke Kamahori and Shihang Li and Simon Peter and Baris Kasikci},
  110. year={2026},
  111. eprint={2605.06068},
  112. archivePrefix={arXiv},
  113. primaryClass={cs.AI},
  114. url={https://arxiv.org/abs/2605.06068},
  115. }
  116. ```

Ratings and comments

Sign in to rate this skill.

Sign in

Free to use. If it’s helped, pass it on.