Skip to content
Back to search

codex-autoresearch

"Run repeated, measured Git experiments toward a numeric target; keep improvements and revert failures. Use for autonomous optimization or managing an autoresearch run, not one-shot edits."

Safety scan

A static text scan, not a guarantee. Select the badge to see what was scanned and every finding.

Checked today
Badge for your README
findsafeskills safety rating

Markdown

[![findsafeskills safety rating](/api/listings/leo-lilinxiao-codex-autoresearch/badge.svg)](/listings/leo-lilinxiao-codex-autoresearch)

HTML

<a href="/listings/leo-lilinxiao-codex-autoresearch"><img src="/api/listings/leo-lilinxiao-codex-autoresearch/badge.svg" alt="findsafeskills safety rating" /></a>

Install

i

Install instructions from the README

As written by the repo's author. findsafeskills has not verified these steps.

Install from Codex:

$skill-installer install https://github.com/leo-lilinxiao/codex-autoresearch

Open a clean Git repository with Full Access:

codex --dangerously-bypass-approvals-and-sandbox

Then invoke the skill:

You:   $codex-autoresearch
       Reduce `python3 scripts/score.py` error_count to 0.

Codex: Baseline: 5
       Target: 0 (lower is better)
       Scope: src/
       Verify: python3 scripts/score.py, JSON key error_count
       Guard: none
       Run in foreground or background?

You:   Background. Go.

Codex launches the confirmed run. No Codex configuration changes or special prompt syntax are required.

See [Installation](docs/INSTALL.md) for manual and development installs.

What was scanned

The files the safety scan read, as written by the repository's author and shown as plain text. Lines with a finding are marked.

SKILL.md · Manifest · 45 lines

View this file on GitHub (read at the repository's current version: the scan did not record which commit it read)

  1. ---
  2. name: codex-autoresearch
  3. description: "Run repeated, measured Git experiments toward a numeric target; keep improvements and revert failures. Use for autonomous optimization or managing an autoresearch run, not one-shot edits."
  4. metadata:
  5. short-description: "Run measurable autonomous experiments"
  6. ---
  7. # Codex Autoresearch
  8. Improve a repository through repeated, reversible experiments:
  9. `hypothesize -> change -> measure -> learn -> keep or revert -> repeat`
  10. Codex chooses hypotheses and makes code changes. The control script owns measurement, commits, rollback, and the event history.
  11. ## Respond To The Request
  12. Resolve `<control>` to this skill's own `scripts/autoresearch.py`; do not assume it is installed in the target repository.
  13. For a status or results request, run the corresponding command directly:
  14. | Request | Command |
  15. |---|---|
  16. | Status | `python3 <control> status --repo <repo>` |
  17. | History | `python3 <control> history --repo <repo>` |
  18. | TSV export | `python3 <control> history --repo <repo> --format tsv` |
  19. | HTML report | `python3 <control> report --repo <repo>` |
  20. Report the requested result without starting or resuming experiments. Return the generated path for an HTML report.
  21. Before starting or resuming an experiment, read the applicable workflow unless it is already in context:
  22. - [Workflow](references/workflow.md): starting, continuing, resuming, or changing an experiment.
  23. - [Background](references/background.md): launching or controlling a detached run.
  24. ## Experiment Boundary
  25. - One run owns one Git repository, one numeric metric, one confirmed target, and approved path scopes.
  26. - For a new foreground run, initialize successfully before creating its Goal; the Goal identifies the returned run id. If initialization reports `complete`, no Goal is needed.
  27. - Use `finish` to finalize each coherent experiment. Do not manually commit, revert, or edit run artifacts during an active run.
  28. - Only a verified target can mean `complete`; an iteration limit, error, or external blocker has its own status.
  29. - Validate state through the control script on entry or resume. Use recorded experiments rather than guessing from conversation.
  30. - Surface command and state errors with their diagnostic paths. Never bypass validation, fabricate metrics, or reconstruct missing events.
  31. Foreground continuation belongs to the official Codex Goal. Background continuation belongs to the detached controller. An already launched background worker follows its supplied experiment contract, without starting a new launch flow.
README.md · README · 251 lines · 1 finding marked

View this file on GitHub (read at the repository's current version: the scan did not record which commit it read)

  1. <p align="center">
  2. <img src="image/banner.png" width="700" alt="Codex Autoresearch">
  3. </p>
  4. <h2 align="center"><b>Aim. Iterate. Arrive.</b></h2>
  5. <p align="center">
  6. <i>Autonomous, measurable experimentation for Codex.</i>
  7. </p>
  8. <p align="center">
  9. <a href="https://developers.openai.com/codex/skills"><img src="https://img.shields.io/badge/Codex-Skill-blue?logo=openai&logoColor=white" alt="Codex Skill"></a>
  10. <a href="https://github.com/leo-lilinxiao/codex-autoresearch"><img src="https://img.shields.io/github/stars/leo-lilinxiao/codex-autoresearch?style=social" alt="GitHub Stars"></a>
  11. <a href="LICENSE"><img src="https://img.shields.io/badge/License-MIT-green.svg" alt="MIT License"></a>
  12. </p>
  13. <p align="center">
  14. <b>English</b> ·
  15. <a href="docs/i18n/README_ZH.md">中文</a> ·
  16. <a href="docs/i18n/README_JA.md">日本語</a> ·
  17. <a href="docs/i18n/README_KO.md">한국어</a> ·
  18. <a href="docs/i18n/README_FR.md">Français</a> ·
  19. <a href="docs/i18n/README_DE.md">Deutsch</a> ·
  20. <a href="docs/i18n/README_ES.md">Español</a> ·
  21. <a href="docs/i18n/README_PT.md">Português</a> ·
  22. <a href="docs/i18n/README_RU.md">Русский</a>
  23. </p>
  24. ---
  25. Tell Codex what measurable result you want. Codex inspects the repository, confirms the experiment with you, changes one thing, verifies it, keeps improvements, reverts failures, and repeats until the target is reached.
  26. Autoresearch works for test failures, coverage, type errors, warnings, latency, binary size, reproducible security findings, and any other outcome a command can measure.
  27. ## Quick Start
  28. Install from Codex:
  29. ```text
  30. $skill-installer install https://github.com/leo-lilinxiao/codex-autoresearch
  31. ```
  32. Open a clean Git repository with Full Access:
  33. ```bash
  34. codex --dangerously-bypass-approvals-and-sandbox
  35. ```
  36. Then invoke the skill:
  37. ```text
  38. You: $codex-autoresearch
  39. Reduce `python3 scripts/score.py` error_count to 0.
  40. Codex: Baseline: 5
  41. Target: 0 (lower is better)
  42. Scope: src/
  43. Verify: python3 scripts/score.py, JSON key error_count
  44. Guard: none
  45. Run in foreground or background?
  46. You: Background. Go.
  47. ```
  48. Codex launches the confirmed run. No Codex configuration changes or special prompt syntax are required.
  49. See [Installation](docs/INSTALL.md) for manual and development installs.
  50. ## The Loop
  51. ```text
  52. inspect evidence
  53. |
  54. change one focused thing
  55. |
  56. commit and measure
  57. |
  58. +-- improved + guard passes --> keep
  59. |
  60. +-- otherwise ---------------> revert
  61. |
  62. append an audit event
  63. |
  64. repeat until target
  65. ```
  66. The control script owns commits, verification, rollback, and state. Codex owns the hypotheses and code changes.
  67. ## Foreground And Background
  68. | | Foreground | Background |
  69. |---|---|---|
  70. | Runs in | Current Codex task | Detached controller |
  71. | Continuation | Official Codex Goal | One `codex exec` worker per iteration |
  72. | Best for | Watching and steering live | Long or overnight runs |
  73. | Control | Codex Goal pause/resume | Ask `$codex-autoresearch` for status, stop, or resume |
  74. Foreground and background use the same experiment rules. A run uses one mode at a time. Foreground continuation uses a Codex Goal; background continuation belongs to the detached controller.
  75. ## What Gets Confirmed
  76. Before the first write, Codex shows:
  77. - the goal and numeric target;
  78. - repository-relative paths it may change;
  79. - the metric command and explicit parser;
  80. - an optional regression guard;
  81. - foreground or background mode;
  82. - an optional iteration limit.
  83. Initialization requires a clean named Git branch. One run manages one repository.
  84. Codex reuses choices and approval you have already given. Once the experiment is confirmed, it keeps working without asking you to approve each iteration.
  85. ## Results
  86. Run artifacts live in `autoresearch-results/` and stay uncommitted:
  87. | Path | Purpose |
  88. |---|---|
  89. | `run.json` | Immutable confirmed configuration |
  90. | `events.jsonl` | Append-only baseline, iteration, stop, and completion history |
  91. | `logs/` | Full metric, guard, and background worker output |
  92. | `runtime.json` | Background process state |
  93. | `runtime.log` | Background controller lifecycle events |
  94. | `report.html` | Optional, regenerated visual snapshot |
  95. `events.jsonl` is the state history. Missing, malformed, contradictory, or partial state is an error; the skill never guesses a result from old files or conversational memory.
  96. ## Review Results
  97. Ask the skill to show the validated experiment history:
  98. ```text
  99. $codex-autoresearch show experiment history
  100. ```
  101. ```text
  102. Codex Autoresearch
  103. Run: 0a516883 Status: complete Mode: foreground
  104. Metric: error_count 2 -> 0 Target: 0 (lower is better)
  105. SEQ ITER EVENT PREVIOUS TRIAL RETAINED DESCRIPTION
  106. --- ---- -------- -------- ----- -------- ------------------------------------
  107. 0 0 baseline - - 2 Initial measurement
  108. 1 1 discard 2 3 2 Broaden parser fallback
  109. 2 2 keep 2 1 1 Fix nested parser branch
  110. 3 3 keep 1 0 0 Remove final parser error
  111. 4 3 complete - - 0 retained metric satisfies the target
  112. ```
  113. The same validated events can be exported as TSV or rendered as a self-contained static report:
  114. ```text
  115. $codex-autoresearch export experiment history as TSV
  116. $codex-autoresearch generate an HTML report
  117. ```
  118. The report is written to `autoresearch-results/report.html`. It is a replaceable snapshot, not runtime state.
  119. <p align="center">
  120. <img src="image/autoresearch-report.png" width="900" alt="Codex Autoresearch HTML report showing metric trajectory and experiment history">
  121. </p>
  122. ## Safety Model
  123. - Every trial is a Git commit.
  124. - A non-improving trial or failed guard is reverted with `git revert`.
  125. - Out-of-scope edits, branch changes, HEAD drift, malformed metrics, command failures, timeouts, and generated byproducts stop the run with an exact error and log path.
  126. - Autoresearch artifacts are never staged.
  127. - A run reports `complete` only when the retained metric reaches the confirmed target.
  128. - A genuine external blocker is reported explicitly; a difficult or unsuccessful hypothesis is not treated as blocked.
  129. This strictness is intentional. Silent recovery makes long autonomous runs impossible to trust.
  130. ## Good Metrics
  131. The verify command must exit successfully and place one finite number on its final non-empty stdout line. It may instead print a JSON object on that line when Codex names one numeric key explicitly.
  132. ```text
  133. 7
  134. ```
  135. ```json
  136. {"error_count": 7, "passed": 12}
  137. ```
  138. Use a guard for behavior the metric does not protect, such as a test suite around a latency benchmark. The guard must pass at baseline.
  139. ## Documentation
  140. | Guide | Contents |
  141. |---|---|
  142. | [Installation](docs/INSTALL.md) | Install, update, and verify the skill |
  143. | [User Guide](docs/GUIDE.md) | Configuration, lifecycle, state, and troubleshooting |
  144. | [Examples](docs/EXAMPLES.md) | Practical prompts and metric patterns |
  145. | [Contributing](CONTRIBUTING.md) | Architecture and validation for contributors |
  146. ## FAQ
  147. **Does installation change my Codex settings?**
  148. No. Installation copies the skill files. Use a current Codex release so foreground runs can use the built-in Goal capability.
  149. **Why Full Access?**
  150. Each iteration creates or reverts a Git commit. Restricted sandboxes may block writes under `.git`. Background runs therefore default to Full Access; `workspace-write` remains an explicit option when its limitations are acceptable.
  151. **Can I stop and resume?**
  152. Yes. Interrupt or pause a foreground Goal. For background, invoke `$codex-autoresearch` and ask for status, stop, or resume. A new direction is optional; "resume" continues the confirmed experiment.
  153. **Can it run without Git or across several repos?**
  154. No. Git is the experiment memory and rollback boundary. Use one run per repository so commit ownership and metrics remain unambiguous.
  155. **Is this only for small changes?**
  156. No. One experiment should test one coherent hypothesis. Its size should match the hypothesis, while still being independently measurable and reversible.
  157. ## Acknowledgments
  158. Inspired by [Karpathy's autoresearch](https://github.com/karpathy/autoresearch), generalized for Codex and software repositories.
  159. ## Citation
  160. ```bibtex
  161. @misc{codex-autoresearch,
  162. author = {Li, Linxiao},
  163. title = {Codex Autoresearch: Autonomous Goal-Driven Experimentation for Codex},
  164. year = {2026},
  165. publisher = {GitHub},
  166. url = {https://github.com/leo-lilinxiao/codex-autoresearch}
  167. }
  168. ```
  169. GitHub also reads [CITATION.cff](CITATION.cff) for its **Cite this repository** menu.
  170. ## Star History
  171. <a href="https://www.star-history.com/?repos=leo-lilinxiao%2Fcodex-autoresearch&type=timeline&legend=top-left">
  172. <picture>
  173. <source media="(prefers-color-scheme: dark)" srcset="https://api.star-history.com/image?repos=leo-lilinxiao/codex-autoresearch&type=timeline&theme=dark&legend=top-left" />
  174. <source media="(prefers-color-scheme: light)" srcset="https://api.star-history.com/image?repos=leo-lilinxiao/codex-autoresearch&type=timeline&legend=top-left" />
  175. <img alt="Star History Chart" src="https://api.star-history.com/image?repos=leo-lilinxiao/codex-autoresearch&type=timeline&legend=top-left" />
  176. </picture>
  177. </a>
  178. ## License
  179. MIT, see [LICENSE](LICENSE).

Ratings and comments

Sign in to rate this skill.

Sign in

Free to use. If it’s helped, pass it on.