Skip to content
Back to search

ShinkaEvolve

ShinkaEvolve: Towards Open-Ended and Sample-Efficient Program Evolution ๐Ÿงฌ

Safety scan

A static text scan, not a guarantee. Select the badge to see what was scanned and every finding.

Checked today

Check it deeper on your machine

It has changed since its scan on 11 Nov 2026.

As a Claude Code plugin, install it and then ask Claude Code:

/plugin marketplace add keithmackay/mackayi/plugin install scanskillsafety@mackayiDo a deep scan of https://github.com/SakanaAI/ShinkaEvolve with scanskillsafety

Or from a terminal, clone it and run it:

git clone https://github.com/keithmackay/scanskillsafety-skill ~/.claude/skills/scanskillsafetynode ~/.claude/skills/scanskillsafety/cli.cjs --deep 'https://github.com/SakanaAI/ShinkaEvolve'

It checks the exact commit: every file, its packages against OSV (api.osv.dev), how it compares with this page's scan (findsafeskills.com receives the repository's address), and, if you agree, Claude reading every file with your own Claude subscription. It never runs the code.

Badge for your README
findsafeskills safety rating

Markdown

[![findsafeskills safety rating](/api/listings/sakanaai--shinkaevolve/badge.svg)](/listings/sakanaai--shinkaevolve)

HTML

<a href="/listings/sakanaai--shinkaevolve"><img src="/api/listings/sakanaai--shinkaevolve/badge.svg" alt="findsafeskills safety rating" /></a>

Install

i

Install instructions from the README

As written by the repo's author. findsafeskills has not verified these steps.

# Install from PyPI
pip install shinka-evolve

# Or with uv
uv pip install shinka-evolve

# Run your first evolution experiment
shinka_launch variant=circle_packing_example

The distribution name is `shinka-evolve`; Python imports stay `import shinka`.

`shinka_launch` still supports the original shorthand group overrides:

shinka_launch variant=circle_packing_example
shinka_launch task=novelty_generator database=island_small

Built-in Hydra presets ship inside the package under `shinka/configs/`. To add your own presets from a PyPI install without cloning the repo, place them in your own config directory and pass `--config-dir`:

mkdir -p ~/my-shinka-configs/variant
$EDITOR ~/my-shinka-configs/variant/my_variant.yaml
shinka_launch --config-dir ~/my-shinka-configs variant=my_variant

For development installs from source:

git clone https://github.com/SakanaAI/ShinkaEvolve
cd ShinkaEvolve
uv venv --python 3.11
source .venv/bin/activate  # On Windows: .venv\Scripts\activate
uv pip install -e .

For detailed installation instructions and usage examples, see the [Getting Started Guide](https://sakanaai.github.io/ShinkaEvolve/getting_started/).

What was scanned

The files the safety scan read, as written by the repository's author and shown as plain text. Lines with a finding are marked.

README.md ยท README ยท 581 lines

View this file on GitHub (read at the repository's current version: the scan did not record which commit it read)

  1. <h1 align="center">
  2. <a href="https://github.com/SakanaAI/ShinkaEvolve"><img src="https://raw.githubusercontent.com/SakanaAI/ShinkaEvolve/main/shinka/favicon.png" width="180" /></a><br>
  3. <b><code>ShinkaEvolve</code>: Towards Open-Ended and Sample-Efficient Program Evolution ๐Ÿงฌ</b><br>
  4. </h1>
  5. <p align="center">
  6. <img src="https://img.shields.io/badge/python-%3E%3D3.10-blue" />
  7. <a href="https://pypi.org/project/shinka-evolve/"><img src="https://img.shields.io/pypi/v/shinka-evolve.svg" /></a>
  8. <a href="https://github.com/SakanaAI/ShinkaEvolve/blob/main/LICENSE"><img src="https://img.shields.io/badge/license-Apache2.0-blue.svg" /></a>
  9. <a href="https://github.com/astral-sh/ruff"><img src="https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json" /></a>
  10. <a href="https://arxiv.org/abs/2509.19349"><img src="http://img.shields.io/badge/paper-arxiv.2509.19349-B31B1B.svg" /></a>
  11. <a href="https://sakana.ai/shinka-evolve/"><img src="https://img.shields.io/badge/Blog%20%7C%20Sakana-0A66C2.svg" /></a>
  12. <a href="https://colab.research.google.com/github/SakanaAI/ShinkaEvolve/blob/main/examples/shinka_tutorial.ipynb"><img src="https://colab.research.google.com/assets/colab-badge.svg" /></a>
  13. </p>
  14. [`shinka`](https://sakana.ai/shinka-evolve/) is a framework that combines Large Language Models (LLMs) with evolutionary algorithms to drive scientific discovery. By leveraging the creative capabilities of LLMs and the optimization power of evolutionary search, `shinka` enables automated exploration and improvement of scientific code. The system is inspired by the [AI Scientist](https://sakana.ai/ai-scientist/), [AlphaEvolve](https://deepmind.google/discover/blog/alphaevolve-a-gemini-powered-coding-agent-for-designing-advanced-algorithms/) and the [Darwin Goedel Machine](https://sakana.ai/dgm/): It maintains a population of programs that evolve over generations, with an ensemble of LLMs acting as intelligent mutation operators that suggest code improvements.
  15. ---
  16. **May 2026 Update**: Added [Headless](https://github.com/RobertTLange/headless-cli) CLI-backed mutation models for subscription-backed agent usage. Use model strings such as `headless/codex@gpt-5.5?effort=high` or `headless/claude`. Check the [example](https://github.com/SakanaAI/ShinkaEvolve/tree/main/examples/sine_approx_headless) for more detail.
  17. **Apr 2026 Update**: Added the new [documentation website](https://sakanaai.github.io/ShinkaEvolve/) with guides for getting started, configuration, async evolution, local models, WebUI usage, and agentic workflows.
  18. **Mar 2026 Update**: Refactored API and unified runner `ShinkaEvolveRunner` (replacing `EvolutionRunner` and `AsyncEvolutionRunner`). You can now install `shinka` via PyPI and `uv`: `pip install shinka-evolve`.
  19. **Feb 2026 Update**: Added [agent skills](https://sakanaai.github.io/ShinkaEvolve/agentic_usage/) for using `shinka` within coding agents (Claude Code, Codex, etc.) for new task generation ([`shinka-setup`](https://github.com/SakanaAI/ShinkaEvolve/blob/main/skills/shinka-setup/SKILL.md)), converting your repo ([`shinka-convert`](https://github.com/SakanaAI/ShinkaEvolve/blob/main/skills/shinka-convert/SKILL.md)), evolution ([`shinka-run`](https://github.com/SakanaAI/ShinkaEvolve/blob/main/skills/shinka-run/SKILL.md)), and result inspection ([`shinka-inspect`](https://github.com/SakanaAI/ShinkaEvolve/blob/main/skills/shinka-inspect/SKILL.md)). Install them via `npx`:
  20. ```
  21. npx skills add SakanaAI/ShinkaEvolve --skill '*' -a claude-code -a codex -y
  22. ```
  23. **Jan 2026 Update**: ShinkaEvolve was accepted at ICLR 2026 and we [released an update](https://github.com/SakanaAI/ShinkaEvolve/blob/main/CHANGELOG.md) with new features.
  24. **Nov 2025 Update**: Rob gave several public talks about our ShinkaEvolve effort ([Official](https://x.com/SakanaAILabs/status/1989352976792846356?s=20), [AutoML Seminar](https://www.youtube.com/watch?v=dAOIer_1INo)).
  25. **Oct 2025 Update** ShinkaEvolve supported Team Unagi in winning the [ICFP 2025 Programming Contest](https://sakana.ai/icfp-2025/).
  26. ---
  27. The framework supports **parallel evaluation of candidates** locally or on a Slurm cluster. It maintains an archive of successful solutions, enabling knowledge transfer between different evolutionary islands. `shinka` is particularly well-suited for scientific tasks where there is a verifier available and the goal is to optimize performance metrics while maintaining code correctness and readability.
  28. ![](https://raw.githubusercontent.com/SakanaAI/ShinkaEvolve/main/docs/media/conceptual.png)
  29. ## Documentation ๐Ÿ“
  30. | Guide | Description | What You'll Learn |
  31. |-------|-------------|-------------------|
  32. | ๐Ÿš€ **[First steps](https://sakanaai.github.io/ShinkaEvolve/getting_started/)** | Installation, basic usage, and examples | Setup, first evolution run, core concepts |
  33. | ๐Ÿ““ **[Tutorial](https://github.com/SakanaAI/ShinkaEvolve/blob/main/examples/shinka_tutorial.ipynb)** | Interactive walkthrough of Shinka | Hands-on examples, config, best practices |
  34. | โš™๏ธ **[Config](https://sakanaai.github.io/ShinkaEvolve/configuration/)** | Comprehensive config reference | All config options & advanced features |
  35. | ๐ŸŽจ **[WebUI](https://sakanaai.github.io/ShinkaEvolve/webui/)** | Interactive visualization and monitoring | Real-time tracking, result analysis, debugging |
  36. | โšก **[Async Evo](https://sakanaai.github.io/ShinkaEvolve/async_evolution/)** | High-perf. throughput (5-10x speedup) | Concurrent processing, proposal/eval tuning |
  37. | ๐Ÿง  **[Local Models](https://sakanaai.github.io/ShinkaEvolve/support_local_models/)** | How to use local LLMs and embeddings with Shinka | Running open-source models & integration tips |
  38. | ๐Ÿค– **[Agentic Use](https://sakanaai.github.io/ShinkaEvolve/agentic_usage/)** | Run Shinka with Claude/Codex skills | CLI install, skill placement, setup/run workflows |
  39. ## Installation & Quick Start ๐Ÿš€
  40. ```bash
  41. # Install from PyPI
  42. pip install shinka-evolve
  43. # Or with uv
  44. uv pip install shinka-evolve
  45. # Run your first evolution experiment
  46. shinka_launch variant=circle_packing_example
  47. ```
  48. The distribution name is `shinka-evolve`; Python imports stay `import shinka`.
  49. `shinka_launch` still supports the original shorthand group overrides:
  50. ```bash
  51. shinka_launch variant=circle_packing_example
  52. shinka_launch task=novelty_generator database=island_small
  53. ```
  54. Built-in Hydra presets ship inside the package under `shinka/configs/`. To add your own presets from a PyPI install without cloning the repo, place them in your own config directory and pass `--config-dir`:
  55. ```bash
  56. mkdir -p ~/my-shinka-configs/variant
  57. $EDITOR ~/my-shinka-configs/variant/my_variant.yaml
  58. shinka_launch --config-dir ~/my-shinka-configs variant=my_variant
  59. ```
  60. For development installs from source:
  61. ```bash
  62. git clone https://github.com/SakanaAI/ShinkaEvolve
  63. cd ShinkaEvolve
  64. uv venv --python 3.11
  65. source .venv/bin/activate # On Windows: .venv\Scripts\activate
  66. uv pip install -e .
  67. ```
  68. For detailed installation instructions and usage examples, see the [Getting Started Guide](https://sakanaai.github.io/ShinkaEvolve/getting_started/).
  69. ## Examples ๐Ÿ“–
  70. | Example | Description | Environment Setup |
  71. |---------|-------------|-------------------|
  72. | โญ• [Circle Packing](https://github.com/SakanaAI/ShinkaEvolve/tree/main/examples/circle_packing) | Optimize circle packing to maximize radii. | `LocalJobConfig` |
  73. | ๐ŸŽฎ [Game 2048](https://github.com/SakanaAI/ShinkaEvolve/tree/main/examples/game_2048) | Optimize a policy for the Game of 2048. | `LocalJobConfig` |
  74. | โˆ‘ [Julia Prime Counting](https://github.com/SakanaAI/ShinkaEvolve/tree/main/examples/julia_prime_counting) | Optimize a Julia solver for prime-count queries. | `LocalJobConfig` |
  75. | ๐Ÿ”ฅ [Fortran Heat Diffusion](https://github.com/SakanaAI/ShinkaEvolve/tree/main/examples/fortran_heat_diffusion) | Optimize a compiled Fortran stencil solver. | `LocalJobConfig` |
  76. | ๐Ÿงฎ [Wolfram GCD Sum](https://github.com/SakanaAI/ShinkaEvolve/tree/main/examples/wolfram_gcd_sum) | Optimize a Wolfram Language GCD-sum solver. | `LocalJobConfig` |
  77. | โœจ [Novelty Generator](https://github.com/SakanaAI/ShinkaEvolve/tree/main/examples/novelty_generator) | Generate creative, surprising outputs (e.g., ASCII art). | `LocalJobConfig` |
  78. | โˆฟ [Sine Approx Headless](https://github.com/SakanaAI/ShinkaEvolve/tree/main/examples/sine_approx_headless) | Evolve a bounded sine approximation using Headless subscription-backed mutation calls. | `LocalJobConfig` |
  79. | โšก [RTLLM PPA](https://github.com/SakanaAI/ShinkaEvolve/tree/main/examples/rtllm) | Evolve Verilog RTL for power/performance/area under a fixed spec (RTLLM v2.0). Requires `iverilog` + `yosys` + `OpenSTA`. | `LocalJobConfig` |
  80. ## `shinka` Run with Python API ๐Ÿ
  81. For the simplest setup with default settings, you only need to specify the evaluation program:
  82. ```python
  83. from shinka.core import ShinkaEvolveRunner, EvolutionConfig
  84. from shinka.database import DatabaseConfig
  85. from shinka.launch import LocalJobConfig, SlurmCondaJobConfig, SlurmDockerJobConfig
  86. # Minimal - only specify what's required
  87. job_conf = LocalJobConfig(eval_program_path="evaluate.py")
  88. # Or source a uv/venv environment per job:
  89. # job_conf = LocalJobConfig(
  90. # eval_program_path="evaluate.py",
  91. # activate_script=".venv/bin/activate",
  92. # )
  93. # Or run evaluations on SLURM:
  94. # job_conf = SlurmCondaJobConfig(
  95. # eval_program_path="evaluate.py",
  96. # partition="gpu",
  97. # time="01:00:00",
  98. # cpus=1,
  99. # gpus=1,
  100. # mem="8G",
  101. # conda_env="shinka",
  102. # )
  103. # Or run evaluations in a Docker container on SLURM:
  104. # job_conf = SlurmDockerJobConfig(
  105. # eval_program_path="evaluate.py",
  106. # image="ubuntu:latest",
  107. # partition="gpu",
  108. # time="01:00:00",
  109. # cpus=1,
  110. # gpus=1,
  111. # mem="8G",
  112. # )
  113. db_conf = DatabaseConfig()
  114. evo_conf = EvolutionConfig(init_program_path="initial.py")
  115. runner = ShinkaEvolveRunner(
  116. evo_config=evo_conf,
  117. job_config=job_conf,
  118. db_config=db_conf,
  119. max_evaluation_jobs=2,
  120. max_proposal_jobs=3, # modest oversubscription when proposal generation is slower than eval
  121. max_db_workers=4,
  122. )
  123. runner.run()
  124. ```
  125. ### Live model pricing
  126. Shinka refreshes supported model metadata and token prices from
  127. [`models.dev`](https://models.dev) when a new run starts. Requests use HTTP
  128. cache validation, then fall back to the last validated user-cache response or
  129. the packaged snapshot when offline. The exact catalog used by a run is written
  130. to `pricing_snapshot.json` in its results directory and reused when that run is
  131. resumed.
  132. Set `SHINKA_PRICING_MODE=offline` to skip the network check, or
  133. `SHINKA_PRICING_MODE=required` to fail startup when live pricing cannot be
  134. validated. Run `shinka_models --verbose` to inspect catalog provenance and the
  135. models available for configured provider credentials.
  136. ### Weights & Biases logging
  137. Install the optional W&B integration and enable it for a run:
  138. ```bash
  139. pip install 'shinka-evolve[wandb]'
  140. # Authenticate online runs. In CI, provide this through a secret manager.
  141. export WANDB_API_KEY=<your-api-key>
  142. shinka_run --task-dir examples/circle_packing \
  143. --results_dir results/circle_wandb \
  144. --num_generations 20 \
  145. --set evo.enable_wandb_logging=true \
  146. --set evo.wandb_project=shinka-evolve
  147. ```
  148. W&B logging is additive: the existing SQLite database and WebUI logging remain
  149. enabled. Population and island snapshots use the monotonic
  150. `population/evaluated_count` axis for counts, correctness, scores, cumulative
  151. `cost/*`, and cumulative `timing/*_total`. Raw evaluated candidates use
  152. `score/individual` and `individual/*` fields without binding those events to a
  153. generation step. Administrative island copies remain in population/table counts
  154. but are excluded from evaluated count, candidate history, and costs. The final
  155. population is available as `population/final_table`, without program code or
  156. embeddings. Resuming the same results directory reuses its persisted W&B run ID
  157. by default. Online mode uses credentials from `wandb login` or `WANDB_API_KEY`;
  158. use `wandb_mode=offline` to record locally without uploading. See
  159. [Configuration](docs/configuration.md#evolutionconfig-shinkacoreconfigevolutionconfig)
  160. for all W&B options.
  161. Shinka disables W&B console, Git, code, requirements, machine, and system-stat
  162. capture. Run configuration contains an explicit allowlist of scalar experiment
  163. settings plus the recursively redacted `wandb_config` extension. Arbitrary
  164. `wandb_config` values cross the upload boundary when their keys are not
  165. recognized as sensitive, so do not put secrets or private data there. Candidate
  166. telemetry excludes private metrics and logs at most 64 public numeric metrics
  167. with keys no longer than 128 characters; secret-like metric names are dropped.
  168. Population snapshots use a dedicated serialized telemetry worker and a compact
  169. database aggregate, throttled to at most once every five seconds. Overlapping
  170. requests are coalesced so telemetry cannot stall active job processing. The
  171. bounded candidate queue drops the newest raw event with a rate-limited warning
  172. if saturated; population and final snapshots remain authoritative.
  173. <details>
  174. <summary><strong>EvolutionConfig Parameters</strong> (click to expand)</summary>
  175. Class defaults below come from `shinka/core/config.py` (`EvolutionConfig`). Hydra presets and CLI overrides can replace these values. Concurrency lives on `ShinkaEvolveRunner` via `max_evaluation_jobs`, `max_proposal_jobs`, and `max_db_workers`; the shared Hydra async launch path currently defaults to `4/6/2` for evaluation/proposal/DB workers.
  176. | Key | Default Value | Type | Explanation |
  177. |-----|---------------|------|-------------|
  178. | `task_sys_msg` | `"You are an expert optimization and algorithm design assistant. Improve the program while preserving correctness and immutable regions."` | `Optional[str]` | System message describing the optimization task |
  179. | `patch_types` | `["diff", "full", "cross"]` | `List[str]` | Types of patches to generate: "diff", "full", "cross" |
  180. | `patch_type_probs` | `[0.6, 0.3, 0.1]` | `List[float]` | Probabilities for each patch type |
  181. | `num_generations` | `50` | `int` | Number of evolution generations to run |
  182. | `max_patch_resamples` | `3` | `int` | Max times to resample a patch if it fails |
  183. | `max_patch_attempts` | `1` | `int` | Max attempts to generate a valid patch |
  184. | `job_type` | `"local"` | `str` | Job execution type: "local", "slurm_docker", "slurm_conda" |
  185. | `language` | `"python"` | `str` | Programming language for evolution |
  186. | `llm_models` | `["gpt-5-mini", "gemini-3-flash-preview", "gemini-3.1-pro-preview", "gpt-5.4"]` | `List[str]` | List of LLM models for code generation |
  187. | `llm_dynamic_selection` | `"ucb"` | `Optional[Union[str, BanditBase]]` | Dynamic model selection strategy |
  188. | `llm_dynamic_selection_kwargs` | `{"cost_aware_coef": 0.5}` | `dict` | Kwargs for dynamic selection |
  189. | `llm_kwargs` | `{"temperatures": [0.0, 0.5, 1.0], "max_tokens": 16384}` | `dict` | Additional kwargs for LLM calls |
  190. | `meta_rec_interval` | `10` | `Optional[int]` | Interval for meta-recommendations |
  191. | `meta_llm_models` | `None` | `Optional[List[str]]` | LLM models for meta-recommendations |
  192. | `meta_llm_kwargs` | `{}` | `dict` | Kwargs for meta-recommendation LLMs |
  193. | `meta_max_recommendations` | `5` | `int` | Max number of meta-recommendations |
  194. | `sample_single_meta_rec` | `True` | `bool` | Sample a single recommendation from meta output when enabled |
  195. | `embedding_model` | `"text-embedding-3-small"` | `Optional[str]` | Model for code embeddings. Also accepts `local/<model>@http(s)://host[:port]/v1` for local OpenAI-compatible embedding servers, with optional `?api_key_env=ENV_VAR` for per-model credentials. |
  196. | `init_program_path` | `"initial.py"` | `Optional[str]` | Path to initial program to evolve |
  197. | `results_dir` | `None` | `Optional[str]` | Directory to save results (auto-generated if None) |
  198. | `enable_wandb_logging` | `False` | `bool` | Mirror evolution metrics to W&B without disabling SQLite or WebUI logging |
  199. | `wandb_project` | `"shinka-evolve"` | `Optional[str]` | W&B project used when logging is enabled |
  200. | `wandb_entity` | `None` | `Optional[str]` | Optional W&B entity or team |
  201. | `wandb_group` | `None` | `Optional[str]` | Optional W&B run group |
  202. | `wandb_name` | `None` | `Optional[str]` | Optional run name; defaults to the results directory name |
  203. | `wandb_mode` | `None` | `Optional[str]` | Optional W&B mode such as `offline` or `disabled` |
  204. | `wandb_tags` | `[]` | `List[str]` | Optional W&B tags |
  205. | `wandb_notes` | `None` | `Optional[str]` | Optional W&B run notes |
  206. | `wandb_dir` | `None` | `Optional[str]` | Optional local W&B directory; defaults to `results_dir` |
  207. | `wandb_run_id` | `None` | `Optional[str]` | Optional W&B run ID; otherwise generated and persisted in the results directory |
  208. | `wandb_resume` | `"allow"` | `str` | W&B resume policy used with the persisted run ID |
  209. | `wandb_config` | `{}` | `Dict[str, Any]` | Extra values merged into the W&B run config |
  210. | `max_novelty_attempts` | `3` | `int` | Max attempts for novelty generation |
  211. | `code_embed_sim_threshold` | `0.99` | `float` | Similarity threshold for code embeddings |
  212. | `novelty_llm_models` | `None` | `Optional[List[str]]` | LLM models for novelty judgment |
  213. | `novelty_llm_kwargs` | `{}` | `dict` | Kwargs for novelty LLMs |
  214. | `use_text_feedback` | `False` | `bool` | Whether to use text feedback in evolution |
  215. | `max_api_costs` | `None` | `Optional[float]` | Total API budget cap (USD); async runner stops new proposals at cap |
  216. | `enable_controlled_oversubscription` | `False` | `bool` | Enable bounded proposal oversubscription when proposal generation is slower than evaluation. |
  217. | `proposal_target_mode` | `'adaptive'` | `str` | Proposal target controller mode (`adaptive` or `fixed`). |
  218. | `proposal_target_min_samples` | `5` | `int` | Minimum completed timing samples before adaptive targeting activates. |
  219. | `proposal_target_ratio_cap` | `2.0` | `float` | Maximum sampling/evaluation ratio used by the adaptive controller. |
  220. | `proposal_buffer_max` | `2` | `int` | Maximum extra proposal jobs beyond evaluation concurrency. |
  221. | `proposal_target_hard_cap` | `None` | `Optional[int]` | Absolute cap for the adaptive proposal target. |
  222. | `proposal_target_ewma_alpha` | `0.3` | `float` | EWMA smoothing factor for proposal/evaluation timing estimates. |
  223. | `inspiration_sort_order` | `"ascending"` | `str` | Inspiration ordering (`"ascending"`, `"chronological"`, `"none"`) |
  224. | `evolve_prompts` | `False` | `bool` | Enable meta-prompt evolution loop |
  225. | `prompt_patch_types` | `["diff", "full"]` | `List[str]` | Patch formats used for prompt evolution |
  226. | `prompt_patch_type_probs` | `[0.7, 0.3]` | `List[float]` | Sampling probabilities for prompt patch formats |
  227. | `prompt_evolution_interval` | `None` | `Optional[int]` | Prompt-evolution cadence in generations (`None` disables periodic updates) |
  228. | `prompt_archive_size` | `10` | `int` | Size of system-prompt archive |
  229. | `prompt_llm_models` | `None` | `Optional[List[str]]` | LLM models for prompt evolution (`None` falls back to `llm_models`) |
  230. | `prompt_llm_kwargs` | `{}` | `dict` | Extra kwargs for prompt-evolution LLM calls |
  231. | `prompt_ucb_exploration_constant` | `1.0` | `float` | UCB exploration constant for prompt sampling |
  232. | `prompt_epsilon` | `0.1` | `float` | Epsilon-greedy exploration probability for prompt sampling |
  233. | `prompt_evo_top_k_programs` | `3` | `int` | Number of top programs used as context in prompt evolution |
  234. | `prompt_percentile_recompute_interval` | `20` | `int` | Generations between prompt percentile recomputations |
  235. </details>
  236. <details>
  237. <summary><strong>DatabaseConfig Parameters</strong> (click to expand)</summary>
  238. Class defaults below come from `shinka/database/dbase.py` (`DatabaseConfig`). Hydra presets and CLI overrides can replace these values.
  239. | Key | Default Value | Type | Explanation |
  240. |-----|---------------|------|-------------|
  241. | `db_path` | `None` | `Optional[str]` | Database file path (auto-generated if None) |
  242. | `num_islands` | `2` | `int` | Number of evolution islands for diversity |
  243. | `archive_size` | `40` | `int` | Global archive size cap |
  244. | `elite_selection_ratio` | `0.3` | `float` | Proportion of elite programs for inspiration |
  245. | `num_archive_inspirations` | `1` | `int` | Number of archive programs to use as inspiration |
  246. | `num_top_k_inspirations` | `1` | `int` | Number of top-k programs for inspiration |
  247. | `migration_interval` | `10` | `int` | Generations between island migrations |
  248. | `migration_rate` | `0.0` | `float` | Proportion of island population to migrate |
  249. | `island_elitism` | `True` | `bool` | Keep best programs on their original islands |
  250. | `enforce_island_separation` | `True` | `bool` | Enforce full separation between islands |
  251. | `island_selection_strategy` | `"uniform"` | `str` | Island sampler (`"uniform"`, `"equal"`, `"proportional"`, `"weighted"`) |
  252. | `enable_dynamic_islands` | `False` | `bool` | Enable stagnation-triggered island spawning |
  253. | `stagnation_threshold` | `100` | `int` | Generations without improvement before spawning a new island |
  254. | `island_spawn_strategy` | `"initial"` | `str` | New-island seed strategy (`"initial"`, `"best"`, `"archive_random"`) |
  255. | `island_spawn_subtree_size` | `1` | `int` | Number of programs copied when spawning an island |
  256. | `parent_selection_strategy` | `"weighted"` | `str` | Parent selection: "weighted", "power_law", "beam_search" |
  257. | `exploitation_alpha` | `1.0` | `float` | Power-law exponent (0=uniform, 1=power-law) |
  258. | `exploitation_ratio` | `0.2` | `float` | Chance to pick parent from archive |
  259. | `parent_selection_lambda` | `10.0` | `float` | Sharpness of sigmoid for weighted selection |
  260. | `num_beams` | `5` | `int` | Number of beams for beam search selection |
  261. | `archive_selection_strategy` | `"fitness"` | `str` | Archive replacement strategy (`"fitness"` or `"crowding"`) |
  262. | `archive_criteria` | `{"combined_score": 1.0}` | `Dict[str, float]` | Weighted ranking criteria used by fitness archive updates |
  263. </details>
  264. <details>
  265. <summary><strong>JobConfig Parameters</strong> (click to expand)</summary>
  266. **LocalJobConfig** (for local execution):
  267. | Key | Default Value | Type | Explanation |
  268. |-----|---------------|------|-------------|
  269. | `eval_program_path` | `"evaluate.py"` | `Optional[str]` | Path to evaluation script |
  270. | `extra_cmd_args` | `{}` | `Dict[str, Any]` | Additional command line arguments |
  271. | `time` | `None` | `Optional[str]` | Time limit for job execution |
  272. | `conda_env` | `None` | `Optional[str]` | Conda environment to run jobs in |
  273. | `activate_script` | `None` | `Optional[str]` | Sourceable env script path, e.g. `.venv/bin/activate` |
  274. **SlurmDockerJobConfig** (for SLURM with Docker):
  275. | Key | Default Value | Type | Explanation |
  276. |-----|---------------|------|-------------|
  277. | `eval_program_path` | `"evaluate.py"` | `Optional[str]` | Path to evaluation script |
  278. | `extra_cmd_args` | `{}` | `Dict[str, Any]` | Additional command line arguments |
  279. | `image` | `"ubuntu:latest"` | `str` | Docker image to use |
  280. | `image_tar_path` | `None` | `Optional[str]` | Path to Docker image tar file |
  281. | `docker_flags` | `""` | `str` | Additional Docker flags |
  282. | `partition` | `"gpu"` | `str` | SLURM partition to use |
  283. | `time` | `"01:00:00"` | `str` | Job time limit |
  284. | `cpus` | `1` | `int` | Number of CPUs to request |
  285. | `gpus` | `1` | `int` | Number of GPUs to request |
  286. | `mem` | `"8G"` | `Optional[str]` | Memory to request |
  287. **SlurmCondaJobConfig / SlurmEnvJobConfig** (for SLURM with sourced or Conda environments):
  288. | Key | Default Value | Type | Explanation |
  289. |-----|---------------|------|-------------|
  290. | `eval_program_path` | `"evaluate.py"` | `Optional[str]` | Path to evaluation script |
  291. | `extra_cmd_args` | `{}` | `Dict[str, Any]` | Additional command line arguments |
  292. | `conda_env` | `""` | `str` | Conda environment name |
  293. | `activate_script` | `None` | `Optional[str]` | Sourceable env script path, e.g. `.venv/bin/activate` |
  294. | `modules` | `[]` | `Optional[List[str]]` | Environment modules to load |
  295. | `partition` | `"gpu"` | `str` | SLURM partition to use |
  296. | `time` | `"01:00:00"` | `str` | Job time limit |
  297. | `cpus` | `1` | `int` | Number of CPUs to request |
  298. | `gpus` | `1` | `int` | Number of GPUs to request |
  299. | `mem` | `"8G"` | `Optional[str]` | Memory to request |
  300. `conda_env` and `activate_script` are mutually exclusive.
  301. </details>
  302. ### Evaluation Setup & Initial Solution ๐Ÿƒ
  303. To use `ShinkaEvolveRunner`, you need two key files: The **`evaluate.py`** script defines how to test and score your programs - it runs multiple evaluations, validates results, and aggregates them into metrics that guide the `shinka` evolution loop. The **`initial.py`** file contains your starting solution with the core algorithm that will be iteratively improved by LLMs across generations.
  304. <table>
  305. <tr>
  306. <td width="50%">
  307. **`evaluate.py` - Evaluation Script**
  308. ```python
  309. from shinka.core import run_shinka_eval
  310. def main(program_path: str,
  311. results_dir: str):
  312. metrics, correct, err = run_shinka_eval(
  313. program_path=program_path,
  314. results_dir=results_dir,
  315. experiment_fn_name="run_experiment",
  316. num_runs=3, # Multi-evals to aggreg.
  317. run_workers=1, # >1 => per-run parallel
  318. get_experiment_kwargs=get_kwargs,
  319. aggregate_metrics_fn=aggregate_fn,
  320. validate_fn=validate_fn, # Optional
  321. )
  322. def get_kwargs(run_idx: int) -> dict:
  323. return {"param1": "value", "param2": 42}
  324. def aggregate_fn(results: list) -> dict:
  325. score = results[0]
  326. text = results[1]
  327. return {
  328. "combined_score": float(score),
  329. "public": {...}, # shinka-visible
  330. "private": {...}, # shinka-invisible
  331. "extra_data": {...}, # store as pkl
  332. "text_feedback": text, # str fb
  333. }
  334. if __name__ == "__main__":
  335. # argparse program path & dir
  336. main(program_path, results_dir)
  337. ```
  338. </td>
  339. <td width="50%">
  340. **`initial.py` - Starting Solution**
  341. ```python
  342. # EVOLVE-BLOCK-START
  343. def advanced_algo():
  344. # This will be evolved
  345. return solution
  346. # EVOLVE-BLOCK-END
  347. def run_experiment(**kwargs):
  348. """Main called by evaluator"""
  349. result = solve_problem(kwargs)
  350. return result
  351. def solve_problem(params):
  352. solution = advanced_algo()
  353. return solution
  354. ```
  355. **Key Points:**
  356. - Eval name matches `experiment_fn_name`
  357. - Use `EVOLVE-BLOCK-START` and `EVOLVE-BLOCK-END` to mark evolution sections
  358. - Return format matches validation expectations
  359. - Dependencies must be available in env
  360. - Results can be unpacked for metrics
  361. - Auto-stores several results in `results_dir`
  362. - Can add text feedback in `shinka` loop
  363. - Higher `combined_score` values indicate better performance (maximization)
  364. </td>
  365. </tr>
  366. </table>
  367. ## `shinka` Launcher with Hydra ๐Ÿš€
  368. `shinka` Launcher utilizes [Hydra](https://hydra.cc/) to configure and launch evolutionary experiments effortlessly. It supports concise configuration via Hydra's powerful override syntax, making it easy to manage and iterate scientific explorations.
  369. ```bash
  370. # Run with the shared default baseline
  371. shinka_launch
  372. # Run with custom parameters
  373. shinka_launch \
  374. task=circle_packing \
  375. database=island_large \
  376. evolution=small_budget \
  377. cluster=local \
  378. evo_config.num_generations=20
  379. ```
  380. For comprehensive configuration options and advanced usage, see the [Configuration Guide](https://sakanaai.github.io/ShinkaEvolve/configuration/).
  381. ## `shinka_run` Agent CLI ๐Ÿค–
  382. `shinka_run` is a task-directory launcher for async evolution. It is designed for agent workflows and does not require Hydra config files.
  383. ```bash
  384. # Inspect full interface (detailed help)
  385. shinka_run --help
  386. # Minimal run
  387. shinka_run \
  388. --task-dir examples/circle_packing \
  389. --results_dir results/circle_agent_run \
  390. --num_generations 20
  391. # Run with keyword overrides
  392. shinka_run \
  393. --task-dir examples/circle_packing \
  394. --results_dir results/circle_agent_custom \
  395. --num_generations 50 \
  396. --max-evaluation-jobs 6 \
  397. --set db.num_islands=2 \
  398. --set job.time=00:10:00 \
  399. --set job.activate_script=.venv/bin/activate \
  400. --set evo.llm_models='["gpt-5-mini","gemini-3-flash-preview"]'
  401. # Load optional YAML config (relative to --task-dir), then override via --set
  402. shinka_run \
  403. --task-dir examples/circle_packing \
  404. --config-fname shinka_small.yaml \
  405. --results_dir results/circle_agent_from_yaml \
  406. --num_generations 50 \
  407. --set db.num_islands=2
  408. ```
  409. `--task-dir` must contain `evaluate.py` and `initial.<ext>`.
  410. `--config-fname` can define `evo/db/job` (or `evo_config/db_config/job_config`) plus `max_evaluation_jobs/max_proposal_jobs/max_db_workers` and `verbose/debug`.
  411. Precedence: config YAML < `--set` < authoritative flags.
  412. `--results_dir` and `--num_generations` are authoritative and always override config/`--set` values for `evo.results_dir` and `evo.num_generations`.
  413. ### Headless Agent Models
  414. Use `headless/<agent>` model strings to route mutation calls through the local Headless CLI instead of provider API clients. Shinka uses `npx -y @roberttlange/headless` by default and runs `headless --check` before evolution starts.
  415. ```bash
  416. shinka_run \
  417. --task-dir examples/sine_approx_headless \
  418. --results_dir results/sine_approx_headless \
  419. --num_generations 5 \
  420. --max-evaluation-jobs 1 \
  421. --max-proposal-jobs 1 \
  422. --set evo.llm_models='["headless/codex@gpt-5.5?effort=high"]' \
  423. --set evo.embedding_model=null \
  424. --set evo.patch_types='["full", "diff"]' \
  425. --set evo.patch_type_probs='[0.5, 0.5]'
  426. ```
  427. For a Python runner using both Codex and Claude through Headless:
  428. ```bash
  429. python examples/sine_approx_headless/run_evo.py
  430. ```
  431. ## Interactive WebUI ๐ŸŽจ
  432. Monitor your evolution experiments in real-time with Shinka's interactive web interface! The WebUI provides live visualization of the evolutionary process, genealogy trees, and performance metrics.
  433. ![WebUI Screenshot](https://raw.githubusercontent.com/SakanaAI/ShinkaEvolve/main/docs/media/webui.png)
  434. Launch the WebUI alongside your evolution experiment:
  435. ```bash
  436. # Start your evolution experiment
  437. shinka_launch
  438. # In another terminal, launch the WebUI
  439. shinka_visualize --port 8888 --open
  440. ```
  441. For detailed WebUI documentation, see the [WebUI Guide](https://sakanaai.github.io/ShinkaEvolve/webui/).
  442. ## Local Docs Development
  443. To preview the documentation site locally on `localhost`:
  444. ```bash
  445. uv sync --group docs
  446. uv run --group docs mkdocs serve --dev-addr 127.0.0.1:8000
  447. ```
  448. Then open `http://127.0.0.1:8000/`.
  449. ## Contributing ๐Ÿ‘ฅ
  450. Contributions are welcome across code, docs, representative benchmarks, and bug reports.
  451. Please read the [contribution guide](https://github.com/SakanaAI/ShinkaEvolve/blob/main/CONTRIBUTING.md) before opening an issue or pull request. It documents the expected issue and PR structure, local checks, and the extra evidence required for changes to the core program evolution pipeline.
  452. If you propose a change to the core evolution pipeline, please include results on a representative runnable task that highlights the new capability and compare them against a baseline. Please do not add random benchmark tasks just to justify a PR.
  453. ## Related Open-Source Projects ๐Ÿง‘โ€๐Ÿ”ง
  454. - [OpenEvolve](https://github.com/codelion/openevolve): An open-source implementation of AlphaEvolve
  455. - [LLM4AD](https://github.com/Optima-CityU/llm4ad): A Platform for Algorithm Design with Large Language Model
  456. ## Citation โœ๏ธ
  457. If you use `ShinkaEvolve` in your research, please cite it as follows:
  458. ```
  459. @article{lange2025shinka,
  460. title={ShinkaEvolve: Towards Open-Ended And Sample-Efficient Program Evolution},
  461. author={Lange, Robert Tjarko and Imajuku, Yuki and Cetin, Edoardo},
  462. journal={arXiv preprint arXiv:2509.19349},
  463. year={2025}
  464. }
  465. ```

Ratings and comments

Sign in to rate this skill.

Sign in

Free to use. If itโ€™s helped, pass it on.